Emoji seem like a simple idea, but there's actually a ton of complexity that goes into an effective emoji implementation. This document discusses a number of these issues.
Currently, Zulip uses the Noto (Android) emoji set, but we are close to being able to support the user choosing which emoji set they want to use.
The Unicode standard has various ranges of characters set aside for emoji. So you can put emoji in your terminal using actual unicode characters like 😀 and 👍. If you paste those into Zulip, Zulip will render them as the corresponding emoji image.
However, the Unicode committee did not standardize on a set of
human-readable names for emoji. So, for example, when using the
: based style for entering emoji from the keyboard, we have
to decide whether to use
:angry_face: to represent an
angry face. Different products use different approaches, but for
purposes like emoji pickers or autocomplete, you definitely want to
pick exactly one of these names, since otherwise users will always be
seeing duplicates of a given emoji next to each other.
Picking which emoji name to use is surprisingly complicated! Zulip
has a nice library,
tools/setup/emoji/emoji_setup_utils.py, which we
use to make these decisions systematically, with a relatively small
list of hand-coded exceptions.
Zulip supports custom user-uploaded emoji. We manage those by having the name of the emoji be its "emoji code", and using an emoji_type field to keep track of it. We are in the progress of migrating Zulip to refer to these emoji only by ID, which is a requirement for being able to support deprecating old realm emoji in a sensible way.
We use the iamcal emoji data package to provide sprite
sheets and individual images for our emoji, as well as a data set of
emoji categories, code points, names, etc. The sprite sheets are used
by the Zulip webapp to display emoji in messages, emoji reactions,
etc. However, we can't use the sprite sheets in some contexts, such
as missed-message and digestemails, that need to have self-contained
assets. For those, we use individual emoji files under
static/generated/emoji. The structure of that repository contains
both files named after the unicode representation of emoji (as actual
image files) as well as symlinks pointing to those emoji.
We need to maintain those both for the names used in the iamcal emoji
data set as well as our old emoji data set (
has a tool,
tools/setup/emoji/build_emoji, that combines the
emoji.json file from iamcal with the old
emoji-map.json data set
to construct the various symlink farms and output files described
below that support our emoji experience.
build_emoji tool generates the set of files under
static/generated/emoji (or really, it generates the
/srv/zulip-emoji-cache/<sha1>/emoji tree, and
static/generated/emoji is a symlink to that tree; we do this in
order to cache old versions to make provisioning and production
deployments super fast in the common case that we haven't changed the
emoji tooling). See our dependencies document
for more details on this strategy.
The emoji tree generated by this process contains several import elements:
emoji_codes.js: A set of mappings used by the Zulip frontend to understand what unicode emoji exist and what their shortnames are, used for autocomplete, emoji pickers, etc. This has been deduplicated using the logic in
tools/setup/emoji/emoji_setup_utils.pyto generally only have
:angry:and not also
:angry_face:, since having both is ugly and pointless for purposes like autocomplete and emoji pickers.
images/emoji/unicode/*.png: A farm of emoji
images/emoji/*.png: A farm of symlinks from emoji names to the
images/emoji/unicode/tree. This is used to serve individual emoji images, as well as for the backend markdown processor to know which emoji names exist and what unicode emoji / images they map to. In this tree, we currently include all of the emoji in
emoji-map.json; this means that if you send
:angry_face:, it won't autocomplete, but will still work (but not in previews).
- Some CSS and PNGs for the emoji spritesheets, used in Zulip for emoji pickers where we would otherwise need to download over 1000 of individual emoji images (which would cause a browser performance problem). We have multiple spritesheets: one for each emoji provider that we support (Google, Twitter, EmojiOne, etc.).