Static asset pipeline¶
This page documents additional information that may be useful when developing new features for Zulip that require front-end changes, especially those that involve adding new files. For a more general overview, see the new feature tutorial.
Our dependencies documentation has useful relevant background as well.
Primary build process¶
Most of the existing JS in Zulip is written in
IIFE-wrapped modules, one per file
static/js directory. We will over time migrate these to
Typescript modules. Stylesheets are written in the Sass extension of
CSS (with the scss syntax), they are converted from plain CSS and we
have yet to take full advantage of the features Sass offers. We use
Webpack to transpile and build JS and CSS bundles that the browser can
understand, one for each entry points specifed in
tools/webpack.assets.json; source maps are generated in the process
for better debugging experience.
In development mode, bundles are built and served on the fly using
webpack-dev-server with live reloading. In production mode (and when creating a
release tarball using
tools/update-prod-static tool (called by both
tools/upgrade-zulip-from-git) is responsible for orchestrating the
webpack build, JS minification and a host of other steps for getting the assets
ready for deployment.
Adding static files¶
first add it to the appropriate place under
- Third-party packages from the NPM repository should be added to
package.jsonfor management by yarn, this allows them to be upgraded easily and not bloat our codebase. Run
./tools/provisionfor yarn to install the new packages and update its lock file. You should also update
version.pyin the same commit. When adding modules to
package.json, please pin specific versions of them (don’t using carets
~, etc). We prefer fixed versions so that when the upstream providers release new versions with incompatible APIs, it can’t break Zulip. We update those versions periodically to ensure we’re running a recent version of third-party libraries.
- Third-party files that we have patched should all go in
static/third/. Tag the commit with “[third]” when adding or modifying a third-party package. Our goal is to the extent possible to eliminate patched third-party code from the project.
static/js. Ideally, new modules should be written in TypeScript (details on this policy below).
- CSS/Sass files lives under
- Custom SVG graphics living under
static/assets/iconsare compiled into custom icon webfonts that live under
tools/setup/generate-custom-icon-webfontaccording to the
For your asset to be included in a development/production bundle, it
needs to be accessible from one of the entry points defined in
- If you plan to only use the file within the app proper, and not on the login
page or other standalone pages, put it in the
appbundle by importing it in
- If it needs to be available both in the app and all
logged-out/portico pages, import it to
static/js/bundles/common.jswhich itself is imported to the
- If it’s just used on a single standalone page (e.g.
/stats), create a new entry point in
tools/webpack.assets.json. Use the
render_bundlefunction in the relevant Jinja2 template to inject the compiled JS and CSS.
If you want to test minified files in development, look for the
PIPELINE_ENABLED = line in
zproject/settings.py and set it to
– or just set
DEBUG = False.
How it works in production¶
You can learn a lot from reading about django-pipeline, but a few useful notes are:
- Zulip installs static assets in production in
/home/zulip/prod-static. When a new version is deployed, before the server is restarted, files are copied into that directory.
- We use the VFL (Versioned File Layout) strategy, where each file in
the codebase (e.g.
favicon.ico) gets a new name (e.g.
favicon.c55d45ae8c58.ico) that contains a hash in it. Each deployment, has a manifest file (e.g.
/home/zulip/deployments/current/staticfiles.json) that maps codebase filenames to serving filenames for that deployment. The benefit of this VFL approach is that all the static files for past deployments can coexist, which in turn eliminates most classes of race condition bugs where browser windows opened just before a deployment can’t find their static assets. It also is necessary for any incremental rollout strategy where different clients get different versions of the site.
- Some paths for files (e.g. emoji) are stored in the
rendered_contentof past messages, and thus cannot be removed without breaking the rendering of old messages (or doing a mass-rerender of old messages).
Webpack provides seemless interoperability between different module
systems such as CommonJS, AMD and ES6. Our JS files are written in the
CommonJS format, which specifies public functions and variables as
properties of the special
module.exports object. We also currently
assign said object to the global
window variable, which is a hack
allowing us to use modules without importing them with the
TypeScript provides more accurate information to development tools,
allowing for better refactoring, auto-completion and static
analysis. TypeScript uses an ES6-like module system. Any declaration
can be made public by adding the
export keyword. Consuming
variables, functions, etc exported from another module should be done
import statement as oppose to accessing them from the
window scope. Internally our typescript compiler is
configured to transpile TS to the ES6 module system.
Read more about these module systems here: