Zulip Documentation · 2019-04-02 · 4.8 Administration and management. . . . . . . . . . . . . ....

Zulip DocumentationRelease 1.4.0

The Zulip Team

Jun 07, 2017

Overview

1 Zulip overview 31.1 Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Installing the Zulip Development environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Running Zulip in production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Ways to contribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Google Summer of Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.6 How to get involved with contributing to Zulip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.7 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Zulip architectural overview 72.1 Key Codebases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Usage assumptions and concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.4 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Directory structure 133.1 Core Python files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 HTML Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.3 JavaScript and other static assets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.5 Management commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.6 Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.7 API and Bots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.8 Production puppet configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.9 Additional Django apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.10 Jinja2 Compatibility Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.11 Translation files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.12 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4 Zulip Roadmap 174.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.2 Major projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.3 Core User Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.4 Social features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.5 Real-time sync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.6 Onboarding issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.7 Production installation issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

i

4.8 Administration and management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.9 Scalability and performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.10 Technology improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.11 Technical Debt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.12 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.13 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.14 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.15 Integrations and bots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.16 Android app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.17 iOS app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.18 Server/webapp support for mobile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.19 Desktop apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.20 Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5 Version History 235.1 Unreleased . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.2 1.6.0 – 2017-06-06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.3 1.5.2 – 2017-06-01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265.4 1.5.1 – 2017-02-07 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265.5 1.5.0 – 2017-02-06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265.6 1.4.3 - 2017-01-29 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.7 1.4.2 - 2016-09-27 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.8 1.4.1 - 2016-09-03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.9 1.4.0 - 2016-08-25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295.10 1.3.13 - 2016-06-21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325.11 1.3.12 - 2016-05-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.12 1.3.11 - 2016-05-02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.13 1.3.10 - 2016-01-21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.14 1.3.9 - 2015-11-16 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.15 1.3.8 - 2015-11-15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.16 1.3.7 - 2015-10-19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

6 Requirements 376.1 Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376.2 Credentials needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

7 Production Installation 397.1 Step 0: Subscribe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.2 Step 1: Install SSL Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.3 Step 2: Download and install latest release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.4 Step 3: Configure Zulip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.5 Step 4: Initialize Zulip database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.6 Step 5: Create a Zulip organization and login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.7 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8 Troubleshooting 438.1 Using supervisorctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438.2 Troubleshooting services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

9 Customize Zulip 479.1 Mobile and desktop apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479.2 Terms of service and Privacy policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479.3 Miscellaneous server settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489.4 Zulip announcement list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489.5 Enjoy your Zulip installation! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

ii

10 Mobile push notification service 4910.1 How to sign up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4910.2 Why this is necessary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4910.3 Security and privacy implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

11 Secure, maintain, and upgrade 5111.1 Upgrading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5111.2 Upgrading from a git repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5311.3 Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5311.4 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5511.5 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5511.6 Securing your Zulip server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5611.7 Management commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

12 Security Model 5912.1 Secure your Zulip server like your email server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5912.2 Encryption and Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5912.3 Messages and History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6012.4 Users and Bots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6112.5 User-uploaded content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6112.6 Final notes and security response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

13 Authentication methods 6313.1 Adding additional methods using python-social-auth . . . . . . . . . . . . . . . . . . . . . . . . . . 6313.2 Remote User SSO Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

14 Postgres database details 6714.1 Remote Postgres database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6714.2 Debugging postgres database issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6814.3 Stopping the Zulip postgres database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6814.4 Debugging issues starting postgres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6814.5 Postgres Vacuuming alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

15 Development environment installation 7115.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7115.2 Recommended setup (Vagrant) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7115.3 Advanced setup (non-Vagrant) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7115.4 Slow internet connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7215.5 Installing remotely . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7215.6 Next steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

16 Vagrant environment setup tutorial 7316.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.2 Step 1: Install Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.3 Step 2: Get Zulip Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7716.4 Step 3: Start the development environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7716.5 Step 4: Developing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8116.6 Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8316.7 Troubleshooting & Common Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8316.8 Specifying a proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

17 Zulip development environment setup without Vagrant 9117.1 Installing directly on Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9117.2 Installing manually on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9117.3 Using Docker (experimental) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

iii

18 Using the Development Environment 99

19 Developing on a remote machine 10119.1 Connecting to the remote environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10119.2 Setting up the development environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10119.3 Running the development server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10219.4 Making changes to code on your remote development server . . . . . . . . . . . . . . . . . . . . . . 103

20 Writing a new integration 10720.1 Types of integrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10720.2 General advice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10820.3 Webhook integrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10820.4 Python script and plugin integrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10920.5 Documenting your integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

21 Webhook walkthrough 11121.1 Step 0: Create fixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11121.2 Step 1: Initialize your webhook python package . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11221.3 Step 2: Create main webhook code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11221.4 Step 3: Create an api endpoint for the webhook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11321.5 Step 4: Create tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11421.6 Step 5: Create documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11621.7 Step 5: Preparing a pull request to zulip/zulip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11721.8 Advanced topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

22 Writing a new application feature 12122.1 General Process in brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12122.2 Example Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

23 Writing views in Zulip 12923.1 What this covers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12923.2 What is a view? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12923.3 Modifying urls.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12923.4 Writing human-readable views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13023.5 Writing API REST endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13023.6 Legacy endpoints used by the web client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13423.7 Webhook integration endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

24 Life of a Request 13524.1 A request is sent to the server, and handled by Nginx . . . . . . . . . . . . . . . . . . . . . . . . . . 13524.2 Nginx secures traffic with SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13524.3 Static files are served directly by Nginx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13524.4 Nginx routes other requests between django and tornado . . . . . . . . . . . . . . . . . . . . . . . . 13624.5 Django routes the request to a view in urls.py files . . . . . . . . . . . . . . . . . . . . . . . . . . . 13624.6 Views serving HTML are internationalized by server path . . . . . . . . . . . . . . . . . . . . . . . 13624.7 API endpoints use REST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13624.8 Django calls rest_dispatch for REST endpoints, and authenticates . . . . . . . . . . . . . . . . . . . 13724.9 The view will authorize the user, extract request variables, and validate them . . . . . . . . . . . . . 13824.10 Results are given as JSON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

25 Reading list 13925.1 General programming/IT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13925.2 Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14025.3 Java/Android . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14025.4 JavaScript/ECMAScript . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

iv

25.5 Git/Version Control Systems (VCS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14025.6 Computer Science/Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14025.7 Community experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14125.8 Competitions/Camps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14125.9 Massive Open Online Courses (MOOC) Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

26 Git & GitHub Guide 14326.1 Quick start: How Zulip uses Git and GitHub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14326.2 Set up Git . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14426.3 How Git is different . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14426.4 Important Git terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14526.5 Get Zulip code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14726.6 Using Git as you work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14826.7 Create a pull request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15526.8 Update a pull request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15726.9 Collaborate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15726.10 Review changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15826.11 Get and stay out of trouble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15826.12 Zulip-specific tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

27 Version control 16527.1 Commit Discipline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16527.2 Commit Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

28 Code style and conventions 16928.1 Be consistent! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16928.2 Lint tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16928.3 Secrets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17028.4 Dangerous constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17028.5 JS array/object manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17228.6 More arbitrary style things . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

29 Python static type checker (mypy) 17729.1 type_debug.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17729.2 Zulip goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17829.3 Installing mypy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17829.4 Running mypy on Zulip’s code locally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17829.5 Excluded files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17929.6 Mypy is there to find bugs in Zulip before they impact users . . . . . . . . . . . . . . . . . . . . . . 17929.7 Annotating strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

30 Reviewing Zulip server code 18130.1 Things to look for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18130.2 Tooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18230.3 Additional Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

31 The chat.zulip.org community 18331.1 This is a bleeding edge development server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18331.2 Community conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18331.3 Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18431.4 Chat meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

32 Using zulipbot 18732.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

v

33 Accessibility 18933.1 Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18933.2 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18933.3 GitHub Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19033.4 Additional Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

34 Testing and writing tests 19134.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19134.2 Running tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19134.3 Schema and initial data changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19234.4 Wiping the test databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19234.5 Local browser testing (local app + web browser) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19334.6 Python 3 Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

35 Linters 19535.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19535.2 Running the linters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19535.3 General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19635.4 Lint checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19635.5 lint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19735.6 Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

36 Backend Django tests 20136.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20136.2 Running tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20136.3 How to write tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20236.4 Zulip Testing Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20236.5 Testing considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

37 JavaScript unit tests 20737.1 HTML output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20737.2 Coverage reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20837.3 Handling dependencies in unit tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20837.4 Creating new test modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

38 Web frontend black-box casperjs tests 21138.1 Debugging Casper.JS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21138.2 Writing Casper tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

39 Travis CI 21539.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21539.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21539.3 Useful debugging tips and tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21639.4 Performance optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

40 Manual testing 21940.1 Basic Stuff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

41 Settings system 23141.1 Server settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23141.2 Realm settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

42 Real-time Push and Events 23542.1 Generation system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23642.2 Delivery system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

vi

42.3 The initial data fetch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

43 Queue processors 23943.1 Adding a new queue processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23943.2 Publishing events into a queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24043.3 Clearing a RabbitMQ queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

44 Zulip bot system 24144.1 The bots system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24144.2 How to run a bot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24244.3 How to develop a bot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24344.4 Bot API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24444.5 Common problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24644.6 Future direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

45 Custom Apps 24945.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24945.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24945.3 A quick note on bots/integrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24945.4 Categories of custom apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25045.5 World Reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25145.6 Zulip Reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25345.7 Deployment issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

46 Unread counts and the pointer 25746.1 Pointer logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25746.2 Unread count logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25846.3 Testing and development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

47 Markdown implementation 26147.1 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26147.2 Changing Zulip’s markdown processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26247.3 Zulip’s Markdown philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26247.4 Zulip’s Changes to Markdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

48 Realms in Zulip 26548.1 Creating Realms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26548.2 Subdomains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

49 Static asset pipeline 26749.1 Primary build process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26749.2 Adding static files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26749.3 How it works in production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26849.4 Webpack/CommonJS/ES6/Typescript modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

50 Schema Migrations 269

51 HTML and CSS 27151.1 Zulip CSS organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27151.2 Editing Zulip CSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27151.3 CSS Style guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27251.4 Validating CSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

52 URL hashes and deep linking 27352.1 Hashchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27352.2 Server-initiated reloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

vii

52.3 All reloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

53 Emoji 27553.1 Emoji codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27553.2 Tooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

54 Full-text search 27754.1 The default full-text search implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27754.2 An optional full-text search implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

55 Analytics 27955.1 Analytics backend overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27955.2 The *Count database tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28055.3 CountStats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28055.4 The FillState table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28055.5 Performance strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28055.6 Backend Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28155.7 LoggingCountStats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28155.8 Analytics UI development and testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

56 Translating Zulip 28356.1 Translation style guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28356.2 Translation process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28456.3 Translators’ workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28456.4 Testing translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28556.5 Setting the default language in Zulip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28656.6 Translation resource files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28656.7 Backend translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28656.8 Frontend translations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28756.9 Transifex config . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28856.10 Transifex CLI setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

57 Clients in Zulip 28957.1 Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28957.2 Integrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

58 Logging and Performance Debugging 291

59 Zulip server release checklist 29359.1 A week before the release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29359.2 Final release preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29359.3 Executing the release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29459.4 Post-release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

60 Zulip PyPI package release checklist 295

61 Documentation 29761.1 Developer and sysadmin documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29761.2 Core website documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29861.3 General user documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

62 General user guide documentation 29962.1 Editing and testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29962.2 Writing documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29962.3 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30162.4 Documentation template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

viii

Zulip Documentation, Release 1.4.0

Zulip is a powerful, open source group chat application. Written in Python and using the Django framework, Zulipsupports both private messaging and group chats via conversation streams.

Zulip also supports fast search, drag-and-drop file uploads, image previews, group private messages, audible notifica-tions, missed-message emails, desktop apps, and much more.

Further information on the Zulip project and its features can be found at https://www.zulip.org and in these docs. Ourcode is available at our GitHub repository.

This set of documents covers installation and contribution instructions.

Contents:

• Overview

• Zulip in production

• Development environment

• Developer tutorials

• Code contribution guide

• Code testing

• Subsystem documentation

Zulip overview | Community | Installing for dev | Installing for production | Ways to contribute | How to get involved| License

Overview 1

https://www.zulip.orghttps://github.com/zulip/


2 Overview

CHAPTER 1

Zulip overview

Zulip is a powerful, open source group chat application. Written in Python and using the Django framework, Zulipsupports both private messaging and group chats via conversation streams.

Zulip also supports fast search, drag-and-drop file uploads, image previews, group private messages, audible notifica-tions, missed-message emails, desktop apps, and much more.

Further information on the Zulip project and its features can be found at https://www.zulip.org.

Community

There are several places online where folks discuss Zulip.

• The primary place is the Zulip development community Zulip server.

• For Google Summer of Code students and applicants, we have a mailing list for help, questions, and announce-ments. But it’s often simpler to visit chat.zulip.org instead.

• We have a public mailing list that is currently pretty low traffic because most discussions happen in our publicZulip instance. We use it to announce Zulip developer community gatherings and ask for feedback on majortechnical or design decisions. It has several hundred subscribers, so you can use it to ask questions about featuresor possible bugs, but please don’t use it ask for generic help getting started as a contributor (e.g. because youwant to do Google Summer of Code). The rest of this page covers how to get involved in the Zulip project indetail.

• Zulip also has a blog and twitter account.

• Last but not least, we use GitHub to track Zulip-related issues (and store our code, of course). Anybody with aGitHub account should be able to create Issues there pertaining to bugs or enhancement requests. We also usePull Requests as our primary mechanism to receive code contributions.

The Zulip community has a Code of Conduct.

3

https://www.zulip.orghttps://zulip.readthedocs.io/en/latest/chat-zulip-org.htmlhttps://groups.google.com/forum/#!forum/zulip-gsochttps://groups.google.com/forum/#!forum/zulip-develhttps://blog.zulip.org/https://twitter.com/zuliposshttps://github.com/zulip/zuliphttps://zulip.readthedocs.io/en/latest/code-of-conduct.html


Installing the Zulip Development environment

The Zulip development environment is the recommended option for folks interested in trying out Zulip. This isdocumented in the developer installation guide.

Running Zulip in production

Zulip in production supports Ubuntu 14.04 Trusty and Ubuntu 16.04 Xenial. Work is ongoing on adding support foradditional platforms. The installation process is documented at https://zulip.org/server.html and in more detail in thedocumentation.

Ways to contribute

Zulip welcomes all forms of contributions! This page documents the Zulip development process.

• Pull requests. Before a pull request can be merged, you need to sign the Dropbox Contributor License Agree-ment. Also, please skim our commit message style guidelines. We encourage early pull requests for work inprogress. Prefix the title of your pull request with [WIP] and reference it when asking for community feedback.When you are ready for final review, remove the [WIP].

• Testing. The Zulip automated tests all run automatically when you submit a pull request, but you can also runthem all in your development environment following the instructions in the testing docs. You can also try outour new desktop client, which is in alpha; we’d appreciate testing and feedback.

• Developer Documentation. Zulip has a growing collection of developer documentation on Read The Docs.Recommended reading for new contributors includes the directory structure and new feature tutorial. You canalso improve Zulip.org.

• Mailing lists and bug tracker. Zulip has a development discussion mailing list and uses GitHub issues . Thereare also lists for the Android and iOS apps. Feel free to send any questions or suggestions of areas whereyou’d love to see more documentation to the relevant list! Please report any security issues you discover [email protected].

• App codebases. This repository is for the Zulip server and web app (including most integrations); the ReactNative Mobile iOS app, Android app, new Electron desktop app, and legacy QT-based desktop app are allseparate repositories.

• Glue code. We maintain a Hubot adapter and several integrations (Phabricator, Jenkins, Puppet, Redmine,and Trello), plus node.js API bindings, an isomorphic JavaScript library, and a full-text search PostgreSQLextension, as separate repos.

• Translations. Zulip is in the process of being translated into 10+ languages, and we love contributions to ourtranslations. See our translating documentation if you’re interested in contributing!

• Code Reviews. Zulip is all about community and helping each other out. Check out #code review onchat.zulip.org to help review PRs and give comments on other people’s work. Everyone is welcome to par-ticipate, even those new to Zulip! Even just checking out the code, manually testing it, and posting on whetheror not it worked is valuable.

4 Chapter 1. Zulip overview

https://zulip.readthedocs.io/en/latest/dev-overview.htmlhttps://zulip.org/server.htmlhttps://zulip.readthedocs.io/en/latest/prod-install.htmlhttps://zulip.readthedocs.io/en/latest/prod-install.htmlhttps://opensource.dropbox.com/cla/https://opensource.dropbox.com/cla/http://zulip.readthedocs.io/en/latest/version-control.html#commit-messageshttp://zulip.readthedocs.io/en/latest/testing.htmlhttps://github.com/zulip/zulip-electronhttps://github.com/zulip/zulip-electron/issues/newhttps://zulip.readthedocs.io/http://zulip.readthedocs.io/en/latest/directory-structure.htmlhttp://zulip.readthedocs.io/en/latest/new-feature-tutorial.htmlhttps://github.com/zulip/zulip.github.iohttps://github.com/zulip/zulip/issueshttps://groups.google.com/forum/#!forum/zulip-androidhttps://groups.google.com/forum/#!forum/zulip-ioshttps://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-androidhttps://github.com/zulip/zulip-electronhttps://github.com/zulip/zulip-desktophttps://github.com/zulip/hubot-zuliphttps://github.com/zulip/phabricator-to-zuliphttps://github.com/zulip/zulip-jenkins-pluginhttps://github.com/matthewbarr/puppet-zuliphttps://github.com/zulip/zulip-redmine-pluginhttps://github.com/zulip/trello-to-zuliphttps://github.com/zulip/zulip-nodehttps://github.com/zulip/zulip-jshttps://github.com/zulip/tsearch_extrashttps://github.com/zulip/tsearch_extrashttps://zulip.readthedocs.io/en/latest/translating.html#testing-translationshttps://chat.zulip.org/#narrow/stream/code.20reviewhttps://zulip.readthedocs.io/en/latest/chat-zulip-org.html


Google Summer of Code

We participated in GSoC in 2016 (with great results) and are participating in 2017 as well. For guidance, please readour GSoC instructions and ideas page and feel free to email our GSoC mailing list.

Note: For GSoC 2017, we will be focused on making our React Native app better rather than developing the JavaAndroid app and React Native app in parallel. You can review our detailed plan for further details on the motivationand logistics.

How to get involved with contributing to Zulip

First, subscribe to the Zulip development discussion mailing list.

The Zulip project uses a system of labels in our issue tracker to make it easy to find a project if you don’t have yourown project idea in mind or want to get some experience with working on Zulip before embarking on a larger projectyou have in mind:

• Integrations. Integrate Zulip with another piece of software and contribute it back to the community! Writingan integration can be a great first contribution. There’s detailed documentation on how to write integrations inthe Zulip integration writing guide.

• Bite Size: Smaller projects that might be a great first contribution.

• Documentation: The Zulip project loves contributions of new documentation.

• Help Wanted: A broader list of projects that nobody is currently working on.

• Platform support: These are open issues about making it possible to install Zulip on a wider range of platforms.

• Bugs: Open bugs.

• Feature requests: Browsing this list can be a great way to find feature ideas to implement that other Zulip usersare excited about.

• 2016 roadmap milestone: The projects that are priorities for the Zulip project. These are great projects if you’relooking to make an impact.

Another way to find issues in Zulip is to take advantage of our “area:” convention in separating out issues. We partitionall of our issues into areas like admin, compose, emoji, hotkeys, i18n, onboarding, search, etc. You can see this here:

https://github.com/zulip/zulip/labels

Click on any of the “area:” labels and you will see all the tickets related to your area of interest.

If you’re excited about helping with an open issue, make sure to claim the issue by commenting the following in thecomment section: “@zulipbot claim”. @zulipbot will assign you to the issue and label the issue as in progress. Formore details, check out @zulipbot.

You’re encouraged to ask questions on how to best implement or debug your changes – the Zulip maintainers areexcited to answer questions to help you stay unblocked and working efficiently. It’s great to ask questions in commentson GitHub issues and pull requests, or on chat.zulip.org. We’ll direct longer discussions to Zulip chat, but please posta summary of what you learned from the chat, or link to the conversation, in a comment on the GitHub issue.

We also welcome suggestions of features that you feel would be valuable or changes that you feel would make Zulip abetter open source project, and are happy to support you in adding new features or other user experience improvementsto Zulip.

If you have a new feature you’d like to add, we recommend you start by opening a GitHub issue about the featureidea explaining the problem that you’re hoping to solve and that you’re excited to work on it. A Zulip maintainer willusually reply within a day with feedback on the idea, notes on any important issues or concerns, and and often tips on

1.5. Google Summer of Code 5

https://developers.google.com/open-source/gsoc/https://blog.zulip.org/2016/10/13/static-types-in-python-oh-mypy/https://github.com/zulip/zulip.github.io/blob/master/gsoc-ideas.mdhttps://groups.google.com/forum/#!forum/zulip-gsochttps://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-androidhttps://github.com/zulip/zulip-androidhttps://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-android/blob/master/android-strategy.mdhttps://github.com/zulip/zulip/issueshttps://github.com/zulip/zulip/labels/area%3A%20integrationshttps://zulip.readthedocs.io/en/latest/integration-guide.htmlhttps://github.com/zulip/zulip/labels/bite%20sizehttps://github.com/zulip/zulip/labels/area%3A%20documentationhttps://github.com/zulip/zulip/labels/help%20wantedhttps://github.com/zulip/zulip/labels/Platform%20supporthttps://github.com/zulip/zulip/labels/bughttps://github.com/zulip/zulip/labels/enhancementhttp://zulip.readthedocs.io/en/latest/roadmap.htmlhttps://zulip.readthedocs.io/en/latest/roadmap.htmlhttps://github.com/zulip/zulip/labelshttps://github.com/zulip/zulipbothttps://zulip.readthedocs.io/en/latest/chat-zulip-org.html


how to implement or test it. Please feel free to ping the thread if you don’t hear a response from the maintainers – wetry to be very responsive so this usually means we missed your message.

For significant changes to the visual design, user experience, data model, or architecture, we highly recommendposting a mockup, screenshot, or description of what you have in mind to zulip-devel@ to get broad feedback beforeyou spend too much time on implementation details.

Finally, before implementing a larger feature, we highly recommend looking at the new feature tutorial and codingstyle guidelines on ReadTheDocs.

Feedback on how to make this development process more efficient, fun, and friendly to new contributors is verywelcome! Just send an email to the Zulip Developers list with your thoughts.

When you feel like you have completed your work on an issue, post your PR to the #code review stream onchat.zulip.org. This is our lightweight process that gives other developers the opportunity to give you commentsand suggestions on your work.

License

Copyright 2011-2017 Dropbox, Inc., Kandra Labs, Inc., and contributors

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance withthe License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an“AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See theLicense for the specific language governing permissions and limitations under the License.

The software includes some works released by third parties under other free and open source licenses. Those worksare redistributed under the license terms under which the works were received. For more details, see the docs/THIRDPARTY file included with this distribution.

6 Chapter 1. Zulip overview

https://chat.zulip.org/#narrow/stream/code.20reviewhttps://zulip.readthedocs.io/en/latest/chat-zulip-org.html

CHAPTER 2

Zulip architectural overview

Key Codebases

The core Zulip application is at https://github.com/zulip/zulip and is a web application written in Python 2.7 (soon toalso support Python 3) and using the Django framework. That codebase includes server-side code and the web client,as well as Python API bindings and most of our integrations with other services and applications (see the directorystructure guide).

We maintain several separate repositories for integrations and other glue code: a Hubot adapter; integrations withPhabricator, Jenkins, Puppet, Redmine, and Trello; node.js API bindings; and our full-text search PostgreSQL exten-sion.

Our mobile clients are separate code repositories: Android and React Native iOS app. Our legacy desktop application(implemented in QT/WebKit) and our new, alpha cross-platform desktop app (implemented in Electron) are alsoseparate repositories.

We use Transifex to do translations.

In this overview, we’ll mainly discuss the core Zulip server and web application.

Usage assumptions and concepts

Zulip is a real-time web-based chat application meant for companies and similar groups ranging in size from a smallteam to more than a thousand users. It features real-time notifications, message persistence and search, public groupconversations (streams), invite-only streams, private one-on-one and group conversations, inline image previews, teampresence/buddy lists, a rich API, Markdown message support, and numerous integrations with other services. Themaintainer team aims to support users who connect to Zulip using dedicated iOS, Android, Linux, Windows, andmacOS clients, as well as people using modern web browsers or dedicated Zulip API clients.

A server can host multiple Zulip realms (organizations) at the same domain, each of which is a private chamber withits own users, streams, customizations, and so on. This means that one person might be a user of multiple Zuliprealms. The administrators of a realm can choose whether to allow anyone to register an account and join, or onlyallow people who have been invited, or restrict registrations to members of particular groups (using email domain

7

https://github.com/zulip/zuliphttps://github.com/zulip/zuliphttps://github.com/zulip/hubot-zuliphttps://github.com/zulip/phabricator-to-zuliphttps://github.com/zulip/zulip-jenkins-pluginhttps://github.com/matthewbarr/puppet-zuliphttps://github.com/zulip/zulip-redmine-pluginhttps://github.com/zulip/trello-to-zuliphttps://github.com/zulip/zulip-nodehttps://github.com/zulip/tsearch_extrashttps://github.com/zulip/tsearch_extrashttps://github.com/zulip/zulip-androidhttps://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-desktophttps://github.com/zulip/zulip-desktophttps://github.com/zulip/zulip-electronhttps://www.transifex.com/zulip/zulip/


names or corporate single-sign-on login for verification). For more on security considerations, see the security modelsection.

The default Zulip home screen is like a chronologically ordered inbox; it displays messages, starting at the oldestmessage that the user hasn’t viewed yet (for more on that logic, see the guide to the pointer and unread counts). Thehome screen displays the most recent messages in all the streams a user has joined (except for the streams they’vemuted), as well as private messages from other users, in strict chronological order. A user can narrow to view only themessages in a single stream, and can further narrow to focus on a topic (thread) within that stream. Each narrow hasits own URL. The user can quickly see what conversation they’re in – the stream and topic, or the names of the user(s)they’re private messaging with – using the recipient bar displayed atop each conversation.

Zulip’s philosophy is to provide sensible defaults but give the user fine-grained control over their incoming informationflow; a user can mute topics and streams, and can make fine-grained choices to reduce real-time notifications they findirrelevant.

Components

Django and Tornado

Zulip is primarily implemented in the Django Python web framework. We also make use of Tornado for the real-timepush system.

Django is the main web application server; Tornado runs the server-to-client real-time push system. The app serversare configured by the Supervisor configuration (which explains how to start the server processes; see “Supervisor”below) and the nginx configuration (which explains which HTTP requests get sent to which app server).

8 Chapter 2. Zulip architectural overview

https://www.djangoproject.com/http://www.tornadoweb.org


Tornado is an asynchronous server and is meant specifically to hold open tens of thousands of long-lived (long-pollingor websocket) connections – that is to say, routes that maintain a persistent connection from every running client. Forthis reason, it’s responsible for event (message) delivery, but not much else. We try to avoid any blocking calls inTornado because we don’t want to delay delivery to thousands of other connections (as this would make Zulip verymuch not real-time). For instance, we avoid doing cache or database queries inside the Tornado code paths, since thoseblocking requests carry a very high performance penalty for a single-threaded, asynchronous server.

The parts that are activated relatively rarely (e.g. when people type or click on something) are processed by the Djangoapplication server. One exception to this is that Zulip uses websockets through Tornado to minimize latency on thecode path for sending messages.

There is detailed documentation on the real-time push and event queue system; most of the code is in zerver/tornado.

nginx

nginx is the front-end web server to all Zulip traffic; it serves static assets and proxies to Django and Tornado. Ithandles HTTP requests according to the rules laid down in the many config files found in zulip/puppet/zulip/files/nginx/.

zulip/puppet/zulip/files/nginx/zulip-include-frontend/app is the most important of thesefiles. It explains what happens when requests come in from outside.

• In production, all requests to URLs beginning with /static/ are served from the corresponding files in /home/zulip/prod-static/, and the production build process (tools/build-release-tarball)compiles, minifies, and installs the static assets into the prod-static/ tree form. In development, files areserved directly from /static/ in the git repository.

• Requests to /json/events, /api/v1/events, and /sockjs are sent to the Tornado server. These arerequests to the real-time push system, because the user’s web browser sets up a long-lived TCP connectionwith Tornado to serve as a channel for push notifications. nginx gets the hostname for the Tornado server viapuppet/zulip/files/nginx/zulip-include-frontend/upstreams.

• Requests to all other paths are sent to the Django app via the UNIX socket unix:/home/zulip/deployments/uwsgi-socket (defined in puppet/zulip/files/nginx/zulip-include-frontend/upstreams). We use zproject/wsgi.py to implement uWSGIhere (see django.core.wsgi).

Supervisor

We use supervisord to start server processes, restart them automatically if they crash, and direct logging.

The config file is zulip/puppet/zulip/templates/supervisor/zulip.conf.template.erb. Thisis where Tornado and Django are set up, as well as a number of background processes that process event queues. Weuse event queues for the kinds of tasks that are best run in the background because they are expensive (in terms ofperformance) and don’t have to be synchronous — e.g., sending emails or updating analytics. Also see the queuingguide.

memcached

memcached is used to cache database model objects. zerver/lib/cache.py and zerver/lib/cache_helpers.py manage putting things into memcached, and invalidating the cache when values change. Thememcached configuration is in puppet/zulip/files/memcached.conf.

2.3. Components 9

https://en.wikipedia.org/wiki/Push_technology#Long_pollinghttp://supervisord.org/


Redis

Redis is used for a few very short-term data stores, such as in the basis of zerver/lib/rate_limiter.py, aper-user rate limiting scheme example), and the email-to-Zulip integration.

Redis is configured in zulip/puppet/zulip/files/redis and it’s a pretty standard configuration except forthe last line, which turns off persistence:

# Zulip-specific configuration: disable saving to disk.save ""

memcached was used first and then we added Redis specifically to implement rate limiting. We’re discussing switchingeverything over to Redis.

RabbitMQ

RabbitMQ is a queueing system. Its config files live in zulip/puppet/zulip/files/rabbitmq. Initialconfiguration happens in zulip/scripts/setup/configure-rabbitmq.

We use RabbitMQ for queuing expensive work (e.g. sending emails triggered by a message, push notifications, someanalytics, etc.) that require reliable delivery but which we don’t want to do on the main thread. It’s also used forcommunication between the application server and the Tornado push system.

Two simple wrappers around pika (the Python RabbitMQ client) are in zulip/zerver/lib/queue.py.There’s an asynchronous client for use in Tornado and a more general client for use elsewhere. Most of the pro-cesses started by Supervisor are queue processors that continually pull things out of a RabbitMQ queue and handlethem; they are defined in zerver/worker/queue_processors.py.

Also see the queuing guide.

PostgreSQL

PostgreSQL (also known as Postgres) is the database that stores all persistent data, that is, data that’s expected to livebeyond a user’s current session.

In production, Postgres is installed with a default configuration. The directory that would contain configuration files(puppet/zulip/files/postgresql) has only a utility script and a custom list of stopwords used by a Post-gresql extension.

In a development environment, configuration of that postgresql extension is handled by tools/postgres-init-dev-db (invoked by tools/provision). That file also manages setting up the developmentpostgresql user.

tools/provision also invokes tools/do-destroy-rebuild-database to create the actual databasewith its schema.

Nagios

Nagios is an optional component used for notifications to the system administrator, e.g., in case of outages.

zulip/puppet/zulip/manifests/nagios.pp installs Nagios plugins from puppet/zulip/files/nagios_plugins/.

This component is intended to install Nagios plugins intended to be run on a Nagios server; most of the Zulip Nagiosplugins are intended to be run on the Zulip servers themselves, and are included with the relevant component of


http://blog.domaintools.com/2013/04/rate-limiting-with-redis/https://zulipchat.com/integrations/#emailhttps://github.com/zulip/zulip/issues/16https://github.com/zulip/zulip/issues/16


the Zulip server (e.g. puppet/zulip/manifests/postgres_common.pp installs a few under /usr/lib/nagios/plugins/zulip_postgres_common).

Glossary

This section gives names for some of the elements in the Zulip UI used in Zulip development conversations. Contri-butions to extend this list are welcome!

• chevron: A small downward-facing arrow next to a message’s timestamp, offering contextual options, e.g.,“Reply”, “Mute [this topic]”, or “Link to this conversation”. To avoid visual clutter, the chevron only appears inthe web UI upon hover.

• huddle: What the codebase calls a “group private message”.

• message editing: If the realm admin allows it, then after a user posts a message, the user has a few minutes toclick “Edit” and change the content of their message. If they do, Zulip adds a marker such as “(EDITED)” atthe top of the message, visible to anyone who can see the message.

• realm: What the codebase calls an “organization” in the UI.

• recipient bar: A visual indication of the context of a message or group of messages, displaying the stream andtopic or private message recipient list, at the top of a group of messages. A typical 1-line message to a newrecipient shows to the user as three lines of content: first the recipient bar, second the sender’s name and avataralongside the timestamp (and, on hover, the star and the chevron), and third the message content. The recipientbar is or contains hyperlinks to help the user narrow.

• star: Zulip allows a user to mark any message they can see, public or private, as “starred”. A user can easily ac-cess messages they’ve starred through the “Starred messages” link in the menu near “Home”, or use “is:starred”as a narrow or a search constraint. Whether a user has or has not starred a particular message is private; otherusers and realm admins don’t know whether a message has been starred, or by whom.

• subject: What the codebase calls a “topic” in many places.

• bankruptcy: When a user has been off Zulip for several days and has hundreds of unread messages, they areprompted for whether they want to mark all their unread messages as read. This is called “declaring bankruptcy”(in reference to the concept in finance).

2.4. Glossary 11

CHAPTER 3

Directory structure

This page documents the Zulip directory structure, where to find things, and how to decide where to put a file.

You may also find the new application feature tutorial helpful for understanding the flow through these files.

Core Python files

Zulip uses the Django web framework, so a lot of these paths will be familiar to Django developers.

• zproject/urls.py Main Django routes file. Defines which URLs are handled by which view functions ortemplates.

• zerver/models.py Main Django models file. Defines Zulip’s database tables.

• zerver/lib/actions.py Most code doing writes to user-facing database tables.

• zerver/views/*.py Most Django views.

• zerver/webhooks/ Webhook views and tests for Zulip webhook integrations.

• zerver/tornado/views.py Tornado views.

• zerver/worker/queue_processors.py Queue workers.

• zerver/lib/*.py Most library code.

• zerver/lib/bugdown/ Backend Markdown processor.

• zproject/backends.py Authentication backends.

HTML Templates

See our translating docs for details on Zulip’s templating systems.

13

https://docs.djangoproject.com/en/1.8/https://docs.djangoproject.com/en/1.8/topics/http/urls/https://docs.djangoproject.com/en/1.8/topics/db/models/https://docs.djangoproject.com/en/1.8/topics/http/views/https://docs.djangoproject.com/en/1.8/topics/auth/customizing/


• templates/zerver/ For Jinja2 templates for the backend (for zerver app).

• static/templates/ Handlebars templates for the frontend.

JavaScript and other static assets

• static/js/ Zulip’s own JavaScript.

• static/styles/ Zulip’s own CSS.

• static/images/ Zulip’s images.

• static/third/ Third-party JavaScript and CSS that has been vendored.

• node_modules/ Third-party JavaScript installed via npm.

• static/assets/ For assets not to be served to the web (e.g. the system to generate our favicons).

Tests

• zerver/tests/ Backend tests.

• frontend_tests/node_tests/ Node Frontend unit tests.

• frontend_tests/casper_tests/ Casper frontend tests.

• tools/test-* Developer-facing test runner scripts.

Management commands

These are distinguished from scripts, below, by needing to run a Django context (i.e. with database access).

• zerver/management/commands/Management commands one might run at a production deployment site(e.g. scripts to change a value or deactivate a user properly).

Scripts

• scripts/ Scripts that production deployments might run manually (e.g., restart-server).

• scripts/lib/ Scripts that are needed on production deployments but humans should never run directly.

• scripts/setup/ Scripts that production deployments will only run once, during installation.

• tools/ Scripts used only in a Zulip development environment. These are not included in production releasetarballs for Zulip, so that we can include scripts here one wouldn’t want someone to run in production acciden-tally (e.g. things that delete the Zulip database without prompting).

14 Chapter 3. Directory structure

http://jinja.pocoo.org/http://handlebarsjs.com/


• tools/setup/ Subdirectory of tools/ for things only used during the development environment setupprocess.

• tools/travis/ Subdirectory of tools/ for things only used to setup and run our tests in Travis CI. Actualtest suites should go in tools/.

API and Bots

• api/ Zulip’s Python API bindings (released separately).

• api/examples/ API examples.

• api/integrations/ Integrations distributed as part of the Zulip API bundle.

• api/bots/ Bots distributed as part of the Zulip API bundle.

• api/bots_api/ Framework for running and testing bots in api/bots/.

Production puppet configuration

This is used to deploy essentially all configuration in production.

• puppet/zulip/ For configuration for production deployments.

• puppet/zulip/manifests/voyager.pp Main manifest for Zulip standalone deployments.

Additional Django apps

• confirmation Email confirmation system.

• analytics Analytics for the Zulip server administrator (needs work to be useful to normal Zulip sites).

• corporate The old Zulip.com website. Not included in production distribution.

• zilencer Primarily used to hold management commands that aren’t used in production. Not included inproduction distribution.

Jinja2 Compatibility Files

• zproject/jinja2/__init__.py Jinja2 environment.

• zproject/jinja2/backends.py Jinja2 backend.

• zproject/jinja2/compressors.py Jinja2 compatible functions of Django-Pipeline.

3.7. API and Bots 15


Translation files

• locale/ Backend (Django) translations data files.

• static/locale/ Frontend translations data files.

Documentation

• docs/ Source for this documentation.

You can consult the repository’s .gitattributes file to see exactly which components are excluded from produc-tion releases (release tarballs are generated using tools/build-release-tarball).

16 Chapter 3. Directory structure

CHAPTER 4

Zulip Roadmap

Introduction

Zulip has received a great deal of interest and attention since it was released as free and open source software byDropbox. That attention has come with a lot of active development work from members of the Zulip community.From when Zulip was released as open source in late September 2015 through today (early November, 2016), morethan 150 people have contributed over 1000 pull requests to the various Zulip repositories, the vast majority of whichwere submitted by Zulip’s users around the world (as opposed to the small core team that reviews and merges the pullrequests).

In any project, there can be a lot of value in periodically putting together a roadmap detailing the major areas where theproject is hoping to improve. This can be especially important in an open source project like Zulip, where developmentis distributed across many people around the world. This roadmap is intended to organize a list of the most importantimprovements that should be made to Zulip in the relatively near future. Our aim is to complete most of theseimprovements by February 2017 and then prepare a new roadmap then.

This document is not meant to constrain in any way what contributions to Zulip will be accepted; instead, it willbe used by the Zulip core team to prioritize our efforts, measure progress on improving the Zulip product and holdourselves accountable for making Zulip improve rapidly.

This roadmap is the best place for contributors to look for substantial projects that will definitely be of value to thecommunity (if you’re looking for a starter project, see the guide to getting involved with Zulip).

We periodically update this roadmap by adding strikethrough to issues that have been resolved, but the linked GitHubissues are the most up-to-date source for that information.

Update: As of May 2017, we are approaching the point where we need to update the roadmap due to much of it beingcompleted.

Without further ado, below is the current Zulip roadmap.

Major projects

There are 2 huge projects that Zulip is working on right now that are too big to have a coherent GitHub issue:

17

https://github.com/zulip/zulip#how-to-get-involved-with-contributing-to-zulip


• We are working with a world-class designer on a major visual redesign of the Zulip webapp. Already completeis completely redesining the streams and settings UIs, logged-out pages, and various other major components.

• We are writing a new React Native iOS app for Zulip to replace the old iOS app. The new app is progressingrapidly, but is not yet feature complete. We expect it to be in the app store in May 2017.

Core User Experience

• Provide shorter UI/Keyboard sequence to edit the last message

• Better drafts management

• Make clicking on desktop notifications renarrow properly

• Add pretty bubbles for recipients in the compose box

• Make right sidebar buddy list UI scale well to large teams

• Display stream descriptions more prominently

• Add support for managing uploaded files

Social features

• Add support for showing “user is typing” notifications, at least for private messages

• Support lightweight emoji “reactions”

• Open graph previews of generic websites

• Add a “join Zulip chat” badge for projects that use Zulip to document that nicely

Real-time sync

The overall goal is to eliminate the few known issues where Zulip does not provide a seamless real-time sync experi-ence.

• Notification bot advertisements for new streams don’t handle stream renames

• Avatar/name changes don’t propagate to already-sent messages

• Advance the pointer / where we load the user to based on unread counts in home view

• Fix the known bug where messages could be incorrectly marked as read

Onboarding issues

This category focuses on issues users experience when installing a new Zulip server, setting up a new Zulip realm, orstarting to use Zulip.

• Move Zulip’s prompt for permission to display notifications to be manually triggered

• Add a mechanism for deleting early test messages (e.g., administrators can hard-delete messages)

• Allow customizing emails when inviting new users

18 Chapter 4. Zulip Roadmap

https://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip/issues/1147https://github.com/zulip/zulip/issues/1717https://github.com/zulip/zulip/issues/1996https://github.com/zulip/zulip/issues/595https://github.com/zulip/zulip/issues/236https://github.com/zulip/zulip/issues/164https://github.com/zulip/zulip/issues/454https://github.com/zulip/zulip/issues/150https://github.com/zulip/zulip/issues/541https://github.com/zulip/zulip/issues/406https://github.com/zulip/zulip/issues/2270https://github.com/zulip/zulip/issues/426https://github.com/zulip/zulip/issues/1932https://github.com/zulip/zulip/issues/1529https://github.com/zulip/zulip/issues/2091https://github.com/zulip/zulip/issues/1189https://github.com/zulip/zulip/issues/135https://github.com/zulip/zulip/pull/1409


Production installation issues

• Document or better script solution to rabbitmq startup issues

• Merge a supported way to use Zulip in Docker in production implementation.

Administration and management

• Make list of allowed domains web-configurable

• Statistics display for realm and server administrators

• Keep track of which users added which realm emoji

• Add setting to enable any user to add new realm emoji

• Make realm filters web-configurable

• Improve administrative controls for managing streams

• Enhance the LDAP integration and make it web-configurable

• Add a SAML integration for Zulip

Scalability and performance

Scalability and performance are not currently major problems for Zulip; it already scales well to thousands of usersand is significantly faster than proprietary alternatives. So, this is not a major focus area for the project.

• Make the Zulip Tornado service support horizontal scaling

• Make presence system scale well to 10000 users in a realm.

• Support running queue workers multithreaded in production to decrease minimum memory footprint

• Improve @-mentioning syntax based on stronger unique identifiers

Technology improvements

• Add support for Zulip running purely on Python 3

• Automatic thumbnailing of uploaded images’ previews to save bandwidth

• Upgrade Zulip to use Django 1.10. The patches needed to run Zulip were merged into mainline Django inDjango 1.10, so this will mean we don’t need to use a fork of Django anymore.

• Upgrade and remove from codebase all unnecessarily vendored JS libraries

• Add support for changing users’ email addresses

• Migrate from jslint to eslint

• Replace the slow closure-compiler based static asset toolchain

• Use a modern JavaScript bundler like webpack

• Add support for building frontend features in something like React

4.7. Production installation issues 19

https://github.com/zulip/zulip/issues/465https://github.com/zulip/zulip/pull/450https://github.com/zulip/zulip/issues/651https://github.com/zulip/zulip/issues/2052https://github.com/zulip/zulip/issues/984https://github.com/zulip/zulip/issues/978https://github.com/zulip/zulip/pull/544https://github.com/zulip/zulip/issues/3783https://github.com/zulip/zulip/issues/715https://github.com/zulip/zulip/issues/716https://github.com/zulip/zulip/issues/445https://github.com/zulip/zulip/issues/728https://github.com/zulip/zulip/issues/34https://github.com/zulip/zulip/issues/374https://github.com/zulip/zulip/issues/256https://github.com/zulip/zulip/issues/432https://github.com/zulip/zulip/issues/3https://github.com/zulip/zulip/issues/3https://github.com/zulip/zulip/issues/1709https://github.com/zulip/zulip/issues/734https://github.com/zulip/zulip/issues/535https://github.com/zulip/zulip/issues/693https://github.com/zulip/zulip/issues/695https://github.com/zulip/zulip/issues/694


Technical Debt

While the Zulip server has a great codebase compared to most projects of its size, it takes work to keep it that way.

• Migrate most web routes to REST API

• Split Tornado subsystem into a separate Django app

• Refactor zulip.css to be broken into components

Security

• Add support for 2-factor authentication on all platforms

• Add support for stronger security controls for uploaded files (The LOCAL_UPLOADS_DIR file uploads back-end only supports world-readable uploads)

• Fix requirement to set a password when creating account via Google

• Add a retention policy feature that automatically deletes old messages

• Add UI for viewing and cancelling open Zulip invitations

Testing

• Extend Zulip’s automated test coverage to include all API endpoints

• Build automated tests for the client API bindings

• Add automated tests for the production upgrade process

Documentation

• Add an in-app mechanism for updating users about new Zulip features

• Significantly expand documentation of the Zulip API and integrating with Zulip.

• Write a visual design / frontend style guide for Zulip

• Update all screenshots to show the current Zulip UI

Nice to have

• Expand library of documentation on Zulip’s feature set. Currently most documentation is for either developersor system administrators.

Integrations and bots

Integrations are essential to Zulip. While we currently have a reasonably good framework for writing new webhookintegrations for getting notifications into Zulip, it’d be great to streamline that process and make bots that receivemessages just as easy to build.


https://github.com/zulip/zulip/issues/611https://github.com/zulip/zulip/issues/729https://github.com/zulip/zulip/issues/731https://github.com/zulip/zulip/pull/1747https://github.com/zulip/zulip/issues/320https://github.com/zulip/zulip/issues/320https://github.com/zulip/zulip/issues/1633https://github.com/zulip/zulip/issues/106https://github.com/zulip/zulip/issues/1180https://github.com/zulip/zulip/issues/1441https://github.com/zulip/zulip/issues/713https://github.com/zulip/zulip/issues/306https://github.com/zulip/zulip/issues/2187https://github.com/zulip/zulip/issues/672https://github.com/zulip/zulip/issues/979https://github.com/zulip/zulip/issues/599https://github.com/zulip/zulip/issues/675https://github.com/zulip/zulip/issues/675


• Add an outgoing webhook integration system

• Make setting up a new integration a smooth flow

• Default new incoming webhooks to permissions-limited incoming webhook bots

• Change how Zulip displays bot names to distinguish them from human users

Android app

• Add support for narrowing to @-mentions

• Support having multiple Zulip realms open simultaneously

iOS app

For the new React Native iOS app, the major goal for it is to be released into the app store. Since it is moving quickly,we’re tracking its roadmap via GitHub milestones.

Server/webapp support for mobile

To support a great mobile experiences, we need to make some improvements in the Zulip server.

• Push notifications bouncer service for GCM and APNS

• A slick process for doing mobile login without typing your password on your phone

• @here mention support (that doesn’t spam people not currently online, i.e. no email/push notifications)

• Fix sending messages from mobile web

Desktop apps

The new cross-platform desktop app is implemented in Electron, and primarily needs work on installer tooling to finishreplacing the old app.

• Finish releasing the Electron app to replace the old desktop app

• Support having multiple Zulip realms open simultaneously

Community

These don’t get GitHub issues since they’re not technical projects, but they are important goals for the project.

• Expand the number of core developers able to do code reviews

• Have a successful season with Zulip’s Outreachy participants

• Have a successful season with Google Code In.

4.16. Android app 21

https://github.com/zulip/zulip/issues/735https://github.com/zulip/zulip/issues/692https://github.com/zulip/zulip/issues/2186https://github.com/zulip/zulip/issues/1107https://github.com/zulip/zulip-android/issues/39https://github.com/zulip/zulip-android/issues/47https://github.com/zulip/zulip-mobilehttps://github.com/zulip/zulip-mobile/milestoneshttps://github.com/zulip/zulip/issues/1767https://github.com/zulip/zulip/issues/2185https://github.com/zulip/zulip/issues/2183https://github.com/zulip/zulip/issues/2184https://github.com/zulip/zulip-electronhttp://electron.atom.io/https://github.com/zulip/zulip-electron/issues/1

CHAPTER 5

Version History

All notable changes to the Zulip server are documented in this file.

Unreleased

1.6.0 – 2017-06-06

Highlights:

• A complete visual redesign of the logged-out pages, including login, registration, integrations, etc.

• New visual designs for numerous UI elements, including the emoji picker, user profile popovers, sidebars,compose, and many more.

• A complete redesign of the Zulip settings interfaces to look a lot nicer and be easier to navigate.

• Organization admins can now configure the login and registration pages to show visitors a nice organizationprofile with custom text and images, written in Markdown.

• Massively improved performance for presence and settings pages, especially for very large organizations (1000+users).

• A dozen useful new keyboard shortcuts, from editing messages to emoji reactions to drafts and managingstreams.

• Typing notifications for private message threads.

• Users can now change their own email address.

• New saved-drafts feature.

• The server can now run on a machine with as little as 2GB of RAM.

• The new Electron desktop app and new React Native mobile app for iOS are now the recommended Zulip apps.

• Mobile web now works much better, especially on iOS.

23

https://github.com/zulip/zulip-electron/releaseshttps://itunes.apple.com/us/app/zulip/id1203036395


• Support for sending mobile push notifications via a new forwarding service

• Complete translations for Spanish, German, and Czech (and expanded partial translations for Japanese, Chinese,French, Hungarian, Polish, Dutch, Russian, Bulgarian, Portuguese, Serbian, Malayalam, Korean, and Italian).

Full feature Changelog:

• Added Basecamp, Gogs, Greenhouse, Home Assistant, Slack, Splunk, and WordPress webhook integrations.

• Added LaTeX support to the markdown processor.

• Added support for filtering branches to all Git integrations.

• Added read-only access to organization-level settings for all users.

• Added UI for managing muted topics and uploaded files.

• Added UI for displaying message edit history.

• Added support for various features needed by new mobile app.

• Added deep links for settings/subscriptions interfaces.

• Added an animation when messages are edited.

• Added support for registration with GitHub auth (not just login).

• Added tracking of uploaded file quotas.

• Added option to display emoji as their alt codes.

• Added new audit log table, to eventually support an auditing UI.

• Added several new permissions-related organization settings.

• Added new endpoint for fetching presence data, useful in employee directories.

• Added typeahead for language for syntax highlighting in code blocks.

• Added support for basic markdown in stream descriptions.

• Added email notifications on new Zulip logins.

• Added security hardening before serving uploaded files.

• Added new PRIVACY_POLICY setting to provide a Markdown privacy policy.

• Added an icon to distinguish bot users as message senders.

• Added a command-line Slack importer tool using the API.

• Added new announcement notifications on stream creation.

• Added support for some newer unicode emoji code points.

• Added support for users deleting realm emoji they themselves uploaded.

• Added support for organization administrators deleting messages.

• Extended data available to mobile apps to cover the entire API.

• Redesigned bots UI. Now can change owners and reactivate bots.

• Redesigned the visuals of code blocks to be prettier.

• Changed right sidebar presence UI to only show recently active users in large organizations. This has a hugeperformance benefit.

• Changed color for private messages to look better.

• Converted realm emoji to be uploaded, not links, for better robustness.

24 Chapter 5. Version History

https://zulip.readthedocs.io/en/latest/prod-mobile-push-notifications.html


• Switched the default password hasher for new passwords to Argon2.

• Increased the paragraph spacing, making multi-paragraph.

• Improved formatting of all Git integrations.

• Improved the UI of the /stats analytics pages.

• Improved search typeahead to support group private messages.

• Improved logic for when the compose box should open/close.

• Improved lightbox to support scrolling through images.

• Improved markdown support for bulleted lists.

• Improved copy-to-clipboard support in various places.

• Improved subject lines of mis

Zulip Documentation · 2019-04-02 · 4.8 Administration and management. . . . . . . . . . . . . ....

Documents

Transcript of Zulip Documentation · 2019-04-02 · 4.8 Administration and management. . . . . . . . . . . . . ....