Slack

Slack is a web based chat platform which features file sharing, voip calls, text messaging, and most importantly, emoji reactions.

The site is an SPA backend by a JSON api built using React, Redux, PHP.

• Data Fetching – XHR and Websockets
  • Pagination
  • Timings and Response sizes
  • An API request
  • Handling Connection Loss
• State Management
• CDN, Caches, Servers
• Bundling
• UI
• Handlebars
• Search
• LocalStorage
• Analytics
• Version updates
• Integrations
• Slack’s Forgotten Pages
• Conclusion

Data Fetching – XHR and Websockets

The API setup is a mix of websockets and XHR requests.

Most XHR requests sent by the slack client are POST requests and send request data as a form with a token and the request data as JSON. The response is JSON with a ok key and the desired data. Websocket messages are entirely JSON.

The api has an RPC like structure, /api/chat.attachmentAction, /api/client.counts, etc.

Once data is fetched for a specific channel, user, etc. via XHR, Slack will not refetch the data until a substantial amount of time has passed.

The websocket responses follow a general structure with with type key and their respective data. Some requests have an additional subtype key.

When the client sends a request via the websocket it includes an id key. The server will then reply with a reply_to key with the same value as the request’s id key. Websocket reply messages also include an ok key as well.

Pagination

Slack will paginate requests at 42 response items.

To indicate pagination the server returns a has_more key set to true and additional response_metadata key with a next_cursor as a subkey.

{
    "ok": true,
    "has_more": true,
    "messages": [...], // or whatever data the client is looking for
    "response_metadata": {
        "next_cursor": "bmV4dF90czoxNTUwMTc1ODE4MDExMTAw"
    }
}

Timings and Response sizes

From testing, we’ve seen api response times the range of 40ms to ~1.8s.

Additionally, requests sometimes queue, which is most noticible on load when there are ~8 requests made.

Response sizes tend to range from as little as 700 bytes to 20 kilobytes.

Most of the data transfer is done through XHR and websockets are used pub/sub updates from the server like user presence. Even sending messages is done via XHR. These server sent updates tend to be less than 1KB.

An API request

For example, if I update my user profile status from 🐛to 🎉 then Slack will first send an XHR request and get a response from the server containing the updated user info. Then the client will recieve the same update via the websocket since the client is subscribed to user changes.

Likewise, if another user on Slack changes their user profile then we will receive the update through the websocket.

Notably, there isn’t any sort of diffing between local and server state like we saw in the Dropbox paper post, the server just sends the client the entire user as JSON.

The client also makes XHR requests for navigation analytics, i18n, and inital load metrics.

For more information about the API see the official Slack docs.

Handling Connection Loss

When Slack loses connection it indicates so to the user via a toast and a “Slack is trying to connect” message next to the chat box.

Slack then tries to test its connection via a POST to https://admithub.slack.com/api/api.test. When doing so, the client surprisingly doesn’t apply exponential backoff and instead sends 1 request per second.

State Management

Slack uses Redux for its state management in addition to it’s window.TS object. The Redux Dev Tools are rather limited with state serialization disabled, but we can still see the actions.

Note that the help message references TS, a reference to Tiny Speck. The TS prefix can be found throughout the codebase.

The structure of the Redux store is nothing special, most items are just keyed by their ID, so rather than having byId and allIds subkeys, Slack just has everything be a subkey.

channels: {
    [key: string]: IChannel
}
// instead of

channels: {
    byId: {[key: string]: IChannel}
    allIds: string[]
}

CDN, Caches, Servers

From Slack’s request headers we can see that Slack uses HAProxy, CloudFront, Varnish, and Apache.

Additionally, Slack includes a header for request tracking: x-slack-req-id

Bundling

Slack uses both Rollup and Webpack for bundling JS and CSS.

Slack also uses code splitting with chunks ranging from around 50KB to 1.6MB in size.

UI

The Slack frontend is an SPA built with React and Redux.

Slack doesn’t use server side rendering but the index.html has a static skeleton UI similar to Twitch.

Slack follows BEM for its CSS along with a spattering of utility classes.

The frontend references CodeMirror which can be accessed via window.TS.codemirror, but the attached methods are both prefixed legacy.

With data fetching, Slack tends to not show spinners, opting to delay pop-ins for things like modals until the content has loaded (similar to Trello’s fetch before nav).

For channels in particular, Slack won’t show a spinner for the message content unless the request takes more than a few hundred milliseconds and instead shows a blank space. Also, the spinner is quite small when Slack does show it, at around 20x20px.

Handlebars

In addition to React, Slack uses handlebars templates.

Considering that Slack was first released in August 2013 and that React was only open sourced in May 2013, these templates are probably a remnant from a previous, non-React version of the site.

These templates are fetched from https://admithub.slack.com/templates.php with query parameters specifying the desired subsections, e.g., required,admin,apps,files,helpers.

The endpoints return a JS file with templates in the following format:

TS.raw_templates["comments"] = `
<div class="{{#if file}}{{makeFileCommentsDomId file}} {{/if}}comments">
    {{{comments file}}}
</div>`;

Search

Slack’s search uses a combination of XHR and websockets. The search box loads initial suggestions via XHR requests and then typeahead queries are handled via websocket.

Once the user hits enter, Slack sends an XHR request to /api/search.modules which has the following response:

{
  "ok": true,
  "pagination": {
    "total_count": 21,
    "page": 1,
    "per_page": 20,
    "page_count": 2,
    "first": 1,
    "last": 20
  },
  "query": "bar",
  "filters": [],
  "module": "files",
  "items": []
}

Note the pagination used for the search is different from the next_cursor setup we’ve seen.

In addition to the search request, Slack send an XHR request to /api/search.save with the query text and also makes a request to /api/search.precache.

While suggestions and typehead responses tend to be less than 60ms, the actual search response is an order of magnitude slower around 400ms to 600ms.

LocalStorage

Slack stores data such as session tokens, activity data, message input state, etc.

The keys for user state are prefixed with the user id, so instead of custom_emoji, Slack uses 1O3MS3NCS_custom_emoji.

Surprisingly, Slack stores some localStorage data in a binary format.

On Slack’s engineering blog they mention they use lz-string to compress data for localStorage.

Analytics

Slack uses beacons as well as traditional XHR requests for analytics and logging.

Slack records session lengths, load times, and usage metrics, which are sent as URL parameters in the beacon POST requests. The transmitted data uses a StatsD style format.

For instance

mvp_m_check|count:1

longtask|timing:168.53,108.48;channel_change_react_msgs|timing:317.57,231.78,202.435,232.665,176.045,368.32,281.74,364.41

channel_switch_usable_legacy|timing:863

For logging session start and end times Slack sends GET requests to https://slack.com/clog/track/ with JSON data encoded as a query parameter.

Version updates

While Slack’s browser version gets a new version on each reload, since Slack is an SPA the user may not reload the page for a long period of time. If there is a new release then the browser won’t know.

To fix this situation Slack sends an occasional POST request to https://admithub.slack.com/api/rtm.shouldReload with _x_id and _x_version_ts as query params. The endpoint provides a response of whether or not the browser needs to a full reload.

{ "ok": true, "should_reload": false }

Integrations

Slack supports third party integrations for things like Google Drive, Sentry, etc.

The Sentry integration supports taking action from a given Slack channel.

For instance, if we hit “Ignore” then Slack will send an XHR to https://admithub.slack.com/api/chat.attachmentAction with the following format.

{
  "actions": [
    {
      "id": "2",
      "name": "status",
      "text": "Ignore",
      "type": "button",
      "value": "ignored",
      "style": ""
    }
  ],
  "attachment_id": "1",
  "callback_id": "{\"issue\":950719020}",
  "channel_id": "B0131U34H",
  "message_ts": "1553364633.005800",
  "prompt_app_install": false
}

The server just returns { "ok": true } for the XHR request and the response for the particular action is sent later via the websocket connection.

{
  "type": "message",
  "subtype": "bot_message",
  "text": "Looks like you haven't linked your Sentry account with your Slack identity yet! <https://sentry.io/extensions/slack/link-identity/kIjoiVUFZTUIzQ05TIiwicmVzcG9uc2VfdXJsIjoiaHR0cHM6Ly9ob29rcy5zbGFjay5jb20vYWN0aW9ucy9UMDQ5RUdFMj/|Link your identity now> to perform actions in Sentry through Slack.",
  "is_ephemeral": true,
  "bot_id": "BB2C15R6K",
  "ts": "1553396846.006200",
  "channel": "B0131U34H",
  "event_ts": "1553396846.000500"
}

Slack’s Forgotten Pages

While Slack is primarly is an SPA, a select few pages are not.

For example in the https://slack.com/customize page shown above, the main page is server rendered while the embedded tabs are built with React.

Conclusion

Compared to some previous sites we’ve explored, Slack has a uniform API which powers the site in a thoughtful manner. The combination of XHR with websockets seems to work nicely together.