Design Live Comments: Real-Time Chat on a Stream with Two Million Viewers

Listen

Listen. Viewers post comments on a live stream and see everyone else's comments scroll by live. When they join, they see the most recent fifty or so. The creator and moderators can delete comments. Afterwards, comments replay alongside the recorded video. Non-functional: a comment should be visible to others within about two seconds, and the system must cope with extreme skew, where one stream has millions of viewers.

Scope out the video pipeline and reactions. Then ask the question that unlocks the whole design: must every viewer see every single comment? On a huge stream with twenty thousand comments a minute, no human can read them all, so the answer is no. A representative sample is fine. And ordering only needs to be roughly by time. Getting those two answers early saves you from designing the impossible.

Ask scale

Ask about scale. Ten million concurrent viewers across a hundred thousand live streams, and the biggest stream has two million. If one percent of viewers comment each minute, that is about seventeen hundred comments a second overall, and around three hundred thirty a second on the biggest stream. Writes are tiny and modest in number. Now look at the reads.

Here is the number that decides the design. Sending every comment to every viewer on the top stream is three hundred thirty times two million, six hundred sixty million messages a second. That is not happening. Cap what is shown at about ten comments a second, and batch them into one frame every half second. Now each viewer gets two small frames a second: four million frames a second, spread across many connection servers.

10M concurrent viewers
  across 100k live streams
top stream: 2M viewers

1% of viewers comment per min
  overall: 100k/min ≈ 1,700/s
  top stream: 20k/min ≈ 330/s

writes are small; a single
  partition handles 330/s

top stream, naive fan-out:
  330 comments/s × 2M viewers
  = 660M messages/s  ✗

cap shown comments at 10/s
  10/s × 2M = 20M msgs/s
batch: 1 frame every 500 ms
  2 frames/s × 2M = 4M frames/s
  over ~10 connection servers
  per 200k viewers each

Define API

Define the API. Posting is an ordinary POST with the text and a client-generated ID for dedupe. A paged GET returns recent or older comments. The live feed is a single long-lived stream, and since data flows server to client, Server-Sent Events fits well: each event is a batch of new comments or a delete notice. Delete is its own endpoint for moderators.

POST /v1/streams/{id}/comments
  { text, client_id }
→ 201 { comment_id, ts }

GET /v1/streams/{id}/comments
  ?before=cursor&limit=50

GET /v1/streams/{id}/live
  (SSE: batches of comments,
   deletes)
DELETE /v1/comments/{id}

Data model

The data model. Comments go in a wide-column or key-value store, partitioned by stream ID and minute so a hot stream does not create one giant partition, and clustered by a time-sortable comment ID. Each row keeps the video timestamp for replay and a status for deletes. A Redis list per stream keeps the latest two hundred comments, so joining viewers get recent history without touching the database.

comments  (Cassandra/DynamoDB)
  PK ((stream_id, minute),
      comment_id)
  user_id, text, video_ts,
  status (visible | deleted)

recent cache (Redis list):
  stream:{id}:recent
  LPUSH + LTRIM 0 199

comment_id: time-sortable

High-level design

Draw the high-level design, starting with a viewer joining. The viewer opens the live feed to a connection server. If this is the first viewer of stream five on that server, the server subscribes to the stream five channel in pub-sub; later viewers on the same server reuse that one subscription. The server sends the last fifty comments from the Redis cache, so the chat is never empty on arrival.

Now posting. The comment service validates and filters the text, writes it to the store, pushes it onto the recent list, and publishes it once to the stream's channel. Pub-sub delivers it to each connection server subscribed to that stream, maybe ten servers, not two million viewers. Each server buffers incoming comments and writes one batch to all its local viewers every half second.

Deep dive: fan-out

Deep dive one: two-level fan-out. Level one, pub-sub sends each comment to the connection servers that have viewers of that stream, one subscription per server. Level two, each server loops over its local sockets in memory, which is cheap. To keep level one small, route viewers of the same stream to the same group of servers, for example by hashing the stream ID when they connect. A small stream then lives on one server.

Deep dive: hot streams

Deep dive two: hot streams. When a stream exceeds about ten comments a second, a sampler sits between the raw channel and the connection servers. It picks what to show: the creator and moderators first, then a random sample, dropping repeats like a thousand people typing lol. Every viewer sees the same sample, which is fair and cheap. Your own comment always appears for you immediately, so nobody feels ignored.

Deep dive: writes

Deep dive three: the write path at peak. A product launch can spike one stream to ten thousand comments a second. Partitioning by stream and minute stops one partition growing forever, and if a single minute is still too hot, add a small bucket number to the key. Comments also flow through a Kafka topic partitioned by stream, feeding the sampler. And rate limit each user to roughly one comment every two seconds, which also cuts spam.

Deep dive: moderation

Deep dive four: moderation. Inside the request, run cheap checks that take milliseconds: blocked words, links, banned users. Slower machine learning checks run asynchronously after publishing. When a comment is removed by a model or a moderator, publish a delete event on the same stream channel; every client removes it from the screen, the store marks it deleted, and the recent cache drops it.

Deep dive: replay

Deep dive five: history. Someone joining live gets the latest fifty from Redis. Scrolling up pages the store by comment ID. For replay with the recorded video, each comment carries its offset into the stream, so the player fetches comments for the next minute of video as it plays. Those windows never change once the stream ends, so the CDN can cache them.

Failure modes

Failure handling is gentler here than in WhatsApp, because live chat is lossy by design. If a connection server dies, its viewers reconnect with jitter and miss half a second of comments, which nobody notices. If a pub-sub node is lost, servers resubscribe. A slow viewer on bad mobile data gets only the newest batch. And if the store is slow, queue the write but publish anyway: the live view matters more than the archive's latency.

Trade-offs

The trade-offs. Sampling means not every comment is seen, and you should say that explicitly as a product decision. Batching adds up to half a second of latency in exchange for a hundred times less traffic. One-way SSE is enough because posts go over normal HTTP. Monitor post-to-screen latency, frames per second per server, drop rates and subscriptions per stream. At ten times, sample more aggressively and add regional pub-sub relays.

Interview summary

The summary. Writes are small, under two thousand a second, but naive fan-out on the top stream would be six hundred sixty million messages a second. So comments are published once per stream; each connection server subscribes once and batches to its local viewers every half second. A sampler caps hot streams at about ten visible comments a second. Comments are stored by stream and minute, and deletes travel as events on the same channel.