Design TikTok: Short-Video Uploads, Transcoding and a For You Feed

Listen

Requirements. Creators upload videos from fifteen seconds to a few minutes. Viewers get an endless, personalised For You feed and swipe to the next video. They can like, share and follow. Playback must feel instant. Live streaming, messaging, shopping and on-device editing effects are out of scope; editing happens on the phone before upload, so it doesn't touch our servers.

Qualities. After a swipe, the next video should start in roughly two hundred milliseconds, which is only possible if it's already downloading. The feed should adapt within the same session: skip three cooking videos and you should see fewer. Nothing harmful should reach a big audience before it's checked. Uploads are durable once accepted. Like counts and view counts can lag; that's fine.

Ask scale

Assume a billion daily users who each watch about a hundred short videos. That's a hundred billion video starts a day, around a million a second. The app fetches recommendations in batches of ten, so a hundred thousand feed calls a second. Uploads are much rarer: fifty million a day, about five hundred a second. And every view sends events like watch time, skip and like, so the event pipeline handles millions of messages a second.

Storage: a thirty-second original is about fifteen megabytes, and the transcoded renditions add about the same. Fifty million uploads a day is one and a half petabytes daily, hundreds of petabytes a year. Bandwidth is the real monster: a hundred billion views at a few megabytes each is three hundred petabytes a day, tens of terabits a second. That only works with a CDN serving popular videos from the edge, and with cheaper storage tiers for the long tail.

DAU                  1B
videos watched/day   100 each
  100B starts/day   = ~1M/s
feed calls (10/batch)
  10B / 100,000 s   = 100K rps

uploads/day          50M
  50M / 100,000 s   = 500/s

events (watch, skip, like)
  ~3 per view       = ~3M/s

original 30 s          ~15 MB
renditions (5 sizes)   ~15 MB
50M x 30 MB          = 1.5 PB/day
  x 365              = ~550 PB/yr

per view ~3 MB watched
  (720p, about 15 s)
100B x 3 MB          = 300 PB/day
  = ~3.5 TB/s = ~28 Tbps

-> CDN, popular videos at edge

Define API

The API. Uploads are chunked and resumable, because creators are on mobile networks. Once all chunks land, the client creates the video with a caption and tags. The feed call returns ten items, each with a manifest URL that points at the CDN, a cover image and author info. The bytes never pass through our API. And the client batches engagement events, like how many milliseconds you watched and whether you liked or skipped, every few seconds.

POST /v1/uploads  -> { uploadId,
                    chunkUrls[] }
PUT  chunk N       (resumable)
POST /v1/videos
  { uploadId, caption, tags }
GET  /v1/feed?session=..&n=10
  -> [{ videoId, manifestUrl,
        coverUrl, author }]
POST /v1/events   (batched)
  [{ videoId, watchMs, action }]

Data model

Data model. The video record holds the owner, duration, tags, a list of renditions and a status that walks from uploaded, to transcoded, to reviewed, to live. View and like counts are sharded counters, because a viral video can get thousands of likes a second. Engagement events go to an append-only log. And every video and every user gets an embedding, a vector of numbers, which the recommender uses to find similar items quickly.

video  (videoId PK, authorId,
        status, duration, tags,
        renditions[], createdAt)
  status: uploaded -> transcoded
          -> reviewed -> live
stats  videoId -> views, likes
       (sharded counters)
user   userId -> interests
events append-only log
embed  videoId -> vector[128]

High-level design

The upload path. Chunks go to the upload service and are assembled into the original in object storage. A transcoding job turns it into several resolutions and bitrates, cut into short segments. In parallel, moderation models scan sampled frames, audio and text. Only when transcoding and the first safety checks pass does the video's status flip to live and become eligible for the feed.

The watch path. The app asks the feed service for ten videos. It retrieves candidates near your taste from an embedding index, scores them with features, and returns the top ten. The app plays segments from the CDN and sends engagement events back. Those events update the feature store in near real time and train the models offline, which closes the loop: what you watch now shapes what you see next.

Deep dive: video

Deep dive one: the video pipeline. Split the original into short segments and encode them in parallel, so a one-minute video is ready in seconds rather than a minute. Produce a ladder of resolutions and bitrates, packaged as HLS or DASH: a small manifest plus many segment files. The player picks the right rung for the current network. A cost trick: once a video goes viral, spend more compute on a more efficient codec, because those bytes are sent billions of times.

Deep dive: playback

Deep dive two: instant playback. The trick is that the feed returns a batch, so the app prefetches the first segment or two of the next few videos while you watch the current one. When you swipe, playback starts from local memory. Start at a lower bitrate and step up. On the CDN side, popular videos are pushed to edges early, and a tiered cache shields object storage from misses. Track video start time and rebuffering per region and network.

Deep dive: recommend

Deep dive three: recommendations. Retrieval gathers a couple of thousand candidates from several sources: nearest neighbours of your user embedding in an approximate index, trending videos in your region, creators you follow, and some brand-new videos that need test views. Filter out what you've seen or blocked. A ranking model predicts watch time, likes, shares and skips for a few hundred of them. A final pass adds diversity so you don't get ten of the same thing.

Why does it learn so fast? Because session signals flow through the stream into the feature store within seconds, so the very next feed call knows you skipped three cooking videos. Models retrain often. For cold start, a new user sees popular videos across varied topics, which works as a quick interest probe. A new video goes to a small test audience first, and if people watch it through, it gets shown to progressively larger groups.

Trade-offs

Failures and monitoring. If transcoding backs up, uploads just take longer to go live, so autoscale on queue depth. If the feed service is slow, fall back to a cached regional trending list so people can still watch. If the event pipeline lags, recommendations get stale, but playback is unaffected. Watch video start time, rebuffering, watch-through rate, and how long risky videos wait for review.

Trade-offs to say out loud. Prefetching wastes some bytes when people swipe away, but buys instant starts. More renditions cost storage but give smoother playback. Staged distribution slows down how fast a video goes viral, but limits the damage of a harmful one. At ten times the scale, spend compute on better codecs for the hottest videos, add edge capacity, and run recommendation stacks per region.

Say it like this

The summary. Resumable chunked uploads land in object storage, get transcoded in parallel into a bitrate ladder, and are checked by moderation before going wide. Delivery is a CDN plus client prefetching for instant starts. The feed retrieves candidates from embeddings, trending and fresh videos, ranks them by predicted engagement and diversifies. Engagement events stream back within seconds to keep the feed sharp.