Design Instagram: Photo Uploads, the Home Feed and the Celebrity Problem

Listen

Start by fixing the scope out loud. Users upload a photo with a caption, follow other users, and scroll a home feed of posts from people they follow. Likes and comments exist, but we treat them as counters. Stories, reels, direct messages, ads and search are out of scope, and you say so, so the interviewer can push back now rather than at minute thirty.

Now the qualities. The feed must feel instant, say under five hundred milliseconds at p ninety-nine. Availability beats consistency: showing a feed that's a few seconds stale is far better than an error. So eventual consistency is fine for the feed. The one thing we must never do is lose an uploaded photo once we've told the user it posted.

Ask scale

Assume five hundred million daily users who open the feed ten times a day. That's five billion feed reads a day. Divide by roughly a hundred thousand seconds in a day and you get fifty thousand requests a second, about a hundred thousand at peak. Uploads are a hundred million a day, around a thousand a second. Reads outnumber writes about fifty to one, so we optimise the read path.

Storage: each photo is about two megabytes, plus another megabyte of resized copies. A hundred million a day is three hundred terabytes daily, over a hundred petabytes a year. Metadata is tiny by comparison, about a kilobyte a post. Bandwidth is the scary one: ten petabytes a day of images going out. No origin can serve that. This is why images live in object storage and leave through a CDN.

DAU                 500M
feed opens/user/day 10
feed reads/day      5B
  5B / 100,000 s   = 50K rps
  peak x2          = ~100K rps

uploads/day         100M
  100M / 100,000 s = 1K wps
  peak x3          = ~3K wps

read:write          ~50:1

photo original      2 MB
resized variants    +1 MB
100M x 3 MB       = 300 TB/day
  x 365           = ~110 PB/yr

metadata 1 KB/post = 100 GB/day

feed view: 20 imgs x 100 KB
  = 2 MB per feed open
  5B x 2 MB       = 10 PB/day
  = ~115 GB/s out  -> CDN

Define API

Four or five endpoints cover it. The client asks for an upload slot and gets back a pre-signed URL, then puts the photo bytes directly into object storage, so our API servers never carry megabytes of image data. Then it creates the post with the upload ID and a caption. The feed is a GET with a cursor and a limit. Follow and like are simple POSTs, and every write takes an idempotency key so retries don't double-post.

POST /v1/uploads
  -> { uploadId, presignedUrl }
PUT  presignedUrl   (bytes to storage)
POST /v1/posts
  { uploadId, caption }
  -> { postId }
GET  /v1/feed?cursor=..&limit=20
  -> { posts[], nextCursor }
POST /v1/users/{id}/follow
POST /v1/posts/{id}/like

Data model

The data model. Users and posts are small records. Posts are sharded by post ID, with a time-sortable ID so newest-first is cheap, and an index by author for the profile grid. The follow graph is stored twice, once keyed by follower and once by followee, because we need both directions fast. A wide-column store like Cassandra suits posts and follows: huge volume, simple key lookups. The precomputed feed is just a list of post IDs per user in Redis.

users    (userId PK, name, bio)
posts    (postId PK, authorId,
          mediaKeys, caption, ts)
  shard by postId; index authorId
follows  (followerId, followeeId)
  two tables: by follower and
  by followee
feed     userId -> [postId..]
  in Redis, capped at ~800
likes    counter per postId

High-level design

Draw the upload path first. The app gets a pre-signed URL through the gateway and uploads the photo straight to object storage. Storage fires an object-created event, and media workers resize it into a thumbnail, a feed size and a full size, and strip location metadata. Meanwhile the app creates the post, and the post service writes the metadata row. The post is visible once the resized images exist.

Now the feed path. When a post is created, an event goes onto a queue. Fan-out workers look up the author's followers and push the new post ID onto each follower's feed list in the cache. When you open the app, the feed service reads your list, hydrates the top twenty IDs into full posts from a post cache, and returns image URLs. The app then pulls the images from the CDN, never from our servers.

Deep dive: media

First deep dive: the media pipeline. Resizing is slow and bursty, so it runs on a worker pool fed by a queue, scaled on queue depth. Each photo becomes a few fixed sizes in a modern format, and location data is stripped for privacy. Variants get immutable URLs that include the post ID and size, so the CDN can cache them for a year. A failed resize is just retried, because writing the same variant twice is harmless.

Why bother with pre-signed URLs? Because two gigabytes a second of image bodies, a thousand two-megabyte uploads, flowing through API servers would make them slow and expensive. Direct uploads keep the API small and stateless. On flaky mobile networks, use multipart uploads so a dropped connection resumes instead of restarting. And if someone uploads but never creates the post, a lifecycle rule cleans up the orphan after a day.

Deep dive: fan-out

Second deep dive, the heart of the problem: fan-out. With push, we do the work at write time: the median user has a few hundred followers, so a post means a few hundred small list writes, and every feed read becomes a single cache hit. With pull, we do nothing at write time and merge everyone you follow when you open the app, which is cheap to write but expensive on every read. Since reads dominate fifty to one, push is the default.

Deep dive: celebrities

Push breaks for celebrities. One post from an account with a hundred million followers means a hundred million writes, minutes of lag and a jammed queue. The standard fix is a hybrid. Accounts under a threshold, say ten thousand followers, are pushed. Above it, we skip fan-out. At read time, the feed service takes your precomputed list and merges in recent posts from the handful of celebrities you follow, which are hot and sit in cache anyway.

Deep dive: feed reads

Serving the feed. Each list holds only the newest several hundred IDs, which keeps memory bounded. Paginate with a cursor, the last post ID you saw, not with page numbers, because new posts arriving at the top would shift every page. Hydration is a batched multi-get from a post cache. We don't fan out to users who haven't opened the app in a month; when they return, we rebuild their feed with a pull. Ranking, if asked, scores the top few hundred candidates with a model.

Trade-offs

Trade-offs and failures. If a feed cache node dies, those users get a slower pull-based rebuild once, not an outage. If fan-out workers fall behind, feeds lag but nothing is lost, because the queue is durable. A viral post is a hot key, so replicate it across cache nodes. What to monitor: fan-out lag from post to follower, feed cache hit rate, p ninety-nine feed latency and resize queue depth.

Name what you traded. Precomputed feeds duplicate post IDs millions of times, spending memory to buy fast reads. Eventual consistency spends a few seconds of freshness to buy availability. At ten times the scale, lower the celebrity threshold so fewer posts fan out, shard the feed cache by user ID, run regions that each serve their own users, and move old originals to cheaper cold storage while keeping the small sizes hot.

Say it like this

Here's the summary you'd give. Photos go directly to object storage with a pre-signed URL, get resized by background workers and are served through a CDN. Metadata sits in a sharded store. The feed is fan-out on write into per-user lists of post IDs, with a hybrid pull for celebrity accounts merged at read time. Cursor pagination, batched hydration, and eventual consistency because reads matter more than instant freshness.