Design Spotify: Audio Delivery, Playlists, Offline Play and Accurate Play Counts

Listen

Requirements. Users search the catalog and play any track. They create playlists, follow others' playlists and can edit some together. Premium users download music for offline play. Every qualifying play is counted, because royalties depend on it. And users get personalised weekly mixes. Podcasts, lyrics, the social activity feed and ad serving are out of scope.

Qualities. A track should start in about two hundred milliseconds, and the next one should already be loaded, so there's no gap between songs. Playlist edits should show up on your other devices within seconds. Play counts must be accurate: nothing lost, nothing double counted, because they turn into payments. Licences differ by country, so a track available in one market may be blocked in another. And music should keep playing even if parts of the backend have problems.

Ask scale

Assume two hundred fifty million daily listeners at two and a half hours each: over six hundred million hours a day. At about three and a half minutes per track, that's around ten billion plays a day, roughly a hundred thousand track starts per second. Average concurrent listeners are about twenty-six million, double at peak. At a hundred sixty kilobits a second, peak egress is around eight terabits. Big, but an order of magnitude below video.

Catalog: around a hundred million tracks. Store each at three quality levels, about fifteen megabytes in total per track, so the whole catalog is about one and a half petabytes. The most popular million tracks, which get most of the plays, are only about fifteen terabytes and fit in CDN edge caches easily. The larger data stream is actually play events: ten billion a day at a couple of hundred bytes each is two terabytes daily.

DAU                   250M
listening/user/day    2.5 h
  = 625M hours/day
avg track             3.5 min
  = ~10B plays/day
  / 100,000 s        = 100K/s

avg concurrent = 625M / 24
                      = ~26M
peak x2 at 160 kbps
  = 52M x 160 kbps   = ~8 Tbps

tracks                 ~100M
avg length             3.5 min
qualities 96 + 160 + 320 kbps
  = 576 kbps x 210 s
  = ~15 MB per track, all
100M x 15 MB         = ~1.5 PB

popular 1M tracks
  = 15 TB -> fits in CDN edges
play events 10B x 200 B
  = 2 TB/day

Define API

The API. Search by text and type. Asking to play a track checks rights and returns a short-lived signed CDN URL plus the decryption key and quality. Playlists are read by revision, and edited by sending operations like add, move and remove along with the revision you started from. Plays are reported in batches, each with a unique play ID generated on the device, which is what lets the server deduplicate retries.

GET  /v1/search?q=..&type=track
GET  /v1/tracks/{id}/play
  -> { cdnUrl (signed, 1 h),
       key, quality }
GET  /v1/playlists/{id}?rev=..
POST /v1/playlists/{id}/ops
  { baseRev, ops:[add|move|rm] }
  -> { newRev }
POST /v1/plays  (batched)
  [{ playId, trackId, ms, ts }]

Data model

Data model. The catalog is read-mostly and heavily cached: tracks, albums, artists and the file keys for each quality. Rights rows say whether a track is allowed in a country and for what dates. A playlist has an owner, a revision number and collaborators, and its entries are ordered tracks. Play records are an append-only log keyed by the device-generated play ID. Playlists suit a wide-column store; plays go to a durable stream and then a warehouse.

track   (trackId, albumId,
         artists[], duration,
         files{96,160,320})
rights  (trackId, country,
         allowed, from, until)
playlist(playlistId, ownerId,
         rev, collaborators[])
entry   (playlistId, pos, trackId)
play    (playId PK, userId,
         trackId, msPlayed, ts)

High-level design

The play path. The app searches, then asks to play a track. The catalog service checks the rights service: is this track licensed in the user's country right now? If yes, it returns a signed CDN URL that expires in an hour and the key to decrypt the file. The app streams from the nearest CDN edge. A miss falls through to audio storage, but popular tracks are almost always at the edge.

Now the rest. Playlist edits go to the playlist service and its store. Play reports go into a durable event stream. Two consumers read it: a royalty pipeline that deduplicates and counts qualifying plays, and the recommendation system, which uses listening history to build weekly mixes in a batch job and writes them back as ordinary playlists. That's a nice touch: recommendations reuse the playlist machinery.

Deep dive: delivery

Deep dive one: delivery. Unlike video, a track is a single small file, so there's no need for a segment ladder. Fetch it with range requests so the first seconds arrive immediately and playback starts. About thirty seconds before the end, prefetch the next track in the queue so playback is gapless. Keep a client-side cache of recently played tracks, because people replay favourites constantly, and every replay from cache is free bandwidth.

Offline downloads use the same encrypted files, stored on the device. The decryption key comes with a licence that expires, say after thirty days, so the app must go online now and then to renew it. If the subscription lapses, keys stop renewing and the files become unplayable. Plays made offline are queued on the device with their play IDs and uploaded later, so they still count toward royalties exactly once.

Deep dive: playlists

Deep dive two: playlists. Treat a playlist like a small shared document: a snapshot plus a log of operations, each with a revision number. A client sends its operations with the revision it started from. If someone else edited in between, the server rebases the operations onto the newer revision. Adds and removes rarely conflict, and a move of a track that was deleted is just dropped. The reply includes the new revision and any operations the client missed, so every device converges.

What about a chart playlist followed by tens of millions of people? Don't push its edits to anyone. A follower's library just stores the playlist ID. Each revision of the playlist is immutable, so cache it aggressively. Clients ask a cheap, cacheable question, what's the latest revision, and fetch the new one only when it changes. It's the pull model again, which suits content that a few people write and millions read.

Deep dive: play counts

Deep dive three: play counts. Because counts become money, aim for effectively exactly once. The device creates a unique play ID for each play. Industry practice is that a stream counts after about thirty seconds of listening. The app retries uploads until acknowledged, which means duplicates, so the consumer deduplicates on play ID. A real-time count feeds dashboards, but payouts come from a daily batch recount of the raw log, which also filters fraud like bot accounts looping one track.

Trade-offs

Failures. If the catalog service is slow, the app's queue already has signed URLs and keys for the next few tracks, so music keeps playing. If a CDN edge fails, retry another edge, or even another CDN provider. A playlist conflict is rebased, and an add is never lost. If the event pipeline backs up, counts lag but nothing is lost. Watch track start latency, playback stalls, and the duplicate rate in the counting pipeline as a health signal.

The trade-offs. Client caching and prefetching spend device storage and some wasted bytes to make playback instant. Signed URLs add a little complexity but stop people from hotlinking audio. Operation logs with rebasing are more work than overwriting the whole playlist, but they make collaboration safe. And batch recounting delays the final numbers in exchange for payouts you can defend. At ten times the scale: more edge capacity and bigger client caches.

Say it like this

The summary. About ten billion plays a day, under ten terabits at peak. Playing a track checks country rights, then returns a signed CDN URL; the app range-fetches, prefetches the next track and caches favourites. Offline uses encrypted files with expiring licences. Playlists are snapshots plus operation logs, rebased when edits collide, and popular ones are cached by revision. Plays carry a device-generated ID, count after thirty seconds, are deduplicated, and are recounted daily for payouts.