SYSTEM DESIGN · CONCEPTS · HOW THE WEB WORKS · EPISODE 1

What Happens When You Type a URL?

Follow one request from your keyboard to a data centre and back: URL, DNS, TCP, TLS, HTTP, load balancers, servers, CDNs and rendering.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
1 / 26

Press play to hear the narration, or step through at your own pace. Every picture here is the same one used in the YouTube video.

Notes

The big picture

It starts with you and your browser. The browser is a client: a program whose job is to ask other computers for things.

On the other side is a server: a computer in a data centre whose job is to answer those requests, all day, for millions of clients.

Between them is the internet: a network of networks. Your home router, your internet provider, undersea cables, and the data centre, all passing data along.

People mix up two words. The internet is the roads: the cables, routers, and addresses. The World Wide Web is one kind of traffic on those roads: pages and files, linked together by URLs, and fetched with a protocol called HTTP.

Now, what did you actually type? A URL has parts. HTTPS is the scheme: the protocol to use. www.google.com is the host name. Then an optional path, like /search, and a query, like q=cats.

Internet vs Web
  • Internet = the roads
  • cables, routers, IP addresses
  • Web = one kind of traffic
  • pages and files, linked by URLs,
  • fetched with HTTP
Anatomy of a URL
https://www.google.com/search?q=cats

scheme   https
host     www.google.com
path     /search
query    q=cats

Step 1

Step one. The browser parses what you typed and decides: is this a search, or an address? Google.com looks like an address, so it goes ahead. It also checks: have I been here recently, and is the page still in my cache?

Step 2

Step two. Computers don't route by names. They route by IP addresses: numbers like 142.250, and so on. So the browser asks the Domain Name System, DNS, to translate the name into an address.

DNS answers with the IP address, and the browser and the operating system both cache it, so the next visit skips this step. DNS has its own episode, next, because there's a lot going on inside it.

One more idea before we connect. An IP address finds the machine. A port finds the program on that machine. Web servers listen on port 80 for plain HTTP, and port 443 for HTTPS.

IP address + port
142.250.x.x : 443

IP address → which machine
port       → which program

80   HTTP
443  HTTPS

Step 3

Data doesn't travel as one big block. It's cut into small packets. Each router reads the destination address and forwards the packet one hop closer. A request to Google might cross ten or more routers.

Step three. The browser opens a TCP connection to that address on port 443. TCP starts with a quick three-way handshake: synchronise, synchronise acknowledge, acknowledge. Now both sides know the other is ready.

TCP is what makes the internet feel reliable. It puts packets back in order, re-sends anything that gets lost, and slows down when the network is crowded.

Why TCP
  • packets arrive in order
  • lost packets are re-sent
  • the sender slows down
  • when the network is busy

Step 4

Step four. Because the URL says HTTPS, the two sides run a TLS handshake. The server proves who it is with a certificate, and they agree on secret keys. From now on, everything is encrypted. Episode three shows exactly how.

Step 5

Step five. Now the browser sends the actual HTTP request. A method, GET, a path, slash, and headers: the host name, the kind of browser, what formats it accepts, and any cookies for this site.

HTTP request
GET / HTTP/1.1
Host: www.google.com
User-Agent: Chrome
Accept: text/html
Cookie: …

Step 6

Step six happens inside the data centre. Big sites never rely on one server. The request first reaches a load balancer, which picks a healthy server from many copies. Often it passes through a reverse proxy, like Nginx, on the way.

The application server runs the code. It checks who you are, reads data from a cache or a database, and builds the response. For a search page, that's a lot of work done in a few milliseconds.

Step 7

Step seven. The response travels back the same way. It has a status code, 200 OK, headers that describe it, and the body: here, an HTML page.

HTTP response
HTTP/1.1 200 OK
Content-Type: text/html
Cache-Control: private

<!doctype html>
<html> … </html>

Step 8

Step eight. The browser reads the HTML and finds more to fetch: style sheets, JavaScript, images, fonts. Each is another request, many in parallel, and most come from a CDN: a copy of the files on a server close to you.

Step 9

Step nine. The browser lays out the page, paints the pixels, and runs the JavaScript. You see Google. And all of it usually takes well under a second.

Interview questions

What happens when you type a URL and press Enter?

The browser parses the URL, resolves the host name to an IP address with DNS, opens a TCP connection (port 443), runs a TLS handshake, and sends an HTTP request. A load balancer routes it to an application server, which builds a response from caches and databases. The browser receives HTML, fetches CSS, JavaScript and images (often from a CDN), then lays out, paints and runs scripts.

What is the difference between the internet and the web?

The internet is the global network of networks: cables, routers and IP addressing. The web is one service that runs on it: documents and resources identified by URLs and transferred with HTTP.

What is the difference between an IP address and a port?

The IP address identifies a machine on the network; the port identifies a program (service) on that machine. HTTP uses port 80 and HTTPS uses 443 by default.

Why does a second visit to a site load faster?

Caching at many layers: the DNS answer is cached by the browser and OS, static files are cached by the browser and served from nearby CDN servers, connections may be reused, and the server may answer from an in-memory cache.