Start practising

SYSTEM DESIGN · CONCEPTS · SERVICES AND HOW THEY TALK · EPISODE 14

Service to Service: Sync vs Async, gRPC, Discovery

How services talk to each other: synchronous calls with REST or gRPC, asynchronous events through a queue, the chain-of-calls availability problem, service discovery and client-side load balancing, timeouts and retries, service meshes, and when to choose sync or async in a design.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
1 / 10

Press play to hear the narration, or step through at your own pace. Every picture here is the same one used in the YouTube video.

Notes

Synchronous

A synchronous call means: send a request and wait for the answer. The order service asks inventory to reserve the item, waits, then asks payments to charge the card, and waits again. Use REST for simplicity or gRPC for typed, fast calls. Sync fits when you genuinely need the answer before continuing.

Synchronous calls
  • caller sends a request and
  • waits for the response
  • REST/JSON: simple, universal
  • gRPC/protobuf: typed, fast,
  • HTTP/2, streaming
  • use when you need the answer
  • to continue: "is it in stock?",
  • "did the payment succeed?"

Chains break

The danger is chains. If each service is up ninety-nine point nine percent of the time, a request that passes through five of them in a row succeeds only about ninety-nine point five percent of the time. Latencies add up, and one slow service ties up threads in every caller above it. Keep synchronous chains short.

The chain problem
  • each service: 99.9% available
  • a request through 5 in a row:
  • 0.999⁵ ≈ 99.5%
  • → ~3.6 hours down per month
  • latencies add up too
  • one slow service ties up
  • threads in every caller
  • keep sync chains short

Asynchronous

The alternative is asynchronous. The order service publishes an 'order placed' event to a queue or log and returns right away. Email, analytics and loyalty points each consume it in their own time. If email is down, events simply wait. You can add consumers without touching the order service. The price: things happen eventually, and tracing a request is harder.

Asynchronous events
  • publish "OrderPlaced" to a queue
  • or log and return immediately
  • email, analytics, loyalty points
  • consume it when they can
  • + caller does not wait
  • + consumer down? events wait
  • + add new consumers freely
  • − eventual consistency
  • − harder to follow one request

Discovery

How does the order service even find payments? Payments runs on a dozen instances whose addresses change on every deploy. A service registry keeps the live list, through Kubernetes DNS or tools like Consul. Either you call a load balancer address, or the client fetches the list and picks a healthy instance itself.

Service discovery
payments runs on 12 pods,
IPs change on every deploy

registry: payments →
  10.0.3.17:8080, 10.0.4.2:8080 …
(Kubernetes DNS, Consul, etcd)

server-side: call a load
  balancer address
client-side: client picks an
  instance from the registry

Hygiene

Every network call needs hygiene: a timeout, always. Retries only on safe errors, with backoff and jitter, and idempotency keys on writes. A circuit breaker for dependencies that keep failing. Mutual TLS so services trust each other, and a trace id passed along. A service mesh like Istio can add much of this without code changes.

Every call needs
  • a timeout (always!)
  • retries only on safe errors,
  • with backoff + jitter
  • idempotency keys on writes
  • a circuit breaker for a
  • failing dependency
  • auth between services (mTLS)
  • a trace id passed along
  • a service mesh can add these
  • without changing app code

Workflows

For multi-step workflows there are two styles. Choreography: each service reacts to the previous service's event, loosely coupled but hard to see as a whole. Orchestration: one coordinator calls each step in order, explicit and easy to follow, but central. Long business flows like order fulfilment are often orchestrated.

Choreography vs orchestration
  • choreography: services react to
  • each other's events
  • + loose coupling
  • − flow is hard to see
  • orchestration: one coordinator
  • calls each step in order
  • + flow is explicit
  • − coordinator knows everyone
  • long workflows: often orchestrated

Decision

The rule of thumb: if you need the answer to continue, like a stock check, authentication or a price, call synchronously. If it's a side effect that can happen a second later, like email, search indexing or analytics, publish an event. Async also wins for fan-out to many consumers and for absorbing traffic spikes.

Sync or async?
  • need the answer now → sync
  • stock check, auth, price
  • side effects, can happen later
  • → async: email, search index,
  • analytics, notifications
  • fan-out to many consumers
  • → async (pub-sub)
  • spiky load → async (queue
  • absorbs bursts)

Say it like this

In an interview, label every arrow in your diagram as sync or async. Sync calls over gRPC with timeouts on the user-facing path, and keep that path short. Everything else goes through a Kafka topic as events. Mention retries with idempotency, and tracing across services.

In the interview
  • label every arrow: sync or async
  • sync via gRPC with timeouts
  • async via Kafka topic
  • "order-events"
  • keep the user-facing path short
  • move side effects off it
  • retries + idempotency
  • tracing across services

Interview questions

When should services communicate synchronously vs asynchronously?

Synchronously when the caller needs the result to continue (stock check, authorisation, price). Asynchronously for side effects that can happen later, fan-out to many consumers, or to absorb bursts of load.

Why are long chains of synchronous calls risky?

Availability multiplies (five services at 99.9% give about 99.5%), latencies add up, and a slow downstream service ties up threads and connections in every caller, causing cascading failures.

What is service discovery?

A way for services to find the current network addresses of healthy instances of another service, using a registry such as Kubernetes DNS, Consul or etcd, either behind a load balancer or with client-side selection.

What is the difference between choreography and orchestration?

In choreography, services react to each other's events with no central controller. In orchestration, a coordinator invokes each step explicitly. Choreography is looser; orchestration is easier to understand and change for long workflows.

Why use gRPC between services?

Compact binary messages, strongly typed contracts with generated clients, HTTP/2 multiplexing and streaming, and good performance for high-volume internal calls.

What is a service mesh?

An infrastructure layer, usually sidecar proxies, that handles service-to-service concerns such as mTLS, retries, timeouts, load balancing and telemetry without changing application code.

What must every service-to-service call have?

A timeout, a retry policy limited to safe errors with backoff and jitter, idempotency for writes, a circuit breaker for failing dependencies, authentication and a propagated trace id.