SYSTEM DESIGN · CONCEPTS · DATABASES · EPISODE 18

Transactions, ACID and Isolation Levels

What a transaction promises (atomic, consistent, isolated, durable), how the database keeps those promises with a write-ahead log and locks or MVCC, the four isolation levels and the anomalies each one allows, and the two rules that keep transactions safe at scale.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
1 / 17

Press play to hear the narration, or step through at your own pace. Every picture here is the same one used in the YouTube video.

Notes

The problem

Here's the problem. You debit account one. Then the server crashes before you credit account two. Five hundred rupees have vanished. With many users doing this at once, it gets worse: two transfers can read the same balance and both succeed, spending money that isn't there.

A transaction wraps the updates in begin and commit. Now the database promises: either both updates happen, or neither does. Crash before commit, and it's as if you never started. That promise has a name with four parts: ACID.

Without a transaction
UPDATE accounts
  SET balance = balance - 500
  WHERE id = 1;        ← done

-- server crashes here --

UPDATE accounts
  SET balance = balance + 500
  WHERE id = 2;        ← never runs
With a transaction
BEGIN;
UPDATE accounts SET balance = balance - 500
  WHERE id = 1;
UPDATE accounts SET balance = balance + 500
  WHERE id = 2;
COMMIT;

crash before COMMIT → nothing happened

ACID

Atomic: all or nothing. Consistent: the rules you declared, like a balance never going negative, hold before and after. Isolated: transactions running at the same time don't see each other's half-finished work. Durable: once the database says committed, the data survives a crash. Two of these are easy. Two hide most of the interview.

ACID
  • A · Atomic all or nothing
  • C · Consistent rules always hold
  • (balance ≥ 0, keys exist)
  • I · Isolated concurrent transactions
  • don’t see each other’s
  • half-done work
  • D · Durable once committed, it
  • survives a crash

How durability works

How does the database keep the durability promise without being slow? With a write-ahead log. On commit, it appends a record of the change to a log file and forces it to disk. That's one sequential write, which is fast. The actual table pages are updated later. After a crash, it replays the log, and nothing committed is lost.

Keep that log in mind. It's also what replication streams to follower databases, which is the next episode but one. Durability and replication are the same mechanism.

Why this matters
  • WAL makes commit = 1 sequential write
  • so transactions can be fast
  • the same log feeds replication:
  • followers replay the leader’s log
  • (episode 20)

Isolation

Now isolation, the part that hides the fine print. Transaction A reads a balance. At the same moment, transaction B is updating it. What does A see? The old value, the new value, or a value B will roll back in a second? Perfect isolation would run them one after the other, but that is far too slow. So databases offer levels.

First, the things that can go wrong. A dirty read: you see data another transaction hasn't committed yet, and it might roll back. A non-repeatable read: you read the same row twice and get different values. A phantom: you re-run a query and new rows appear. And a lost update: two transactions read, both write, one overwrites the other.

Anomalies
  • dirty read: see uncommitted data
  • non-repeatable read: read a row
  • twice, get two different values
  • phantom read: re-run a query,
  • new rows appear
  • lost update: two writes,
  • one overwrites the other

Isolation levels

The levels trade safety for speed. Read uncommitted prevents nothing. Read committed, the default in Postgres, prevents dirty reads: you only ever see committed data. Repeatable read also freezes the rows you read for the whole transaction. Serializable prevents everything, as if transactions ran one at a time, but it costs the most and may abort your transaction and ask you to retry.

Isolation levels
level            dirty  non-rep  phantom
READ UNCOMMITTED  ✗      ✗        ✗
READ COMMITTED    ✓      ✗        ✗
REPEATABLE READ   ✓      ✓        ✗*
SERIALIZABLE      ✓      ✓        ✓

✓ = prevented · *Postgres prevents it

MVCC

How does Postgres isolate without locking everyone out? Multi-version concurrency control. Every update writes a new version of the row instead of overwriting it. A reader sees the versions that were committed when it started, a consistent snapshot. Readers never block writers and writers never block readers. Old versions are cleaned up later.

How Postgres does it: MVCC
  • every update writes a new row version
  • readers see the version that was
  • committed when they started
  • readers never block writers
  • writers never block readers
  • old versions are cleaned by VACUUM

Lost updates

Here's the trap interviewers love. Two transactions read a balance of a thousand. A writes a thousand minus five hundred. B writes a thousand minus three hundred. Final balance: seven hundred. It should be two hundred. Read committed, the default, does not prevent this. You have to.

Two fixes. One: don't read then write; let the database do the math in one statement, balance equals balance minus five hundred. Two: when you must read first, lock the row with select for update, or use optimistic locking with a version column and retry if it changed. Say either of these and the interviewer relaxes.

The lost update
A: SELECT balance → 1000
B: SELECT balance → 1000
A: UPDATE … SET balance = 1000 - 500
B: UPDATE … SET balance = 1000 - 300

final: 700. Should be 200.

READ COMMITTED does NOT prevent this
Two fixes
1. atomic update: let the DB do the math
   UPDATE accounts
   SET balance = balance - 500 …

2. lock the row while you decide
   SELECT … FOR UPDATE;
   (or optimistic: WHERE version = 7)

Rules

Three rules for the real world. Keep transactions short: never call another service inside one, because it holds locks while it waits. Touch rows in the same order everywhere, or two transactions will deadlock waiting for each other. And a transaction covers one database. Across services there is no begin and commit; that needs sagas, episode thirty-three.

Rules at scale
  • keep transactions short:
  • no network calls inside them
  • touch rows in the same order
  • (avoids deadlocks)
  • one transaction = one database;
  • across services, use sagas (ep 33)

In the interview

In the interview, say it like this. The transfer is one transaction, so both rows change or neither. I'd write balance equals balance minus amount in the update itself, to avoid lost updates at read committed. I'd keep the transaction short with no external calls, and add a check constraint so a balance can never go negative. That's a complete answer.

Say it like this
  • “The transfer is one transaction, so
  • both rows change or neither. I’d do
  • balance = balance - amount in the
  • UPDATE to avoid lost updates at
  • READ COMMITTED, keep the transaction
  • short with no external calls, and
  • add a CHECK (balance >= 0).”

Interview questions

What does ACID stand for?

Atomic (all changes in the transaction happen or none do), Consistent (declared rules such as constraints hold before and after), Isolated (concurrent transactions do not see each other’s uncommitted work, to the degree the isolation level promises), Durable (once committed, the change survives a crash).

How does a database make commits durable without being slow?

With a write-ahead log: on commit it appends the change to a sequential log and flushes it to disk, which is one fast write. Table pages are updated later. After a crash the log is replayed. The same log is what replication streams to followers.

What are the isolation levels and what does each prevent?

READ UNCOMMITTED prevents nothing. READ COMMITTED (the PostgreSQL default) prevents dirty reads. REPEATABLE READ also prevents non-repeatable reads, and in PostgreSQL phantoms too. SERIALIZABLE makes transactions behave as if run one at a time, at the highest cost, and may abort a transaction so it must be retried.

What is a lost update and how do you prevent it?

Two transactions read the same value, both compute a new value and write it, and one write overwrites the other. READ COMMITTED does not prevent it. Fix it by computing in the statement (SET balance = balance - 500), by locking the row with SELECT … FOR UPDATE, or with optimistic locking on a version column and a retry.

What is MVCC?

Multi-version concurrency control: each update creates a new row version instead of overwriting, and each transaction reads the versions committed as of its snapshot. Readers and writers do not block each other. Old versions are removed later (VACUUM in PostgreSQL).

Why should transactions be short?

A transaction holds locks and keeps row versions alive until it ends. A network call inside it holds those locks for the whole call, which stalls other transactions and can cause deadlocks and timeouts. Do external work before or after the transaction.

Can a transaction span two services or databases?

Not with plain BEGIN/COMMIT. Across services you need two-phase commit (slow and fragile) or, more commonly, a saga: a sequence of local transactions with compensating actions if a later step fails, usually driven by an outbox and a queue.