LOW LEVEL DESIGN · MACHINE CODING · MEDIUM · 17 OF 42

Design a Circuit Breaker: Low-Level Design with Java Code

A circuit breaker designed and coded: the closed, open and half-open state machine, a count-based sliding window with an O(1) failure rate, a minimum number of calls before tripping, slow calls counted as failures, a lazy cool-down timer, a limited number of half-open trials under a race, fallbacks, retry with exponential backoff and full jitter, listeners and the follow-ups, with full Java 17 code.

Reported atUber

Patterns usedObserverStateStrategyDecorator

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
1 / 19

Press play to hear the narration, or step through at your own pace. The full Java code is below.

Full Java solution

One file, Java 17, 274 lines, with a small main() demo. Save it as CircuitBreaker.java, then run javac CircuitBreaker.java && java CircuitBreaker.

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.Function;
import java.util.function.LongConsumer;
import java.util.function.LongSupplier;
import java.util.function.Supplier;

/**
 * A circuit breaker around calls to a remote dependency: CLOSED lets calls through and records outcomes in a
 * count-based sliding window; too many failures (or slow calls) trip it OPEN, where calls fail fast to a
 * fallback; after a cool-down it goes HALF_OPEN and lets a few trial calls decide. Plus retry with jitter.
 */
public final class CircuitBreaker {

    enum State { CLOSED, OPEN, HALF_OPEN }

    record Config(int windowSize, int minCalls, int failureRatePercent, long slowCallMillis,
                  long openMillis, int halfOpenTrials) {
        Config {
            if (windowSize <= 0 || minCalls <= 0 || minCalls > windowSize || halfOpenTrials <= 0)
                throw new IllegalArgumentException("bad config");
        }
    }

    /** Thrown (and passed to the fallback) when the breaker refuses a call without trying it. */
    static final class CallNotPermittedException extends RuntimeException {
        private static final long serialVersionUID = 1L;

        CallNotPermittedException(String name) { super(name + " is open: failing fast"); }
    }

    /** Observer: alerts, dashboards and logs hear about every transition. */
    interface StateListener { void onTransition(String breaker, State from, State to); }

    /** The last N outcomes in a ring buffer, with running counts so the failure rate is O(1). */
    static final class SlidingWindow {
        private final boolean[] failed;
        private int next, size, failures;

        SlidingWindow(int capacity) { failed = new boolean[capacity]; }

        void record(boolean failure) {
            if (size == failed.length && failed[next]) failures--; // the outcome falling out of the window
            else if (size < failed.length) size++;
            failed[next] = failure;
            if (failure) failures++;
            next = (next + 1) % failed.length;
        }

        int size() { return size; }

        int failureRatePercent() { return size == 0 ? 0 : failures * 100 / size; }

        void reset() {
            next = size = failures = 0;
            java.util.Arrays.fill(failed, false);
        }
    }

    // ---------- breaker state, guarded by "this" ----------
    private final String name;
    private final Config config;
    private final LongSupplier clock;
    private final SlidingWindow window;
    private final List<StateListener> listeners = new CopyOnWriteArrayList<>();
    private State state = State.CLOSED;
    private long openedAt;
    private int trialsStarted, trialsSucceeded;

    CircuitBreaker(String name, Config config, LongSupplier clock) {
        this.name = name;
        this.config = config;
        this.clock = clock;
        this.window = new SlidingWindow(config.windowSize());
    }

    void addListener(StateListener l) { listeners.add(l); }

    synchronized State state() {
        maybeHalfOpen();
        return state;
    }

    /** Runs the action if permitted; on refusal or failure returns fallback(error) instead. */
    <T> T call(Supplier<T> action, Function<Throwable, T> fallback) {
        if (!tryAcquirePermission()) return fallback.apply(new CallNotPermittedException(name));
        long start = clock.getAsLong();
        try {
            T result = action.get(); // the remote call runs outside the lock
            boolean slow = clock.getAsLong() - start > config.slowCallMillis();
            onResult(slow);
            return result;
        } catch (RuntimeException e) {
            onResult(true);
            return fallback.apply(e);
        }
    }

    private synchronized boolean tryAcquirePermission() {
        maybeHalfOpen();
        return switch (state) {
            case CLOSED -> true;
            case OPEN -> false;
            case HALF_OPEN -> {
                if (trialsStarted >= config.halfOpenTrials()) yield false; // enough probes in flight
                trialsStarted++;
                yield true;
            }
        };
    }

    private synchronized void onResult(boolean failure) {
        switch (state) {
            case CLOSED -> {
                window.record(failure);
                if (window.size() >= config.minCalls()
                        && window.failureRatePercent() >= config.failureRatePercent()) transition(State.OPEN);
            }
            case HALF_OPEN -> {
                if (failure) transition(State.OPEN); // one failed probe: back to open
                else if (++trialsSucceeded >= config.halfOpenTrials()) transition(State.CLOSED);
            }
            case OPEN -> { } // a call that started before the trip finished late: ignore it
        }
    }

    /** Lazy timer: no background thread, the cool-down is checked whenever someone asks. */
    private void maybeHalfOpen() {
        if (state == State.OPEN && clock.getAsLong() - openedAt >= config.openMillis())
            transition(State.HALF_OPEN);
    }

    private void transition(State to) {
        State from = state;
        state = to;
        switch (to) {
            case OPEN -> openedAt = clock.getAsLong();
            case HALF_OPEN -> trialsStarted = trialsSucceeded = 0;
            case CLOSED -> window.reset();
        }
        for (StateListener l : listeners) l.onTransition(name, from, to);
    }

    // ---------- retry with exponential backoff and full jitter ----------
    /** Retries transient failures, but never a call the breaker refused: that would defeat failing fast. */
    record Retry(int maxAttempts, long baseMillis, long capMillis, Random random, LongConsumer sleeper) {
        <T> T run(Supplier<T> action) {
            RuntimeException last = null;
            for (int attempt = 1; attempt <= maxAttempts; attempt++) {
                try {
                    return action.get();
                } catch (CallNotPermittedException e) {
                    throw e;
                } catch (RuntimeException e) {
                    last = e;
                    if (attempt == maxAttempts) break;
                    long ceiling = Math.min(capMillis, baseMillis << (attempt - 1)); // 100, 200, 400 ...
                    sleeper.accept((long) (random.nextDouble() * ceiling)); // full jitter: 0..ceiling
                }
            }
            throw last;
        }
    }

    // ---------- demo ----------
    static void check(boolean ok, String what) { if (!ok) throw new AssertionError(what); }

    public static void main(String[] args) throws InterruptedException {
        AtomicLong now = new AtomicLong(0);
        Config cfg = new Config(10, 5, 50, 2_000, 30_000, 2);
        CircuitBreaker breaker = new CircuitBreaker("payments-api", cfg, now::get);
        List<String> transitions = new ArrayList<>();
        breaker.addListener((n, from, to) -> {
            transitions.add(from + "->" + to);
            System.out.println("  [" + n + "] " + from + " -> " + to);
        });

        boolean[] healthy = {true};
        AtomicInteger remoteCalls = new AtomicInteger();
        Supplier<String> charge = () -> {
            remoteCalls.incrementAndGet();
            if (!healthy[0]) throw new IllegalStateException("503 from payments");
            return "charged";
        };
        Function<Throwable, String> fallback = e -> "queued for later (" + e.getMessage() + ")";

        System.out.println("Healthy: " + breaker.call(charge, fallback));
        healthy[0] = false;
        for (int i = 1; i <= 5; i++) System.out.println("Call " + i + ": " + breaker.call(charge, fallback));
        check(breaker.state() == State.OPEN, "5 calls, 5 of 6 failed: should be open");

        int before = remoteCalls.get();
        System.out.println("While open: " + breaker.call(charge, fallback));
        check(remoteCalls.get() == before, "open breaker must not call the remote");

        now.addAndGet(30_000);
        System.out.println("After 30 s: " + breaker.state());
        System.out.println("Trial 1: " + breaker.call(charge, fallback));
        check(breaker.state() == State.OPEN, "a failed trial reopens");

        now.addAndGet(30_000);
        healthy[0] = true;
        System.out.println("Trial 1: " + breaker.call(charge, fallback));
        System.out.println("Trial 2: " + breaker.call(charge, fallback));
        check(breaker.state() == State.CLOSED, "two good trials close it");

        // Slow calls count as failures: a dependency that answers in 3 s is as bad as one that errors.
        Supplier<String> slow = () -> {
            now.addAndGet(3_000);
            return "late";
        };
        for (int i = 0; i < 5; i++) breaker.call(slow, fallback);
        System.out.println("After 5 slow calls: " + breaker.state());
        check(breaker.state() == State.OPEN, "slow calls trip it");

        // Half-open lets only 2 trial calls through, even with 8 threads racing.
        now.addAndGet(30_000);
        CountDownLatch go = new CountDownLatch(1), done = new CountDownLatch(8);
        CountDownLatch refused = new CountDownLatch(6);
        AtomicInteger admitted = new AtomicInteger();
        Function<Throwable, String> countRefusal = e -> {
            refused.countDown();
            return "refused";
        };
        ExecutorService pool = Executors.newFixedThreadPool(8);
        for (int t = 0; t < 8; t++) {
            pool.submit(() -> {
                try {
                    go.await();
                    breaker.call(() -> {
                        admitted.incrementAndGet();
                        try {
                            refused.await(); // stay in flight until the other six have been turned away
                        } catch (InterruptedException e) {
                            Thread.currentThread().interrupt();
                        }
                        return "ok";
                    }, countRefusal);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                } finally {
                    done.countDown();
                }
            });
        }
        go.countDown();
        done.await();
        pool.shutdown();
        System.out.println("Half-open race: " + admitted.get() + " of 8 admitted, now " + breaker.state());
        check(admitted.get() == 2 && breaker.state() == State.CLOSED, "only 2 trials");

        // Retry with backoff and full jitter, outside the breaker; refused calls are not retried.
        List<Long> sleeps = new ArrayList<>();
        Retry retry = new Retry(4, 100, 1_000, new Random(7), sleeps::add);
        AtomicInteger attempts = new AtomicInteger();
        String r = retry.run(() -> breaker.call(() -> {
            if (attempts.incrementAndGet() < 3) throw new IllegalStateException("timeout");
            return "charged on attempt " + attempts.get();
        }, e -> {
            throw e instanceof RuntimeException re ? re : new IllegalStateException(e);
        }));
        System.out.println("Retry: " + r + ", slept " + sleeps + " ms");
        check(attempts.get() == 3 && sleeps.size() == 2, "retry twice");
        check(sleeps.get(0) <= 100 && sleeps.get(1) <= 200, "jitter stays under the backoff ceiling");
        System.out.println("Transitions: " + transitions);
    }
}

Notes

Requirements

The requirements. The breaker wraps any call and takes a fallback. It tracks the last N outcomes and trips when the failure rate reaches a threshold, but only after a minimum number of calls, so two early errors do not trip it. Calls slower than a limit count as failures. While open, calls fail fast to the fallback. After a cool-down, a few trial calls decide: all succeed and it closes, one fails and it reopens.

Pin the numbers. A count-based window of the last ten calls, tripping at fifty percent once at least five calls are in. Open for thirty seconds, then half-open with two trial calls; anyone else arriving during the trial fails fast. A failure is an exception or a call slower than two seconds. And there is one breaker per dependency, so a broken payments service never blocks calls to the inventory service.

Requirements
  • wrap any call: call(action, fb)
  • track the last N outcomes
  • trip when failure rate >= X%
  • but only after M calls
  • slow calls count as failures
  • open: fail fast to a fallback
  • after a cool-down: a few trials
  • all pass: close; one fails: open
  • notify on every state change
  • safe from many threads
Clarify first
  • window by count or by time?
  • count: last 10 calls
  • trip threshold?
  • 50% of at least 5 calls
  • how long open?
  • 30 s, then half-open
  • how many trials?
  • 2; extra callers fail fast
  • what is a failure?
  • exception or slower than 2 s
  • one breaker per dependency

Entities

The entities. The breaker holds its current state and a sliding window of recent outcomes. A config record holds the numbers: window size, minimum calls, threshold, slow-call limit, cool-down and trial count. A clock is injected so tests can jump thirty seconds instantly. And each call brings its own fallback, a function from the error to a default answer.

State machine

The heart is a state machine with three states and four transitions. Closed goes to open when the failure rate crosses the threshold. Open goes to half-open when the cool-down has passed. Half-open goes back to closed when every trial succeeds, and back to open on the first failed trial. Entering closed resets the window, so old failures cannot trip it again a moment later. Draw this diagram first in the interview; everything else is bookkeeping.

Transitions
  • CLOSED -> OPEN
  • failure rate >= 50% of >= 5
  • OPEN -> HALF_OPEN
  • 30 s have passed
  • HALF_OPEN -> CLOSED
  • 2 trials succeeded
  • HALF_OPEN -> OPEN
  • any trial failed
  • closed: window reset

Class diagram

In code, the State pattern can be a full class per state, but with three states and two events an enum with an exhaustive switch is clearer, and the compiler checks every state is handled. There are only two events: may I call, and here is the result. The config is a record that validates itself, so a window smaller than the minimum call count is rejected at construction.

State as an enum switch
enum State { CLOSED, OPEN, HALF_OPEN }
record Config(int windowSize,
    int minCalls, int failureRatePercent,
    long slowCallMillis, long openMillis,
    int halfOpenTrials)

switch (state) {
  case CLOSED -> true;
  case OPEN -> false;
  case HALF_OPEN -> trialsStarted++ < 2
}

Sliding window

The sliding window is a ring buffer of the last ten outcomes with a running failure count. Recording overwrites the oldest slot; if the outcome being dropped was a failure, the count goes down first. So the failure rate is one division, with no scan, on every call. If the interviewer prefers a time-based window, keep one bucket per second with counts, and drop buckets older than the window.

SlidingWindow
boolean[] failed; int next, size, fails

record(f):
  if full and failed[next]: fails--
  else if not full: size++
  failed[next] = f; if (f) fails++
  next = (next + 1) % capacity

rate = fails * 100 / size
time-based: buckets per second

Flow: closed to open

The closed flow. Ask for permission, which in closed is always yes. Run the action outside any lock, time it, and record the outcome. In the demo the payments service starts returning errors. After one success and four failures there are five calls in the window at eighty percent failure, which crosses fifty, so the breaker opens. The caller still gets an answer, the fallback, which queues the payment for later.

call(action, fallback)
if (!tryAcquirePermission())
  return fallback(new NotPermitted())
start = now
try {
  r = action.get()  // outside the lock
  onResult(now - start > slowMillis)
  return r
} catch (RuntimeException e) {
  onResult(true); return fallback(e)
}

Flow: open

While open, the remote service is not called at all. The demo checks that the remote call counter does not move. The fallback is a per-call strategy: queue the payment, serve a cached value, return an empty list of recommendations, or hide a widget. The refusal arrives as its own exception type, so the fallback can tell not tried from tried and failed, which matters for logging and for retries.

Fallback strategies
  • queue it for later (payments)
  • serve the cached value
  • return a safe default
  • (empty recommendations)
  • degrade: hide the widget
  • or fail fast with a clear error
  • refusal is an exception type,
  • so the fallback can tell
  • "not tried" from "tried, failed"

Flow: half-open

Half-open. There is no timer thread: whenever someone asks for permission, the breaker checks whether the cool-down has passed, and if so moves to half-open. That keeps the design simple and testable. In the demo the first trial fails, so it reopens for another thirty seconds. Next time two trials succeed and the breaker closes with a fresh window.

Lazy cool-down
void maybeHalfOpen() {
  if (state == OPEN
      && now - openedAt >= openMillis)
    transition(HALF_OPEN)
}
called from tryAcquirePermission()
  and state(): no timer thread

HALF_OPEN: trial fails -> OPEN
  2 trials succeed -> CLOSED

Timeouts

Timeouts. A dependency that answers in three seconds hurts as much as one that errors, because your threads are stuck, so calls slower than the limit count as failures; in the demo, five slow calls trip the breaker. But a call that hangs forever never reports at all, so also bound the wait itself, with a client timeout on the HTTP or database driver, or a future with a timed get. Keep that timeout shorter than the caller's own deadline.

Slow calls and timeouts
  • slow: took > 2 s, count as failed
  • 5 slow calls -> OPEN
  • but a call that never returns
  • is never recorded at all
  • so also bound the wait:
  • client timeout (HTTP, JDBC)
  • or future.get(2, SECONDS)
  • then cancel the task
  • timeout below the caller budget

Retry with jitter

Retries sit outside the breaker. Each failed attempt waits longer, a hundred, two hundred, four hundred milliseconds, capped. Full jitter picks a random wait between zero and that ceiling, so a thousand clients that failed together do not all retry in the same millisecond. Never retry a call the breaker refused, because that defeats failing fast, and only retry calls that are safe to repeat, or that carry an idempotency key.

Retry.run(action)
for attempt in 1..maxAttempts:
  try { return action.get() }
  catch (NotPermitted e) { throw e }
  catch (RuntimeException e) {
    ceil = min(cap, base << (attempt-1))
    sleep(random * ceil)  // full jitter
  }

slept [73, 149] ms: under 100, 200
retry only idempotent calls

Concurrency

Concurrency. The two decisions, may I call and record this result, are short synchronized methods, constant time each. The remote call itself runs with no lock held. In half-open, the breaker counts trials started, not finished, so when eight threads race in at once exactly two are admitted and six fail fast; the demo checks that. A result that arrives after the breaker already tripped is ignored. At very high request rates, use an atomic state and striped counters instead.

Thread safety
  • synchronized: tryAcquirePermission
  • and onResult, both O(1)
  • the remote call: no lock held
  • half-open: count trials started,
  • not finished, so 8 racing
  • threads admit exactly 2
  • late result after a trip: ignore
  • hot path at high QPS: an atomic
  • state + striped counters

Observer

Patterns. State for the three modes, Proxy because the breaker wraps the real call, Strategy for the fallback, and Observer for transitions: an alerting listener pages someone when payments opens, a dashboard colours it red, a log line records it. Retry, breaker and client stack as decorators around one call. Listeners must be quick, or be handed off to another thread, since they run during the transition.

Patterns in play
  • State: CLOSED, OPEN, HALF_OPEN
  • Proxy: call() wraps the action
  • Strategy: the fallback per call
  • Observer: StateListener
  • Decorator: Retry around breaker
  • around the client

Extensibility

Extensions. Swap the count window for per-second buckets over a minute. Add a predicate for which errors count, because a not-found answer is not an outage. Add a bulkhead, a semaphore that limits concurrent calls to one dependency. Grow the cool-down after repeated failed trials. Keep breakers in a registry keyed by dependency name, and give operators a manual force-open switch for incidents.

Extension points
  • time-based window: per-second
  • buckets over the last 60 s
  • ignore some errors: 404 is not
  • an outage (a predicate)
  • bulkhead: max concurrent calls
  • growing cool-down: 30 s, 60 s..
  • registry: one breaker per
  • dependency, made on demand
  • manual force-open for incidents

Testing

The full code is below the player. The demo trips the breaker after four failures, proves the open breaker never calls the service, fails one trial and reopens, then closes after two good trials. Slow calls trip it again. Then eight threads race into half-open and exactly two are admitted, using latches so the test never depends on timing. Finally a seeded retry succeeds on the third attempt with jittered waits under each ceiling.

Run it
Call 3: queued for later (503 from payments)
  CLOSED -> OPEN
While open: queued for later (... fast)
  OPEN -> HALF_OPEN, trial fails
  HALF_OPEN -> OPEN
  2 trials pass: HALF_OPEN -> CLOSED
After 5 slow calls: OPEN
Half-open race: 2 of 8 admitted
Retry: charged on attempt 3,
  slept [73, 149] ms

Follow-ups

Expect follow-ups. How do timeout, breaker and retry relate? They stack: the timeout bounds each attempt, the breaker decides whether to attempt at all, and retry sits outside both. Should breakers share state across service instances? Usually not; each instance's local view reacts fast and needs no coordination. Why a minimum call count? One failure out of two is noise. And service meshes do the same per host as outlier detection.

Follow-ups to expect
  • breaker vs retry vs timeout?
  • they stack: timeout inside,
  • breaker, then retry outside
  • shared across instances?
  • usually not: local view is
  • fast and good enough
  • why minimum calls?
  • 1 of 2 failing is noise
  • where in a service mesh?
  • outlier detection per host

Say it like this

The short version. Three states. Closed records each outcome in a ring buffer with a running failure count and trips at fifty percent of at least five calls, counting slow calls as failures. Open fails fast to a fallback without touching the dependency. After a cool-down, a fixed number of trial calls decide whether to close or reopen. Decisions are short synchronized steps, and retry with jittered backoff wraps the outside.

In the interview
  • "Three states. Closed records
  • outcomes in a ring buffer with a
  • running failure count and trips
  • at 50% of at least 5 calls,
  • slow calls included. Open fails
  • fast to a fallback. After the
  • cool-down, two trials decide.
  • Retry with jitter goes outside."

Interview questions

What problem does a circuit breaker solve?

When a dependency is failing or slow, continued calls tie up threads and connections and add load to a service that needs room to recover. The breaker detects the failures, stops calling for a cool-down period, answers from a fallback, and probes carefully for recovery.

Describe the three states and their transitions.

Closed passes calls and records outcomes; when the failure rate crosses the threshold it opens. Open refuses calls until a cool-down passes, then becomes half-open. Half-open admits a few trial calls: if all succeed it closes and resets the window, and if any fails it opens again.

How do you compute the failure rate efficiently?

A count-based ring buffer of the last N outcomes with a running failure count. Each record overwrites the oldest slot, adjusting the count if a failure falls out, so the rate is one division per call. A time-based variant keeps per-second buckets.

Why require a minimum number of calls before tripping?

With very few calls the rate is noise: one failure out of two is fifty percent. A minimum such as five calls stops the breaker from opening on a single unlucky request after a quiet period.

How do you stop a thundering herd in half-open?

Count trials started, not finished, under the same lock as the state. Only the configured number are admitted; every other caller fails fast until those trials report back, so a recovering service sees two requests rather than thousands.

How do timeouts and slow calls fit in?

Calls slower than a threshold are recorded as failures, since slowness exhausts threads just like errors. A call that hangs would never be recorded, so each attempt also needs a hard client timeout shorter than the caller deadline.

How should retries interact with the breaker?

Retry wraps the breaker, uses exponential backoff with full jitter so clients do not retry in lockstep, never retries a refused call, and only retries idempotent operations or those carrying an idempotency key.

How do you test a circuit breaker without waiting?

Inject the clock as a LongSupplier and advance it by hand to pass the cool-down. Use a controllable fake dependency for failures and slowness, latches to make half-open races deterministic, and a seeded Random and fake sleeper for retry jitter.