LOW LEVEL DESIGN · MACHINE CODING · EASY · 14 OF 42

Design a Logger with Multiple Sinks: LLD in Java

A logging library designed and coded: levels with cheap filtering and lazy messages, per-name level overrides, a LogManager factory, formatters and sinks (console, file, remote with batching) as Strategies, a Chain of Responsibility of handlers, an async dispatcher with a bounded queue, block or drop on overflow, clean shutdown, and the follow-ups, with a full Java 17 solution.

Patterns usedStrategyChain of responsibilityFactory methodSingletonObserver

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
1 / 20

Press play to hear the narration, or step through at your own pace. The full Java code is below.

Full Java solution

One file, Java 17, 334 lines, with a small main() demo. Save it as LoggingFramework.java, then run javac LoggingFramework.java && java LoggingFramework.

import java.io.BufferedWriter;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Instant;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.Supplier;

/** A logger with levels, formatters, several sinks in a chain, and an async buffered dispatcher. */
public class LoggingFramework {

    // ---------- the model ----------
    enum Level {
        DEBUG, INFO, WARN, ERROR;

        boolean atLeast(Level min) { return ordinal() >= min.ordinal(); }
    }

    record LogEvent(Instant time, Level level, String logger, String thread, String message) {}

    // ---------- formatters ----------
    interface Formatter { String format(LogEvent e); }

    static final Formatter PLAIN = e -> e.time() + " " + e.level() + " [" + e.logger() + "] " + e.message();

    static final Formatter JSON = e -> "{\"ts\":\"" + e.time() + "\",\"level\":\"" + e.level()
            + "\",\"logger\":\"" + e.logger() + "\",\"msg\":\"" + e.message().replace("\"", "\\\"") + "\"}";

    // ---------- sinks: where the bytes go ----------
    interface Sink extends AutoCloseable {
        void write(String line);

        default void flush() {}

        default void close() { flush(); }
    }

    static final class ConsoleSink implements Sink {
        public void write(String line) { System.out.println(line); }
    }

    static final class FileSink implements Sink {
        private final BufferedWriter out;

        FileSink(Path path) throws IOException {
            out = Files.newBufferedWriter(path, StandardCharsets.UTF_8);
        }

        public void write(String line) {
            try {
                out.write(line);
                out.newLine();
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        }

        public void flush() {
            try {
                out.flush();
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        }

        public void close() {
            try {
                out.close();
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        }
    }

    /** Remote collector: lines are batched, one network call per batch (simulated by a list of batches). */
    static final class RemoteSink implements Sink {
        final List<List<String>> sentBatches = new CopyOnWriteArrayList<>();
        private final int batchSize;
        private final List<String> pending = new ArrayList<>();

        RemoteSink(int batchSize) { this.batchSize = batchSize; }

        public void write(String line) {
            pending.add(line);
            if (pending.size() >= batchSize) flush();
        }

        public void flush() {
            if (pending.isEmpty()) return;
            sentBatches.add(List.copyOf(pending));
            pending.clear();
        }
    }

    // ---------- chain of responsibility ----------
    /** Each handler writes the event if it is at or above its level, then passes it down the chain. */
    static final class Handler {
        private final Level min;
        private final Formatter formatter;
        private final Sink sink;
        private Handler next;

        Handler(Level min, Formatter formatter, Sink sink) {
            this.min = min;
            this.formatter = formatter;
            this.sink = sink;
        }

        Handler then(Handler n) {
            this.next = n;
            return n;
        }

        void handle(LogEvent e) {
            if (e.level().atLeast(min)) sink.write(formatter.format(e));
            if (next != null) next.handle(e);
        }

        void flushAll() {
            sink.flush();
            if (next != null) next.flushAll();
        }

        void closeAll() {
            sink.close();
            if (next != null) next.closeAll();
        }
    }

    // ---------- async dispatcher ----------
    enum Overflow { BLOCK, DROP }

    /** Callers only enqueue; one background thread drains in batches and runs the chain. */
    static final class AsyncDispatcher {
        private static final LogEvent STOP = new LogEvent(Instant.EPOCH, Level.ERROR, "", "", "stop");
        private final BlockingQueue<LogEvent> queue;
        private final Handler chain;
        private final Overflow overflow;
        private final AtomicLong dropped = new AtomicLong();
        private final Thread worker;

        AsyncDispatcher(Handler chain, int capacity, Overflow overflow) {
            this.chain = chain;
            this.queue = new ArrayBlockingQueue<>(capacity);
            this.overflow = overflow;
            this.worker = new Thread(this::drain, "log-writer");
            worker.setDaemon(true);
        }

        void start() { worker.start(); }

        void submit(LogEvent e) {
            if (overflow == Overflow.DROP) {
                if (!queue.offer(e)) dropped.incrementAndGet(); // never slow the caller down
                return;
            }
            try {
                queue.put(e); // BLOCK: back-pressure instead of losing lines
            } catch (InterruptedException ex) {
                Thread.currentThread().interrupt();
            }
        }

        private void drain() {
            List<LogEvent> batch = new ArrayList<>();
            try {
                while (true) {
                    batch.add(queue.take());
                    queue.drainTo(batch, 255);
                    for (LogEvent e : batch) {
                        if (e == STOP) {
                            chain.flushAll();
                            return;
                        }
                        chain.handle(e);
                    }
                    batch.clear();
                    chain.flushAll(); // one flush per batch, not per line
                }
            } catch (InterruptedException ex) {
                Thread.currentThread().interrupt();
            }
        }

        /** Shutdown: enqueue a stop marker behind everything else, wait for the writer, close the sinks. */
        void close() throws InterruptedException {
            queue.put(STOP);
            worker.join();
            chain.closeAll();
        }

        long dropped() { return dropped.get(); }
    }

    // ---------- configuration and loggers ----------
    /** Root level plus per-name overrides; the longest matching prefix wins. */
    record Config(Level root, Map<String, Level> overrides) {
        Level levelFor(String name) {
            Level best = root;
            int bestLen = -1;
            for (Map.Entry<String, Level> o : overrides.entrySet()) {
                String k = o.getKey();
                boolean matches = name.equals(k) || name.startsWith(k + ".");
                if (matches && k.length() > bestLen) {
                    best = o.getValue();
                    bestLen = k.length();
                }
            }
            return best;
        }
    }

    static final class Logger {
        final String name;
        private final Level level;
        private final AsyncDispatcher out;
        private final Supplier<Instant> clock;

        Logger(String name, Level level, AsyncDispatcher out, Supplier<Instant> clock) {
            this.name = name;
            this.level = level;
            this.out = out;
            this.clock = clock;
        }

        boolean isEnabled(Level l) { return l.atLeast(level); }

        /** The supplier runs only when the level is enabled, so filtered debug lines cost almost nothing. */
        void log(Level l, Supplier<String> msg) {
            if (!isEnabled(l)) return;
            out.submit(new LogEvent(clock.get(), l, name, Thread.currentThread().getName(), msg.get()));
        }

        void debug(Supplier<String> m) { log(Level.DEBUG, m); }

        void info(String m) { log(Level.INFO, () -> m); }

        void warn(String m) { log(Level.WARN, () -> m); }

        void error(String m) { log(Level.ERROR, () -> m); }
    }

    /** Factory: one cached Logger per name, levels resolved from the config once. */
    static final class LogManager {
        private final Config config;
        private final AsyncDispatcher dispatcher;
        private final Supplier<Instant> clock;
        private final Map<String, Logger> loggers = new ConcurrentHashMap<>();

        LogManager(Config config, AsyncDispatcher dispatcher, Supplier<Instant> clock) {
            this.config = config;
            this.dispatcher = dispatcher;
            this.clock = clock;
        }

        Logger get(String name) {
            return loggers.computeIfAbsent(name, n -> new Logger(n, config.levelFor(n), dispatcher, clock));
        }
    }

    // ---------- demo ----------
    static void check(boolean ok, String what) {
        if (!ok) throw new AssertionError(what);
    }

    public static void main(String[] args) throws Exception {
        Path file = Files.createTempFile("app", ".log");
        RemoteSink remote = new RemoteSink(2);
        Handler console = new Handler(Level.WARN, PLAIN, new ConsoleSink());
        console.then(new Handler(Level.DEBUG, PLAIN, new FileSink(file)))
                .then(new Handler(Level.ERROR, JSON, remote));
        AsyncDispatcher dispatcher = new AsyncDispatcher(console, 1024, Overflow.BLOCK);
        dispatcher.start();
        Map<String, Level> overrides = new LinkedHashMap<>();
        overrides.put("payments", Level.DEBUG);
        Instant fixed = Instant.parse("2026-03-02T09:00:00Z");
        LogManager logs = new LogManager(new Config(Level.INFO, overrides), dispatcher, () -> fixed);

        Logger checkout = logs.get("checkout"), card = logs.get("payments.card");
        AtomicInteger built = new AtomicInteger();
        checkout.debug(() -> "cart dump " + built.incrementAndGet()); // filtered: supplier never runs
        checkout.info("order placed");
        card.debug(() -> "token fetched " + built.incrementAndGet());
        checkout.warn("stock low for item 42");
        card.error("gateway timeout");
        card.error("retry failed");
        check(built.get() == 1, "a filtered debug message was still built");

        // Four threads log 2,500 lines each through the same dispatcher.
        List<Thread> workers = new ArrayList<>();
        for (int t = 0; t < 4; t++) {
            Logger lg = logs.get("worker" + t);
            Thread th = new Thread(() -> {
                for (int i = 0; i < 2500; i++) lg.info("job " + i);
            });
            workers.add(th);
            th.start();
        }
        for (Thread th : workers) th.join();
        dispatcher.close();

        List<String> lines = Files.readAllLines(file);
        System.out.println("File sink: " + lines.size() + " lines, first: " + lines.get(0));
        check(lines.size() == 5 + 10_000, "lost or extra lines in the file");
        System.out.println("Remote sink: " + remote.sentBatches.size() + " batch, "
                + remote.sentBatches.get(0).size() + " ERROR lines as JSON");
        System.out.println("  " + remote.sentBatches.get(0).get(1));
        check(remote.sentBatches.size() == 1 && remote.sentBatches.get(0).size() == 2, "remote batching");
        check(logs.get("payments.card") == card, "loggers are cached per name");
        Files.delete(file);

        // DROP policy: a full queue never blocks the caller; the overflow is counted.
        RemoteSink mem = new RemoteSink(1000);
        AsyncDispatcher lossy = new AsyncDispatcher(new Handler(Level.DEBUG, PLAIN, mem), 4, Overflow.DROP);
        Logger noisy = new LogManager(new Config(Level.DEBUG, Map.of()), lossy, Instant::now).get("noisy");
        for (int i = 0; i < 10; i++) noisy.debug(() -> "tick");
        lossy.start();
        lossy.close();
        System.out.println("DROP policy: queue of 4, 10 logged, " + mem.sentBatches.get(0).size()
                + " written, " + lossy.dropped() + " dropped");
        check(lossy.dropped() == 6, "drop count");
    }
}

Notes

Requirements

The requirements. Four levels: debug, info, warn and error. Loggers have names like checkout or payments, and the configuration sets a level per name. Output goes to several sinks, console, file and a remote collector, each with its own minimum level and format, plain text or JSON. Many threads log at once, logging must not block the application, and a clean shutdown must not lose lines. File rotation is out of scope.

Ask the questions that change the design. Synchronous or asynchronous writes? Asynchronous, so callers only enqueue. What happens when the buffer is full? Configurable: block the caller, or drop and count. Must lines be ordered across threads? Order within each thread is enough. If the remote collector is down, buffer and retry, but never throw into business code.

Requirements
  • levels: DEBUG, INFO, WARN, ERROR
  • named loggers: checkout, payments
  • level per logger name, by config
  • sinks: console, file, remote
  • each sink: own level and format
  • formats: plain text, JSON
  • many threads log at once
  • logging must not block the app
  • no lost lines on clean shutdown
  • out of scope: log rotation
Clarify first
  • sync or async writes?
  • async: callers only enqueue
  • queue full?
  • configurable: block or drop
  • order across threads?
  • per thread order is enough
  • remote sink down?
  • buffer, retry, never throw
  • level change at runtime?
  • nice to have, via config

Entities

The entities. A log manager hands out named loggers. A logger checks the level and turns a call into a log event. The event goes to a dispatcher, which runs it through a chain of handlers. Each handler has a minimum level, a formatter that turns the event into text, and a sink that writes the text somewhere. Separating formatting from writing is the key move: any format can go to any destination.

Class diagram

In code: the level is an enum ordered by severity, with an at-least check. A log event is an immutable record of the time, level, logger name, thread and message, so it can safely cross threads. A formatter turns an event into a string, and a sink writes a string and can flush and close. Three tiny interfaces are the whole extension surface of the library.

Types
enum Level { DEBUG, INFO, WARN, ERROR;
  boolean atLeast(Level min) }
record LogEvent(Instant time, Level level,
    String logger, String thread,
    String message)
interface Formatter {
  String format(LogEvent e); }
interface Sink extends AutoCloseable {
  void write(String line);
  default void flush() {} }

Cheap filtering

Most log calls are filtered out, so filtering must be nearly free. The logger compares the level before doing anything else. Messages can be passed as a supplier, a small lambda, so an expensive message like a cart dump is only built if the line will actually be written. The demo proves it: a filtered debug call never runs its supplier. The clock is injected too, which keeps tests deterministic.

Logger.log()
void log(Level l, Supplier<String> msg) {
  if (!l.atLeast(level)) return;
  out.submit(new LogEvent(clock.get(), l,
      name, thread, msg.get()));
}

debug(() -> "cart " + dumpCart())
  logger at INFO: dumpCart() never runs
demo asserts the supplier ran
  once, not twice

Configuration

Configuration is a root level plus overrides by logger name, and the most specific match wins. With the root at info and payments at debug, checkout logs at info, while payments dot card inherits debug from its parent. Match whole name segments, key plus a dot, so a logger called payments x does not accidentally inherit. In a real library this config comes from a file and can be reloaded at runtime.

Config.levelFor(name)
root = INFO
overrides: payments = DEBUG

checkout       -> INFO (root)
payments       -> DEBUG
payments.card  -> DEBUG (prefix)
paymentsx      -> INFO (not a child)

match: name equals key, or
  starts with key + "."

Factory

Loggers come from a factory. The log manager caches one logger per name with compute-if-absent, so asking twice returns the same instance, and the level is resolved once instead of on every call. Real libraries make the manager a process-wide singleton for convenience. In the design, say that you pass it explicitly so each test can build its own with a fake clock and memory sinks.

LogManager.get(name)
loggers.computeIfAbsent(name, n ->
  new Logger(n, config.levelFor(n),
             dispatcher, clock))

same name: same Logger instance
level resolved once, not per call

usually a singleton per process;
  passed explicitly here so tests
  can build their own

Strategy: formats

Formatting is a strategy. Plain text puts the time, level, logger name and message on one line for humans. JSON emits one object per line for machines, so a log platform can search by field, and it must escape quotes inside the message. A pattern-string formatter or logfmt is another implementation. Because the handler holds the formatter, the same event can be plain on the console and JSON to the collector.

Formatters
PLAIN: time level [name] message
  2026-03-02T09:00:00Z INFO
  [checkout] order placed

JSON: {"ts":..., "level":"ERROR",
  "logger":"payments.card",
  "msg":"retry failed"}

escape quotes in JSON
logfmt, pattern strings: new ones

Sinks

Sinks are where the text goes. The console sink prints. The file sink wraps a buffered writer and flushes once per batch rather than once per line, which is a big speed difference. The remote sink collects lines and sends them in batches, one network call per batch instead of one per line, and flush sends whatever is left. A database sink or a message queue sink is just one more class.

Sinks
ConsoleSink: System.out.println
FileSink: BufferedWriter,
  flush per batch, close at the end
RemoteSink(batchSize):
  pending.add(line)
  if size >= batchSize: send batch
  flush(): send what is left

one network call per batch,
  not one per line

Chain of Responsibility

Handlers form a Chain of Responsibility. Each handler checks whether the event meets its own minimum level, writes it if so, and always passes the event to the next handler. So the console shows warnings and above, the file keeps everything, and only errors go to the remote collector as JSON. Adding a destination is adding a link, and no handler knows how many others exist.

Handler.handle(e)
if (e.level().atLeast(min))
  sink.write(formatter.format(e));
if (next != null) next.handle(e);

console(WARN, PLAIN)
  .then(file(DEBUG, PLAIN))
  .then(remote(ERROR, JSON))

add a sink: add a link
skip a sink: it just passes on

Flow: one log call

Follow one call. Checkout calls info with order placed. The logger checks the level, builds an immutable event and drops it on the dispatcher's queue, and the caller returns immediately. Later, the single writer thread takes the event and runs it through the chain: the console skips it because it is below warn, the file writes it, and the remote handler skips it. The application thread never touched a disk or a socket.

Async buffering

The dispatcher is the heart of async logging. A bounded blocking queue sits between the application threads and one writer thread. The writer blocks until there is an event, then drains up to two hundred and fifty five more in one go, runs them all through the chain and flushes once. One writer means the sinks are only ever touched by one thread, so they need no locks, and the bound means a logging storm cannot eat all the memory.

AsyncDispatcher
ArrayBlockingQueue<LogEvent> queue
writer thread:
  batch.add(queue.take())
  queue.drainTo(batch, 255)
  for (e : batch) chain.handle(e)
  chain.flushAll()  // once per batch

one writer: sinks need no locks
bounded: memory cannot run away

Overflow policy

When the queue is full, there are two honest choices. Block, which applies back-pressure and never loses a line: right for audit logs. Or drop, using offer, and count what was dropped: right for noisy debug output on a latency critical path. Either way, expose the dropped count as a metric. The demo fills a queue of four with ten lines under the drop policy: four are written and six are counted as dropped.

submit(e)
DROP:  if (!queue.offer(e))
         dropped.incrementAndGet()
BLOCK: queue.put(e)  // back-pressure

DROP for debug noise and
  latency-critical paths
BLOCK for audit logs
report the dropped count
  as a metric

Concurrency

Concurrency is simple because only the queue is shared. Events are immutable, loggers sit in a concurrent map, sinks are touched only by the writer, and the drop counter is atomic. Shutdown is the subtle bit: put a stop marker on the queue behind everything already waiting, join the writer thread and then close the sinks. A boolean flag would lose whatever was still queued. The demo logs ten thousand lines from four threads and checks the file has every one.

Thread safety and shutdown
  • callers share only the queue
  • events: immutable records
  • loggers: ConcurrentHashMap
  • sinks: touched by one thread
  • dropped: AtomicLong
  • shutdown: put a STOP marker
  • behind all pending events,
  • join the writer, close sinks
  • writer is a daemon thread

Extensibility

Extensions fit the same three interfaces. File rotation is a rolling file sink that switches files by size or by day. Remote retries add backoff and spill to a local file when the collector is down. A context map carries a request id on every event. Sampling is a handler that keeps one debug line in a hundred. Reloading configuration updates logger levels in place so existing references keep working.

Extension points
  • rotation: a RollingFileSink
  • by size or by day
  • remote retry: backoff +
  • a local spill file
  • context: MDC map on the event
  • (request id, user id)
  • sampling: a handler that keeps
  • 1 in 100 debug lines
  • reload config: swap levels
  • in place, loggers keep refs

Testing

The full code is below the player. The demo wires the console at warn, the file at debug and the remote sink at error with JSON in batches of two. It proves a filtered debug message is never built, checks the console and remote outputs, logs ten thousand lines from four threads and finds exactly ten thousand and five in the file, and fills a small queue under the drop policy to get exactly six drops.

Run it
2026-03-02T09:00:00Z WARN [checkout]
  stock low for item 42
2026-03-02T09:00:00Z ERROR [payments.card]
  gateway timeout
File sink: 10005 lines, first: ...
  INFO [checkout] order placed
Remote sink: 1 batch, 2 ERROR lines as JSON
DROP policy: queue of 4, 10 logged,
  4 written, 6 dropped

Follow-ups

Expect follow-ups. Why one writer? Sinks stay lock free and per thread order is preserved. What about a crash? Anything still queued is lost, so some teams write errors synchronously. A slow remote collector gets its own queue so it cannot stall the file sink. A sink that itself logs must be guarded against infinite recursion. And one queue keeps each thread's lines in order across every sink.

Follow-ups to expect
  • why one writer thread?
  • what if the app crashes?
  • unflushed lines are lost:
  • flush errors synchronously
  • remote collector is slow?
  • its own queue, never block
  • logging inside a sink?
  • guard against recursion
  • per-thread order across sinks?
  • one queue keeps it

Say it like this

The short version. Loggers check the level first and build messages lazily. Events are immutable records placed on a bounded queue, and a single writer thread drains them in batches through a chain of handlers, each a level, a formatter and a sink. When the queue is full you either block or drop and count, and shutdown uses a stop marker so nothing queued is lost.

In the interview
  • "Loggers check the level first
  • and build messages lazily. Events
  • are immutable and go on a
  • bounded queue; one writer drains
  • batches through a chain of
  • handlers, each a level, a
  • formatter and a sink. Full queue:
  • block or drop and count."

Interview questions

Why separate the Formatter from the Sink?

They change independently: the same event may be plain text on the console and JSON to a collector. With two small interfaces any format can go to any destination, and each piece is tested alone.

Where is Chain of Responsibility used and why?

Handlers are linked; each writes the event if it meets its own minimum level and then passes it to the next. Adding or removing a destination is adding or removing a link, and no handler needs to know about the others.

How do you avoid the cost of log calls that are filtered out?

Check the level before doing any work, and accept the message as a Supplier so expensive string building only happens when the line will be written. Resolve each logger's level once when it is created.

How does asynchronous logging work here?

Callers create an immutable LogEvent and put it on a bounded ArrayBlockingQueue. One writer thread takes an event, drains up to a batch more, runs them through the handler chain and flushes once per batch.

What happens when the log queue is full?

It is a policy: BLOCK uses put() and applies back-pressure so nothing is lost, which suits audit logs; DROP uses offer() and counts the dropped lines, which suits noisy debug output on latency-sensitive paths. Expose the count as a metric.

How do you shut down without losing lines?

Enqueue a STOP marker behind all pending events, join the writer thread, which flushes when it reaches the marker, and then close the sinks. A boolean stop flag could leave events stranded in the queue.

How are per-logger levels configured?

A root level plus overrides keyed by logger name; the longest matching name wins, matching whole dot-separated segments so payments.card inherits from payments but paymentsx does not.

Should the logger be a singleton?

A process-wide LogManager is convenient and common, but the design should not depend on it. Passing the manager (with its clock and sinks) explicitly lets tests build isolated instances with memory sinks and a fixed clock.