Notes
Requirements
The requirements. Callers borrow a connection and give it back by closing it. The pool keeps a minimum number open and warm, and never opens more than a maximum, because the database itself has a session limit. When everything is lent out, a caller waits up to a timeout, then gets a clear error. Dead connections are never handed out, idle ones are closed after a while, long-held ones are reported as leaks, and many threads use it at once.
Clarify the edge cases. When the pool is empty, wait up to a timeout rather than fail at once, and serve waiters fairly in arrival order. Hand out the most recently used idle connection, because it is warm and the others can age out. Validate on borrow and on return. A suspected leak is reported with the holder's name, not forcibly closed, because the holder may still be using it. And the pool grows lazily up to its maximum.
- borrow(): get a connection
- close(): give it back
- minIdle kept open and warm
- maxTotal never exceeded
- wait up to a timeout, then fail
- never hand out a dead connection
- close connections idle too long
- report leases held too long
- safe from many threads
- stats: total, idle, active
- pool empty: wait or fail?
- wait, up to a timeout
- fair to waiters?
- yes, first come first served
- which idle one to lend?
- most recently used: warm
- validate when?
- on borrow and on return
- leak: close it or report it?
- report with the holder name
- grow lazily up to max
Entities
The entities. The pool is the service object. A semaphore holds one permit per connection that may be lent out, which is how the maximum is enforced. An idle deque holds the connections nobody is using. A lease wraps a raw connection while a caller holds it. A factory opens new physical connections. Notice the split: the semaphore answers may I have one, and the deque answers which one.
Class diagram
The types. A raw connection only needs three methods: an id, a validity check and close, so the demo can use a fake and real code can wrap a JDBC connection. The config is a record that rejects a minimum larger than the maximum in its constructor. An idle entry remembers when it was returned, for eviction, and the lease registry remembers who borrowed each connection and when, for leak detection.
interface RawConnection {
String id(); boolean isValid();
void close();
}
record Config(int minIdle, int maxTotal,
long idleTimeoutMillis,
long leakThresholdMillis)
record Idle(RawConnection c, long since)
record LeaseInfo(String connId,
String holder, long borrowedAt) Proxy: the lease
Callers never touch the raw connection. They get a lease, which is the Proxy pattern: it looks like a connection, but its close method returns the connection to the pool instead of closing the socket. An atomic flag makes a second close a no-op, so a double return can never put one connection in the idle deque twice. And because it is auto-closeable, try-with-resources gives it back even when the query throws.
final RawConnection conn;
final AtomicBoolean returned;
public void close() {
if (returned.compareAndSet(false, true))
giveBack(this);
}
try (Lease l = pool.borrow("svc", 100)) {
l.query("SELECT 1");
} Factory
How a connection is opened is a Factory. In production it calls the database driver with a URL and credentials; in tests it returns a fake with a numbered id, so the demo runs with no database at all. The pool as a whole is the Object Pool pattern: when something is expensive to create and cheap to reuse, create a bounded number and lend them out.
interface ConnectionFactory {
RawConnection open();
}
prod: () -> new JdbcConnection(
DriverManager.getConnection(url))
test: () -> new FakeConnection(
"conn-" + ids.incrementAndGet())
the whole pool is the Object Pool
pattern: reuse, do not recreate Bounding with permits
The semaphore starts with max-total permits. Borrowing first takes a permit with a timed try-acquire, which is exactly wait up to the timeout, then give up. Fairness is on, so the longest waiter gets the next free permit. Holding a permit means you may own one connection, either an idle one or a newly opened one, so the total can never pass the maximum. In the demo, three are lent out and the fourth caller fails after fifty milliseconds.
permits = new Semaphore(maxTotal, true)
if (!permits.tryAcquire(timeout, MS))
throw new PoolExhaustedException(
"no connection within 50 ms")
holding a permit means:
you may own one connection,
idle or freshly opened
giveBack: permits.release() Flow: borrow
The borrow flow. Take a permit. Then poll the warm end of the idle deque, and validate each connection; a dead one, say after a database restart, is closed and dropped, and we try the next. If no idle connection is left, open a new one, which the permit already allows. Record the holder and time in the registry and return the lease. If opening fails, release the permit before rethrowing, or the pool would slowly shrink to nothing.
acquire a permit or throw
try {
while (conn == null && (c = pollFirst()))
valid ? conn = c : destroy(c)
if (conn == null) conn = openNew()
leased.put(lease, info(holder, now))
return lease
} catch (RuntimeException e) {
permits.release(); throw e
} Flow: give back
Giving back. Remove the lease from the registry. If the connection broke while in use, or the pool is shutting down, close it; otherwise push it on the warm end of the idle deque with the current time. Release the permit last. The order matters: the connection must already be in the deque when a waiting thread wakes up with that permit, or it would open a new connection for nothing.
leased.remove(lease) if (shutdown || !conn.isValid()) destroy(conn, "broken on return") else idle.addFirst(new Idle(conn, now)) permits.release() // last order matters: the connection is back in the deque before a waiter can take the permit
Idle eviction
Idle eviction. A housekeeper thread runs every thirty seconds or so. Because we always lend and return at the warm end, the cold end holds the connections that have been idle longest. Close them while the deque is above the minimum and the oldest has been idle past the timeout. After a burst grows the pool to three, quiet time shrinks it back to two. Real pools also retire connections after a maximum lifetime, to stay ahead of firewall and server timeouts.
while (idle.size() > minIdle
&& oldest idle >= timeout)
removeLast, destroy it
warm end: addFirst / pollFirst
cold end: peekLast / removeLast
so busy periods reuse the same few
and the extras age out together
also: max lifetime, e.g. 30 min,
to rotate past firewall timeouts Leak detection
Leak detection. The most common pool bug is a caller that borrows and never closes, and after a few of those the whole service hangs waiting for permits. Every loan is in the registry with its holder and time, so the housekeeper reports any lease held past a threshold. In the demo, the checkout service has held connection one for sixty-one seconds. Production pools also capture the stack trace at borrow time, which points straight at the missing close.
for (LeaseInfo i : leased.values())
if (now - i.borrowedAt()
>= leakThresholdMillis)
report(i)
LeaseInfo[conn-1,
checkout-service, borrowedAt=0]
production: capture the stack trace
at borrow time, log it on a leak Concurrency
Concurrency without a global lock. The semaphore is the only place a thread waits. The idle set is a concurrent deque, the registry a concurrent map, the counters atomics, and each lease's return flag an atomic boolean. The permit ties them together: whoever holds one is entitled to exactly one connection. Crucially, we never hold a lock while opening a connection, which can take a hundred milliseconds, and every failure path gives its permit back.
- Semaphore: the only place to wait
- LinkedBlockingDeque: idle set
- ConcurrentHashMap: lease registry
- AtomicInteger: total, created
- AtomicBoolean: return once
- never hold a lock while opening
- a connection (100 ms or more)
- release the permit on every
- failure path, in finally/catch
Extensibility
Extensions. Validating on every borrow costs a round trip, so busy pools validate idle connections in the housekeeper instead. Add a maximum lifetime, prefill to the minimum in the background, or split into per-tenant partitions so one noisy tenant cannot starve others. A metrics listener can record wait times, timeouts and leaks. And with a generic factory, the same skeleton pools sockets, buffers or anything else that is expensive to create.
- test-while-idle: validate in the
- housekeeper, not on every borrow
- max lifetime per connection
- prefill to minIdle in background
- per-tenant pools or partitions
- metrics listener: wait time,
- timeouts, leaks (Observer)
- generic Pool<T> for sockets,
- threads, buffers
Testing
The full code is below the player. The demo starts with two warm connections, lends three, and shows the fourth caller timing out. A returned connection is reused, a connection broken while idle fails validation and is replaced, and a sixty-one-second loan is reported as a leak. Then eight threads run sixteen hundred queries through a pool of three and never see more than three active, idle eviction shrinks it back to two, and shutdown closes everything.
Borrowed conn-1, conn-2, conn-3 4th borrow: no connection within 50 ms After one close: conn-3 ran SELECT 1 closed conn-2: failed validation Leaks: [conn-1, checkout-service] 8 threads: 1600 queries, max active 3 closed conn-4: idle too long Evicted 1 idle: total=2 idle=2 Shut down: total=0 idle=0
Follow-ups
Expect follow-ups. The classic one: now design a thread pool executor. It is the same idea turned around: a bounded task queue and worker threads that loop taking tasks, a core size kept alive and a maximum for bursts, idle workers retired after a keep-alive, and a rejection policy when the queue is full. How big should a connection pool be? Smaller than you think. And remember that every service instance has its own pool, so instances times pool size must fit the database limit.
- now design a thread pool executor
- task queue + N worker threads
- core vs max size, keep-alive
- full queue: reject, caller-runs
- how big should the pool be?
- small: cores x 2 is a start
- caller in a transaction waits
- for a 2nd connection: deadlock
- one pool per service instance x
- instances vs db max sessions
Say it like this
The short version. A fair semaphore with max-total permits bounds the pool and gives borrow its timeout. Idle connections sit in a deque, lent and returned at the warm end, and are validated before they are handed out. Callers get a proxy whose close returns the connection exactly once. A housekeeper closes idle connections down to the minimum and reports leases held too long.
- "A fair semaphore of maxTotal
- permits bounds the pool and gives
- borrow a timeout. Idle ones sit
- in a deque, warm end first,
- validated on borrow. Callers get
- a proxy whose close() returns it.
- A housekeeper evicts idle ones
- and reports long leases."