Request Coalescing

Have you ever asked yourself what actually happens when you fix a concurrency race by throwing a 409 Conflict at the client? I know this sounds like a narrow technical question about HTTP status codes - and on paper, returning 409 is textbook-correct whenever an in-flight distributed lock is already held. But if you look at it from the perspective of an end user on a degraded cellular connection whose mobile app just fired two background refresh calls in an elevator, is failing the second request really a solution - or did you just shift the burden of coordination from your gateway onto someone else's frontend code?

In the previous note on idempotency, the priority was defensive: stop concurrent requests carrying the exact same single-use token from racing into the database at the same time. The atomic Redis lock solved that part cleanly. But once that guard went live, a new friction showed up: the duplicate request didn't corrupt the database anymore, but it did surface as a 409 Conflict on the client. If the frontend didn't have special-case logic to quietly swallow that 409, the user still saw an error banner - and for background operations like session renewal, showing an error for a request that actually succeeded in parallel is just bad engineering.

That gap led to a different question: instead of rejecting the follower with an error, can the gateway coordinate concurrent callers so they share the exact same upstream result transparently?

From rejection to result sharing

Consider what happens during a token refresh. Request 1 arrives at $t=0\text{ms}$, acquires the lock, and begins rotating the token upstream. Request 2 arrives at $t=10\text{ms}$ carrying the identical token because the mobile network dropped an ACK.

Under defensive locking, Request 2 gets an immediate 409 Conflict. But Request 2 didn't want a different token - it wanted the exact same new session that Request 1 is already in the middle of computing. If Request 2 simply pauses for a few milliseconds, waits for Request 1 to finish, and reads Request 1's output from a short-lived cache, both network requests return 200 OK with the identical { accessToken, refreshToken } pair. The client gets what it asked for, the database only runs one mutation, and zero frontend error-handling code is required.

In distributed systems, this pattern is often called Request Coalescing (or SingleFlight). Moving from mutual exclusion to request coalescing changes the entire interaction model:

The architectural trade-off: self-healing promotion vs. strict determinism

When designing the follower behavior, there's a subtle architectural fork in the road: what should happen if the leader crashes or takes an unusually long time to respond?

You could design a "self-healing" state machine where, after a timeout, the follower attempts to promote itself to the new leader and re-executes the upstream call. But if you think about what that actually means under production load - when the upstream database is already crawling and connection pools are saturated - having multiple waiting followers suddenly "promote" themselves and fire duplicate mutations into an already-struggling service is how minor latency hiccups cascade into full upstream outages.

We chose Strict Determinism over complex self-healing:

Two roles, no promotion race, and a state machine you can hold in your head.

One honest limit on that last claim, though. "Exactly one execution per token" holds as long as the leader finishes inside the lock's TTL. If the lock is set for 5 seconds and the leader takes 6, the lock lapses and a request arriving at second 5.1 will acquire it and execute the mutation a second time. The window is narrow and the TTL is chosen to sit well above the upstream's realistic worst case - but it is a real ceiling, not an absence of one, and a system that genuinely cannot tolerate a second execution needs idempotency at the upstream, not just coalescing at the gateway.

The implementation

The entire mechanism is encapsulated in a reusable gateway service. The key is derived by hashing the sensitive input (the refresh token, interim MFA token, or user ID) using SHA-256 so that raw credential material is never stored as Redis keys:

import { Injectable, Logger } from '@nestjs/common';
import { createHash } from 'crypto';

export class GatewayTimeoutException extends Error {
  constructor(message = 'Upstream processing timeout') {
    super(message);
    this.name = 'GatewayTimeoutException';
  }
}

@Injectable()
export class SingleUseTokenLockService {
  private readonly logger = new Logger(SingleUseTokenLockService.name);

  constructor(private readonly redisClient: any) {}

  /**
   * Executes an operation with strict request coalescing.
   * @param scope Domain namespace (e.g. "refresh", "mfa-verify", "mfa-enable")
   * @param rawKey The unique input to hash (token or user ID)
   * @param fn The upstream operation to execute if leader
   * @param ttlSeconds TTL for lock and result cache (default 5s)
   */
  async executeWithCoalescing<T>(
    scope: string,
    rawKey: string,
    fn: () => Promise<T>,
    ttlSeconds = 5,
  ): Promise<T> {
    const keyHash = createHash('sha256').update(rawKey).digest('hex');
    const lockKey = `lock:${scope}:${keyHash}`;
    const resultKey = `result:${scope}:${keyHash}`;

    // 1. Attempt atomic leader acquisition
    let isLeader = false;
    try {
      const acquired = await this.redisClient.set(lockKey, '1', 'EX', ttlSeconds, 'NX');
      isLeader = acquired === 'OK';
    } catch (err) {
      this.logger.warn(`Redis connection error, falling back to direct execution: ${err}`);
      return fn(); // Fail-open: infrastructure outage must not block real users
    }

    if (isLeader) {
      try {
        const result = await fn();
        try {
          await this.redisClient.set(resultKey, JSON.stringify(result), 'EX', ttlSeconds);
        } catch (cacheErr) {
          this.logger.error(`Failed to cache leader result: ${cacheErr}`);
        }
        return result;
      } catch (execErr) {
        // Release lock immediately on failure so future attempts aren't blocked
        await this.redisClient.del(lockKey).catch(() => null);
        throw execErr;
      }
    }

    // 2. FOLLOWER: poll for the leader's cached result (up to 3000ms).
    const pollIntervalMs = 100;
    const maxWaitMs = 3000;
    const startTime = Date.now();

    while (true) {
      try {
        // Check BEFORE sleeping - the leader may already be done.
        const cached = await this.redisClient.get(resultKey);
        if (cached) return JSON.parse(cached) as T;

        // Lock gone but no result? The leader failed and released it.
        // Nothing is coming; don't hold the caller for the full timeout.
        if (!(await this.redisClient.exists(lockKey))) {
          const late = await this.redisClient.get(resultKey); // one last look
          if (late) return JSON.parse(late) as T;
          throw new LeaderFailedException('Upstream operation failed');
        }
      } catch (err) {
        if (err instanceof LeaderFailedException) throw err;
        this.logger.warn(`Error reading cached result: ${err}`);
      }

      if (Date.now() - startTime >= maxWaitMs) break;
      await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
    }

    // 3. Strict failure: timeout rather than double-executing
    throw new GatewayTimeoutException(
      'Processing is taking longer than expected. Please retry.',
    );
  }
}

Two bugs that were in this loop, and what they cost

Both of these came out of running the lab rather than reading the code, and both are the same kind of mistake: the loop was written as "wait, then check," when the correct shape is "check, then wait."

Sleeping before the first check. The original loop called setTimeout at the top of the body, so a follower always paid one full poll interval - even when the leader had already finished and the result was sitting in Redis before the follower ever arrived. Measured against real Redis with the result pre-cached, a follower that should have returned instantly took 101ms at a 100ms interval. Moving the GET above the sleep brings the same case down to 1ms. Nothing about the algorithm changed; two lines swapped order.

No way to notice the leader had died. The leader deletes its lock on failure, and the original article described that as making sure "subsequent retries aren't artificially blocked" - which is true, and also beside the point for the follower already waiting. That follower is polling result:, not lock:, and after a leader failure no result will ever be written. So it kept polling until the timeout expired: 3,012ms measured, for an answer that was already decided in the first 60. Checking whether the lock still exists turns that into a 66ms failure with a 503, which is both faster and more truthful - the request didn't time out, the thing it was waiting for failed.

The second one is worth dwelling on because the failing path was invisible in every normal test. Leaders succeed almost always, so the follower's failure branch essentially never executes in a happy-path demo, and the cost of getting it wrong is only paid by users during an incident - exactly when a three-second stall is most expensive.

The three protected invariants

We applied this abstraction across three distinct authentication boundaries:

1. POST /auth/sessions/refresh (keyed on sha256(refreshToken)): Eliminates the "double-refresh logout" bug entirely. Both in-flight network requests return 200 OK with the exact same rotated token pair.
2. POST /auth/mfa/verify (keyed on sha256(interimToken)): Stops double-clicks on OTP verification from racing through the database, preserving recovery codes and avoiding fragmented session records.
3. POST /auth/mfa/enable (keyed on sha256(userId)): Fixes the "Stale QR Secret" race - when a user repeatedly taps "Enable 2FA" on a sluggish UI, only one TOTP secret is generated and stored in the database, ensuring the QR code shown on screen matches the backend secret 100% of the time.

What I learned

Looking back at the progression from retry semantics to defensive locking and finally to request coalescing, the most interesting realization was how each "solution" naturally reveals the next layer of the problem.

First, we realized retries without status-code semantics cause self-inflicted load. Then, we saw that safe retries require mutual exclusion on single-use mutations. And finally, we realized that mutual exclusion alone isn't enough for good user experience - because rejecting valid callers with 409 Conflict is just an incomplete implementation of what should have been result sharing from the start.

Writing this down is part of that same process: it forces you to trace whether a design decision was an intentional trade-off or just an accident of what seemed easiest to write at the time.

Runnable Reproduction

A complete, production-grade reproduction running against real PostgreSQL 16 (ACID transactions, row-level locks) and real Redis 7 (atomic SET NX EX) is available in the lab repository:

• engineering-notebook-labs/request-coalescing