Concepts — A Server-State Query Cache

Phase 5 · Spec area §35. Prerequisites: fw-08.

1. Why this matters

Server state is not client state, and treating it as such is one of the most expensive architectural mistakes in frontend work. Server state is shared, stale by default, and asynchronously invalidated by actors you cannot see. A query cache is the machinery that admits this.

This is also the densest concentration of race conditions in the track. Build the races deliberately.

2. Architecture

query key -> cache entry { data, status, updatedAt, subscribers }
          -> fetch (deduplicated, cancellable)
          -> notify subscribers

3. Build order

1. Key normalisation (structural equality, stable ordering).
2. Cache with status: fresh | stale | fetching | error.
3. Deduplication: N simultaneous subscribers, one request.
4. Stale-while-revalidate.
5. Retries with backoff; distinguish retriable from terminal errors.
6. Cancellation on unsubscribe.
7. Invalidation, exact and by key prefix.
8. Background refetch (focus, reconnect, interval).
9. Optimistic updates with rollback.
10. Garbage collection of unobserved entries.
11. Pagination and dependent queries.

4. Failure Lab — build every race

1. Out-of-order responses. Request A then B for the same key; A resolves last. Show A's data winning. Fix with a sequence number; explain why timestamps are insufficient.
2. Optimistic rollback onto a changed base. Apply an optimistic update, then a different server update arrives, then the optimistic one fails. Roll back to what?
3. Dedup + cancellation. Three subscribers share one request; one unsubscribes. Does the request abort? Should it?
4. Refetch storm. Window focus triggers refetch of 50 queries simultaneously.
5. GC race. An entry is collected while a component is mid-mount.
6. Cross-tab. Two tabs, same key, one mutates.

4.5 Deep dive: why timestamps fail for last-write-wins

The obvious fix for out-of-order responses is "keep the newest." It does not work.

t=0   request A dispatched
t=10  request B dispatched
t=50  response B arrives   (server processed at t=20)
t=90  response A arrives   (server processed at t=15)

Which is newer? By response arrival, A. By server processing, B. By request dispatch, B. Only the third is under your control and monotonic on the client.

Client clocks are also not trustworthy across tabs, and server timestamps have clock skew and insufficient resolution — two writes in the same millisecond are indistinguishable.

The rule: order by a monotonic counter you control, incremented at dispatch. Not by wall clock, not by arrival, not by server time. A sequence number is one integer and it is exactly correct.

Same conclusion as fw-08's navigation ordering, and for the same reason — which is why these two modules are adjacent.

0. Orientation

The 60-second version

Server state is not client state, and treating it as such is one of the most expensive architectural mistakes in frontend work.

A query cache is the machinery that admits this. Every feature in it — deduplication, stale-while-revalidate, background refetch, invalidation, retries — is a consequence of that table's right-hand column.

Putting server state in a client state manager is not a style disagreement. It is asserting the left column about data that lives in the right, and the bugs that follow are exactly the ones you would predict: data that never refreshes, caches with no eviction policy, and hand-rolled deduplication in five places.

The densest concentration of races in the track

Build them deliberately. The one that generalises furthest:

t=0   request A dispatched
t=10  request B dispatched
t=50  response B arrives   (server processed at t=20)
t=90  response A arrives   (server processed at t=15)

Which is newer? By arrival, A. By server processing, B. By dispatch, B. Only the third is under your control and monotonic on the client. Client clocks are untrustworthy across tabs; server timestamps have skew and insufficient resolution.

Order by a monotonic counter you control, incremented at dispatch. Not wall clock, not arrival, not server time. One integer, exactly correct — the same conclusion fw-08 reaches, from a different direction.

The design rule worth carrying into reviews

Optimistic UI is a latency optimisation that trades correctness under concurrency. It is right for low-contention data (your own profile) and wrong for high-contention data (a shared counter, a seat booking). "Is this data contended?" is the question — and it is a product question, not a technical one.

What you should be able to do after this module

4.6 Deep dive: optimistic rollback when the base moved

The genuinely hard case, and the one most implementations get wrong.

state:  { title: "A" }
user edits    → optimistic { title: "B" }
server push   → { title: "C", author: "X" }   (someone else changed it)
your mutation FAILS

Roll back to what? "A" discards the other user's change. Keep "C" and the optimistic edit silently vanishes — which is correct but confusing. Merge and you are writing a CRDT.

The practical answers, in increasing order of honesty:

1. Snapshot-and-restore — restore the pre-mutation value. Simple; loses concurrent updates.
2. Invalidate and refetch — discard local state, ask the server. Correct, costs a round trip, and flickers.
3. Store the optimistic change as a separate layer applied over server state, removed on failure. Correct and composable; substantially more machinery.

Most libraries do (1) and document it. Option (3) is what you need when concurrent editing is real.

Design rule worth stating in a review: optimistic UI is a latency optimisation that trades correctness under concurrency. It is right for low-contention data (your own profile) and wrong for high-contention data (a shared counter, a seat booking). "Is this data contended?" is the question, and it is a product question, not a technical one.

4.7 Deep dive: the cache-key problem

useQuery(['todos', { status: 'done', page: 1 }])
useQuery(['todos', { page: 1, status: 'done' }])   // same query, different object

Keys must be structurally compared with stable ordering, or you get duplicate entries, doubled requests, and invalidations that miss.

The sharp edges:

• Key order must not matter — serialise deterministically.
• Undefined vs missing should usually be the same key.
• Functions and class instances in keys are un-serialisable and usually a design error.
• Partial matching for invalidation (['todos'] invalidates ['todos', ...]) requires the key to be a path, which is why array keys beat string keys.

Note the resemblance to bi-08's constraint-space cache key and fw-07's content hash: the key must capture every input that affects the value, and no more. Too little and you serve stale data; too much and you never hit the cache.

4.8 Deep dive: normalised vs document caches

The trade is: normalised caches give automatic consistency across queries and demand that you describe your data model to the cache. Document caches stay simple and push consistency onto explicit invalidation.

Most libraries chose document caches, and the reason is worth understanding: REST responses have no reliable identity. Without ids and a schema you cannot normalise. GraphQL clients normalise because GraphQL gives them __typename and id for free.

That is a case where the data format determined the client architecture — a good example for any argument about API design having downstream consequences far beyond the wire.

5. Trade-offs

Cache-first vs network-first. Latency vs freshness; the right answer differs per query, which is why these libraries are configuration-heavy.

Normalised vs document cache. Normalisation gives consistency across queries and costs a schema and complexity. Most libraries chose document caches; say why.

Optimistic updates. Better perceived latency, and rollback is genuinely hard when the base moved underneath.

Core terminology

Mental models

A cache key must capture every input that affects the value, and no more. Too little and you serve stale data; too much and you never hit the cache. This is the same statement as fw-07's content hash and bi-08's constraint-space key — the third module in this track to reach it.

Sharp edges specific to query keys:

• Key order must not matter — serialise deterministically.
• undefined vs missing should usually be the same key.
• Functions and class instances in keys are unserialisable and usually a design error.
• Partial matching for invalidation (['todos'] invalidating ['todos', …]) requires the key to be a path, which is why array keys beat string keys.

Rollback is only simple when nothing else changed. The genuinely hard case:

state:  { title: "A" }
user edits    → optimistic { title: "B" }
server push   → { title: "C", author: "X" }   (someone else changed it)
your mutation FAILS

Roll back to "A" and you discard the other user's change. Keep "C" and the optimistic edit silently vanishes — correct but confusing. Merge and you are writing a CRDT. The practical answers, in increasing order of honesty: snapshot-and-restore (simple, loses concurrent updates), invalidate and refetch (correct, costs a round trip, flickers), or store the optimistic change as a separate layer applied over server state (correct and composable, substantially more machinery). Most libraries do the first and document it.

The data format determined the client architecture. Most libraries chose document caches, and the reason is that REST responses have no reliable identity — without ids and a schema you cannot normalise. GraphQL clients normalise because GraphQL gives them __typename and id for free. That is an unusually clean example of API design having consequences far beyond the wire, and it is worth having ready for any argument about schema design.

Configuration-heaviness is a symptom, not a defect. Cache-first versus network-first is a latency/freshness trade whose right answer differs per query — a user's own draft and a shared dashboard have genuinely different requirements. That is why these libraries expose so many knobs.

Common misconceptions

1. "Server state can live in the client store." It is shared, stale by default, and invalidated by actors you cannot see. Putting it in a client store means hand-rolling deduplication, staleness, retries and invalidation in application code.

2. "Keep the newest response." By what clock? Arrival order is the bug. Server timestamps have skew and resolution limits. Use a monotonic dispatch counter.

3. "Optimistic updates are a UX best practice." They trade correctness under concurrency for latency. Right for low-contention data, wrong for a shared counter or a seat booking.

4. "Rolling back means restoring the previous value." Only when nothing else changed underneath. When the base moved, restoring discards someone else's write.

5. "Normalised caches are the sophisticated choice." They demand a schema or reliable id extraction. With REST you usually cannot normalise, which is why most libraries did not.

6. "Cache keys are strings." Array/path keys are what make prefix invalidation possible. A string key cannot express "invalidate everything under todos."

7. "Deduplication means one subscriber owns the request." If it does, an unsubscribe aborts a request three others are waiting on. The ownership question is a design decision, not an implementation detail.

8. "Retries make the app resilient." Retrying a 4xx wastes calls and can duplicate side effects. Retriable and terminal errors must be distinguished, and backoff is what stops a failure becoming a stampede.

Interview talking points

• "Server state is shared, stale by default, and invalidated by actors you can't see. A query cache is the machinery that admits that. Putting it in a client state manager means reimplementing deduplication, staleness, retries and invalidation in application code — usually inconsistently, in five places."
• "Timestamps don't solve last-write-wins. Arrival order is the bug, client clocks differ across tabs, and server clocks have skew and insufficient resolution. Order by a monotonic counter you increment at dispatch — one integer, exactly correct. The router module reaches the same conclusion from a completely different direction."
• "Optimistic UI is a latency optimisation that trades correctness under concurrency. The question I ask is whether the data is contended — your own profile, yes, ship it; a seat booking or a shared counter, no. And that's a product question, not a technical one."
• "The hard case is rollback when the base moved. Restoring the pre-mutation value discards someone else's write; keeping theirs makes the user's edit silently vanish; merging means writing a CRDT. Most libraries snapshot-and-restore and document the limitation, which is the honest choice."
• "Most query libraries chose document caches over normalised ones because REST responses have no reliable identity. GraphQL clients normalise because __typename and id come for free. That's the cleanest example I know of an API format determining client architecture."
• "A cache key must capture every input that affects the value and no more — too little serves stale data, too much never hits. It's the same rule as a build system's content hash and Blink's layout-cache key, which is why I treat 'what's in the key?' as the first question in any caching review."

6. Principal Engineer Review

1. Why is a timestamp insufficient for last-write-wins? Give the failing scenario.
2. Optimistic update rollback when the base changed: specify the correct behaviour, and defend it.
3. Argue that server state should never live in a client state manager. Then give the exception.
4. Design invalidation for a mutation affecting an unknown set of queries. What do you give up?
5. A team reports "the cache shows stale data sometimes." Give your diagnostic procedure.

Connections to other modules

• fw-08 — prerequisite, and the seam: a loader with a server side effect cannot be fixed client-side. Both modules independently conclude "order by a counter you control."
• fw-01 — server state is why a client store is not enough; the exception (and where the two genuinely overlap) is worth arguing explicitly.
• fw-07 — the same cache-key rule, in a build system.
• fw-10 — pagination and windowing interact: a virtualized list over a paginated query is where both modules' edge cases meet.
• bi-08 — the constraint-space cache key; third instance of "capture every input, and no more."
• fe-01 — "completion order is not issue order," and cancellation as a scheduling primitive.
• fe-02 — a cache without an eviction policy is a leak; garbage collection of unobserved entries is the policy this module makes explicit.