Step 6 — Growth Detection: Telling a Leak From Noise

Goal

Turn "the heap went up" into a defensible claim, and design the only automation that catches leaks.

Prerequisites

  • Steps 1–5 complete

Predict first

Two workloads, 12 cycles each, 60 components created and destroyed per cycle, GC forced between every cycle. One registers each component in a module-level array; one does not.

  1. What slope (MB/cycle) do you expect for each?
  2. What R² do you expect for the clean run?

Most people answer the second question "near zero". Commit to a number before running.

Run

npm run growth

Expected output:

  clean  ▁▁▁▁▁▁▁█████  0.68MB -> 0.69MB
         slope 0.001 MB/cycle   R² 0.742

  leaky  ▁▁▂▃▃▄▅▆▆▇██  2.51MB -> 22.71MB
         slope 1.836 MB/cycle   R² 1.000

What just happened

The clean run has R² = 0.742. Over a short series, noise routinely looks strongly correlated. If R² alone were your test, you would have just reported a leak that does not exist.

Slope alone is equally insufficient — caches warming, lazy compilation, and an unsettled heap all produce real positive slopes for a while.

The claim requires both, read together:

slopeR²Interpretation
highhighreal leak driven by the repeated operation
highlowwarming / lazy compilation / unsettled heap — not yet a finding
lowhighsteady tiny growth; quantify against session length before acting
lowlownoise

Two further requirements before the claim is honest:

  • Magnitude against session length. 0.05 MB/cycle means nothing until multiplied by how many cycles a real session performs. 4 MB over a working day does not justify a sprint; 400 MB does.
  • A control. Run the same harness against a path you believe is clean. Without one you cannot distinguish your application leaking from your harness leaking. Harnesses leak.

Three rules the experiment encodes:

  1. Force GC between cycles, or you are measuring GC scheduling, not retention.
  2. Use identical repeated cycles, or slope is meaningless.
  3. Report slope and linearity. Neither alone supports a conclusion.

Design the soak test

Leaks need time, and no normal test spends it. Unit tests do not. Component tests do not. E2E suites rarely exceed a minute. Sketch the automation:

// nightly, NOT PR CI: slow by nature, and mildly flaky by nature
const samples = [];
for (let i = 0; i < 30; i++) {
  await doTheRepeatedThing(page);
  await cdp.send('HeapProfiler.collectGarbage');
  samples.push((await cdp.send('Runtime.getHeapUsage')).usedSize);
}
const { slope, r2 } = regression(samples.slice(5));      // discard warm-up
expect(slope).toBeLessThan(BUDGET_BYTES_PER_CYCLE);
expect(detachedNodeCount).toBe(0);                        // exact where bytes are not

Write down, for an application you actually work on:

  1. Which repeated action would you soak? Why that one?
  2. What is your budget in bytes per cycle, and how did you derive it from an assumed session length rather than from taste?
  3. Where does it run — and why not in PR CI?
  4. What does it assert besides slope? (Detached counts are exact; byte counts are not.)
  5. How do you stop it becoming a flaky test everyone disables within a month?

Question 5 is the one that decides whether this exists in twelve months. A soak test that fails noisily gets muted; one that reports a trend and only fails on a sustained regression survives. Picked up properly in fe-33.

Checkpoint

docs/verification.md Checkpoint 5.