The Garbage Collector Bills You for Survivors, Not Garbage

Here are five runs of the same program, each in a fresh process on Node v22.22.3 (V8 12.4.254.21-node.56). Each run allocates exactly one million identical objects. The only thing that changes between rows is how many of those objects are still referenced when the loop ends. Every cell is the median of seven runs.

Same allocation count in every row, and roughly the same number of collections. But total time in GC goes from 13.4 ms to 333.8 ms — a factor of 25 — and the major collector goes from never running to 172 ms. Your absolute numbers will be different; the ratio between the rows is the finding.

Almost every piece of GC advice you have ever been given is downstream of that ratio. The collector does not walk your garbage. It walks your live objects and copies them somewhere else; the garbage is whatever is left behind, and leaving something behind costs nothing. V8's own writeup says it plainly: "we only pay a cost (for copying) proportional to the number of surviving objects, not the number of allocations."

So the question that actually matters when your p99 goes sideways is not "how much am I allocating." It is "how much of what I allocate is still alive when the collector shows up."

Here is the script that produced the table, if you want to run it yourself:

// row.mjs — one row per process: for k in 0 100 10 2 1; do node row.mjs $k; done
import { PerformanceObserver, constants as PERF } from 'node:perf_hooks';
 
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const keepEvery = Number(process.argv[2]); // 0 keeps nothing, 100 keeps 1%, 1 keeps everything
 
const stats = { minor: 0, minorMs: 0, major: 0, majorMs: 0 };
const obs = new PerformanceObserver((list) => {
  for (const e of list.getEntries()) {
    if (e.detail.kind === PERF.NODE_PERFORMANCE_GC_MINOR) { stats.minor++; stats.minorMs += e.duration; }
    else if (e.detail.kind === PERF.NODE_PERFORMANCE_GC_MAJOR) { stats.major++; stats.majorMs += e.duration; }
  }
});
obs.observe({ entryTypes: ['gc'] });
 
const survivors = [];
const N = 1_000_000;
const t0 = process.hrtime.bigint();
for (let i = 0; i < N; i++) {
  const o = { id: i, name: `user-${i}`, tags: ['a', 'b'], payload: { seq: i, s: 'y'.repeat(64) } };
  if (keepEvery && i % keepEvery === 0) survivors.push(o);
  else globalThis.__sink = o.id;
}
const wallMs = Number(process.hrtime.bigint() - t0) / 1e6;
 
// GC entries are delivered on a later turn. Disconnecting synchronously
// drops every one of them and prints a very convincing zero.
await sleep(60);
obs.disconnect();
 
console.log(JSON.stringify({ survived: survivors.length, wallMs, ...stats }));

Both of the odd-looking lines in that script are there because the obvious version lied to me. The await sleep(60): my first version disconnected the observer right after the loop and reported zero garbage collections for every row. Perfectly plausible-looking output, completely wrong. And one row per process: my first version ran all five in one process, and the arrays it kept alive from earlier rows made every later row's scavenges expensive, which flattened exactly the effect I was trying to measure.

Reachability, in about sixty lines

The collector cannot know what you still need. It approximates that with what you can still reach: start at a set of roots — the execution stack, the global object — follow every pointer, and anything you did not arrive at is garbage.

D and E point at each other and nothing else points at them. A reference-counting scheme would see two objects with a count of 1 each and keep them forever. A tracing collector never arrives, so they die. Here is that, as a program:

// toy-gc.ts — node --experimental-strip-types toy-gc.ts
class HeapObject {
  id: string;
  marked = false;
  references: HeapObject[] = [];
  constructor(id: string) { this.id = id; }
}
 
class ToyRuntime {
  private heap: HeapObject[] = [];
  private roots = new Set<HeapObject>();
 
  allocate(id: string): HeapObject {
    const obj = new HeapObject(id);
    this.heap.push(obj);
    return obj;
  }
  addRoot(obj: HeapObject) { this.roots.add(obj); }
 
  collect() {
    const worklist: HeapObject[] = [...this.roots];
    while (worklist.length > 0) {
      const obj = worklist.pop()!;
      if (obj.marked) continue;
      obj.marked = true;
      for (const ref of obj.references) {
        if (!ref.marked) worklist.push(ref);
      }
    }
 
    const survivors: HeapObject[] = [];
    const freed: string[] = [];
    for (const obj of this.heap) {
      if (obj.marked) { obj.marked = false; survivors.push(obj); }
      else freed.push(obj.id);
    }
    this.heap = survivors;
    return { kept: survivors.map((o) => o.id), freed };
  }
}
 
const rt = new ToyRuntime();
const objA = rt.allocate('A');
const objB = rt.allocate('B');
const objC = rt.allocate('C');
const objD = rt.allocate('D');
const objE = rt.allocate('E');
 
objA.references.push(objB);
objB.references.push(objC);
objD.references.push(objE);
objE.references.push(objD); // a cycle
 
rt.addRoot(objA);
 
const assert = (cond: boolean, msg: string) => { if (!cond) throw new Error(msg); };
const result = rt.collect();
console.log(result);
assert(result.kept.join() === 'A,B,C', 'A, B and C are reachable from the root');
assert(result.freed.join() === 'D,E', 'the D/E cycle is unreachable and must go');
console.log('assertions passed');

Output:

{ kept: [ 'A', 'B', 'C' ], freed: [ 'D', 'E' ] }
assertions passed

Real collectors do this concurrently, which is where it gets interesting. If the collector has already finished scanning object A, and your code then makes A point at some object it hasn't scanned yet — while deleting the only other path to it — the collector will happily free memory that is live. The standard fix is tri-colour marking (white = unvisited, grey = found but not yet scanned, black = fully scanned) plus a write barrier: a small piece of collector code the compiler injects into every pointer write, which re-greys anything a black object starts pointing at. Rick Hudson's ISMM keynote describes Go's version: "The write barrier is on only during the GC. At other times the compiled code loads a global variable and looks at it."

What V8 says about its own heap

Most explainers describe a V8 heap that has not existed for a while. Rather than trust one, ask the runtime. v8.getHeapSpaceStatistics() on a stock node -e process, Node v22.22.3:

space_name                  size_MB  used_MB
read_only_space                0.00     0.00
new_space                      1.00     0.48
old_space                      2.76     2.70
code_space                     0.25     0.04
shared_space                   0.00     0.00
trusted_space                  0.85     0.69
new_large_object_space         0.00     0.00
large_object_space             0.26     0.25
code_large_object_space        0.00     0.00
shared_large_object_space      0.00     0.00
trusted_large_object_space     0.00     0.00

Eleven spaces, not two. new_space is the young generation, internally two semi-spaces: you bump-allocate in From-Space, and when it fills, survivors are evacuated into To-Space and the two swap roles. That is the scavenger. Objects that survive a second scavenge get promoted to old_space instead.

Some of what gets repeated about this heap is out of date, and some of it was never right:

"Orinoco is V8's old-space algorithm." Orinoco is the codename of the project that made V8's collector parallel, incremental and concurrent — it is not an algorithm you can name in a sentence like "V8 uses Orinoco." The major collector is Mark-Compact: concurrent marking, concurrent sweeping, parallel compaction and pointer updating.

"Scavenges stop the thread and run single-threaded." They stop the main thread, but they are parallel. From the V8 blog: "Today, V8 uses parallel scavenging to distribute work across helper threads during the young generation GC."

"new_space is tiny." It is dynamically sized. Under a request-shaped load test on this machine it grew to 32 MB — both semi-spaces — against a default 2000 MB heap limit.

Now the actual pause numbers. Two million simulated request handlers — build a JSON body, parse it, project a DTO — with 5% of the results retained:

// pauses.mjs — node pauses.mjs
import { PerformanceObserver, constants as PERF } from 'node:perf_hooks';
 
const KIND = {
  [PERF.NODE_PERFORMANCE_GC_MINOR]: 'scavenge (minor)',
  [PERF.NODE_PERFORMANCE_GC_INCREMENTAL]: 'incremental marking',
  [PERF.NODE_PERFORMANCE_GC_MAJOR]: 'mark-compact (major)',
};
const durations = new Map();
new PerformanceObserver((list) => {
  for (const e of list.getEntries()) {
    const kind = KIND[e.detail.kind] ?? String(e.detail.kind);
    if (!durations.has(kind)) durations.set(kind, []);
    durations.get(kind).push(e.duration);
  }
}).observe({ entryTypes: ['gc'] });
 
// One "request": build a JSON body, parse it, project a DTO. 5% of DTOs are kept.
const retained = [];
const REQUESTS = 2_000_000;
const startedAt = process.hrtime.bigint();
for (let i = 0; i < REQUESTS; i++) {
  const body = JSON.stringify({ id: i, tags: ['a', 'b', 'c'], nested: { at: Date.now(), blob: 'x'.repeat(200) } });
  const parsed = JSON.parse(body);
  const dto = { id: parsed.id, key: `req-${i}`, tags: parsed.tags.slice(), size: parsed.nested.blob.length };
  if (i % 20 === 0) retained.push(dto);
}
const wallClockMs = Number(process.hrtime.bigint() - startedAt) / 1e6;
 
setTimeout(() => {                       // GC entries are delivered on a later turn
  const q = (a, p) => a[Math.min(a.length - 1, Math.floor(p * a.length))];
  let gcMs = 0;
  for (const [kind, d] of durations) {
    d.sort((a, b) => a - b);
    const total = d.reduce((a, b) => a + b, 0);
    gcMs += total;
    console.log([kind, d.length, total.toFixed(1), (total / d.length).toFixed(3),
      q(d, 0.5).toFixed(3), q(d, 0.99).toFixed(3), d[d.length - 1].toFixed(3)].join('  '));
  }
  const minor = durations.get('scavenge (minor)');
  console.log(`wall ${wallClockMs.toFixed(0)} ms, GC ${gcMs.toFixed(1)} ms (${(100 * gcMs / wallClockMs).toFixed(1)}%), retained ${retained.length}`);
  console.log(`scavenges under 1 ms: ${minor.filter((x) => x < 1).length}/${minor.length}`);
}, 50);

Seven fresh processes on a 4-core arm64 Linux container, every cell the median across them:

The p50 and p99 of two samples are not numbers worth printing, hence the dashes. Total GC time 149.4 ms out of 2663 ms of wall clock — 5.6%.

The scavenge row is where the shape lives. Across those seven processes the median pause never left the range 0.685 to 0.725 ms, a spread of 6%. The worst single pause in a run ranged from 0.97 ms to 18.6 ms. Same script, same machine, same two million requests. The p99 came out above a millisecond in five runs of the seven, and 256 of 260 scavenges in the median run finished under one. "Minor GCs are sub-millisecond" is a claim about the median being repeated as if it were a bound, and the median is the one part of that distribution that holds still. The tail is what shows up in your latency graphs.

The two mark-compacts are worth a second look for the opposite reason. Two million requests, and the entire major-collection bill is 1.8 ms — because the only things that survived were 100,000 flat DTOs. The 10% row of the table at the top of this article also retains 100,000 objects and pays 6.0 ms in majors; its survivors are nested objects carrying a 64-character string each. Same count, heavier survivors, three times the bill.

The allocation the collector never sees

The cheapest object is the one that never reaches the heap. V8, like Go's compiler, does escape analysis: if an object provably cannot outlive the frame that created it, its fields get scalar-replaced into registers and no allocation happens at all.

You can watch this flip. Same loop, same object literal, ten million iterations, GC counted by the same observer as before. The only difference is one line:

// escape.mjs — node escape.mjs
import { PerformanceObserver, constants as GC_KIND } from 'node:perf_hooks';
 
const settle = (ms) => new Promise((r) => setTimeout(r, ms));
 
async function bench(label, body) {
  let scavenges = 0, gcMs = 0;
  const watcher = new PerformanceObserver((list) => {
    for (const e of list.getEntries()) {
      if (e.detail.kind === GC_KIND.NODE_PERFORMANCE_GC_MINOR) { scavenges++; gcMs += e.duration; }
    }
  });
  watcher.observe({ entryTypes: ['gc'] });
 
  const N = 10_000_000;
  const start = process.hrtime.bigint();
  const sum = body(N);
  const nsPerOp = Number(process.hrtime.bigint() - start) / N;
 
  await settle(60);
  watcher.disconnect();
  console.log(`${label}: ${nsPerOp.toFixed(1)} ns/op, ${scavenges} scavenges, ${gcMs.toFixed(1)} ms in GC (sum ${sum})`);
}
 
await bench('stays local', (N) => {
  let sum = 0;
  for (let i = 0; i < N; i++) {
    const o = { a: i, b: i + 1, c: i + 2, d: i + 3, e: i + 4, f: i + 5, g: i + 6, h: i + 7 };
    sum += o.a + o.h;
  }
  return sum;
});
 
await bench('escapes', (N) => {
  let sum = 0;
  for (let i = 0; i < N; i++) {
    const o = { a: i, b: i + 1, c: i + 2, d: i + 3, e: i + 4, f: i + 5, g: i + 6, h: i + 7 };
    globalThis.__escaped = o; // the only difference
    sum += o.a + o.h;
  }
  return sum;
});

Six times slower and 860 collections instead of 2, from one assignment. (Stable to within 10% across three runs.) This is also the reason Go does not have generations, which is usually explained backwards. The Go team built a non-moving generational collector and measured it losing. Hudson, in the same keynote:

It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack. The result is that generational collection is much less effective than you might find in other managed runtime languages.

The blocker was the write barrier: a generational collector needs one on all the time, and Go's is otherwise off outside a GC cycle. That is a measured engineering tradeoff, not a philosophical position about generations — and the traffic runs both ways. ZGC, the OpenJDK collector usually cited as the non-generational low-latency one, went generational in JDK 21. The same wiki page is where its actual promise lives, and it is worth reading precisely: ZGC performs its expensive work concurrently "without stopping the execution of application threads for more than a millisecond," and "works well with heap sizes from a few hundred megabytes to 16TB." Not zero pauses. A bounded one.

The leak your collector is right about

A tracing collector reclaims unreachable memory. It cannot reclaim memory you are still pointing at, and "I forgot I was pointing at it" is what a memory leak looks like in a managed runtime. The canonical version: a cache keyed by a live object.

// weak.mjs — node --expose-gc weak.mjs
import v8 from 'node:v8';
 
const usedMB = () => { global.gc(); global.gc(); return v8.getHeapStatistics().used_heap_size / 1048576; };
const check = (cond, msg) => { if (!cond) throw new Error(msg); };
 
function fillCache(store, n) {
  for (let i = 0; i < n; i++) {
    const session = { id: i, notes: 'z'.repeat(1024) }; // ~1 KB
    store.set(session, { lastSeen: Date.now() });
    // request ends here; nothing else references `session`
  }
}
 
const base = usedMB();
const strongCache = new Map();
fillCache(strongCache, 50_000);
const afterMap = usedMB();
 
const weakCache = new WeakMap();
fillCache(weakCache, 50_000);
const afterWeakMap = usedMB();
 
console.log(`baseline           ${base.toFixed(2)} MB`);
console.log(`after Map          ${afterMap.toFixed(2)} MB  (+${(afterMap - base).toFixed(2)})`);
console.log(`after WeakMap      ${afterWeakMap.toFixed(2)} MB  (+${(afterWeakMap - afterMap).toFixed(2)})`);
console.log(`Map.size           ${strongCache.size}`);
 
check(afterMap - base > 15, 'the Map should retain tens of MB');
check(afterWeakMap - afterMap < 5, 'the WeakMap should retain almost nothing');
console.log('both checks passed');

baseline           3.51 MB
after Map          21.29 MB  (+17.78)
after WeakMap      22.29 MB  (+1.00)
Map.size           50000
both checks passed

Eighteen megabytes held by a cache nobody will ever read from again, because a Map key is a strong reference. The WeakMap doing the identical work holds one. The collector behaved correctly in both runs. Run the file without --expose-gc and it does not quietly measure the wrong thing: global.gc is undefined, line 4 throws TypeError: global.gc is not a function, and nothing is printed at all. That is the failure mode you want from a memory benchmark, and it is worth checking you have it before you trust one.

The natural next thought is WeakRef and FinalizationRegistry — a weak handle plus a callback when the target dies. Be careful what you promise yourself there. This is what actually happened when I ran it:

same turn the WeakRef was created      still alive   finalizers run: 0
next macrotask, no forced GC           still alive   finalizers run: 0
after allocation churn + 1 full GC     still alive   finalizers run: 0
after more churn + 2 more full GCs     still alive   finalizers run: 0
300 ms later                           still alive   finalizers run: 0

Five forced full collections and the object was never reclaimed. The cause is not the API and not the module system — I checked both. It is that every one of those lines ran inside the same still-executing top-level script body that created the object, and that frame kept a reference I could not see. Move the allocation into a function that has already returned and drive the checks from a setInterval instead, and the object is collected on round 1 and the finalizer runs on round 2 — one full turn after the memory was already gone. The same rewrite in a CommonJS file behaves identically, and so does the original shape: I ran both, and the module system changes nothing here.

Twitch's ballast, and why it worked

The best-known production GC story in Go is Ross Engers' 2019 writeup from Twitch. Their API gateway, Visage, was running 8–10 GC cycles per second with a live heap under 450 MiB on a machine with 64 GiB of RAM, and spending 30% of its CPU in GC-related functions. The opening line of the post:

We recently rolled out a small change that reduced the CPU utilization of our API frontend servers at Twitch by ~30% and reduced overall 99th percentile API latency during peak load by ~45%.

Go's pacer triggers a cycle when the heap reaches roughly twice the live set (GOGC=100). A small live set means a small trigger, which means constant collection. So they made the live set artificially large:

package main
 
import "runtime"
 
func main() {
	// 10 GiB of nothing, held for the lifetime of main
	ballast := make([]byte, 10<<30)
 
	serve() // the actual application
 
	runtime.KeepAlive(ballast)
}

Two details do the work. The allocation is virtual: ps on their test program showed a 100 MiB slice producing 108 MB of VSZ and 4.8 MB of RSS, because untouched pages are never faulted in. And a []byte contains no pointers, so the marker handles the whole thing in one step — their words: "the GC can mark the entire object in O(1) time."

The part everyone gets wrong is why latency improved. It was not shorter pauses. Twitch measured that directly: "the GC pause times before and after the change were not significantly different," and their pauses were, in their words, "on the order of single digit milliseconds, not the 100s of milliseconds improvement we saw at peak load." The win came from mark assists — when a goroutine allocates during an active GC cycle, Go charges it allocation debt and makes it do marking work before the allocation proceeds. Fewer cycles means fewer requests paying that tax.

The malloc myth, and its cousin

"malloc and free are expensive system calls." They are library functions.

// m.c — gcc -O2 -o m m.c && ./m && strace -c ./m
#include <stdlib.h>
#include <stdio.h>
#include <time.h>
int main(void) {
  struct timespec a, b; void *sink = 0;
  clock_gettime(CLOCK_MONOTONIC, &a);
  for (long i = 0; i < 10000000; i++) {
    void *p = malloc(64); ((char *)p)[0] = 1; sink = p; free(p);
  }
  clock_gettime(CLOCK_MONOTONIC, &b);
  double ns = ((b.tv_sec - a.tv_sec) * 1e9 + (b.tv_nsec - a.tv_nsec)) / 1e7;
  printf("%.1f ns per malloc+free pair (sink=%p)\n", ns, sink);
  return 0;
}

Ten million pairs: 7.8 ns each, and strace -c counts 32 syscalls for the whole process — every one of them from startup, three of them brk. The loop reaches the kernel zero times. Manual allocation is not slow because of syscalls, and a bump-pointer nursery is not fast because it avoids them. The real difference is that free does per-object bookkeeping in userspace while a copying collector does none.

"Calling global.gc() / System.gc() / runtime.GC() will help." It replaces a heuristic that has your allocation history with your guess. The one legitimate use is what I did above: forcing a known-good baseline inside a measurement script run with --expose-gc. Not in production.

Quick check

So what changes on Monday?

Less than you would like. A GC PerformanceObserver in staging, bucketing scavenge and mark-compact durations by kind, gets you a p99 to look at, and the mean will always look fine so do not look at that. When the p99 turns ugly the first question is what is surviving, not how much is being allocated: a per-request object that ends up in a module-level Map, a cache with no eviction, an array appended to and never truncated. Those are what turn free garbage into copied survivors, and the table at the top of this article is the price list. A WeakMap fixes the first of the three in one line, and keeping an object local enough for escape analysis to delete the allocation outright is worth 2.2 ns against 13.8.

Then there is what none of this reaches. Every number here comes off one machine, one Node version, and a loop that does nothing but allocate. Scavenge cost moves with the semi-space size V8 chooses at startup, which moves with the machine. The Twitch result is a Go program and I am relaying it, not reproducing it — Go has no generations, so almost none of the mechanism in this article applies to it at all. What survives all of that is the ratio between rows, which is why this article keeps pointing at ratios. The milliseconds are mine.