Skip to content

feat(jsc): add byte-based heap allocation sampling - #2

Merged
steipete merged 1 commit into
openclaw/mainfrom
claude/w144-allocation-sampling
Oct 4, 2026
Merged

steipete merged 1 commit into
openclaw/mainfrom
claude/w144-allocation-sampling

Conversation

@steipete

@steipete steipete commented Oct 4, 2026

Copy link
Copy Markdown

OpenClaw's catalog performance consumers require byte-based allocation profiles that survive garbage collection. JSC's existing snapshots and CPU profiler cannot recover the allocation sites of objects already collected.

Add a VM-owned allocation sampler to HeapProfiler. It limits the existing free-list range to route selected C++ and generated allocations through the slow path, preserving the disabled fast-path instructions. Exponential byte intervals and inverse inclusion-probability weights produce allocation-site trees and samples. Precise allocations, auxiliary growth and relocation, and deduplicated JS string backing allocations participate; sampled addresses are reconciled before sweeping without retaining objects. Stack capture is limited to 128 frames. Collection inclusion follows the actual major/minor collection type.

The sampling contract follows Node/V8's implementation. Measurements cover JSC cells, newly reported string backing storage, and GC-managed auxiliary storage. Other native allocations and external buffers are outside this profile. This changes engine layouts and requires a joint Bun rebuild and the matching in-process inspector adapter.

Validation on one direct AWS Linux c7a.24xlarge using the unchanged WebKit lane recipe:

  • JSC regression passes with default tiers, interpreter, baseline JIT, and eager FTL. It covers warmed allocation paths, all major/minor inclusion combinations, collection/reuse, live profiles, sample IDs/ordinals, weighted totals, growth, an 8 MB auxiliary allocation, and short/long string payloads (including reports below the GC accounting threshold). The old engine fails because the sampler is absent.
  • Node 24.21.0 and matching Bun pass the shared callback/promise protocol and independent-worker fixtures. The surrounding inspector suite passes 78 tests.
  • The fork's 44-file Linux selection and two dependency checks pass on both baseline and candidate: 46/46, zero regressions.
  • Complete scoped P2 review is clean.

ABBA measurements use eight samples per arm, pinned to one CPU, at the default 32 KiB interval. Ratios compare allocation-loop medians; setup, final profile retrieval/GC, and serialization are outside the timed loops.

Hot path Candidate/baseline, sampling disabled Enabled/disabled on candidate
Objects 1.002× 1.279×
Arrays 0.998× 1.345×
JSON parsing 1.004× 1.013×
Escaped-string parsing 1.0004× 1.007×

Both unmodified OpenClaw catalog consumers pass on snapshot e9edda349faee43c13244cf3f4547fcbcb9c37fe; baseline Bun fails both at missing HeapProfiler.startSampling, and Node 24 passes both. The gateway observes about 1.17 MB/list and 524 CPU samples with both profilers active. The native ordinary/catalog walks observe about 403 MB/5.82 MB, each over 3,000 rows and 47 pages. Node's corresponding allocation estimates are 209 MB/10.20 MB; runtime allocation strategies and layouts differ, so these values are not expected to match numerically.

The existing read-only engine PR workflow now runs the sampler regression across those tiers using the same Linux artifact as namespace qualification, before its existing paired Bun/fork gate. Artifact publication remains the separate coordinator-controlled workflow; this PR does not dispatch it. The matching Bun change stays draft until the batch-2 engine is published and pinned.

Record real GC-cell, auxiliary, and reported JS string storage allocations with allocating stacks and inverse-probability byte weights. Use existing free-list boundaries and a variable-size byte budget, reconcile weak owners before sweeping, and support independent major/minor collection inclusion. Add interpreter/JIT/large/string regressions and reuse the existing read-only engine CI artifact. The distribution and protocol semantics follow Node/V8; a matching Bun adapter and joint rebuild are required.
@steipete
steipete merged commit 594794f into openclaw/main Oct 4, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant