runtime,cl: add Go-compatible sampled memory profiling - #2027
Open
cpunion wants to merge 8 commits into
Open
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
cpunion
force-pushed
the
codex/stage5-memprofile
branch
4 times, most recently
from
July 4, 2026 12:52
c647b14 to
2b22d07
Compare
cpunion
force-pushed
the
codex/stage5-memprofile
branch
2 times, most recently
from
July 8, 2026 07:02
4695ad3 to
9d9f6b0
Compare
cpunion
force-pushed
the
codex/stage5-memprofile
branch
3 times, most recently
from
July 9, 2026 05:52
e6bbe61 to
2e36dbf
Compare
8 tasks
cpunion
added
go-test-compat
39 tasks
cpunion
force-pushed
the
codex/stage5-memprofile
branch
from
July 18, 2026 23:37
2e36dbf to
ef62b2d
Compare
cpunion
changed the title
cpunion
force-pushed
the
codex/stage5-memprofile
branch
from
July 29, 2026 01:50
3eff4be to
a00bc0c
Compare
cpunion
force-pushed
the
codex/stage5-memprofile
branch
from
August 1, 2026 15:06
7b22159 to
0604edf
Compare
Collaborator
Author
|
Rebased the independent memprofile root onto current Conflict handling kept current-main panic/recover snapshot hooks in Fresh validation:
|
LLGo baseline benchmarks
Program measurements
Core language and compiler benchmarks
Compared only with the latest matching platform in the main series. |
cpunion
force-pushed
the
codex/stage5-memprofile
branch
3 times, most recently
from
August 3, 2026 22:54
857d2a8 to
1b77778
Compare
9 tasks
Replaces the size-class counters with gc-shaped heap profiling: sampled allocations are attributed to physical call stacks at exact statement lines, and records hold RAW sampled counts — consumers (pprof, goroot heapsampling.go) apply the Poisson correction themselves, exactly as with gc. - Sampling mirrors gc's mcache.nextSample: bytes count down to an exponentially distributed threshold (mean MemProfileRate), sample once on crossing, redraw. The memoryless distribution is load-bearing: with any bounded-support threshold a near-periodic allocation pattern phase-locks the sample points onto the large sites (observed 1.6x per-site skew on heapsampling's interleaved sizes). ln() is a small local approximation — the runtime core cannot import math. - Stacks come from the FP walk at sample time (fpCallers via a hook the public runtime registers), bucketed by stack hash; allocator plumbing (including __llgo_stub. wrapper frames of the hook) is trimmed at read time. A reentrancy flag spans the whole decision path: threshold drawing and bucket allocation themselves allocate, and a recursive sample overflows the stack. - Heap allocations get statement anchors in tracked functions, and a package that reads the memory profile (runtime.MemProfile / MemProfileRate under either the "runtime" or the patched lib-runtime spelling) pins all its trackable functions: per-site attribution loses sites to inlining otherwise. Profiling packages are rare and accuracy beats inlining there; gc gets both via its inline tree (P4). goroot heapsampling.go passes on darwin/arm64 and linux/arm64 (the latter was a pre-existing platform gap). Depends on --icf=none from the line-directive PR: heapsampling's three identical wrapper functions must keep distinct pcs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An acceptance regression asserts exact per-line attribution at rate=1 (raw counts are exact there), and a cl unit test covers the memprofile-package pinning criterion under both runtime spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… capture path The frame-table init allocates; when one of those allocations crossed the sampling threshold, captureMemProfileStack -> fpCallers re-entered initRuntimeFuncPCFramesSlow on the thread that already held the Busy latch and usleep-spun forever. First testing.callerName call of a test binary triggers the init, so whole test binaries hung at startup — which packages hit it depends on the deterministic threshold sequence meeting the binary's pre-init allocation volume: net/rpc and net/rpc/jsonrpc under go1.24 stdlib, net/http/expvar/cookiejar under go1.26 (CI shard timeouts on ubuntu, both attempts). Entering an Uninit latch from the capture path is safe (the whole sample runs under memProfileInSample, so init's own allocations cannot re-sample); only Busy must not be waited on. Drop that one sample.
cpunion
force-pushed
the
codex/stage5-memprofile
branch
from
August 10, 2026 02:27
1b77778 to
cc755a4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Go-compatible, stack-keyed sampled heap profiles for LLGo.
Problem
LLGo's
runtime.MemProfileonly kept size-class counters, so heap profiles had no allocation stacks or source-line attribution. Lazy frame-metadata initialization from the first sampledAllocZ/AllocUcould also re-enter allocation and corrupt reflected closure calls.Changes
runtime.MemProfileRate, keyed by physical call stackheapsampling.goexpectations covered by this implementationThe small
hasPrefixrelocation only shares an existing runtime helper across native runtime files; this PR does not change wasm behavior or CI configuration.Validation on current main
packageReadsMemProfilecoverage is 95.7%go test ./internal/runtime ./internal/lib/runtimepasses in the runtime module./test/go/memprofilepassestest/heapsampling.gopasses on darwin/arm64Uses the frame-pointer/funcinfo infrastructure already merged in #2024.