runtime: add configurable pclntab packaging

[cpunion]

Summary

Add an LLGo-specific policy for packaging runtime function/line metadata:

• -pclntab=embedded keeps metadata in the executable and remains the default.
• -pclntab=external writes an optional <executable>.pclntab sidecar and loads it once, lazily, from allocation/I/O-safe symbolization paths.
• -pclntab=none does not generate the metadata and does not link the sidecar probe/loader path.

Proposal

#2113 has merged. GitHub reports this PR clean, and its effective diff now contains only the pclntab implementation.

Closes #2112.

The change also finishes the command/build boundary needed by that stack: Go-compatible build syntax is parsed in internal/goflags, flags.txt uses Go-native flags, and the legacy LLGO_DEBUG / LLGO_DEBUG_SYMBOLS switches are removed. Because the existing DWARF path still changes program correctness and LTO validity, an absent -w temporarily defaults to omitting DWARF; explicit -w=false enables the runnable debug path.

User-visible semantics

PCLN placement is independent of Go's linker flags. In particular, -s and -w never silently select -pclntab=none.

Go linker flags	DWARF in a native executable	Native symbol table in this phase	PCLN
default	omitted temporarily while DI correctness work is outstanding	unchanged	selected explicitly; embedded by default
-w	omitted	unchanged	selected explicitly
-s	omitted because -s implies -w	omission intent recorded, but native stripping is not implemented yet	selected explicitly
-s -w=false	emitted; explicit -w wins regardless of order	omission intent recorded, but native stripping is not implemented yet	selected explicitly

Explicit -w semantics remain Go-compatible: -s implies -w, and an explicit -w value wins regardless of order. The temporary absent-flag default is intentionally different from Go: LLGo currently behaves as if -w=true, whereas Go behaves as if -w=false. Thus the default, -w, and -s have the same sizes today; -w=false and -s -w=false form the DWARF-enabled pair. Actual native symbol-table deletion remains a later phase and must run after external PCLN analysis.

Initial external support is restricted to native Darwin/Linux amd64/arm64 executables without -target. Unsupported generation, archive/shared, named-target, and platform combinations fail with a capability diagnostic rather than silently changing behavior.

Runtime and artifact behavior

• The v2 sidecar binds to one exact executable identity, stores image-relative PCs, separates callable entry sites from closure ABI stubs, and resolves PC-line ownership against final linked symbols.
• The host writer validates ELF/Mach-O ranges, bounds every section, stages publication atomically, clears link-only data, and signs the final mutated Mach-O.
• A missing, stale, truncated, corrupt, wrong-target, or wrong-build sidecar is a nonfatal terminal result. Success and failure are both cached; the filesystem is not probed repeatedly.
• Fatal signal handling neither starts I/O nor waits for an in-progress load. Safe concurrent first callers share one load result.
• The loader uses one bounded read capped at 512 MiB in this phase; it does not yet use mmap.
• none follows a generation-time omission path. It does not generate metadata and does not contain the external probe/loader code.
• Pure lib/c programs retain the executable identity even when the LLGo runtime itself is not linked.

Go flag and debug migration

• Move Go build/gc/link flag parsing from cmd/internal/flags to shared internal/goflags; internal/build consumes typed configuration and no longer parses command strings.
• flags.txt accepts blank lines, comments, shell-style quoting/backslashes, multiple ordinary flags, -flag=value, -flag value, and single/double-dash spellings. Argument-list flags such as -ldflags=-s -w=false preserve the Go-style list. The removed private -dbg spelling reports a migration error.
• Only -s and -w linker semantics are translated in this phase. Other -ldflags tokens remain in the normalized raw list for future support; unpatterned and all= forms are interpreted, while other package patterns are rejected rather than applied globally.
• Native executable builds temporarily omit DWARF when -w is absent. -w=false explicitly enables the existing runnable debug path; ModeGen also emits DWARF only for explicit -w=false.
• Remove LLGO_DEBUG and LLGO_DEBUG_SYMBOLS from implementation, build fingerprints, LLDB scripts, and documentation. Use -w=false, -w, and -O0 instead; no replacement environment variable is introduced.

This reuses LLGo's existing runnable DWARF implementation. It currently disables LLVM/C-ABI optimization while emitting DWARF, and this PR deliberately does not attempt a broad DWARF repair. Broad CI exercises the DWARF-free default; the debug IR fixture and LLDB suite explicitly use -w=false. The underlying follow-ups are recorded separately:

• pclntab: preserve precise statement PCs with Darwin DWARF #2115 — Darwin DWARF/PCLN statement-PC interaction.
• DWARF: break recursive type cycles in DI generation #2116 — recursive DI type cycles.
• DWARF: handle frontend-specific types without compiler panic #2117 — frontend-specific DI types.
• DWARF: emit verifier-valid call locations under LTO #2118 — verifier-valid LTO call locations.
• build cache: cold and warm DWARF builds differ in artifact size #2119 — cold/warm DWARF artifact-size nondeterminism.
• DWARF: avoid aggregate-return segfault in cabisret #2121 — aggregate-return segfault in the DWARF-enabled codegen path.
• DWARF: preserve crypto/x509 OID.Equal semantics #2122 — crypto/x509 OID equality changes under the DWARF-enabled codegen path.

The earlier default-DWARF CI run https://github.com/xgo-dev/llgo/actions/runs/29549966202 made the scope concrete: normal Linux demos reached 101/103, full-LTO demos only 43/103, crypto/x509.TestOID changed behavior, and recursive DI types overflowed the compiler stack. The affected normal cases pass with -w. This PR therefore keeps DWARF explicitly available and continuously tested, but does not make the provisional path the broad default. No new default-control environment variable is added; restoring Go's absent-flag default later is one typed build-policy change.

Fixed size workloads

Add source-only workloads under benchmark/binary_size/ for:

• lib/c.Printf hello world;
• built-in println hello world;
• fmt.Printf hello world;
• a heavier text/template program;
• an import-free nil-pointer fault path.

The lib/c workload is tagged llgo so ordinary go install ./... does not try to link LLGo-only C intrinsics; LLGo supplies that build tag automatically.

Collection/history/CI tracking is intentionally deferred to a separate PR. Every variant below was fully warmed first and then built at the same exact output path. Executable and sidecar byte sizes were read from the filesystem and external totals were summed independently. Normal outputs were executed, and all nil-fault variants produced a nonzero diagnostic. External nil-fault variants were tested both with and without the sidecar.

On both platforms, the default, -w, and -s have matching sizes in every workload/mode combination. -w=false and -s -w=false also match each other. This reconciles all reported executable and sidecar sizes with the effective DWARF policy.

The smallest Darwin cprintf case also exposes initial fixed overhead clearly. Its external executable contains no LLGo runtime and has the same 100-byte __text as embedded, so the increase is not lazy-loader code. It comes primarily from making a 16 KiB __DATA page file-backed plus Mach-O re-signing/__LINKEDIT growth. Its roughly 41 KiB sidecar currently retains the complete roughly 684-entry prebuilt func/symbol index although only two final sites survive. Live-record pruning and compact Darwin re-signing are follow-up size optimizations; external mode's initial contract is optional deployment, not a guarantee that executable plus sidecar is smaller than embedded.

Darwin arm64 — default / -w / -s (bytes)

macOS 26.2, Go 1.26.5, LLVM/Clang/LLD 19.1.7.

workload	mode	executable	sidecar	total
cprintf	embedded	84,912	—	84,912
cprintf	external	68,512	41,256	109,768
cprintf	none	33,968	—	33,968
println	embedded	128,768	—	128,768
println	external	112,384	42,976	155,360
println	none	94,608	—	94,608
fmtprintf	embedded	2,298,592	—	2,298,592
fmtprintf	external	2,004,064	352,664	2,356,728
fmtprintf	none	1,897,008	—	1,897,008
texttemplate	embedded	3,145,296	—	3,145,296
texttemplate	external	2,785,008	458,520	3,243,528
texttemplate	none	2,616,432	—	2,616,432
nilfault	embedded	128,784	—	128,784
nilfault	external	112,384	42,976	155,360
nilfault	none	94,608	—	94,608

Darwin arm64 — -w=false / -s -w=false (bytes)

workload	mode	executable	sidecar	total
cprintf	embedded	34,480	—	34,480
cprintf	external	69,040	41,256	110,296
cprintf	none	34,480	—	34,480
println	embedded	123,456	—	123,456
println	external	141,056	43,728	184,784
println	none	123,456	—	123,456
fmtprintf	embedded	2,858,112	—	2,858,112
fmtprintf	external	2,697,344	359,784	3,057,128
fmtprintf	none	2,560,896	—	2,560,896
texttemplate	embedded	3,922,512	—	3,922,512
texttemplate	external	3,722,640	466,920	4,189,560
texttemplate	none	3,542,624	—	3,542,624
nilfault	embedded	123,456	—	123,456
nilfault	external	141,056	43,728	184,784
nilfault	none	123,456	—	123,456

Linux arm64 — default / -w / -s (bytes)

Ubuntu kernel 6.8, Go 1.26.5, LLVM/Clang/LLD 19.1.7.

workload	mode	executable	sidecar	total
cprintf	embedded	19,840	—	19,840
cprintf	external	9,040	40,856	49,896
cprintf	none	7,632	—	7,632
println	embedded	83,824	—	83,824
println	external	73,024	42,576	115,600
println	none	67,016	—	67,016
fmtprintf	embedded	2,456,184	—	2,456,184
fmtprintf	external	2,165,400	360,232	2,525,632
fmtprintf	none	1,968,056	—	1,968,056
texttemplate	embedded	3,307,688	—	3,307,688
texttemplate	external	2,936,112	465,736	3,401,848
texttemplate	none	2,666,376	—	2,666,376
nilfault	embedded	83,752	—	83,752
nilfault	external	73,016	42,584	115,600
nilfault	none	67,008	—	67,008

Linux arm64 — -w=false / -s -w=false (bytes)

workload	mode	executable	sidecar	total
cprintf	embedded	21,288	—	21,288
cprintf	external	10,552	40,856	51,408
cprintf	none	9,192	—	9,192
println	embedded	442,600	—	442,600
println	external	431,800	43,328	475,128
println	none	425,184	—	425,184
fmtprintf	embedded	5,365,288	—	5,365,288
fmtprintf	external	5,091,248	367,512	5,458,760
fmtprintf	none	4,867,808	—	4,867,808
texttemplate	embedded	6,956,984	—	6,956,984
texttemplate	external	6,602,104	474,312	7,076,416
texttemplate	none	6,304,672	—	6,304,672
nilfault	embedded	442,672	—	442,672
nilfault	external	431,872	43,336	475,208
nilfault	none	425,192	—	425,192

Local validation

Darwin arm64:

• LLDB suite with explicit -w=false: 194/194 PASS in 27.491s.
• Final default-policy caller/lineinfo/recursive integration group: PASS, 366.667s, with no test-specific -w override.
• Full-LTO regression with no test-specific DWARF override: PASS, 124.587s.
• Final post-review command/build packages: PASS; internal/build 219.234s, with command packages and internal/llgen also passing.
• Full internal/cabi suite after restricting the temporary default to linked modes: PASS, 102.472s, 96.4% coverage.
• Go 1.24.11 full go test -p 1 ./... -count=1: PASS on the preceding implementation commit; the final policy-only delta is covered by the focused and integration runs above.

Native Linux arm64:

• Final Go 1.26.5 default-policy tests plus internal/goflags, internal/pclnmap, and internal/pclnpost: PASS.
• Full internal/cabi suite: PASS, 40.085s, 82.7% coverage.
• Relevant command/build package run: all functional tests PASS. The root container can read a chmod-000 fixture, so one permission-assumption assertion fails as root; the exact test passes when rerun as nobody.
• PCLN native integration: PASS, package 93.870s / real 94.54s.
• go test -race ./internal/pclnmap ./internal/pclnpost: PASS.

Local Linux/amd64 under QEMU was not a reliable test environment: the Go compiler/cgo/vet/assembler processes themselves hit nondeterministic SIGSEGVs. Native amd64 is therefore left to GitHub CI.

Changed-package coverage:

• internal/build: 72.8%; every function in link_options.go is 100% covered, and the new PCLN build/post-link functions range from 72.2% to 100%.
• internal/goflags: 99.2%
• cmd/internal/flags: 54.7%
• internal/llgen: 23.2%
• internal/pclnmap: 98.8%
• internal/pclnpost: 89.6%
• internal/cabi: 96.4% on Darwin and 82.7% on Linux; this caught and now covers the ModeGen/linked-default boundary.

The final review removed the now-redundant golden/LTO/caller test-specific -w overrides and their helper test. It also removed a proposed general -size text/JSON schema expansion (the fixed workloads and measured PR data are sufficient here), deleted unreachable duplicate flag-lexer checks and redundant post-load section validation, and added malformed Mach-O chained-fixup bounds tests. Broad tests now exercise the real default, while LLDB remains the explicit-DWARF acceptance suite. go test -race passes for the map/postprocessor packages, go vet passes for all changed Go packages, and git diff --check is clean.

GitHub CI coverage and timing

On final head b6d8aa872, Codecov patch coverage is 90.50715%, above the 88.92% target, and the codecov/patch check passes. The patch report contains 696 hits, 38 misses, and 35 partial lines; the bot combines misses and partials into its "73 lines" review message. The remaining gaps are concentrated in filesystem/signing failure returns, malformed cross-format file reads, and overflow guards; no I/O mock layer was added solely to inflate coverage.

The default-DWARF-off Dev LTO job passes 103/103 demos in 18m04s. The earlier default-DWARF experiment passed only 43/103, so the temporary default restores the full-LTO acceptance rate while explicit -w=false remains covered by LLDB and focused tests.

Execution time versus the #2113 baseline (runner queue time excluded):

job	#2113	this PR	change
Ubuntu Go coverage	15m51s	17m28s	+10.2%
Ubuntu LLGo Go 1.24 main	11m24s	10m13s	-10.4%
Ubuntu LLGo Go 1.26 main	11m15s	11m14s	-0.1%
Ubuntu Go 1.24 four shards, summed	47m13s	47m42s	+1.0%
Ubuntu Go 1.26 four shards, summed	46m17s	45m30s	-1.7%
macOS Go coverage	24m31s	16m56s	-30.9%
macOS LLGo Go 1.24 main	14m33s	15m08s	+4.0%
macOS LLGo Go 1.26 main	15m56s	15m21s	-3.7%
macOS Go 1.24 two shards, summed	38m52s	44m42s	+15.0%
macOS Go 1.26 two shards, summed	49m06s	39m02s	-20.5%

All 47 non-skipped checks passed with no failures; the release publishing job was expectedly skipped for the PR. This includes all Ubuntu and macOS Go 1.24/1.26 shards, main, Targets, build-cache, docs, release-artifact, stdlib-coverage, Dev LTO, format, and Codecov checks. Runner queue delay is excluded from the table.

[codecov]

Codecov Report

❌ Patch coverage is 90.50715% with 73 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/pclnpost/external.go	77.12%	22 Missing and 21 partials ⚠️
internal/pclnpost/binary.go	72.50%	11 Missing and 11 partials ⚠️
internal/goflags/ldflags.go	95.65%	3 Missing and 1 partial ⚠️
internal/pclnmap/pclnmap.go	98.03%	2 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

This PR adds a configurable LLGo policy for packaging runtime function/line metadata (pclntab) with embedded (default), external sidecar loading, and none omission, and completes the command/build boundary for Go-compatible -gcflags/-ldflags parsing (including typed -s/-w handling and removal of legacy LLGO_DEBUG* switches). It also introduces a versioned external sidecar format plus post-link analysis/detachment tooling, updates runtime symbolization entry points to lazily install external metadata on safe paths, and adds fixed binary-size workloads and updated documentation.

Changes:

• Add -pclntab={embedded,external,none} support end-to-end (build config, post-link externalization, runtime lazy loader/probe, and sidecar format).
• Move Go build/link flag parsing into internal/goflags and wire all relevant commands to apply typed linker/debug intent (-s/-w), removing LLGO_DEBUG / LLGO_DEBUG_SYMBOLS.
• Add tests/fixtures/docs plus binary-size benchmark workloads to validate behavior and track size impact.

Reviewed changes

Copilot reviewed 73 out of 73 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
runtime/internal/runtime/z_rt.go	Invoke a runtime hook on recovered panics to clear auxiliary traceback state.
runtime/internal/lib/runtime/unwind_llgo.go	Install panic hooks and ensure external pclntab is loaded only on I/O-safe panic paths.
runtime/internal/lib/runtime/pprof_runtime_stub_llgo.go	Ensure external metadata is installed before consulting FuncForPC caches.
runtime/internal/lib/runtime/pclntab_static.go	Stub ensureRuntimePCLN/runtimePCLNReady for non-external builds.
runtime/internal/lib/runtime/pclntab_external_linux.go	Linux executable-path discovery for sidecar loading.
runtime/internal/lib/runtime/pclntab_external_darwin.go	Darwin executable-path discovery for sidecar loading.
runtime/internal/lib/runtime/pclntab_external_arm64.go	External sidecar GOARCH encoding for arm64.
runtime/internal/lib/runtime/pclntab_external_amd64.go	External sidecar GOARCH encoding for amd64.
runtime/internal/lib/runtime/fault_unwind_llgo.go	Track/clear “fault-active” state to preserve async-signal no-I/O behavior through fatal panics.
runtime/internal/lib/runtime/extern.go	Ensure metadata is installed before Caller/Callers fast paths; adjust Caller PC semantics.
internal/pclnpost/macho_fixture_test.go	Extend Mach-O fixtures to cover external pclntab sections and PAGEZERO base-discovery behavior.
internal/pclnpost/external.go	Add post-link external analysis, binary detachment, section clearing, and (Darwin) re-signing flow.
internal/pclnpost/elf_fixture_test.go	Extend ELF fixtures to include optional pcline + identity sections for externalization tests.
internal/pclnpost/binary.go	Strengthen binary loading/bounds checks; track pointer size/endianness and new sections.
internal/pclnpost/binary_test.go	Add tests for bounds helpers and Mach-O chained-fixups parsing robustness.
internal/pclnmap/pclnmap.go	Define and implement the versioned external sidecar encoding + validation rules.
internal/pclnmap/pclnmap_test.go	Add encode/parse determinism + corruption/overflow/misalignment tests for sidecar format.
internal/llgen/llgenf.go	Switch golden-test flags parsing to Go-compatible flags.txt via internal/goflags.
internal/llgen/llgenf_test.go	Verify flags.txt parsing/application and error-path diagnostics include the file path.
internal/goflags/normalize.go	Normalize Go build flags into canonical -flag=value form, rejecting removed -dbg.
internal/goflags/ldflags.go	Parse Go -ldflags argument-list syntax into typed build.LinkOptions (+ ignored tokens).
internal/goflags/ldflags_test.go	Validate precedence/order/quoting/errors for typed -s/-w parsing.
internal/goflags/gobuild.go	Apply normalized Go build flags atomically into build.Config (raw + typed semantics).
internal/goflags/gobuild_test.go	Test normalization + atomicity guarantees for ApplyBuildFlags.
internal/goflags/gcflags.go	Map supported -gcflags subset into typed frontend config (Go version / O0).
internal/goflags/flagfile.go	Parse flags.txt with comments/quoting/backslashes while preserving argument-list flags.
internal/goflags/flagfile_test.go	Test flags.txt parsing formats, comment rules, escapes, and error cases.
internal/crosscompile/crosscompile.go	Introduce typed linker debug-info capability policy (DebugInfoPolicy).
internal/crosscompile/crosscompile_test.go	Verify native and fixed-target debug-info policies are exported correctly.
internal/build/testdata/ldflagsstrip/main.go	New runtime symbolization fixture for -s/-w integration coverage.
internal/build/pclntab_external.go	Implement external sidecar writing + detachment in the build finalization phase.
internal/build/pcln_mode.go	Add PCLNMode typed config, legacy-env normalization, and external capability validation.
internal/build/pcln_mode_test.go	Test mode validity, legacy precedence, site enabling rules, and sidecar cleanup.
internal/build/outputs.go	Add sidecar output path (.pclntab) and export it via environment maps.
internal/build/outputs_test.go	Validate sidecar path derivation and env-map wiring.
internal/build/link_options.go	Add typed DWARF intent, safe default policy, and linker-arg selection for omission.
internal/build/link_options_test.go	Test DWARF omission/preserve behavior and native-input DWARF suppression integration (ELF).
internal/build/funcinfo_table.go	Emit external-mode identity slot + keep payload out of IR; prepare build-side sidecar data.
internal/build/funcinfo_table_test.go	Ensure external mode avoids embedding funcinfo payloads in LLVM IR and keeps identity/live markers.
internal/build/fingerprint.go	Include EmitDWARF and PCLN mode in build fingerprints/manifests.
internal/build/collect.go	Record effective DWARF emission and effective PCLN mode into fingerprints/manifests.
internal/build/collect_test.go	Test fingerprint coverage for DWARF and PCLN policy, plus env canonicalization rules.
internal/build/clean.go	Remove .pclntab sidecars alongside executables on clean.
internal/build/clean_test.go	Verify cleaning removes executable + sidecar artifacts but not unrelated files.
internal/build/build.go	Wire typed flags/policies through build orchestration, tags, linking, and PCLN finalization.
internal/build/build_test.go	Add integration tests for typed LinkOptions effects on symbolization and debug artifacts.
doc/design/pclntab-packaging.md	Document the pclntab packaging modes, boundaries, and ordering/identity semantics.
cmd/internal/test/test.go	Capture and apply Go build flags via internal/goflags for llgo test.
cmd/internal/test/test_test.go	Assert build-flag wiring is correctly bound for the test command.
cmd/internal/run/run.go	Capture/apply Go build flags independently for run and cmptest.
cmd/internal/run/run_test.go	Verify run and cmptest have independent build-flag collectors and register -ldflags.
cmd/internal/install/install.go	Capture and apply Go build flags for llgo install.
cmd/internal/install/install_test.go	Assert build-flag wiring is correctly bound for the install command.
cmd/internal/flags/pclntab.go	Add the -pclntab CLI flag and map it to typed build.PCLNMode.
cmd/internal/flags/gobuild.go	Implement Go build-flag capture + typed application (ApplyGoBuildFlags).
cmd/internal/flags/flags.go	Register -pclntab and plumb it into build.Config update flow.
cmd/internal/flags/flags_test.go	Test Go build-flag application atomicity, -pclntab parsing, and legacy precedence.
cmd/internal/build/build.go	Switch llgo build to typed Go build-flag parsing/application.
cmd/internal/build/build_test.go	Update build command tests to include -ldflags capture behavior.
cmd/internal/base/pass.go	Remove legacy PassBuildFlags helper (replaced by flags.CaptureGoBuildFlags).
cl/_testdata/debug/flags.txt	Migrate debug fixture to Go-compatible -ldflags=-w=false spelling.
benchmark/binary_size/README.md	Add fixed workloads and measurement guidance for size comparisons.
benchmark/binary_size/cprintf/main.go	Add smallest lib/c.Printf workload (LLGo-only tag).
benchmark/binary_size/println/main.go	Add built-in println workload.
benchmark/binary_size/fmtprintf/main.go	Add fmt.Printf workload.
benchmark/binary_size/texttemplate/main.go	Add heavier text/template workload.
benchmark/binary_size/nilfault/main.go	Add import-free nil-deref failure workload for runtime diagnostics.
_lldb/README.md	Update LLDB instructions to use -ldflags=-w=false and explain the temporary DWARF default.
_lldb/common.sh	Update LLDB test harness to build with -ldflags=-w=false instead of env vars.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.