test: parallelize llgo test packages

[cpunion]

Summary

• implement Go-compatible package parallelism for llgo test through the -p build flag
• isolate package builds in child llgo processes so LLVM/compiler state is not shared concurrently
• keep -parallel as the test-binary concurrency flag and centralize -p parsing in internal/goflags
• keep tests with shared profile outputs, compile-only/output modes, file arguments, fuzzing, and embedded targets sequential
• buffer and flush each completed package output as one unit with explicit ok/FAIL package diagnostics
• use one macOS shard with three workers and two Ubuntu shards with four workers, reducing the test matrix from 10 jobs to 5 without dropping packages or std build-mode checks

Parallelism and resource bounds

The parent owns package fan-out. Each child receives -p=1, preventing N parent workers from each starting N go/packages build/load actions.

An initial macOS process-isolation prototype measured 114.77s sequential versus 63.44s at parent -p=2 on the same four packages with the LLGo cache disabled. That prototype still forwarded -p=2 to children. After review, children are deliberately limited to -p=1, so the PR CI run is the authoritative validation for final wall time and runner memory rather than reusing that 44.7% figure.

Against the exact merge-base run, the changed test matrix drops from 10 jobs to 5. Job runner time falls from 161m20s to 99m39s (-38.2%), and the run-llgo-test steps fall from 133m24s to 87m18s (-34.6%).

On macOS, two baseline shards used 30m08s of test time and 36m42s of job time. The final single shard uses 25m05s of test time and 27m55s of job time, reducing those totals by 16.8% and 23.9%. The tradeoff is critical-path latency: the longest baseline macOS test step was 18m35s, while the single shard takes 25m05s (+35.0%). Three workers are required to retain enough margin under the unchanged 30-minute job timeout; two workers timed out on a slower hosted runner.

Ubuntu keeps two shards because the Go 1.26 primary lane also runs serial c-shared/c-archive std build-mode checks. Those checks consumed about 25.5 runner-minutes across four baseline shards and are intentionally outside this change.

Validation

• go test -race ./cmd/internal/test ./internal/goflags
• GOTOOLCHAIN=go1.24.2 go test ./cmd/internal/test ./internal/goflags
• go test ./cmd/internal/...
• go test ./internal/build -run ^TestDefaultBuildTags$ -count=1
• go vet ./cmd/internal/test ./internal/goflags
• real llgo test -p=2 execution and Go-compatible -p=0 failure
• go list -- compatibility verified with Go 1.24.2 and Go 1.26.5
• workflow YAML parsing and shard simulation: macOS 192/192 packages; Ubuntu 96+96/192 packages

Coverage for internal/goflags is 98.9% and normalize.go is 100%; the package scheduler is covered at 92.0%, output reporting and child argument rewriting at 100%.

[fennoai]

Review: parallelize llgo test packages

The change is well-scoped and the parallel scheduler in runTestPackages is correct and leak-free: the results channel is buffered to parallelism, each worker sends exactly once, a result is drained before a replacement worker starts, and fail-fast drains all in-flight workers before returning. -p parsing, atomic-on-error config updates, and listTestPackages dedup are also sound, and test coverage for the new paths is good. Security is a non-issue: both subprocess spawns use the argv-slice form of os/exec, so there is no shell/command injection surface.

A few concrete items below are worth addressing before this lands. None are blocking correctness bugs; the top two affect output readability and resource usage under load.

Key points

• Interleaved child output (cmd/internal/test/test.go:93): parallel children write straight to the shared os.Stdout/os.Stderr, so output from different packages interleaves (possibly mid-line, since cross-process writes aren't line-atomic). cmd/go buffers each package's output and flushes it atomically. Consider per-package buffering, or streaming only when parallelism == 1.
• -p re-forwarded to each child (cmd/internal/test/test.go:88): flagArgs retains the user's -p=N and passes it to every child, where it flows into packages.Config.BuildFlags (internal/build/build.go:361-365) as the child's own build/load concurrency. With up to N children each running go/packages loading at -p=N, this can oversubscribe the host. Consider stripping -p (or passing -p=1) to children so the parent fan-out is the only package-level parallelism.

Diagnosability / fail-fast

• When fail-fast triggers, already-running children are not cancelled and skipped packages are not reported, and on a normal test failure (*exec.ExitError) the parent prints no per-package summary — identification relies entirely on the child's own FAIL <pkg> line surviving the interleave (compounds the first point). A brief end-of-run per-package result summary would help.

Minor

• listTestPackages appends user patterns after subcommand flags without a -- terminator (cmd/internal/test/test.go:162); a pattern beginning with - is interpreted by go list as a flag. Inserting a literal "--" before the patterns removes the ambiguity.
• The Config.BuildParallelism doc comment (internal/build/build.go:173) reads as if the build pipeline honors -p, but the field is only consumed by llgo test. Worth clarifying scope.
• effectiveParallelism is computed twice (in canRunPackagesInParallel and again in runCmd); minor duplication.
• CI shard-halving in .github/workflows/llgo.yml assumes near-linear -p speedup; worth validating peak RSS on the runners given the -p re-forwarding above.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!