Repository navigation
CI scheduling: per-shard RSS measurement plumbing (Node A) - #5792
Merged
Merged
Conversation
gunbai-bot Bot
pushed a commit
that referenced
this pull request
Jun 25, 2026
Design converged 2026-06-25 with sharp-stag-782: PerformanceReceipt.cost: CostAccount<Nano> is the single measured authority for Time+Space+Energy. The [calibration]/[measurement] text lines are Lossless transport projections of that receipt, not the authority themselves. wall_duration removed as stored field; becomes a pure projection fn. cache_state_summary stays top-level (it is a measurement-context tag, not a cost axis). nimble-tern-908 authors the shape PR; #5792 (gentle-newt-542) carries the Rust plumbing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
briansrls
pushed a commit
that referenced
this pull request
Jun 25, 2026
…5789) * v1 interp: memoize per-call parameter-name resolution (ctx-owned cache) call_function re-derived each parameter's authored name by re-slicing the source span (authored_name_at) on every call — invariant per fn_node but recomputed per call, so inside folds/loops it repeated per iteration. This was the dominant ExprCall self-time in the CI floor's eval phase. Memoize the derivation in a per-InterpContext cache keyed by fn_node pointer identity — sound because the ctx owns fn_nodes (pointers stable for the cache's lifetime, cache dies with the ctx), the same discipline as the existing data_cache. Behavior is identical (same node-eval counts, same witness results). Measured on ci_yaml_serializer_keystone (505k node-evals): ExprCall self-time 255ms->126ms (-51%), total witness eval 602ms->408ms (-32%). Full floor (975 witnesses) green, eval phase 1428s->981s. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v1 interp: extend symbol-reuse memoization to ExprVar and callee names Follow-on to the per-call param-name cache. The same chokepoint (authored_name_at re-slicing a name from its source span, then ctx.sym re-interning it) was hit on every ExprVar read and every call's callee-name resolution: - eval_var: cache the resolved Symbol per ExprVar node (var_sym_cache); eval then skips the slice + re-intern and goes straight to env.lookup(sym). Keyword/variant checks use ctx.sym_eq on the cached symbol; the name String is materialized lazily only on the registry slow path. - eval_call: cache the decoded callee name per call node (call_func_name_cache). - call_function: slice each param name once into ; reuses it instead of re-slicing (addresses review note on redundant per-param slicing). All caches are per-InterpContext fields keyed by node pointer identity — sound for the ctx lifetime, the same discipline as data_cache/param_name_cache. Behavior is identical: same node-eval counts, same witness results. Measured on ci_yaml_serializer_keystone (505,325 node-evals, vs the param-only base): ExprVar self-time 82ms->29ms (-64%), ExprCall 126ms->106ms; cumulative witness eval 602ms->322ms (-47%) across both commits. node-eval count unchanged (correctness). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: CI profiling * v1 resolve cache: always-on via temp_dir default (drop env-var gate) GUNBC_RESOLVED_GRAPH_CACHE_DIR still overrides the path; without it the cache defaults to $TMPDIR/gunbc-rg-cache. Eliminates the 447s resolve serial sum in the CI floor corpus run — type inference for each unique closure now runs once and is reused across all witnesses sharing that closure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WIP: CI profiling * Scope resource-aware scheduler (nodes A-D) + ROADMAP update Four-node plan for making spawn_width self-calibrating from per-shard measured RSS rather than a static data row: - Node A: measurement plumbing (claim_batch emit, claim_executor collect) - Node B: CostEstimate.space on Runnable in std substrate - Node C: width derivation moves into scheduler, side-channel deleted - Node D: calibration loop retires static data rows Also updates ROADMAP §1 scheduling band (new resource-aware bullet) and §2 P3/P4 (resolve-cache always-on via #5789, P4 space+time arms). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Restore foreign files reverted by stale branch base typescript_import_emit_by_execution_test.dag, typescript.dag, and target_model.dag exist on main but were absent/different on this session branch — restoring to match origin/main so the PR does not delete them on merge. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * resource-aware-scheduler plan: lock PerformanceReceipt as cost authority Design converged 2026-06-25 with sharp-stag-782: PerformanceReceipt.cost: CostAccount<Nano> is the single measured authority for Time+Space+Energy. The [calibration]/[measurement] text lines are Lossless transport projections of that receipt, not the authority themselves. wall_duration removed as stored field; becomes a pure projection fn. cache_state_summary stays top-level (it is a measurement-context tag, not a cost axis). nimble-tern-908 authors the shape PR; #5792 (gentle-newt-542) carries the Rust plumbing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * resource-aware-scheduler plan: add cost-authority re-role and P0-P6 roadmap Derivation is the authority (CostBasis.Predicted); measurement is the §5 falsifier + physical-constant calibration (CostBasis.Measured). P0/P1 decouple cleanly: P0 = PerformanceReceipt.cost shape (nimble-tern #5798), P1 = InputEnvelope (sharp-stag-782), both parallel. P2 symbolic- cost fold follows P1; P3 min_bytes dissolve after P0; P4 BestEffort width swaps static row for Predicted once P2 ships; P5 falsifier compares. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WIP: CI profiling * WIP: CI profiling --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
briansrls
pushed a commit
that referenced
this pull request
Jun 25, 2026
…tes) derives width from live cgroup budget (#5825) * v1 interp: memoize per-call parameter-name resolution (ctx-owned cache) call_function re-derived each parameter's authored name by re-slicing the source span (authored_name_at) on every call — invariant per fn_node but recomputed per call, so inside folds/loops it repeated per iteration. This was the dominant ExprCall self-time in the CI floor's eval phase. Memoize the derivation in a per-InterpContext cache keyed by fn_node pointer identity — sound because the ctx owns fn_nodes (pointers stable for the cache's lifetime, cache dies with the ctx), the same discipline as the existing data_cache. Behavior is identical (same node-eval counts, same witness results). Measured on ci_yaml_serializer_keystone (505k node-evals): ExprCall self-time 255ms->126ms (-51%), total witness eval 602ms->408ms (-32%). Full floor (975 witnesses) green, eval phase 1428s->981s. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v1 interp: extend symbol-reuse memoization to ExprVar and callee names Follow-on to the per-call param-name cache. The same chokepoint (authored_name_at re-slicing a name from its source span, then ctx.sym re-interning it) was hit on every ExprVar read and every call's callee-name resolution: - eval_var: cache the resolved Symbol per ExprVar node (var_sym_cache); eval then skips the slice + re-intern and goes straight to env.lookup(sym). Keyword/variant checks use ctx.sym_eq on the cached symbol; the name String is materialized lazily only on the registry slow path. - eval_call: cache the decoded callee name per call node (call_func_name_cache). - call_function: slice each param name once into ; reuses it instead of re-slicing (addresses review note on redundant per-param slicing). All caches are per-InterpContext fields keyed by node pointer identity — sound for the ctx lifetime, the same discipline as data_cache/param_name_cache. Behavior is identical: same node-eval counts, same witness results. Measured on ci_yaml_serializer_keystone (505,325 node-evals, vs the param-only base): ExprVar self-time 82ms->29ms (-64%), ExprCall 126ms->106ms; cumulative witness eval 602ms->322ms (-47%) across both commits. node-eval count unchanged (correctness). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP: CI profiling * v1 resolve cache: always-on via temp_dir default (drop env-var gate) GUNBC_RESOLVED_GRAPH_CACHE_DIR still overrides the path; without it the cache defaults to $TMPDIR/gunbc-rg-cache. Eliminates the 447s resolve serial sum in the CI floor corpus run — type inference for each unique closure now runs once and is reused across all witnesses sharing that closure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WIP: CI profiling * Scope resource-aware scheduler (nodes A-D) + ROADMAP update Four-node plan for making spawn_width self-calibrating from per-shard measured RSS rather than a static data row: - Node A: measurement plumbing (claim_batch emit, claim_executor collect) - Node B: CostEstimate.space on Runnable in std substrate - Node C: width derivation moves into scheduler, side-channel deleted - Node D: calibration loop retires static data rows Also updates ROADMAP §1 scheduling band (new resource-aware bullet) and §2 P3/P4 (resolve-cache always-on via #5789, P4 space+time arms). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Restore foreign files reverted by stale branch base typescript_import_emit_by_execution_test.dag, typescript.dag, and target_model.dag exist on main but were absent/different on this session branch — restoring to match origin/main so the PR does not delete them on merge. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * resource-aware-scheduler plan: lock PerformanceReceipt as cost authority Design converged 2026-06-25 with sharp-stag-782: PerformanceReceipt.cost: CostAccount<Nano> is the single measured authority for Time+Space+Energy. The [calibration]/[measurement] text lines are Lossless transport projections of that receipt, not the authority themselves. wall_duration removed as stored field; becomes a pure projection fn. cache_state_summary stays top-level (it is a measurement-context tag, not a cost axis). nimble-tern-908 authors the shape PR; #5792 (gentle-newt-542) carries the Rust plumbing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * resource-aware-scheduler plan: add cost-authority re-role and P0-P6 roadmap Derivation is the authority (CostBasis.Predicted); measurement is the §5 falsifier + physical-constant calibration (CostBasis.Measured). P0/P1 decouple cleanly: P0 = PerformanceReceipt.cost shape (nimble-tern #5798), P1 = InputEnvelope (sharp-stag-782), both parallel. P2 symbolic- cost fold follows P1; P3 min_bytes dissolve after P0; P4 BestEffort width swaps static row for Predicted once P2 ships; P5 falsifier compares. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WIP: CI profiling * WIP: CI profiling * WIP: CI profiling --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Node A of the resource-aware scheduler plan (
docs/plans/resource-aware-scheduler.md).Makes per-shard RSS observable as structured output:
claim_batch: emits[measurement] per-shard-peak-rss: N bytesunconditionally at process exit using VmHWM from/proc/self/status. Eachclaim_batchinvocation is one shard; its process-level peak is the per-shard peak.claim_executor: after the floor walk, emits[calibration] max-per-shard-peak-rss: N bytes at spawn_width=W(computed asceil(process_peak / width)). Makes the per-shard estimate observable as a structured line so future Node D work can updategunbc_ci_floor_measured_peakfrom real CI measurements rather than thewhole_run / concurrencyheuristic.No substrate changes. No
.dagchanges. The static data rowgunbc_ci_floor_measured_peakis unchanged — that is Node D work.Dissolution trigger (from plan doc): when
RunnablecarriesCostEstimate.space(Node B) and the scheduler derives width from it, this calibration plumbing and the static data row retire.🤖 Generated with Claude Code