Skip to content

§5 CI-tooling gap: runner-context env scope — discriminating witnesses on generator output - #5485

Merged
briansrls merged 7 commits into
mainfrom
session/warm-crab-895
Jun 21, 2026
Merged

briansrls merged 7 commits into
mainfrom
session/warm-crab-895

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • yaml_check.rs is a YAML-syntax-only fallback (parse only, no semantic checks — this is correct per §5/§3)
  • The §5 gap was real: ${{ env.RUNNER_TEMP }} passes YAML parse but is semantically wrong in GitHub Actions (RUNNER_TEMP is not declared in any env: block; ${{ runner.temp }} is the correct runner-context path)
  • yaml_check.rs hardening is superseded: the generated ci.yml is correct-by-construction (ci_workflow.dag uses runner.temp throughout); a hand-edited bad ci.yml fails the drift gate first (ci_yml_drifted)
  • Adds two discriminating witnesses + two red controls in dsl/test/claim/ci_yaml_serializer_witness_test.dag, operating on expected_ci_yml() (the live generator output), auto-enrolled by the CI floor discovery scan

Changes

dsl/test/claim/ci_yaml_serializer_witness_test.dag — four additions:

  • runner_env_scope_ok(yaml) / no_runner_env_context_vars(yaml) — extracted predicate helpers
  • witness_cache_uses_runner_temp_not_env_cargo_home() — asserts generator emits runner.temp path, not env.CARGO_HOME
  • witness_cache_runner_env_scope_red_control() — feeds a bad yaml string (env.CARGO_HOME), asserts guard fires → proves witness 1 is non-vacuous
  • witness_no_runner_env_vars_accessed_via_env_context() — asserts generator never emits env.RUNNER_TEMP/OS/ARCH
  • witness_no_runner_env_context_red_control() — appends ${{ env.RUNNER_TEMP }} to good yaml, asserts guard fires → proves witness 2 is non-vacuous
  • All four wired into ci_yaml_serializer_keystone_holds

yaml_check.rs is unchanged (pure YAML syntax, §3 single authority, SCAFFOLD marker intact).

Test plan

🤖 Generated with Claude Code

briansrls and others added 2 commits June 21, 2026 21:20
The binary previously did YAML-syntax-only parsing and returned exit 0
for ci.yml files that referenced `${{ env.VARNAME }}` where the var was
never declared in any static env: block (e.g. `${{ env.RUNNER_TEMP }}`
instead of `${{ runner.temp }}`), or used invalid runner context
properties (e.g. `${{ runner.notaprop }}`).

Now:
- Collects declared env vars from workflow/job/step env: blocks and
  from `echo "VAR=..." >> "$GITHUB_ENV"` writes in run: scripts.
- Rejects any `${{ env.VARNAME }}` not covered by the above.
- Rejects any `${{ runner.PROP }}` not in the documented property set.

The current .github/workflows/ci.yml passes (runner.temp, runner.os,
runner.arch all valid; env refs are CARGO_TERM_COLOR/RUSTFLAGS from
static workflow env: block).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review June 21, 2026 21:23
briansrls and others added 3 commits June 21, 2026 21:31
…tion

Review feedback (claude-opus-4-7): unquoted `echo VAR=value >> "$GITHUB_ENV"`
was not matched, causing false-positive env-not-declared errors for any
future step using that style. Now strips the optional leading quote before
extracting the variable name, covering all three common forms:
  echo "VAR=..." >> "$GITHUB_ENV"
  echo 'VAR=...' >> "$GITHUB_ENV"
  echo VAR=...   >> "$GITHUB_ENV"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…5 construction in .dag)

Addresses review REQUEST_CHANGES: the env-scope constraint belongs at the
generator/model level (.dag world), not in the yaml_check fallback Rust
bin. Reverts yaml_check to pure YAML syntax parse (original role).

Adds two discriminating witnesses to the existing keystone test in
ci_yaml_serializer_witness_test.dag, operating on expected_ci_yml() —
the same authority the generator produces:

- witness_cache_uses_runner_temp_not_env_cargo_home: asserts the cache
  path uses ${{ runner.temp }} (runner context) and not ${{ env.CARGO_HOME }}
  (env context, set at runtime via $GITHUB_ENV). Goes RED if the generator
  switches to env context, catching the false-green class at the source.

- witness_no_runner_env_vars_accessed_via_env_context: asserts that
  runner-provided env vars (RUNNER_TEMP, RUNNER_OS, RUNNER_ARCH) are never
  accessed via the env context (${{ env.RUNNER_* }}). Goes RED if the
  generator emits these incorrect forms.

Both witnesses use string_contains on the generated output — same mechanism
as the existing cache-path and sccache witnesses — so they are automatically
enrolled in the CI floor via the *_test.dag / test fn discovery.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the REQUEST_CHANGES by reversing the approach entirely — the reviewer was right on all three counts.

What changed:

  1. yaml_check reverted to pure YAML syntax — the 150-LOC Rust env-scope checker is gone. yaml_check is the fallback parser; adding a second representation of the env-scope constraint in Rust (when the generator/model already owns that knowledge) was a §3/§5 parallel-representation violation, not a §5 fix.

  2. Constraint moved to the .dag world — two discriminating witnesses added to ci_yaml_serializer_witness_test.dag, operating on expected_ci_yml() (the generator's output):

    • witness_cache_uses_runner_temp_not_env_cargo_home — asserts the cache path uses ${{ runner.temp }} (runner context), not ${{ env.CARGO_HOME }} (env context). Goes RED if the generator emits the wrong form.
    • witness_no_runner_env_vars_accessed_via_env_context — asserts ${{ env.RUNNER_TEMP/OS/ARCH }} never appear. Goes RED on the exact false-green class the original task targeted.

Both are enrolled automatically by *_test.dag / test fn discovery (no hand-registration). The heuristic-shell-parsing problem (find #2) disappears because the witnesses operate on the modeled output, not a Rust scan of bash scripts.

— sent from warm-crab-895

@gunbai-bot gunbai-bot Bot changed the title yaml_check false-greens invalid ci.yml (misses runner-context env scope) — §5 CI-tooling gap §5 CI-tooling gap: runner-context env scope — discriminating witnesses on generator output Jun 21, 2026
@briansrls
briansrls merged commit 0ccb2e9 into main Jun 21, 2026
1 check passed
@briansrls
briansrls deleted the session/warm-crab-895 branch June 21, 2026 23:49
gunbai-bot Bot pushed a commit that referenced this pull request Jun 21, 2026
…onical per quick-ant directive)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jun 22, 2026
…ner-env-var witness as a model walk - first instance of the medium-as-Node guard (#5499)

* WIP: Model GitHub Actions expression contexts as Nodes and rewrite the runner

* WIP: Model GitHub Actions expression contexts as Nodes and rewrite the runner

* Resolve merge: drop #5485 string_contains guard, keep model-walk (canonical per quick-ant directive)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: re-trigger on merged head (workflow did not fire on the origin/main merge commit)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Brian Searls <briansearls1@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant