ci(test): enforce hermetic deterministic Reborn suite - #6883
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
🚅 Deployed to the ironclaw-pr-6883 environment in ironclaw-ci-preview
|
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds hermetic process isolation, native network enforcement, deterministic Reborn suite orchestration, CI integration, E2E environment forwarding, centralized package discovery, and deterministic loopback-based network-failure tests. ChangesHermetic deterministic Reborn testing
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related issues
Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant CI
participant SuiteRunner as run-hermetic-deterministic-suite.sh
participant ProcessRunner as run-hermetic-test-process.sh
participant TestCommand
participant NetworkGuard as hermetic-network-guard.c
CI->>SuiteRunner: invoke deterministic stage
SuiteRunner->>ProcessRunner: run stage command
ProcessRunner->>TestCommand: provide isolated environment
TestCommand->>NetworkGuard: attempt outbound network operation
NetworkGuard-->>TestCommand: allow loopback or reject non-loopback
TestCommand-->>CI: return test result
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
scope: ci
🔎 Review · PR #6883
2 actionable findings →The new hermetic suite has two blocking gaps in its core guarantees: non-loopback enforcement is skipped for many supported command launchers, and the advertised Rust clock/random controls have no consumers. Other changed workflow, documentation, environment-forwarding, and test areas were reviewed without additional material findings. Automatic · PR opened + CI failed · attempt 1 of 3 · completed in 2m 5s Run details
|
![ironloopai[bot]](https://avatars.githubusercontent.com/in/4116490?s=60&v=4){kind=small}
There was a problem hiding this comment.
🔍 Review complete · PR #6883
The new hermetic suite has two blocking gaps in its core guarantees: non-loopback enforcement is skipped for many supported command launchers, and the advertised Rust clock/random controls have no consumers. Other changed workflow, documentation, environment-forwarding, and test areas were reviewed without additional material findings.
Findings
- 🔴 High · Network guard is bypassed for unrecognized command launchers —
scripts/ci/run-hermetic-test-process.sh:140-148
Details are attached to the relevant diff. - 🟠 Medium · Injected Rust clock and random seed variables are unused —
scripts/ci/run-hermetic-test-process.sh:118-124
Details are attached to the relevant diff.
Validation and technical details
- Reviewed the complete refs/ironloop/base (bed3f68) to refs/ironloop/head (ffd67b7) comparison across all 15 changed files.
- Ran
git diff --check; it passed. - Ran Bash syntax checks for
scripts/ci/*.sh; they passed. - Compiled the four changed/added Python modules with
python3 -m py_compile; they passed. - Repository-wide search confirmed
IRONCLAW_TEST_RANDOM_SEEDandIRONCLAW_TEST_CLOCKoccur only in the new runner and its self-test. - Attempted
scripts/ci/test-hermetic-test-process.sh; this sandbox lacks a mounted/proc/self/fd, so Bash process substitution failed at runner line 69 before the behavioral probes could execute. - Base:
main - Head:
codex/ws1-hermetic-suiteatffd67b7 - Run:
21b7a2e7-77a9-4490-b474-49173c96fb9e
e91f5b4 to
1614044
Compare
b301d15 to
35a7982
Compare
6674927 to
bb0e867
Compare
bb0e867 to
38ab508
Compare
38ab508 to
d3251d3
Compare
d3251d3 to
608f8cf
Compare
546eb1b to
62b6375
Compare
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
🔎 Review · PR #6883
1 actionable findings →Reviewed the complete trusted base-to-head comparison across all 20 changed files. The hermetic suite has one credential-isolation gap: agent sockets remain available inside guarded test processes. No other material correctness, CI-wiring, or test-fixture defects were identified. Automatic · PR opened · attempt 1 of 3 · completed in 1m 56s Run details
|
There was a problem hiding this comment.
🔍 Review complete · PR #6883
💬 1 finding
Reviewed the complete trusted base-to-head comparison across all 20 changed files. The hermetic suite has one credential-isolation gap: agent sockets remain available inside guarded test processes. No other material correctness, CI-wiring, or test-fixture defects were identified.
Findings
- 🟠 Medium · Hermetic tests retain access to ambient credential-agent sockets —
scripts/ci/run-hermetic-test-process.sh:52-68
Details are attached to the relevant diff.
Validation and technical details
- Inspected the complete refs/ironloop/base..refs/ironloop/head diff and surrounding workflow, runner, network-interposer, Rust-test, and Python E2E code.
- Shell syntax checks passed for all changed CI scripts.
- Python compilation passed for all changed E2E modules.
- git diff --check passed for the trusted comparison.
- The hermetic self-test could not execute in this review environment because
rustcis unavailable; it exited at guard setup before running assertions. - Base:
main - Head:
codex/ws1-hermetic-suiteat62b6375 - Run:
3e7e864a-09fd-444f-a2e9-c783695dafe6
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/reborn-tests.yml (1)
286-293: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winClose the unguarded E2E test bypass.
webui-v2-smokestill runspytestdirectly at lines 254-260. Only the later E2E stages are wrapped here, leaving a merge-gating test outside the scrubbed environment and network boundary. Route that validation invocation throughrun-hermetic-deterministic-suite.sh commandtoo. As per coding guidelines, guardrails must enforce their stated guarantees; the suite documentation says every test stage runs through the hermetic process boundary.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/reborn-tests.yml around lines 286 - 293, Update the webui-v2-smoke validation step to invoke pytest through run-hermetic-deterministic-suite.sh command, matching the guarded E2E stages. Ensure every test stage uses this hermetic process boundary and no direct pytest invocation remains in that workflow path.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/ci/hermetic-network-guard.c`:
- Around line 297-339: Update guarded_write and guarded_writev to avoid calling
non_loopback_destination for regular files, pipes, stdout, and stderr; first
determine whether fd is a socket using a cached descriptor classification or
equivalent socket check, invalidating cached entries when descriptors close.
Only perform the peer lookup and violation handling for socket descriptors,
while preserving the existing platform-specific write/writev dispatch.
In `@scripts/ci/run-hermetic-test-process.sh`:
- Around line 50-69: Replace the denylist-based environment filtering in the
env_args construction loop with an explicit allowlist of variables required by
the hermetic toolchain, including PATH, CC, CARGO_*, RUSTUP_*, RUNNER_*, CI,
TERM, and LD_LIBRARY_PATH. Unset every other inherited variable by default,
while preserving the existing sabotage=env bypass and explicitly retaining only
the documented test-control variables needed by the script.
- Around line 125-136: The hermetic runner must not require Rust for non-Rust
stages. In scripts/ci/run-hermetic-test-process.sh lines 125-136, always add
IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY to env_args, but derive and add the
CARGO_TARGET_*_RUNNER entry only when command -v rustc succeeds; in
.github/workflows/code_style.yml line 236, make no direct change if this guarded
runner removes the dependency, otherwise add an explicit Rust toolchain setup
for the static-checks job.
In `@tests/e2e/hermetic_process.py`:
- Around line 32-36: Update the macOS branch in the preload environment setup to
unconditionally assign DYLD_FORCE_FLAT_NAMESPACE the required value of "1"
instead of preserving an existing value via setdefault. Keep the existing
_prepend_preload behavior and Linux handling unchanged.
---
Outside diff comments:
In @.github/workflows/reborn-tests.yml:
- Around line 286-293: Update the webui-v2-smoke validation step to invoke
pytest through run-hermetic-deterministic-suite.sh command, matching the guarded
E2E stages. Ensure every test stage uses this hermetic process boundary and no
direct pytest invocation remains in that workflow path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: dc9009dd-5e80-4d51-933b-ac48bcf129dc
📒 Files selected for processing (20)
.github/workflows/README.md.github/workflows/code_style.yml.github/workflows/reborn-e2e.yml.github/workflows/reborn-tests.ymlcrates/ironclaw_host_runtime/src/sandbox_process/connect.rscrates/ironclaw_llm/src/github_copilot_auth.rscrates/ironclaw_skills/src/catalog.rsdocs/internal/hermetic-deterministic-suite.mdscripts/ci/discover-reborn-package-crates.shscripts/ci/hermetic-network-guard.cscripts/ci/hermetic-network-probe.cscripts/ci/hermetic-network-runner.shscripts/ci/reborn-local-coverage-ratchet.shscripts/ci/run-hermetic-deterministic-suite.shscripts/ci/run-hermetic-test-process.shscripts/ci/test-hermetic-test-process.shtests/e2e/conftest.pytests/e2e/hermetic_process.pytests/e2e/reborn_webui_harness.pytests/e2e/scenarios/test_reborn_responses_api.py
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
scripts/ci/run-hermetic-test-process.sh (2)
72-86: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftDon’t propagate host Cargo/Rustup directories into the hermetic shell.
Lines 72-86 derive
CARGO_HOMEandRUSTUP_HOMEfrom ambient values or the original$HOME, then re-export them afterHOME/XDG roots move underhermetic_root. Wrapped tests can read/write host toolchain/install and registry cache files, including~/.cargo/credentials.toml. Reuse dependency state outside this boundary or provide sanitized/read-only copies and test that contract explicitly.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/run-hermetic-test-process.sh` around lines 72 - 86, The hermetic test setup must not derive or re-export host Cargo and Rustup directories. Update the environment initialization around original_home, cargo_home, and rustup_home so CARGO_HOME and RUSTUP_HOME point only to sanitized hermetic locations or are omitted, while preserving any intentional dependency reuse through explicit read-only copies outside the host paths.
52-69: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winMake environment enumeration multiline-safe.
Lines 52-69 parse
envoutput with line-delimited reads, so a value containing\nPATH=...is seen as a variable name and can be-ued. Use exported names/NUL-delimited input and add a self-test case with ambient env containing a multiline value.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/run-hermetic-test-process.sh` around lines 52 - 69, Update the environment enumeration loop to consume exported variable names using a NUL-delimited, multiline-safe source instead of parsing line-delimited env output, while preserving the existing case-based allowlist and unset rules. Add a self-test covering an ambient exported variable whose value contains a newline and text resembling another variable assignment, verifying that only the intended variable is unset.Source: Coding guidelines
♻️ Duplicate comments (2)
scripts/ci/run-hermetic-test-process.sh (2)
57-69: 🔒 Security & Privacy | 🟠 MajorReplace the denylist with an allowlist.
PERPLEXITY_KEY,DEEPSEEK_APIKEY,HF_TOKEN_FILE,SSH_AUTH_SOCK,GPG_AGENT_INFO,BASH_ENV, and mutable-path overrides such asCARGO_TARGET_DIRdo not match these cases and remain available to the guarded process. That permits ambient credential/agent access, inherited shell startup code, and writes outside the hermetic root. Build the child environment from the minimal toolchain/test-control allowlist and add fixtures for unlisted providers and agent variables.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/run-hermetic-test-process.sh` around lines 57 - 69, Replace the environment-variable denylist in the shell script’s case statement with an explicit allowlist containing only the minimal toolchain and test-control variables required by the guarded process. Ensure variables such as PERPLEXITY_KEY, DEEPSEEK_APIKEY, HF_TOKEN_FILE, SSH_AUTH_SOCK, GPG_AGENT_INFO, BASH_ENV, and CARGO_TARGET_DIR are excluded by default, and add fixtures covering unlisted providers, agent variables, startup hooks, and mutable-path overrides.
126-137: 🩺 Stability & Availability | 🟠 MajorMake Rust runner wiring conditional on Rust availability.
The suite routes Python, frontend, QA, and E2E stages through this wrapper, but every normal invocation executes
rustc -vVfirst. On a runner without Rust, non-Rust commands fail before they start. Export the native guard library unconditionally and deriveCARGO_TARGET_*_RUNNERonly whencommand -v rustcsucceeds.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/run-hermetic-test-process.sh` around lines 126 - 137, Update the environment setup around cargo_runner_key so IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY is added unconditionally, while rustc host-triple detection and CARGO_TARGET_*_RUNNER wiring occur only when command -v rustc succeeds. Preserve the existing sabotage network condition and failure handling for available Rust installations, but allow non-Rust stages to run without rustc.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@scripts/ci/run-hermetic-test-process.sh`:
- Around line 72-86: The hermetic test setup must not derive or re-export host
Cargo and Rustup directories. Update the environment initialization around
original_home, cargo_home, and rustup_home so CARGO_HOME and RUSTUP_HOME point
only to sanitized hermetic locations or are omitted, while preserving any
intentional dependency reuse through explicit read-only copies outside the host
paths.
- Around line 52-69: Update the environment enumeration loop to consume exported
variable names using a NUL-delimited, multiline-safe source instead of parsing
line-delimited env output, while preserving the existing case-based allowlist
and unset rules. Add a self-test covering an ambient exported variable whose
value contains a newline and text resembling another variable assignment,
verifying that only the intended variable is unset.
---
Duplicate comments:
In `@scripts/ci/run-hermetic-test-process.sh`:
- Around line 57-69: Replace the environment-variable denylist in the shell
script’s case statement with an explicit allowlist containing only the minimal
toolchain and test-control variables required by the guarded process. Ensure
variables such as PERPLEXITY_KEY, DEEPSEEK_APIKEY, HF_TOKEN_FILE, SSH_AUTH_SOCK,
GPG_AGENT_INFO, BASH_ENV, and CARGO_TARGET_DIR are excluded by default, and add
fixtures covering unlisted providers, agent variables, startup hooks, and
mutable-path overrides.
- Around line 126-137: Update the environment setup around cargo_runner_key so
IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY is added unconditionally, while rustc
host-triple detection and CARGO_TARGET_*_RUNNER wiring occur only when command
-v rustc succeeds. Preserve the existing sabotage network condition and failure
handling for available Rust installations, but allow non-Rust stages to run
without rustc.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 0b00804b-4700-4bc3-8c35-21959523a157
📒 Files selected for processing (2)
scripts/ci/run-hermetic-test-process.shscripts/ci/test-hermetic-test-process.sh
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/e2e/hermetic_process.py (1)
29-34: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winFail closed when the guard library is empty.
An empty
IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARYpasses this check, then_prepend_preload()writes an empty loader value; nested E2E processes run without network enforcement. Reject whitespace/empty configuration instead of silently proceeding.Proposed fix
guard_library = source_env.get("IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY") if guard_library is None: return + if not guard_library.strip(): + raise ValueError("Hermetic network guard library must be non-empty")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/e2e/hermetic_process.py` around lines 29 - 34, Update the guard_library validation in the hermetic process setup to treat empty or whitespace-only IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY values as invalid and fail closed before calling _prepend_preload(). Preserve the existing behavior for a valid library path and the non-macOS flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@tests/e2e/hermetic_process.py`:
- Around line 29-34: Update the guard_library validation in the hermetic process
setup to treat empty or whitespace-only IRONCLAW_HERMETIC_NETWORK_GUARD_LIBRARY
values as invalid and fail closed before calling _prepend_preload(). Preserve
the existing behavior for a valid library path and the non-macOS flow.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1e2f0a01-c394-4e4f-9aac-b21ab7084ad7
📒 Files selected for processing (9)
.github/workflows/reborn-e2e.ymldocs/internal/hermetic-deterministic-suite.mdscripts/ci/hermetic-network-guard.cscripts/ci/hermetic-network-probe.cscripts/ci/hermetic-network-runner.shscripts/ci/run-hermetic-deterministic-suite.shscripts/ci/run-hermetic-test-process.shscripts/ci/test-hermetic-test-process.shtests/e2e/hermetic_process.py
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
scripts/ci/test-hermetic-test-process.sh (1)
70-73: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winAssert
CARGO_TARGET_DIRis inside the hermetic root.Lines [70]-[73] reject only the seeded
/developer/targetvalue and otherwise accept any path ending in/target;/tmp/targetor/developer/other/targetwould pass. This can let a regression leak Cargo state outside the isolated root without failing the self-test.As per path instructions, CI guardrail behavior requires regression coverage that validates the enforced boundary.
Suggested fix
- if [[ "${CARGO_TARGET_DIR}" != */target || "${CARGO_TARGET_DIR}" == "/developer/target" ]]; then - echo "ambient CARGO_TARGET_DIR leaked into the hermetic process" >&2 - exit 40 - fi + case "${CARGO_TARGET_DIR}" in + "${IRONCLAW_HERMETIC_ROOT}"/*) ;; + *) + echo "CARGO_TARGET_DIR is outside the hermetic root: ${CARGO_TARGET_DIR}" >&2 + exit 40 + ;; + esac🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci/test-hermetic-test-process.sh` around lines 70 - 73, Update the CARGO_TARGET_DIR validation in the hermetic process self-test to require the resolved path to be within the hermetic root, rather than merely ending in /target or excluding /developer/target. Preserve rejection of external paths such as /tmp/target and /developer/other/target, and add regression coverage exercising both an in-root value and representative out-of-root values.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@scripts/ci/test-hermetic-test-process.sh`:
- Around line 70-73: Update the CARGO_TARGET_DIR validation in the hermetic
process self-test to require the resolved path to be within the hermetic root,
rather than merely ending in /target or excluding /developer/target. Preserve
rejection of external paths such as /tmp/target and /developer/other/target, and
add regression coverage exercising both an in-root value and representative
out-of-root values.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2e499f87-37f4-4643-8c40-90912e831167
📒 Files selected for processing (2)
scripts/ci/test-hermetic-test-process.shtests/e2e/hermetic_process.py
|
Final maintainer readiness audit — head
|
* ci(test): enforce hermetic deterministic suite * fix(ci): close hermetic process guard gaps * fix(ci): preserve frontend package-manager selection * fix(ci): prepare frontend dependencies outside guard * fix(ci): avoid races in network interposer lookup * fix(ci): prefetch WebUI build toolchain for Cargo * test(ci): keep failure probes on loopback * test(ci): isolate Copilot refresh failures * fix(ci): preserve isolated Playwright toolchain * fix(ci): close connected UDP write bypasses * fix(ci): expose Linux sendmmsg probe API * fix(ci): allow IPv4-mapped loopback fakes * fix(ci): preserve guard and one-time setup * fix(ci): prepare coverage WebUI inputs once * fix(ci): prepare QA WebUI inputs once * fix(ci): preserve prepared Corepack cache * fix(ci): prefetch hermetic Postgres image * fix(ci): prepare direct WebUI crate bucket * fix(ci): share Reborn package discovery * fix(ci): preserve explicit Emulate fixture bearer * fix(ci): close hermetic review gaps * fix(e2e): reject empty hermetic guard path
This branch was successfully deployed
1 participant
Summary
scripts/ci/run-hermetic-deterministic-suite.sh allas the canonical local composition of the deterministic Reborn merge gates, and route the corresponding CI jobs through the same checked-in stages.sendfilealongside the existing socket-write paths.Change Type
Linked Issue
Related #6524 (WS1)
Overlap reconciliation:
41f0c9284c88b5bd6614542ca060075226952fb5. It isolates direct execution of the Trace Commons coverage test in its own child process. This PR does not copy that test-specific harness; the canonical suite instead supplies an isolated process root to all registered lanes. The changes are complementary and neither PR depends on the other.Validation
cargo fmt --all -- --checkcargo clippy -p ironclaw_skills -p ironclaw_host_runtime --all-targets --all-features -- -D warningsandcargo clippy -p ironclaw_llm --all-targets --all-features -- -D warnings; the new C guard is compiled with-Wall -Wextra -Werrorby the self-test.cargo build— not run as a standalone command; the representative Rust E2E substrate lane compiled every exercised target through the canonical runner.scripts/ci/test-hermetic-test-process.shscripts/ci/run-hermetic-deterministic-suite.sh rust-e2e substrates(262 passed)scripts/ci/run-hermetic-deterministic-suite.sh frontendafter rebasing onto current main (111 files, 914 tests passed)Reborn WebUI v2 smoke(43 guarded browser tests, 403 harvested QA/provider tests, and 21 Responses API tests passed)ironclaw_skills::catalog::test_search_returns_error_on_network_failure(1 passed)ironclaw_host_runtime::connect_override_proceeds_past_trust_boundary_with_opt_in(1 passed)cargo test -p ironclaw_llm --lib(995 passed; Copilot refresh-failure tests verified through loopback-only proxy)scripts/ci/run-hermetic-deterministic-suite.sh command cargo test -p ironclaw_common env_helpers::tests::runtime_override_round_trip -- --exact --nocapture(1 passed)scripts/ci/test-classify-test-scope.shscripts/ci/test-check-hermetic-env.sh(8/8)scripts/pre-commit-safety.shPLAYWRIGHT_BROWSERS_PATHpropagation through the hermetic self-testcargo test --features integration— not applicable: no database-backed product behavior or schema changed.31, temp=32, python-seed=34, and network=1with the intended diagnostics; confirmed loopback remains allowed, route-only UDP association emits no packet, and TCP plus connected UDPsend/sendmmsg/write/writevpaths are rejected.review-prorpr-shepherd --fixwas run before requesting review —pr-shepherdplus the IronClaw maintainer review lens found two blocking gaps; both were fixed ina59a3ab3aand revalidated locally.Test Strategy
User behavior:
Local deterministic Reborn testing now uses one documented command whose stages and configuration are also invoked by CI. Tests cannot silently inherit real provider credentials, developer homes/databases/workspaces, or external network access.
Risk areas:
Tests added or updated:
scripts/ci/test-hermetic-test-process.shcompiles/probes the guard, validates the default-deny environment allowlist (including unlisted providers, multiline values, token files, credential-agent sockets, and shell startup state), sanitized toolchain homes, unique and parallel roots, fixed inputs, workflow/stage parity, loopback allowance, nested minimal-env E2E propagation, and all four mutation modes.scripts/ci/run-hermetic-deterministic-suite.sh rust-e2e substratespassed 262 filesystem, event, projection, network, secret, resource, approval, and authorization contract tests through the new boundary.scripts/ci/check-reborn-qa-fixtures.shpassed for all 61 fixtures through the guardedqastage. The replay target rebuilt through the guarded WebUI/Corepack path and passed: 41 passed, 12 intentionally ignored, 0 failed.Reborn WebUI v2 smokejob: 43 guarded browser tests, 403 harvested QA/provider tests, and 21 Responses API tests. CI installs and caches Chromium at an explicit runner-temporaryPLAYWRIGHT_BROWSERS_PATHthat the guard preserves as immutable tooling input. The product-surface evidence contracts and all merge-gating launchers use the hermetic boundary, and the canonical local Python stage includes the SSO smoke.ironclaw_common::env_helpers::tests::runtime_override_round_trippassed through thecommandstage; the complete substrate group passed throughrust-e2e.What the tests prove:
FakeClock/FixedClockseams, configured deterministic jitter is zero, and security/identity randomness remains OS-backed rather than being weakened by a process-global seed.Commands run:
Checkbox-to-test/PR map:
envsabotage (this PR)PYTHONHASHSEEDconsumer pluspython-seedsabotage; live-code audit confirmed Rust time-sensitive behavior already uses typed clock seams and deterministic Reborn jitter defaults to zero, while cryptographic/identity randomness must remain OS-backednetworksabotage (this PR)Security Impact
Test-only security boundary added. It removes provider credentials and unrelated behavioral environment, disables OS keychain access, isolates file-backed mutable state, and interposes test-binary/Python/Node IP syscalls to deny and record non-loopback egress. Unix sockets and loopback fakes remain allowed. Product authorization, provider mediation, runtime policy, and production networking are unchanged.
Reborn Trust-Boundary Checklist
serde(default)fields fail closed or have migration tests: N/A — no serialization changed.Transient,Permanent,Misconfigured,PolicyDeniedor equivalent): N/A — no driver errors changed.hermetic-network-guard/hermetic-test-processand are test-only.Database Impact
None. No migration, schema, or backend implementation changed. Test database paths are redirected into per-invocation temporary roots.
Blast Radius
Merge-gating Reborn CI jobs and developers using the canonical local suite. The main risks are over-scrubbing a required test variable, blocking a test that intentionally reaches a non-loopback service, or platform loader differences. The allowlist, explicit live-lane exclusions, Linux/macOS guard implementations, localhost probes, nested-child probe, and draft CI run constrain those risks.
Compatibility: existing individual test commands still work; the canonical path adds stronger environment/process controls. CI matrix sharding and existing test discovery scripts remain authoritative.
Rollback Plan
Revert this twenty-two-commit focused stack to restore the previous direct workflow commands. No persistent data or schema rollback is required. If one lane proves intentionally non-hermetic, it can temporarily return to its previous direct command while a narrow, documented exemption is reviewed.
Review Follow-Through
Reviewer judgment is requested on the generic secret scrub suffixes and the deliberate boundary between guarded test execution and unguarded dependency-download/compiler-cache setup. CI validates the Linux
LD_PRELOADpath; local mutation testing validated the macOS interposer plus SIP-resistant process-sandbox fallback. The guard now wraps arbitrary launchers, forces Cargo offline after a checked locked fetch, and disables remote compiler wrappers inside the boundary.The maintainer self-review found no remaining correctness or security issue after the arbitrary-launcher, Linux fortified-build, unsupported-clock-claim, Corepack isolation, interposer race, intentional TEST-NET and Copilot refresh-failure fixture fixes, isolated Playwright browser-toolchain path, one-time CI dependency preparation, preload-chain preservation, packet-free UDP route-probe handling, IPv4-mapped loopback allowance, connected-UDP
write/writev/sendmmsg/sendfilebypass closure, shared local/CI package discovery, default-deny environment construction, credential-agent removal, sanitized Cargo/Rustup state, Rust-less execution, SSO/product-evidence parity, forced macOS loader mode, and fail-closed empty nested-guard validation. All readiness-triggered review threads and outside-diff findings were verified, fixed where valid, and resolved. No proven WS1 item is left unimplemented in this PR's process/CI boundary. #6714 is a complementary direct-test defense, not a blocker or copied dependency.Review track: C (security/runtime/DB/CI)