Skip to content

test(approval): patch _YOLO_MODE_FROZEN directly in test_yolo_overrides_cron_deny - #32019

Merged
teknium1 merged 1 commit into
mainfrom
fix/yolo-frozen-test-isolation
May 25, 2026
Merged

test(approval): patch _YOLO_MODE_FROZEN directly in test_yolo_overrides_cron_deny#32019
teknium1 merged 1 commit into
mainfrom
fix/yolo-frozen-test-isolation

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

test_yolo_overrides_cron_deny now passes deterministically in CI. Pre-existing test-isolation bug exposed by CI's clean-env conftest — blocks every PR routed to the test (3) / test (4) shard.

Root cause

tools.approval._YOLO_MODE_FROZEN is intentionally frozen at module import time to prevent prompt-injection escalation:

# tools/approval.py line 29
# Freeze YOLO mode at module import time. Reading os.environ on every call
# would allow any skill running inside the process to set this variable and
# instantly bypass all approval checks — a prompt-injection escalation path.
_YOLO_MODE_FROZEN: bool = is_truthy_value(os.getenv("HERMES_YOLO_MODE", ""))

The test set HERMES_YOLO_MODE=1 via monkeypatch.setenv AFTER the module had already been imported, so the frozen value stayed False and the yolo-bypass path never activated. Result: check_dangerous_command("rm -rf /tmp/stuff", "local") returned approved=False, the test assert failed.

Why local passes: the conftest setup leaks a non-empty HERMES_YOLO_MODE into the import-time env on developer machines. CI's env -i HOME=$HOME PATH=$PATH clean-env path exposes the bug deterministically.

Fix

Patch the module attribute directly via mock.patch.object, simulating process-startup with HERMES_YOLO_MODE=1 regardless of import order. The behavior under test (yolo bypasses cron_mode=deny for non-hardline commands) is unchanged. The security invariant (_YOLO_MODE_FROZEN can't be set at runtime by skills) is preserved.

Changes

  • tests/tools/test_cron_approval_mode.pytest_yolo_overrides_cron_deny adds mock.patch.object(tools.approval, "_YOLO_MODE_FROZEN", True) alongside the existing _get_cron_approval_mode patch.

Validation

Before After
env -i HOME=$HOME PATH=$PATH pytest tests/tools/test_cron_approval_mode.py 1 failed, 23 passed 24 passed
CI test (3) / test (4) shard fails on every PR passes
Behavior under test identical identical
Security invariant _YOLO_MODE_FROZEN frozen preserved preserved

Why fix-it-first PR

This is the second night in a row a different PR has been blocked by this same test failure (PR #32013 Windows freeze, PR #32017 KBI guard, PR #32018 bracketed-paste timeout all hit it). Per references/green-ci-policy.md, pre-existing failures get a separate fix-it PR so the affected PRs can rebase onto a clean main.

Will rebase #32013 / #32017 / #32018 onto this once it lands.

Infographic

yolo-frozen-test-isolation

https://v3b.fal.media/files/b/0a9b9e23/O6oTzesAd4CJlfSUJvOMc_C0Mg0cuv.png

…es_cron_deny

The test set HERMES_YOLO_MODE=1 via monkeypatch.setenv, expecting
check_dangerous_command() to honor yolo and bypass cron_mode=deny. But
tools.approval._YOLO_MODE_FROZEN is intentionally frozen at module
import time (security: prevents prompt-injection runtime escalation).
When CI imports the module BEFORE the test sets the env, the frozen
value stays False and the yolo bypass never activates.

Local runs missed this because the conftest leaked a non-empty
HERMES_YOLO_MODE into the import-time env. CI's clean-env path exposed
the bug deterministically on test (3) / test (4) shards.

Fix: patch the module attribute directly via mock.patch.object so the
test simulates process-startup-with-yolo regardless of import order.
The behavior under test (yolo bypasses cron_mode=deny for non-hardline
commands) is unchanged; the security invariant (_YOLO_MODE_FROZEN can't
be set at runtime by skills) is preserved.

Reproduced locally with: env -i HOME=$HOME PATH=$PATH python3 -m pytest
  tests/tools/test_cron_approval_mode.py -o 'addopts=' -v
Without the fix: 1 failed, 23 passed. With the fix: 24 passed.
@alt-glitch alt-glitch added type/test Test coverage or test infrastructure P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/auth Authentication, OAuth, credential pools labels May 25, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #32014 — both fix the same test (test_yolo_overrides_cron_deny) by patching _YOLO_MODE_FROZEN directly on the module. #32014 uses the more idiomatic monkeypatch.setattr approach.

@teknium1
teknium1 merged commit 79799c8 into main May 25, 2026
22 checks passed
@teknium1
teknium1 deleted the fix/yolo-frozen-test-isolation branch May 25, 2026 12:07
@github-actions

Copy link
Copy Markdown
Contributor

🔎 Lint report: fix/yolo-frozen-test-isolation vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9256 on HEAD, 9256 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 4909 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants