Skip to content

feat(smoke): consult runtime_wedge after execute() to catch SDK init wedges - #2473

Merged
HongmingWang-Rabbit merged 1 commit into
stagingfrom
feat/universal-turn-smoke-runtime-wedge
May 2, 2026
Merged

HongmingWang-Rabbit merged 1 commit into
stagingfrom
feat/universal-turn-smoke-runtime-wedge

Conversation

@HongmingWang-Rabbit

Copy link
Copy Markdown
Contributor

Summary

  • Adds a runtime_wedge.is_wedged() consultation at the end of every result path in run_executor_smoke
  • Upgrades the provisional PASS-on-timeout / PASS-on-non-import-error branches to FAIL when an adapter has marked the runtime wedged
  • Closes the gap that let PR-25-class regressions (claude-agent-sdk init wedge from a malformed CLI argv) ship to GHCR — the SDK's 60s initialize() timeout used to read as "network boundary, imports healthy" through the 5s outer wait_for

Why

Today's incident: a wheel-side change (template repo PR #25) shipped extra_args={"dangerously-load-development-channels": None}. The CLI rejected the bare --flag, the SDK timed out at initialize, every A2A turn wedged. Workspace dd40faf8 dead-locked.

The existing import-smoke (issue #2275, task #126) catches lazy-import regressions but treats every other timeout / non-ImportError as PASS. That was an explicit choice — adapter-level errors are caught by adapter-level tests — but it makes the publish gate blind to wedges that surface as the executor returning a 'success-shaped' control flow.

Design

Universal, opt-in via the existing runtime_wedge module (already documented as the cross-cutting wedge state holder; already read by heartbeat to flip the workspace to degraded). Adapters call runtime_wedge.mark_wedged(reason) from their executor's wedge catch arm; the smoke reads is_wedged() and overrides PASS → FAIL.

  • Additive: adapters that don't integrate with runtime_wedge keep working as before.
  • Fail-open on import: a corrupt install where runtime_wedge is missing reads as 'no wedge info' (same posture heartbeat takes), so observability never crashes the smoke gate.
  • Calibrated timeout: CI usage example sets MOLECULE_SMOKE_TIMEOUT_SECS=90 to outlast the SDK's 60s initialize() handshake — comments call this out so a future contributor doesn't shrink it accidentally.

Test plan

  • pytest tests/test_smoke_mode.py — 28 passed, 2 skipped (real-SDK-only)
  • pytest tests/test_smoke_mode.py tests/test_runtime_wedge.py tests/test_heartbeat.py — 57 passed, 2 skipped
  • Regression-injection: silencing wedge_reason_str = _check_runtime_wedge() makes both wedge-shape tests fail at unit-test time
  • Wired into publish-template-image.yml smoke step in a follow-up molecule-ci PR (this PR ships the runtime-side change so the wheel publishes first)
  • Wired into claude-code template's _mark_sdk_wedged in a follow-up template-repo PR (so the heartbeat + this gate both see the same wedge)

Follow-ups (separate PRs by design — this PR is the foundation):

  1. molecule-ci: extend publish-template-image.yml to run a MOLECULE_SMOKE_MODE=1 boot with MOLECULE_SMOKE_TIMEOUT_SECS=90 between import-smoke and GHCR push
  2. molecule-ai-workspace-template-claude-code: have _mark_sdk_wedged also call runtime_wedge.mark_wedged so the universal hook sees claude wedges (heartbeat already needs this — the local sticky flag isn't currently observed)

…wedges

Timeout-as-PASS in run_executor_smoke missed the PR-25-class
regression: claude-agent-sdk takes 60s to time out on a malformed
argv, our outer wait_for fires at 5s default and reports "imports
healthy, hit a network boundary." A broken image then ships to GHCR.

Universal fix uses the existing runtime_wedge module (already
documented as the cross-cutting wedge holder, already read by
heartbeat). Adapters opt-in by calling runtime_wedge.mark_wedged()
from their executor's wedge catch arm; the smoke now consults
runtime_wedge.is_wedged() at the end of every result path and
upgrades a provisional PASS to FAIL when the flag is set. Non-opt-in
adapters keep working as before — the check is additive.

CI uses MOLECULE_SMOKE_TIMEOUT_SECS=90 to outlast the SDK's 60s
initialize() handshake so the wedge marks before our outer wait_for
fires. Module + helper docstrings call out the calibration so a
future contributor doesn't lower it without thinking through what
that wins back vs. what it loses.

Tests: 7 new cases pinning the wedge-aware paths — mark+raise (PR-25
shape), mark+block (still-running execute that wait_for cuts short),
clean+clean (additive contract), import-resilience (fail-open when
runtime_wedge unimportable). Regression-injection-checked: silencing
the new check fails both wedge-shape tests at unit-test time.
@HongmingWang-Rabbit
HongmingWang-Rabbit added this pull request to the merge queue May 2, 2026
Merged via the queue into staging with commit b39dc62 May 2, 2026
20 checks passed
@HongmingWang-Rabbit
HongmingWang-Rabbit deleted the feat/universal-turn-smoke-runtime-wedge branch May 2, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant