feat(smoke): consult runtime_wedge after execute() to catch SDK init wedges - #2473
Merged
HongmingWang-Rabbit merged 1 commit intoMay 2, 2026
Merged
Conversation
…wedges Timeout-as-PASS in run_executor_smoke missed the PR-25-class regression: claude-agent-sdk takes 60s to time out on a malformed argv, our outer wait_for fires at 5s default and reports "imports healthy, hit a network boundary." A broken image then ships to GHCR. Universal fix uses the existing runtime_wedge module (already documented as the cross-cutting wedge holder, already read by heartbeat). Adapters opt-in by calling runtime_wedge.mark_wedged() from their executor's wedge catch arm; the smoke now consults runtime_wedge.is_wedged() at the end of every result path and upgrades a provisional PASS to FAIL when the flag is set. Non-opt-in adapters keep working as before — the check is additive. CI uses MOLECULE_SMOKE_TIMEOUT_SECS=90 to outlast the SDK's 60s initialize() handshake so the wedge marks before our outer wait_for fires. Module + helper docstrings call out the calibration so a future contributor doesn't lower it without thinking through what that wins back vs. what it loses. Tests: 7 new cases pinning the wedge-aware paths — mark+raise (PR-25 shape), mark+block (still-running execute that wait_for cuts short), clean+clean (additive contract), import-resilience (fail-open when runtime_wedge unimportable). Regression-injection-checked: silencing the new check fails both wedge-shape tests at unit-test time.
HongmingWang-Rabbit
requested a review
from hongmingwang-moleculeai
as a code owner
May 2, 2026 00:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runtime_wedge.is_wedged()consultation at the end of every result path inrun_executor_smokeinitialize()timeout used to read as "network boundary, imports healthy" through the 5s outer wait_forWhy
Today's incident: a wheel-side change (template repo PR #25) shipped
extra_args={"dangerously-load-development-channels": None}. The CLI rejected the bare--flag, the SDK timed out atinitialize, every A2A turn wedged. Workspace dd40faf8 dead-locked.The existing import-smoke (issue #2275, task #126) catches lazy-import regressions but treats every other timeout / non-ImportError as PASS. That was an explicit choice — adapter-level errors are caught by adapter-level tests — but it makes the publish gate blind to wedges that surface as the executor returning a 'success-shaped' control flow.
Design
Universal, opt-in via the existing
runtime_wedgemodule (already documented as the cross-cutting wedge state holder; already read by heartbeat to flip the workspace todegraded). Adapters callruntime_wedge.mark_wedged(reason)from their executor's wedge catch arm; the smoke readsis_wedged()and overrides PASS → FAIL.runtime_wedgekeep working as before.runtime_wedgeis missing reads as 'no wedge info' (same posture heartbeat takes), so observability never crashes the smoke gate.MOLECULE_SMOKE_TIMEOUT_SECS=90to outlast the SDK's 60sinitialize()handshake — comments call this out so a future contributor doesn't shrink it accidentally.Test plan
pytest tests/test_smoke_mode.py— 28 passed, 2 skipped (real-SDK-only)pytest tests/test_smoke_mode.py tests/test_runtime_wedge.py tests/test_heartbeat.py— 57 passed, 2 skippedwedge_reason_str = _check_runtime_wedge()makes both wedge-shape tests fail at unit-test timepublish-template-image.ymlsmoke step in a follow-up molecule-ci PR (this PR ships the runtime-side change so the wheel publishes first)_mark_sdk_wedgedin a follow-up template-repo PR (so the heartbeat + this gate both see the same wedge)Follow-ups (separate PRs by design — this PR is the foundation):
publish-template-image.ymlto run aMOLECULE_SMOKE_MODE=1boot withMOLECULE_SMOKE_TIMEOUT_SECS=90between import-smoke and GHCR push_mark_sdk_wedgedalso callruntime_wedge.mark_wedgedso the universal hook sees claude wedges (heartbeat already needs this — the local sticky flag isn't currently observed)