Repository navigation
refactor(wedge): extract claude_sdk_executor wedge state into runtime_wedge module #2154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
HongmingWang-Rabbit
merged 4 commits into
staging
from
refactor/extract-wedge-state-from-claude-sdk
Apr 27, 2026
Merged
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
1d231ed
refactor(wedge): extract claude_sdk_executor wedge state into runtime…
rabbitblood cd899c9
docs(wedge): integration recipe for adapters that want to flip-to-deg…
rabbitblood feb5449
refactor(wedge): address review feedback — class wrap + import-path d…
rabbitblood 5e04924
refactor(wedge): mark re-exports explicit via __all__
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| """Per-process runtime-wedge state. | ||
|
|
||
| Adapter executors that hit a non-recoverable wedge (e.g. claude-agent-sdk's | ||
| `Control request timeout: initialize` corrupting the client process's | ||
| internal state) call mark_wedged(reason). The heartbeat task reads | ||
| is_wedged() / wedge_reason() and forwards them in the heartbeat payload's | ||
| runtime_state field — the platform then flips workspace status to | ||
| `degraded` so the canvas surfaces a Restart hint instead of leaving the | ||
| user staring at a green dot while every chat hangs. | ||
|
|
||
| Module scope (not instance scope) is deliberate: the wedge is a property | ||
| of the Python process, not any particular executor. With one executor | ||
| per workspace process today this is the simplest lock-free | ||
| read+write fit. A future per-org multi-executor design could move this | ||
| to a shared registry. | ||
|
|
||
| This module lives in molecule-runtime (NOT in any adapter / template | ||
| repo) because: | ||
|
|
||
| 1. workspace/heartbeat.py reads it on every heartbeat — cross-cutting | ||
| concern, runtime owns it. | ||
| 2. Multiple adapter executors can mark themselves wedged with their | ||
| own reason; the runtime aggregates one flag for the platform. | ||
| 3. Decoupling from claude_sdk_executor is the prerequisite for the | ||
| universal-runtime refactor (molecule-core task #87) — without | ||
| this extraction, claude_sdk_executor.py couldn't move to its | ||
| template repo because heartbeat would lose access to the wedge | ||
| state. | ||
|
|
||
| Public API: mark_wedged(reason), clear_wedge(), is_wedged(), | ||
| wedge_reason(). The reset_for_test() helper is for unit tests only. | ||
| """ | ||
| from __future__ import annotations | ||
|
|
||
| import logging | ||
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| # Single-flag state. None = healthy; non-empty string = wedged with that | ||
| # human-readable reason. Surfaced verbatim as the canvas's degraded-card | ||
| # banner text via heartbeat.sample_error. | ||
| _wedged_reason: str | None = None | ||
|
|
||
|
|
||
| def is_wedged() -> bool: | ||
| """True if some adapter executor in this process has marked itself | ||
| wedged. Sticky until the same executor calls clear_wedge() on | ||
| observed recovery (or the process restarts).""" | ||
| return _wedged_reason is not None | ||
|
|
||
|
|
||
| def wedge_reason() -> str: | ||
| """Human-readable description of the wedge cause, or empty string | ||
| when not wedged. Surfaced to the canvas via heartbeat sample_error.""" | ||
| return _wedged_reason or "" | ||
|
|
||
|
|
||
| def mark_wedged(reason: str) -> None: | ||
| """Flag the runtime as wedged. Only the FIRST call wins so a | ||
| subsequent identical-class wedge can't overwrite a more specific | ||
| initial reason — the operator-visible banner stays stable. | ||
|
|
||
| Adapters call this from their executor's exception path when the | ||
| SDK has hit a non-recoverable error class. Safe to call multiple | ||
| times; the no-op when already wedged is intentional. | ||
| """ | ||
| global _wedged_reason | ||
| if _wedged_reason is None: | ||
| _wedged_reason = reason | ||
| logger.error( | ||
| "runtime wedge detected: %s — workspace will report degraded until cleared", | ||
| reason, | ||
| ) | ||
|
|
||
|
|
||
| def clear_wedge() -> None: | ||
| """Auto-recovery: adapter calls this after an observed successful | ||
| operation. The original wedge could be transient (single network | ||
| blip during the SDK's first-message handshake), and a sticky-only | ||
| flag would lock the workspace into degraded forever even after the | ||
| SDK started working again. Clearing on observed success means the | ||
| next heartbeat after a working query reports runtime_state empty | ||
| and the platform flips status back to online. | ||
|
|
||
| No-op when not wedged (the common case).""" | ||
| global _wedged_reason | ||
| if _wedged_reason is not None: | ||
| logger.info("runtime wedge cleared after successful operation — workspace will recover to online on next heartbeat") | ||
| _wedged_reason = None | ||
|
|
||
|
|
||
| def reset_for_test() -> None: | ||
| """Test-only escape hatch. Production code clears the wedge via | ||
| clear_wedge() on observed success; this helper is for unit tests | ||
| that need to reset between cases without going through the full | ||
| SDK round-trip.""" | ||
| global _wedged_reason | ||
| _wedged_reason = None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| """Tests for runtime_wedge — the runtime-side wedge-state module that | ||
| heartbeat reads + adapter executors write. Extracted from claude_sdk_ | ||
| executor (task #87 universal-runtime refactor) so the executor can move | ||
| to its template repo without breaking heartbeat. | ||
|
|
||
| The behavior is identical to the prior in-executor implementation; tests | ||
| pin the contract so the re-export shim in claude_sdk_executor.py can | ||
| later be deleted without surprise.""" | ||
| import pytest | ||
|
|
||
| import runtime_wedge | ||
|
|
||
|
|
||
| @pytest.fixture(autouse=True) | ||
| def _reset(): | ||
| """Each test starts with a clean wedge state — production wedges are | ||
| sticky-per-process, but cross-test bleed would couple unrelated cases.""" | ||
| runtime_wedge.reset_for_test() | ||
| yield | ||
| runtime_wedge.reset_for_test() | ||
|
|
||
|
|
||
| class TestRuntimeWedge: | ||
| def test_starts_unwedged(self): | ||
| assert runtime_wedge.is_wedged() is False | ||
| assert runtime_wedge.wedge_reason() == "" | ||
|
|
||
| def test_mark_wedged_sets_flag_and_reason(self): | ||
| runtime_wedge.mark_wedged("SDK init timeout") | ||
| assert runtime_wedge.is_wedged() is True | ||
| assert runtime_wedge.wedge_reason() == "SDK init timeout" | ||
|
|
||
| def test_first_mark_wins(self): | ||
| # Stable banner text is more important than the most-recent | ||
| # cause. A second wedge while already wedged should NOT | ||
| # overwrite — operator sees the original (more diagnosable) | ||
| # reason, not whatever the SDK said next. | ||
| runtime_wedge.mark_wedged("SDK init timeout") | ||
| runtime_wedge.mark_wedged("Subsequent identical-class wedge") | ||
| assert runtime_wedge.wedge_reason() == "SDK init timeout" | ||
|
|
||
| def test_clear_wedge_restores_healthy(self): | ||
| # Auto-recovery: when the SDK starts working again, the next | ||
| # heartbeat must report empty runtime_state so the platform | ||
| # flips status from degraded back to online. | ||
| runtime_wedge.mark_wedged("transient blip") | ||
| runtime_wedge.clear_wedge() | ||
| assert runtime_wedge.is_wedged() is False | ||
| assert runtime_wedge.wedge_reason() == "" | ||
|
|
||
| def test_clear_wedge_when_not_wedged_is_noop(self): | ||
| # No-op safety — production calls clear_wedge() on every | ||
| # successful query (~thousands of times per session); throwing | ||
| # or logging when not wedged would spam. | ||
| runtime_wedge.clear_wedge() | ||
| runtime_wedge.clear_wedge() # still safe twice in a row | ||
| assert runtime_wedge.is_wedged() is False | ||
|
|
||
| def test_re_marking_after_clear_is_allowed(self): | ||
| # Real production path: SDK wedges, recovers, wedges again. | ||
| # Each cycle should land cleanly (not silently drop). | ||
| runtime_wedge.mark_wedged("first wedge") | ||
| runtime_wedge.clear_wedge() | ||
| runtime_wedge.mark_wedged("second wedge — different reason") | ||
| assert runtime_wedge.is_wedged() is True | ||
| assert runtime_wedge.wedge_reason() == "second wedge — different reason" | ||
|
|
||
|
|
||
| class TestClaudeSdkExecutorReExportShim: | ||
| """claude_sdk_executor.py keeps re-exporting the old names for one | ||
| release cycle so any third-party adapter copying our wedge convention | ||
| has time to migrate. These tests pin the shim — when removed, the | ||
| test file goes too.""" | ||
|
|
||
| def test_is_wedged_re_exported(self): | ||
| from claude_sdk_executor import is_wedged | ||
| assert is_wedged is runtime_wedge.is_wedged | ||
|
|
||
| def test_wedge_reason_re_exported(self): | ||
| from claude_sdk_executor import wedge_reason | ||
| assert wedge_reason is runtime_wedge.wedge_reason | ||
|
|
||
| def test_internal_helpers_re_exported(self): | ||
| # Keep the underscore names too — claude_sdk_executor's own | ||
| # _run_query calls _mark_sdk_wedged / _clear_sdk_wedge_on_success | ||
| # via these re-exports. | ||
| from claude_sdk_executor import ( | ||
| _mark_sdk_wedged, | ||
| _clear_sdk_wedge_on_success, | ||
| _reset_sdk_wedge_for_test, | ||
| ) | ||
| assert _mark_sdk_wedged is runtime_wedge.mark_wedged | ||
| assert _clear_sdk_wedge_on_success is runtime_wedge.clear_wedge | ||
| assert _reset_sdk_wedge_for_test is runtime_wedge.reset_for_test | ||
|
|
||
| def test_re_export_state_is_shared(self): | ||
| # The shim isn't a copy — both names refer to the same module | ||
| # state. Marking via the executor name must be observable via | ||
| # the runtime_wedge name (and vice versa). | ||
| from claude_sdk_executor import _mark_sdk_wedged | ||
| _mark_sdk_wedged("via executor shim") | ||
| assert runtime_wedge.is_wedged() is True | ||
| assert runtime_wedge.wedge_reason() == "via executor shim" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.