refactor(python): remove deprecated server stack - #343
Merged
Conversation
Signed-off-by: nachiketb <nachiketb@nvidia.com>
|
ayushag-nv
approved these changes
Aug 10, 2026
sabhatinas
reviewed
Aug 11, 2026
sabhatinas
requested changes
Aug 11, 2026
sabhatinas
left a comment
sabhatinas
left a comment
Contributor
There was a problem hiding this comment.
Had one comment about stage router skill, rest looks good to me!
ayushag-nv
marked this pull request as ready for review
August 11, 2026 17:27
ayushag-nv
enabled auto-merge (squash)
August 11, 2026 17:28
sabhatinas
approved these changes
Aug 11, 2026
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (239)
💤 Files with no reviewable changes (106)
WalkthroughThe pull request removes the deprecated Python server and component stack, simplifies Python bindings, adds a PTY-based launcher terminal, switches workflows to ChangesNative serving transition
Estimated code review effort: 5 (Critical) | ~120 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci.yml (1)
216-227: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winValidate the top-level command allowlist.
The current checks reject selected removed commands but do not prove that
switchyard --helpexposes onlylaunch. An additional stale top-level command can pass both checks.
.github/workflows/ci.yml#L216-L227: runswitchyard --helpand assert thatlaunchis the only available subcommand.tests/test_cli_reference_docs.py#L50-L56: assert that_subparsers(_build_parser())has exactly{"launch"}.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 216 - 227, The CLI validation must prove that the top-level command allowlist contains only launch. In .github/workflows/ci.yml lines 216-227, add a switchyard --help check that confirms launch is the sole available subcommand; in tests/test_cli_reference_docs.py lines 50-56, assert that _subparsers(_build_parser()) equals {"launch"}.
🧹 Nitpick comments (1)
crates/switchyard-py/src/libsy_bindings.rs (1)
31-39: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider naming the failing header in the error message.
The
httpcrate errors are generic, for example "invalid HTTP header name". A caller that passes many headers cannot tell which entry failed. Include the header name in the message. Do not include the header value, because header values often carry API keys.
try_appendis safe here. The input is aHashMap, so keys are unique, andtry_appendfails only when the map exceeds its maximum size.♻️ Proposed error-context improvement
for (name, value) in headers { - let name = HeaderName::from_bytes(name.as_bytes()) - .map_err(|error| PyValueError::new_err(error.to_string()))?; - let value = HeaderValue::from_str(value) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + let header_name = HeaderName::from_bytes(name.as_bytes()) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; + // Never include the header value in the error: values can carry credentials. + let value = HeaderValue::from_str(value) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; result - .try_append(name, value) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + .try_append(header_name, value) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/switchyard-py/src/libsy_bindings.rs` around lines 31 - 39, Update the header parsing errors in the loop over headers to include the failing header name alongside the underlying error message, for both HeaderName::from_bytes and HeaderValue::from_str failures. Do not include the header value, and preserve the existing try_append handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Line 142: Update the architecture and project-structure fenced blocks in
AGENTS.md to use the text language identifier on both opening fences, resolving
the markdownlint MD040 violations while leaving their contents unchanged.
In `@crates/switchyard-py/src/errors.rs`:
- Around line 10-19: Add a deprecation and migration record for the removed PyO3
exports, covering SwitchyardRuntimeError and the typed Switchyard*Error classes,
before finalizing the new LibsyError-only API in register and py_libsy_error.
Document the replacement behavior and migration path using the repository’s
established deprecation-plan location and format.
In `@crates/switchyard-server/README.md`:
- Around line 168-169: Update the README latency definition around
switchyard_total_latency_ms to state that timing ends after the wrapped stream
completes, not when the stream handle arrives. Keep the response-completion
wording consistent with stream completion and replace “connection accept” with
“connection acceptance.”
In `@docs/routing_algorithms/stage_router_routing.md`:
- Around line 154-157: Replace the universal lowest-threshold guidance in the
benchmark calibration section with picker-specific rules: for capable_first,
lower confidence_threshold only enough to rescue RESCUE without over-escalating
harmful LOSS downgrades; for efficient_first, calibrate it to increase
escalation for beneficial LOSS cases while avoiding unnecessary RESCUE changes.
Keep the corroborative scorer context and require selecting the best threshold
independently for each picker mode.
In `@switchyard_rust/__init__.py`:
- Line 4: Before removing the package-root exports from switchyard_rust, add an
explicit deprecation plan for the 34 previously public names. Preserve the
current exports during the deprecation period and document the planned removal
timeline and migration path for callers.
In `@switchyard/__init__.py`:
- Around line 4-14: Document this as a breaking release in CHANGELOG.md,
including symbol-level migration guidance for the removed package-level exports,
quiet_dependency_loggers, and SwitchyardRuntimeError base. Cover
switchyard/__init__.py, switchyard/cli/command_utils.py, and
switchyard_rust/libsy.py; no direct code changes are required at these sites
unless you choose to retain compatibility shims through a documented removal
release.
In `@switchyard/cli/launchers/launcher_runtime.py`:
- Around line 88-91: Update configure_debug_file_logging to attach a
logging.NullHandler to the root logger immediately after clearing its handlers,
preventing logging.lastResort from writing dependency warnings to stderr. Revise
silence_launch_loggers’ docstring to accurately describe the terminal-logging
behavior and new scope.
In `@switchyard/cli/launchers/shell_tui.py`:
- Around line 172-175: Snapshot a bounded footer height once per paint in the
shell TUI layout flow, and use that same value for footer rendering, shell row
calculation, scroll-region setup, and child PTY sizing. When the snapshot
differs from the previous layout, update TIOCSWINSZ and the scroll region
immediately rather than waiting for an outer-terminal resize. Add a regression
test that changes footer height between paints and verifies the child PTY and
rendered layout stay synchronized.
- Around line 388-392: Update the stdin forwarding loop around os.read and
master_fd so partial writes and BlockingIOError from os.write are handled
without losing bytes. Maintain a pending-output buffer, append newly read stdin
data, and monitor master_fd for writability until all buffered bytes are
written, while preserving the existing EOF break behavior.
- Around line 316-419: Refactor ShellTUI.run and its blocking event loop to an
async lifecycle, replacing the background footer thread and blocking
select/waits with asyncio-compatible tasks and awaits while preserving terminal
setup, child I/O, cleanup, and exit-code behavior. Update synchronous launcher
boundaries that invoke run to call the async lifecycle through asyncio.run(),
and adjust related lifecycle helpers to remain consistently awaitable.
---
Outside diff comments:
In @.github/workflows/ci.yml:
- Around line 216-227: The CLI validation must prove that the top-level command
allowlist contains only launch. In .github/workflows/ci.yml lines 216-227, add a
switchyard --help check that confirms launch is the sole available subcommand;
in tests/test_cli_reference_docs.py lines 50-56, assert that
_subparsers(_build_parser()) equals {"launch"}.
---
Nitpick comments:
In `@crates/switchyard-py/src/libsy_bindings.rs`:
- Around line 31-39: Update the header parsing errors in the loop over headers
to include the failing header name alongside the underlying error message, for
both HeaderName::from_bytes and HeaderValue::from_str failures. Do not include
the header value, and preserve the existing try_append handling.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8b113f39-f5b8-46c9-9c29-1081242baa7a
⛔ Files ignored due to path filters (2)
Cargo.lockis excluded by!**/*.lock,!Cargo.lockuv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (239)
.agents/skills/switchyard-stage-router-scorer/SKILL.md.github/ISSUE_TEMPLATE/bug_report.md.github/workflows/ci.yml.github/workflows/perf.ymlAGENTS.mdCHANGELOG.mdCargo.tomlINSTALLATION.mdREADME.mdbenchmark/README.mdbenchmark/score_staged_run.pycrates/libsy-llm-client/src/client.rscrates/libsy-llm-client/src/error.rscrates/switchyard-components/Cargo.tomlcrates/switchyard-components/src/backends/anthropic.rscrates/switchyard-components/src/backends/common.rscrates/switchyard-components/src/backends/context_overflow.rscrates/switchyard-components/src/backends/mod.rscrates/switchyard-components/src/backends/multi.rscrates/switchyard-components/src/backends/openai.rscrates/switchyard-components/src/backends/selection.rscrates/switchyard-components/src/backends/stats.rscrates/switchyard-components/src/contracts/backend.rscrates/switchyard-components/src/contracts/context.rscrates/switchyard-components/src/contracts/error.rscrates/switchyard-components/src/contracts/ids.rscrates/switchyard-components/src/contracts/mod.rscrates/switchyard-components/src/contracts/roles.rscrates/switchyard-components/src/contracts/types.rscrates/switchyard-components/src/dimension_collector/mod.rscrates/switchyard-components/src/dimension_collector/response/checks.rscrates/switchyard-components/src/dimension_collector/response/mod.rscrates/switchyard-components/src/dimension_collector/tool_signals.rscrates/switchyard-components/src/lib.rscrates/switchyard-components/src/request_processors/dimension_collector.rscrates/switchyard-components/src/request_processors/mod.rscrates/switchyard-components/src/request_processors/random_routing.rscrates/switchyard-components/src/request_processors/stats.rscrates/switchyard-components/src/response_processors/mod.rscrates/switchyard-components/src/response_processors/response_signals.rscrates/switchyard-components/src/response_processors/stats.rscrates/switchyard-components/src/stage_router.rscrates/switchyard-components/src/stats/accumulator.rscrates/switchyard-components/src/stats/cache_eligibility.rscrates/switchyard-components/src/stats/context.rscrates/switchyard-components/src/stats/cost.rscrates/switchyard-components/src/stats/mod.rscrates/switchyard-components/src/stats/usage.rscrates/switchyard-components/src/telemetry.rscrates/switchyard-components/tests/adversarial_multi_llm_backend.rscrates/switchyard-components/tests/adversarial_native_backends.rscrates/switchyard-components/tests/adversarial_random_routing.rscrates/switchyard-components/tests/contracts.rscrates/switchyard-components/tests/stats_accumulator.rscrates/switchyard-components/tests/stats_processors.rscrates/switchyard-components/tests/stats_usage_shapes.rscrates/switchyard-components/tests/support/config.rscrates/switchyard-components/tests/support/mod.rscrates/switchyard-py/Cargo.tomlcrates/switchyard-py/src/component_bindings.rscrates/switchyard-py/src/component_bindings/backends.rscrates/switchyard-py/src/component_bindings/config.rscrates/switchyard-py/src/component_bindings/dimension_collector.rscrates/switchyard-py/src/component_bindings/request_processors.rscrates/switchyard-py/src/component_bindings/response_processors.rscrates/switchyard-py/src/component_bindings/stage_router.rscrates/switchyard-py/src/component_bindings/stats.rscrates/switchyard-py/src/errors.rscrates/switchyard-py/src/interop.rscrates/switchyard-py/src/interop/context.rscrates/switchyard-py/src/interop/request.rscrates/switchyard-py/src/interop/response.rscrates/switchyard-py/src/interop/roles.rscrates/switchyard-py/src/interop/subagent.rscrates/switchyard-py/src/lib.rscrates/switchyard-py/src/libsy_bindings.rscrates/switchyard-py/src/py_serde.rscrates/switchyard-py/src/translation.rscrates/switchyard-server/README.mddocs/cli_reference.mddocs/getting_started.mddocs/internal/metrics_reference.mddocs/routing_algorithms/stage_router_routing.mdexamples/minimal.pyexamples/route.yamlexamples/utils.pypyproject.tomlswitchyard/__init__.pyswitchyard/cli/command_utils.pyswitchyard/cli/launchers/claude_code_launcher.pyswitchyard/cli/launchers/codex_cli_launcher.pyswitchyard/cli/launchers/cost_estimator.pyswitchyard/cli/launchers/launcher_runtime.pyswitchyard/cli/launchers/live_stats_footer.pyswitchyard/cli/launchers/openclaw_launcher.pyswitchyard/cli/launchers/session_summary.pyswitchyard/cli/launchers/shell_tui.pyswitchyard/cli/model_catalog/__init__.pyswitchyard/cli/model_catalog/model_discovery.pyswitchyard/cli/route_bundle.pyswitchyard/cli/switchyard_cli.pyswitchyard/lib/__init__.pyswitchyard/lib/backends/__init__.pyswitchyard/lib/backends/anthropic_native_llm_backend.pyswitchyard/lib/backends/backend_format_resolver.pyswitchyard/lib/backends/llm_target.pyswitchyard/lib/backends/multi_llm_backend.pyswitchyard/lib/backends/openai_llm_backend.pyswitchyard/lib/backends/openai_native_backend.pyswitchyard/lib/backends/stats_llm_backend.pyswitchyard/lib/chat_request/__init__.pyswitchyard/lib/chat_request/anthropic.pyswitchyard/lib/chat_request/base.pyswitchyard/lib/chat_request/openai_chat.pyswitchyard/lib/chat_request/openai_responses.pyswitchyard/lib/chat_response/__init__.pyswitchyard/lib/chat_response/anthropic.pyswitchyard/lib/chat_response/base.pyswitchyard/lib/chat_response/openai_chat.pyswitchyard/lib/chat_response/openai_responses.pyswitchyard/lib/chat_response/streaming_response_accumulator.pyswitchyard/lib/conversation_turn.pyswitchyard/lib/endpoints/__init__.pyswitchyard/lib/endpoints/anthropic_messages_endpoint.pyswitchyard/lib/endpoints/base.pyswitchyard/lib/endpoints/dispatch.pyswitchyard/lib/endpoints/error_envelope.pyswitchyard/lib/endpoints/models_endpoint.pyswitchyard/lib/endpoints/openai_chat_endpoint.pyswitchyard/lib/endpoints/outcome_metrics.pyswitchyard/lib/endpoints/prometheus_emitter.pyswitchyard/lib/endpoints/responses_endpoint.pyswitchyard/lib/endpoints/route_selection.pyswitchyard/lib/endpoints/routing_log_stats_endpoint.pyswitchyard/lib/endpoints/sse_helpers.pyswitchyard/lib/endpoints/stats_endpoint.pyswitchyard/lib/endpoints/upstream_error.pyswitchyard/lib/endpoints/upstream_error_log.pyswitchyard/lib/llm_client.pyswitchyard/lib/model_listing.pyswitchyard/lib/processors/__init__.pyswitchyard/lib/processors/format_translate.pyswitchyard/lib/processors/model_rewrite_request_processor.pyswitchyard/lib/processors/rl_logging_request_processor.pyswitchyard/lib/processors/rl_logging_response_processor.pyswitchyard/lib/processors/routing_log_response_processor.pyswitchyard/lib/processors/stats_request_processor.pyswitchyard/lib/processors/stats_response_processor_accumulator.pyswitchyard/lib/prometheus_exposition.pyswitchyard/lib/proxy_context.pyswitchyard/lib/request_metadata.pyswitchyard/lib/roles.pyswitchyard/lib/route_table.pyswitchyard/lib/startup_timing.pyswitchyard/lib/stats_accumulator.pyswitchyard/lib/switchyard.pyswitchyard/lib/tracing.pyswitchyard/server/__init__.pyswitchyard/server/server_util.pyswitchyard/server/switchyard_app.pyswitchyard/telemetry.pyswitchyard_rust/__init__.pyswitchyard_rust/_native.pyswitchyard_rust/components.pyswitchyard_rust/components.pyiswitchyard_rust/core.pyswitchyard_rust/libsy.pyswitchyard_rust/server.pyswitchyard_rust/translation.pytests/_chain_test_helpers.pytests/conftest.pytests/contract/__init__.pytests/contract/test_platform_imports.pytests/contract/test_proxy_context.pytests/contract/test_request_response_types.pytests/e2e/_helpers.pytests/e2e/conftest.pytests/e2e/test_passthrough_e2e.pytests/e2e/test_passthrough_responses_e2e.pytests/e2e_multiturn_responses.pytests/getting_started/test_getting_started.pytests/readme/test_readme.pytests/test_anthropic_native_llm_backend.pytests/test_anthropic_openai_translation.pytests/test_anthropic_output_config_strip.pytests/test_anthropic_probe.pytests/test_backend_format_resolver.pytests/test_build_and_serve.pytests/test_chat_request.pytests/test_chat_response.pytests/test_cli_reference_docs.pytests/test_codex_multiturn_traces.pytests/test_context_error_translation.pytests/test_context_window_exceeded_endpoint.pytests/test_cost_estimator_gemini.pytests/test_endpoint_state_contract.pytests/test_error_source_annotation.pytests/test_format_translate_processor.pytests/test_inference_e2e.pytests/test_infra.pytests/test_init_all_exports.pytests/test_launchers.pytests/test_live_stats_footer.pytests/test_llm_client.pytests/test_metrics_endpoint.pytests/test_no_stale_module_paths.pytests/test_outcome_metrics.pytests/test_prometheus_emitter.pytests/test_prometheus_exposition.pytests/test_python_server_passthrough.pytests/test_request_metadata.pytests/test_request_translation_engine.pytests/test_request_translation_engine_to_any_of.pytests/test_response_translation_engine.pytests/test_responses_openai_translation.pytests/test_rl_logging.pytests/test_rl_logging_e2e.pytests/test_route_bundle.pytests/test_route_selection_headers.pytests/test_route_table.pytests/test_routing_log_response_processor.pytests/test_shell_tui.pytests/test_sse_stream_close.pytests/test_stats_accumulator.pytests/test_stream_close_chain.pytests/test_stream_leak_repro.pytests/test_switchyard.pytests/test_switchyard_app_factory.pytests/test_switchyard_app_lifecycle.pytests/test_switchyard_rust_component_bindings.pytests/test_switchyard_rust_core_bindings.pytests/test_telemetry.pytests/test_tool_result_signal_collector.pytests/test_tracing.pytests/test_translation_engine_chaos.pytests/test_upstream_error_log.pytests/test_upstream_error_passthrough.pytests/translation/__init__.pytests/translation/test_format_fidelity_contract.py
💤 Files with no reviewable changes (106)
- Cargo.toml
- switchyard/lib/backends/openai_llm_backend.py
- crates/switchyard-components/src/response_processors/mod.rs
- switchyard/lib/init.py
- .agents/skills/switchyard-stage-router-scorer/SKILL.md
- switchyard/lib/chat_request/base.py
- switchyard/lib/chat_request/openai_responses.py
- switchyard/lib/chat_request/anthropic.py
- crates/switchyard-components/src/dimension_collector/mod.rs
- switchyard/lib/backends/anthropic_native_llm_backend.py
- switchyard/lib/chat_response/openai_chat.py
- switchyard/lib/endpoints/routing_log_stats_endpoint.py
- crates/switchyard-components/src/contracts/roles.rs
- switchyard/lib/chat_response/openai_responses.py
- switchyard/lib/chat_response/anthropic.py
- switchyard/cli/model_catalog/model_discovery.py
- switchyard/lib/backends/openai_native_backend.py
- benchmark/README.md
- switchyard/lib/backends/multi_llm_backend.py
- crates/switchyard-components/src/request_processors/dimension_collector.rs
- switchyard/lib/backends/stats_llm_backend.py
- crates/switchyard-py/src/component_bindings/request_processors.rs
- crates/switchyard-py/src/interop/roles.rs
- crates/switchyard-components/src/contracts/context.rs
- crates/switchyard-py/src/interop.rs
- crates/switchyard-components/src/backends/context_overflow.rs
- crates/switchyard-components/Cargo.toml
- crates/switchyard-py/src/component_bindings.rs
- examples/route.yaml
- crates/switchyard-components/src/stats/usage.rs
- crates/switchyard-components/src/backends/stats.rs
- crates/switchyard-py/src/component_bindings/stage_router.rs
- crates/switchyard-py/Cargo.toml
- crates/switchyard-py/src/component_bindings/config.rs
- switchyard/lib/chat_request/openai_chat.py
- crates/switchyard-components/tests/adversarial_native_backends.rs
- crates/switchyard-py/src/interop/request.rs
- crates/switchyard-py/src/lib.rs
- crates/switchyard-components/src/contracts/backend.rs
- switchyard/lib/endpoints/dispatch.py
- crates/switchyard-components/src/backends/multi.rs
- switchyard/cli/model_catalog/init.py
- switchyard/lib/endpoints/responses_endpoint.py
- switchyard/lib/chat_response/base.py
- switchyard/lib/endpoints/error_envelope.py
- crates/switchyard-components/tests/contracts.rs
- crates/switchyard-components/src/response_processors/stats.rs
- crates/switchyard-components/tests/adversarial_random_routing.rs
- crates/switchyard-py/src/interop/subagent.rs
- crates/switchyard-py/src/interop/context.rs
- crates/switchyard-components/src/contracts/error.rs
- crates/switchyard-components/src/dimension_collector/tool_signals.rs
- crates/switchyard-py/src/component_bindings/dimension_collector.rs
- crates/switchyard-components/src/contracts/types.rs
- switchyard/lib/conversation_turn.py
- crates/switchyard-components/src/dimension_collector/response/mod.rs
- switchyard/lib/endpoints/models_endpoint.py
- switchyard/lib/endpoints/route_selection.py
- crates/switchyard-components/src/stats/mod.rs
- switchyard/lib/endpoints/prometheus_emitter.py
- crates/switchyard-components/src/backends/mod.rs
- crates/switchyard-components/tests/support/config.rs
- crates/switchyard-components/src/lib.rs
- switchyard/cli/route_bundle.py
- crates/switchyard-components/src/backends/selection.rs
- switchyard/lib/endpoints/base.py
- crates/switchyard-components/src/stats/accumulator.rs
- examples/utils.py
- crates/switchyard-components/src/response_processors/response_signals.rs
- crates/switchyard-components/src/request_processors/random_routing.rs
- switchyard/lib/chat_request/init.py
- crates/switchyard-components/src/stats/cache_eligibility.rs
- switchyard/lib/endpoints/outcome_metrics.py
- crates/switchyard-py/src/translation.rs
- switchyard/lib/chat_response/init.py
- crates/switchyard-components/src/request_processors/stats.rs
- crates/switchyard-components/tests/adversarial_multi_llm_backend.rs
- benchmark/score_staged_run.py
- switchyard/lib/endpoints/anthropic_messages_endpoint.py
- switchyard/lib/endpoints/openai_chat_endpoint.py
- switchyard/lib/backends/backend_format_resolver.py
- crates/switchyard-py/src/component_bindings/response_processors.rs
- crates/switchyard-components/src/backends/anthropic.rs
- switchyard/lib/endpoints/init.py
- crates/switchyard-components/src/contracts/mod.rs
- crates/switchyard-py/src/component_bindings/stats.rs
- crates/switchyard-components/src/telemetry.rs
- crates/switchyard-components/tests/stats_accumulator.rs
- crates/switchyard-py/src/interop/response.rs
- switchyard/lib/backends/init.py
- crates/switchyard-components/src/stats/context.rs
- crates/switchyard-py/src/component_bindings/backends.rs
- crates/switchyard-components/src/contracts/ids.rs
- crates/switchyard-components/src/stage_router.rs
- switchyard/lib/chat_response/streaming_response_accumulator.py
- crates/switchyard-components/src/backends/common.rs
- examples/minimal.py
- crates/switchyard-components/tests/support/mod.rs
- crates/switchyard-components/src/backends/openai.rs
- crates/switchyard-py/src/py_serde.rs
- crates/switchyard-components/src/dimension_collector/response/checks.rs
- switchyard/lib/backends/llm_target.py
- crates/switchyard-components/src/stats/cost.rs
- crates/switchyard-components/tests/stats_processors.rs
- crates/switchyard-components/src/request_processors/mod.rs
- crates/switchyard-components/tests/stats_usage_shapes.rs
| Everything flows through a fixed-shape chain enforced at construction time: | ||
| The supported serving path is native Rust: | ||
|
|
||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add language identifiers to the new fenced blocks.
markdownlint-cli2 reports MD040 for the architecture and project-structure fences. Add text to both opening fences.
Proposed fix
-```
+```textAlso applies to: 158-158
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)
[warning] 142-142: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@AGENTS.md` at line 142, Update the architecture and project-structure fenced
blocks in AGENTS.md to use the text language identifier on both opening fences,
resolving the markdownlint MD040 violations while leaving their contents
unchanged.
Source: Linters/SAST tools
Comment on lines
+10
to
19
| create_exception!(_switchyard_rust, LibsyError, PyRuntimeError); | ||
|
|
||
| /// Converts libsy execution failures into one stable Python exception. | ||
| pub(crate) fn py_libsy_error(error: impl std::fmt::Display) -> PyErr { | ||
| LibsyError::new_err(error.to_string()) | ||
| } | ||
|
|
||
| /// Converts core Switchyard errors into typed Python runtime errors. | ||
| /// | ||
| /// `ContextWindowExceeded` and `ContextPoolExhausted` carry typed fields | ||
| /// (`target_id`, `model`, `last_target_id`, `reason`) — we attach those as | ||
| /// Python attributes on the raised exception so callers can inspect them | ||
| /// programmatically and `backend_error_with_ctx` can recover the typed | ||
| /// variant when a Python `LLMBackend` re-raises through the Rust chain. | ||
| /// Without these attrs the variant collapses to the string message and the | ||
| /// compatibility retry code stamps `"unknown"` for `target_id`/`model`. | ||
| pub(crate) fn py_core_error(error: SwitchyardError) -> PyErr { | ||
| let message = error.to_string(); | ||
| match error { | ||
| SwitchyardError::InvalidConfig(_) => SwitchyardConfigError::new_err(message), | ||
| SwitchyardError::InvalidId(_) => SwitchyardInvalidIdError::new_err(message), | ||
| SwitchyardError::DuplicateRegistration { .. } => { | ||
| SwitchyardDuplicateRegistrationError::new_err(message) | ||
| } | ||
| SwitchyardError::ModelNotFound { .. } => SwitchyardModelNotFoundError::new_err(message), | ||
| SwitchyardError::UnsupportedRequestType { .. } => { | ||
| SwitchyardUnsupportedRequestTypeError::new_err(message) | ||
| } | ||
| SwitchyardError::InvalidRequest(_) => SwitchyardInvalidRequestError::new_err(message), | ||
| SwitchyardError::Processor(_) => SwitchyardProcessorError::new_err(message), | ||
| SwitchyardError::Backend(_) => SwitchyardBackendError::new_err(message), | ||
| SwitchyardError::Upstream(_) => SwitchyardUpstreamError::new_err(message), | ||
| SwitchyardError::UpstreamHttp { | ||
| status_code, body, .. | ||
| } => { | ||
| let err = SwitchyardUpstreamError::new_err(message); | ||
| attach_upstream_http_attrs(&err, status_code, &body); | ||
| err | ||
| } | ||
| SwitchyardError::ContextWindowExceeded { | ||
| target_id, model, .. | ||
| } => { | ||
| let err = SwitchyardContextWindowExceededError::new_err(message); | ||
| attach_attrs(&err, &[("target_id", &target_id), ("model", &model)]); | ||
| err | ||
| } | ||
| SwitchyardError::ContextPoolExhausted { | ||
| last_target_id, | ||
| reason, | ||
| } => { | ||
| let err = SwitchyardContextPoolExhaustedError::new_err(message); | ||
| attach_attrs( | ||
| &err, | ||
| &[("last_target_id", &last_target_id), ("reason", &reason)], | ||
| ); | ||
| err | ||
| } | ||
| SwitchyardError::Other(_) => SwitchyardRuntimeError::new_err(message), | ||
| } | ||
| } | ||
|
|
||
| /// Set string attributes on a freshly-constructed `PyErr`'s exception value. | ||
| /// Errors are intentionally ignored — attribute attachment is best-effort | ||
| /// diagnostic metadata, never the path-of-correctness for raising the error. | ||
| fn attach_attrs(err: &PyErr, attrs: &[(&str, &str)]) { | ||
| Python::attach(|py| { | ||
| let bound = err.value(py); | ||
| for (name, value) in attrs { | ||
| let _ = bound.setattr(*name, *value); | ||
| } | ||
| }); | ||
| } | ||
|
|
||
| /// Attach typed upstream HTTP details for endpoint error handling. | ||
| fn attach_upstream_http_attrs(err: &PyErr, status_code: u16, body: &str) { | ||
| Python::attach(|py| { | ||
| let bound = err.value(py); | ||
| let _ = bound.setattr("status_code", status_code); | ||
| let _ = bound.setattr("body", body); | ||
| }); | ||
| } | ||
|
|
||
| /// Registers typed exception classes in the native extension module. | ||
| pub(crate) fn register(module: &Bound<'_, PyModule>) -> PyResult<()> { | ||
| let py = module.py(); | ||
| module.add( | ||
| "SwitchyardRuntimeError", | ||
| py.get_type::<SwitchyardRuntimeError>(), | ||
| )?; | ||
| module.add("LibsyError", py.get_type::<LibsyError>())?; | ||
| module.add( | ||
| "SwitchyardConfigError", | ||
| py.get_type::<SwitchyardConfigError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardInvalidIdError", | ||
| py.get_type::<SwitchyardInvalidIdError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardDuplicateRegistrationError", | ||
| py.get_type::<SwitchyardDuplicateRegistrationError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardModelNotFoundError", | ||
| py.get_type::<SwitchyardModelNotFoundError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardUnsupportedRequestTypeError", | ||
| py.get_type::<SwitchyardUnsupportedRequestTypeError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardInvalidRequestError", | ||
| py.get_type::<SwitchyardInvalidRequestError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardProcessorError", | ||
| py.get_type::<SwitchyardProcessorError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardBackendError", | ||
| py.get_type::<SwitchyardBackendError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardUpstreamError", | ||
| py.get_type::<SwitchyardUpstreamError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardContextWindowExceededError", | ||
| py.get_type::<SwitchyardContextWindowExceededError>(), | ||
| )?; | ||
| module.add( | ||
| "SwitchyardContextPoolExhaustedError", | ||
| py.get_type::<SwitchyardContextPoolExhaustedError>(), | ||
| )?; | ||
| Ok(()) | ||
| module.add("LibsyError", module.py().get_type::<LibsyError>()) | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find residual references to removed typed exceptions and check the exported error surface.
# All exception types still created or registered in the PyO3 crate.
rg -nP --type=rust 'create_exception!|add\("(\w*Error)"' crates/switchyard-py/src
# Any Python/stub/doc reference to a Switchyard-specific error name other than LibsyError.
rg -nP -g '!target' -g '*.py' -g '*.pyi' -g '*.md' '\b[A-Z]\w*Error\b' \
| rg -v '\b(LibsyError|RuntimeError|ValueError|TypeError|OSError|KeyError|FileNotFoundError)\b'
# Confirm the declared exception surface in the Python stubs.
fd -t f -e py -e pyi . switchyard_rust --exec rg -n 'Error' {}Repository: NVIDIA-NeMo/Switchyard
Length of output: 483
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- relevant change summary ---'
git status --short
git diff --stat -- crates/switchyard-py/src/errors.rs switchyard_rust crates
printf '%s\n' '--- current error bindings ---'
cat -n crates/switchyard-py/src/errors.rs
rg -n -C 3 'LibsyError|Error' crates/switchyard-py/src/libsy_bindings.rs switchyard_rust 2>/dev/null || true
printf '%s\n' '--- removed exception identifiers from the diff ---'
git diff --unified=80 -- crates/switchyard-py/src/errors.rs crates/switchyard-py/src/libsy_bindings.rs switchyard_rust \
| rg '^[+-].*(create_exception!|Error|exception)' || true
printf '%s\n' '--- repository references to likely typed exceptions ---'
rg -n --hidden -g '!target' -g '!node_modules' \
'(LibsyError|[A-Z][A-Za-z0-9_]*(Error|Exception))' . \
| head -n 500
printf '%s\n' '--- deprecation-plan references ---'
rg -n --hidden -g '!target' -i \
'(deprecat|removed|renamed|typed exception|exception surface)' \
README.md docs crates switchyard_rust 2>/dev/null | head -n 300 || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- commit and error-surface diff ---'
git rev-parse HEAD
git diff --unified=20 HEAD^ HEAD -- crates/switchyard-py/src/errors.rs crates/switchyard-py/src/libsy_bindings.rs switchyard_rust/libsy.py switchyard/libsy/__init__.py
printf '%s\n' '--- previous error binding source ---'
git show HEAD^:crates/switchyard-py/src/errors.rs 2>/dev/null | cat -n || true
printf '%s\n' '--- Python consumers of the binding ---'
rg -n --hidden -g '*.py' -g '*.pyi' -g '*.md' \
'from (switchyard|switchyard_rust)(\.[A-Za-z0-9_]+)* import|import switchyard(_rust)?|LibsyError|NoTargets|TargetNotFound|AlgorithmError|DriverError|MissingFinalResponse|AllTargetsExcluded' \
. | head -n 400
printf '%s\n' '--- public package exports and documentation ---'
rg -n --hidden -g '*.py' -g '*.pyi' -g '*.md' \
'(__all__|LibsyError|NoTargets|TargetNotFound|AlgorithmError|DriverError|MissingFinalResponse|AllTargetsExcluded)' \
switchyard switchyard_rust docs README.md crates/switchyard-py 2>/dev/null | head -n 400 || true
printf '%s\n' '--- likely deprecation records ---'
rg -n --hidden -g '!target' -i \
'(deprecat|breaking change|migration|removed exception|typed exception)' \
CHANGELOG* docs README.md switchyard switchyard_rust crates 2>/dev/null | head -n 300 || true
printf '%s\n' '--- exact current exception declarations and references ---'
python3 - <<'PY'
from pathlib import Path
import re
root = Path(".")
declared = []
for path in root.rglob("*"):
if not path.is_file() or "target" in path.parts or ".git" in path.parts:
continue
try:
text = path.read_text()
except (UnicodeDecodeError, OSError):
continue
for match in re.finditer(r"create_exception!\([^,]+,\s*([A-Za-z_]\w*)", text):
declared.append((match.group(1), str(path)))
print("declared:", declared)
for name, path in declared:
refs = []
pattern = re.compile(rf"\b{re.escape(name)}\b")
for candidate in root.rglob("*"):
if not candidate.is_file() or "target" in candidate.parts or ".git" in candidate.parts:
continue
try:
content = candidate.read_text()
except (UnicodeDecodeError, OSError):
continue
if pattern.search(content):
refs.append(str(candidate))
print(name, "references:", sorted(set(refs)))
PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 32265
Add a deprecation plan for the removed PyO3 exception exports.
The previous binding exported SwitchyardRuntimeError and the typed Switchyard*Error classes. No repository consumer references them, but no deprecation or migration record exists. Add the required plan before removing these public exports.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/switchyard-py/src/errors.rs` around lines 10 - 19, Add a deprecation
and migration record for the removed PyO3 exports, covering
SwitchyardRuntimeError and the typed Switchyard*Error classes, before finalizing
the new LibsyError-only API in register and py_libsy_error. Document the
replacement behavior and migration path using the repository’s established
deprecation-plan location and format.
Source: Coding guidelines
Comment on lines
+168
to
+169
| decoded, so it measures request ingress through response completion. It still excludes connection | ||
| accept and TLS handshake, which hyper completes before the server sees the request. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- README context ---'
sed -n '145,190p' crates/switchyard-server/README.md
printf '%s\n' '--- latency references ---'
rg -n -C 3 'switchyard_total_latency_ms|stream|SSE|total_latency|response completion|connection accept|connection acceptance' cratesRepository: NVIDIA-NeMo/Switchyard
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- README lines 155-185 ---'
sed -n '155,185p' crates/switchyard-server/README.md
printf '%s\n' '--- server latency symbols ---'
rg -n -C 5 'switchyard_total_latency_ms|latency_ms|stream.*(start|end)|SSE|stream handle|response completion|connection accept' crates/switchyard-server
printf '%s\n' '--- server source files ---'
git ls-files crates/switchyard-server | sed -n '1,120p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 39763
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- latency call sites ---'
rg -n -C 8 'record_usage|record_latency|total_latency|Instant::now|elapsed\(\)|RunObservation|ResponseBody|Sse<|frame_stream|stream handle' crates/switchyard-server/src crates/switchyard-server/tests
printf '%s\n' '--- lib outline ---'
ast-grep outline crates/switchyard-server/src/lib.rs
printf '%s\n' '--- usage metrics ---'
cat -n crates/switchyard-server/src/usage_metrics.rs | sed -n '1,190p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -e
set -o pipefail
printf '%s\n' '--- usage_metrics stream wrapper ---'
cat -n crates/switchyard-server/src/usage_metrics.rs | sed -n '16,115p'
printf '%s\n' '--- endpoint flow ---'
cat -n crates/switchyard-server/src/lib.rs | sed -n '445,625p'
printf '%s\n' '--- routed request flow ---'
cat -n crates/switchyard-server/src/lib.rs | sed -n '677,745p'
printf '%s\n' '--- SSE framing ---'
cat -n crates/switchyard-server/src/sse.rs | sed -n '13,75p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 17550
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
usage = Path("crates/switchyard-server/src/usage_metrics.rs").read_text()
lib = Path("crates/switchyard-server/src/lib.rs").read_text()
observe = usage[usage.index("LlmResponse::Stream(mut stream)"):usage.index("\n };\n\n Response", usage.index("LlmResponse::Stream(mut stream)"))]
assert "while let Some(item) = stream.next().await" in observe
assert "record_terminal(&stats, &usage, &model, started, cache_eligible);" in observe
assert observe.index("while let Some(item)") < observe.index("record_terminal(&stats, &usage")
endpoint = lib[lib.index("let response = render_error_response"):lib.index("\n response", lib.index("let response = render_error_response"))]
assert endpoint.index("request_log.emit(&response)") < endpoint.index("response")
record_terminal = usage[usage.index("fn record_terminal"):usage.index("\n}\n\nfn attributes", usage.index("fn record_terminal"))]
assert "started.elapsed()" in record_terminal
print("switchyard_total_latency_ms: recorded after streamed source completion")
print("request log latency: recorded after response construction, before handler return")
PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 249
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
usage = Path("crates/switchyard-server/src/usage_metrics.rs").read_text()
lib = Path("crates/switchyard-server/src/lib.rs").read_text()
stream_start = usage.index("LlmResponse::Stream(mut stream)")
stream_end = usage.index("\n };\n\n let llm_response", stream_start) if "\n };\n\n let llm_response" in usage[stream_start:] else usage.index("\n };\n\n LlmResponse::Stream", stream_start)
stream = usage[stream_start:stream_end]
assert "while let Some(item) = stream.next().await" in stream
assert "record_terminal(&stats, &usage, &model, started, cache_eligible);" in stream
assert stream.index("while let Some(item)") < stream.index("record_terminal(&stats, &usage")
emit = "request_log.emit(&response)"
response_return = "response\n}"
emit_pos = lib.index(emit, lib.index("async fn handle_endpoint_inner"))
return_pos = lib.index(response_return, emit_pos)
assert emit_pos < return_pos
terminal_start = usage.index("fn record_terminal")
terminal_end = usage.index("\n}\n\nfn attributes", terminal_start)
terminal = usage[terminal_start:terminal_end]
assert "started.elapsed()" in terminal
print("switchyard_total_latency_ms: recorded after streamed source completion")
print("request log latency: recorded after response construction, before handler return")
PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 265
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
from pathlib import Path
usage_lines = Path("crates/switchyard-server/src/usage_metrics.rs").read_text().splitlines()
lib_lines = Path("crates/switchyard-server/src/lib.rs").read_text().splitlines()
def line_number(lines, text, start=0):
for n, line in enumerate(lines[start:], start + 1):
if text in line:
return n
raise SystemExit(f"missing source text: {text}")
stream_line = line_number(usage_lines, "LlmResponse::Stream(mut stream)")
poll_line = line_number(usage_lines, "while let Some(item) = stream.next().await", stream_line - 1)
terminal_line = line_number(
usage_lines,
"record_terminal(&stats, &usage, &model, started, cache_eligible);",
poll_line - 1,
)
elapsed_line = line_number(usage_lines, "let total_latency = started.elapsed();")
inner_line = line_number(lib_lines, "async fn handle_endpoint_inner")
emit_line = line_number(lib_lines, "request_log.emit(&response)", inner_line - 1)
return_line = line_number(lib_lines, " response", emit_line - 1)
assert stream_line <= poll_line < terminal_line
assert elapsed_line > 0
assert inner_line < emit_line < return_line
print(f"stream polling begins at usage_metrics.rs:{poll_line}")
print(f"total latency is recorded at usage_metrics.rs:{terminal_line}, after polling")
print(f"request log latency is emitted at lib.rs:{emit_line}, before handler return at lib.rs:{return_line}")
PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 351
Correct the streaming latency definition in lines 178–180.
switchyard_total_latency_ms is recorded after the wrapped stream ends. Do not state that both clocks stop when the stream handle arrives. Keep the response-completion description consistent with stream completion, and use connection acceptance instead of connection accept.
🧰 Tools
🪛 LanguageTool
[grammar] ~169-~169: Use a hyphen to join words.
Context: ...on accept and TLS handshake, which hyper completes before the server sees the req...
(QB_NEW_EN_HYPHEN)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/switchyard-server/README.md` around lines 168 - 169, Update the README
latency definition around switchyard_total_latency_ms to state that timing ends
after the wrapped stream completes, not when the stream handle arrives. Keep the
response-completion wording consistent with stream completion and replace
“connection accept” with “connection acceptance.”
Comment on lines
+154
to
+157
| Sweep a few candidate thresholds in representative benchmark runs. Choose the | ||
| lowest threshold that rescues the RESCUE quadrant without over-escalating the | ||
| LOSS quadrant. Because the scorer is corroborative, a `0.5` threshold takes | ||
| roughly 1.5 signals of agreement. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Calibrate confidence_threshold separately for each picker.
The current rule is not valid for both picker modes. With capable_first, lowering the threshold sends more turns to efficient; RESCUE is a beneficial downgrade, while LOSS is a harmful downgrade. With efficient_first, lowering the threshold sends more turns to capable; LOSS is the case that benefits from escalation. Replace the universal “lowest threshold” rule with picker-specific guidance.
Proposed direction
-Sweep a few candidate thresholds in representative benchmark runs. Choose the
-lowest threshold that rescues the RESCUE quadrant without over-escalating the
-LOSS quadrant.
+Sweep candidate thresholds separately for each picker. With `capable_first`,
+balance beneficial RESCUE downgrades against harmful LOSS downgrades. With
+`efficient_first`, tune escalation for LOSS cases while limiting unnecessary
+escalation. Treat `0.5` as a starting point, not a picker-independent rule.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| Sweep a few candidate thresholds in representative benchmark runs. Choose the | |
| lowest threshold that rescues the RESCUE quadrant without over-escalating the | |
| LOSS quadrant. Because the scorer is corroborative, a `0.5` threshold takes | |
| roughly 1.5 signals of agreement. | |
| Sweep candidate thresholds separately for each picker. With `capable_first`, | |
| balance beneficial RESCUE downgrades against harmful LOSS downgrades. With | |
| `efficient_first`, tune escalation for LOSS cases while limiting unnecessary | |
| escalation. Treat `0.5` as a starting point, not a picker-independent rule. | |
| Because the scorer is corroborative, a `0.5` threshold takes roughly 1.5 signals of agreement. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/routing_algorithms/stage_router_routing.md` around lines 154 - 157,
Replace the universal lowest-threshold guidance in the benchmark calibration
section with picker-specific rules: for capable_first, lower
confidence_threshold only enough to rescue RESCUE without over-escalating
harmful LOSS downgrades; for efficient_first, calibrate it to increase
escalation for beneficial LOSS cases while avoiding unnecessary RESCUE changes.
Keep the corroborative scorer context and require selecting the best threshold
independently for each picker mode.
| "TranslationEngine", | ||
| "is_native_translation_available", | ||
| ] | ||
| """Python wrappers for Switchyard's native libsy and server bindings.""" |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find root-package imports of switchyard_rust that no longer resolve.
# Direct "from switchyard_rust import X" usages (submodule imports are excluded by the dot check below).
rg -nP -g '*.py' -g '*.pyi' -g '*.md' -g '*.rst' -g '*.toml' 'from\s+switchyard_rust\s+import\s+'
# Attribute access on the package object, e.g. switchyard_rust.SomeName
rg -nP -g '*.py' -g '*.pyi' -g '*.md' '\bswitchyard_rust\.(?!libsy\b|server\b|_native\b)\w+'
# Show what the sibling modules actually export.
fd -t f -e py . switchyard_rust --exec rg -n '^__all__|^def |^class ' {}Repository: NVIDIA-NeMo/Switchyard
Length of output: 160
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- repository files ---'
git ls-files | sed -n '1,160p'
printf '%s\n' '--- package files ---'
fd -t f . switchyard_rust 2>/dev/null || true
printf '%s\n' '--- root imports and attribute access across tracked text files ---'
rg -n --hidden -g '! .git' \
-e 'from[[:space:]]+switchyard_rust[[:space:]]+import' \
-e 'import[[:space:]]+switchyard_rust' \
-e '\bswitchyard_rust\.[A-Za-z_][A-Za-z0-9_]*' . || true
printf '%s\n' '--- package initializer and sibling exports ---'
if [ -f switchyard_rust/__init__.py ]; then
cat -n switchyard_rust/__init__.py
fi
for f in switchyard_rust/*.py switchyard_rust/*.pyi; do
[ -f "$f" ] || continue
printf '%s\n' "--- $f ---"
cat -n "$f" | sed -n '1,220p'
doneRepository: NVIDIA-NeMo/Switchyard
Length of output: 16315
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- relevant diff ---'
git diff -- switchyard_rust/__init__.py switchyard_rust/libsy.py switchyard_rust/server.py \
.github/workflows/publish.yml .github/workflows/package-portability.yml || true
printf '%s\n' '--- prior initializer, if available ---'
git show HEAD^:switchyard_rust/__init__.py 2>/dev/null | cat -n || true
printf '%s\n' '--- exact root-import contexts ---'
rg -n -C 3 \
-e 'from[[:space:]]+switchyard_rust[[:space:]]+import' \
-e 'import[[:space:]]+switchyard_rust' \
-g '*.py' -g '*.pyi' -g '*.md' -g '*.rst' -g '*.toml' -g '*.yml' -g '*.yaml' . || true
printf '%s\n' '--- public names exported by package initializer ---'
python3 - <<'PY'
import ast
from pathlib import Path
path = Path("switchyard_rust/__init__.py")
tree = ast.parse(path.read_text())
public = []
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
if not node.name.startswith("_"):
public.append(node.name)
elif isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and not target.id.startswith("_"):
public.append(target.id)
elif isinstance(node, ast.AnnAssign):
if isinstance(node.target, ast.Name) and not node.target.id.startswith("_"):
public.append(node.target.id)
elif isinstance(node, ast.ImportFrom):
public.extend(
alias.asname or alias.name
for alias in node.names
if not (alias.asname or alias.name).startswith("_")
)
print("public_initializer_bindings:", sorted(set(public)))
PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 7700
🏁 Script executed:
#!/bin/bash
set -u
python3 - <<'PY'
import ast
import subprocess
from pathlib import Path
old_text = subprocess.run(
["git", "show", "HEAD^:switchyard_rust/__init__.py"],
check=True,
capture_output=True,
text=True,
).stdout
old_tree = ast.parse(old_text)
old_all = []
for node in old_tree.body:
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id == "__all__":
old_all = [elt.value for elt in node.value.elts if isinstance(elt, ast.Constant)]
current_text = Path("switchyard_rust/__init__.py").read_text()
current_tree = ast.parse(current_text)
current_all = []
for node in current_tree.body:
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id == "__all__":
current_all = [elt.value for elt in node.value.elts if isinstance(elt, ast.Constant)]
removed = sorted(set(old_all) - set(current_all))
print("previous_root_all:", old_all)
print("current_root_all:", current_all)
print("removed_root_exports:", removed)
root_imports = []
attribute_uses = []
for raw in subprocess.run(
["git", "ls-files", "*.py", "*.pyi"],
check=True,
capture_output=True,
text=True,
).stdout.splitlines():
path = Path(raw)
try:
tree = ast.parse(path.read_text(), filename=str(path))
except SyntaxError:
continue
for node in ast.walk(tree):
if isinstance(node, ast.ImportFrom) and node.module == "switchyard_rust":
for alias in node.names:
if alias.name in removed:
root_imports.append((str(path), node.lineno, alias.name))
if isinstance(node, ast.Attribute) and node.attr in removed:
value = node.value
if isinstance(value, ast.Name) and value.id == "switchyard_rust":
attribute_uses.append((str(path), node.lineno, node.attr))
print("removed_root_imports:", root_imports)
print("direct_removed_attribute_uses:", attribute_uses)
PY
printf '%s\n' '--- deprecation or migration references ---'
rg -n -i 'deprecat|backward.compat|breaking.change|migration|package root|root import|switchyard_rust\.(components|core|translation)' \
CHANGELOG.md README.md INSTALLATION.md CONTRIBUTING.md AGENTS.md .github switchyard_rust switchyard tests pyproject.toml \
2>/dev/null | sed -n '1,240p' || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 2547
Add an explicit deprecation plan before removing the package-root exports. The previous root exported 34 public names. No current callers use them, but their removal is still a public Python API reduction.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard_rust/__init__.py` at line 4, Before removing the package-root
exports from switchyard_rust, add an explicit deprecation plan for the 34
previously public names. Preserve the current exports during the deprecation
period and document the planned removal timeline and migration path for callers.
Source: Coding guidelines
Comment on lines
+4
to
+14
| """Switchyard's Python launcher and libsy bindings.""" | ||
|
|
||
| from importlib import metadata as _metadata | ||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| from switchyard.lib.backends import ( | ||
| AnthropicNativeBackend, | ||
| OpenAiNativeBackend, | ||
| ) | ||
| from switchyard.lib.backends.llm_target import ( | ||
| BackendFormat, | ||
| LlmTarget, | ||
| ) | ||
| from switchyard.lib.chat_request import ( | ||
| AnthropicChatRequest, | ||
| OpenAIChatRequest, | ||
| ResponsesChatRequest, | ||
| ) | ||
| from switchyard.lib.chat_response import ( | ||
| AnthropicChatResponse, | ||
| AnthropicResponseStream, | ||
| AnthropicStreamingChatResponse, | ||
| AnyResponseStream, | ||
| CompletionChatResponse, | ||
| ResponsesApiChatResponse, | ||
| ResponsesApiStream, | ||
| ResponsesApiStreamingChatResponse, | ||
| ResponseStream, | ||
| StreamingChatResponse, | ||
| ) | ||
| from switchyard.lib.processors.rl_logging_request_processor import RlLoggingRequestProcessor | ||
| from switchyard.lib.processors.rl_logging_response_processor import RlLoggingResponseProcessor | ||
| from switchyard.lib.request_metadata import RequestMetadata | ||
| from switchyard.lib.roles import ( | ||
| LLMBackend, | ||
| ) | ||
| from switchyard.lib.route_table import RouteTable | ||
| from switchyard.lib.switchyard import Switchyard | ||
| from switchyard_rust.components import RandomRoutingProcessorConfig | ||
| from switchyard_rust.core import ( | ||
| ChatRequest, | ||
| ChatRequestType, | ||
| ChatResponse, | ||
| ChatResponseType, | ||
| ) | ||
| from switchyard_rust.translation import TranslationEngine | ||
|
|
||
| if TYPE_CHECKING: | ||
| from switchyard.lib.endpoints.anthropic_messages_endpoint import ( | ||
| AnthropicMessagesEndpoint, | ||
| ) | ||
| from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint | ||
| from switchyard.lib.endpoints.openai_chat_endpoint import ( | ||
| OpenAIChatEndpoint, | ||
| ) | ||
| from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint | ||
| from switchyard.server.switchyard_app import build_switchyard_app | ||
|
|
||
|
|
||
| def __getattr__(name: str) -> Any: | ||
| """Lazy-load optional server exports that require the ``server`` extra.""" | ||
| if name == "OpenAIChatEndpoint": | ||
| from switchyard.lib.endpoints.openai_chat_endpoint import ( | ||
| OpenAIChatEndpoint, | ||
| ) | ||
|
|
||
| return OpenAIChatEndpoint | ||
| if name == "AnthropicMessagesEndpoint": | ||
| from switchyard.lib.endpoints.anthropic_messages_endpoint import ( | ||
| AnthropicMessagesEndpoint, | ||
| ) | ||
|
|
||
| return AnthropicMessagesEndpoint | ||
| if name == "ResponsesEndpoint": | ||
| from switchyard.lib.endpoints.responses_endpoint import ResponsesEndpoint | ||
|
|
||
| return ResponsesEndpoint | ||
| if name == "ModelsEndpoint": | ||
| from switchyard.lib.endpoints.models_endpoint import ModelsEndpoint | ||
|
|
||
| return ModelsEndpoint | ||
| if name == "build_switchyard_app": | ||
| from switchyard.server.switchyard_app import build_switchyard_app | ||
|
|
||
| return build_switchyard_app | ||
| raise AttributeError(f"module {__name__!r} has no attribute {name!r}") | ||
|
|
||
|
|
||
| __all__ = [ | ||
| # ChatRequest types | ||
| "AnthropicChatRequest", | ||
| "ChatRequest", | ||
| "ChatRequestType", | ||
| "OpenAIChatRequest", | ||
| "ResponsesChatRequest", | ||
| # Chain infrastructure | ||
| "Switchyard", | ||
| "LLMBackend", | ||
| "AnthropicNativeBackend", | ||
| "OpenAiNativeBackend", | ||
| "OpenAIChatEndpoint", | ||
| "AnthropicMessagesEndpoint", | ||
| "ResponsesEndpoint", | ||
| "ModelsEndpoint", | ||
| "build_switchyard_app", | ||
| # Route dispatch table | ||
| "RouteTable", | ||
| "RequestMetadata", | ||
| "RlLoggingRequestProcessor", | ||
| "RlLoggingResponseProcessor", | ||
| # Random Routing usage case | ||
| "BackendFormat", | ||
| "RandomRoutingProcessorConfig", | ||
| "LlmTarget", | ||
| # Deterministic (LLM-classifier) routing usage case | ||
| # Translation engine | ||
| "TranslationEngine", | ||
| # ChatResponse types | ||
| "AnthropicChatResponse", | ||
| "ChatResponse", | ||
| "ChatResponseType", | ||
| "CompletionChatResponse", | ||
| "StreamingChatResponse", | ||
| "ResponsesApiChatResponse", | ||
| "ResponsesApiStreamingChatResponse", | ||
| "AnthropicStreamingChatResponse", | ||
| "ResponseStream", | ||
| "ResponsesApiStream", | ||
| "AnthropicResponseStream", | ||
| "AnyResponseStream", | ||
| ] | ||
|
|
||
| try: | ||
| __version__ = _metadata.version("nemo-switchyard") | ||
| except _metadata.PackageNotFoundError: | ||
| # A source checkout may not have installed distribution metadata. | ||
| __version__ = "0.0.0+unknown" | ||
|
|
||
| __all__ = ["__version__"] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed files ---'
git diff --stat
git diff --name-status
printf '%s\n' '--- relevant file inventory ---'
git ls-files \
'switchyard/__init__.py' \
'switchyard/cli/command_utils.py' \
'switchyard_rust/libsy.py' \
'pyproject.toml' \
'setup.cfg' \
'mypy.ini' \
'docs/**' \
'tests/**' | sed -n '1,240p'
printf '%s\n' '--- package initializer ---'
cat -n switchyard/__init__.py
printf '%s\n' '--- command utilities ---'
cat -n switchyard/cli/command_utils.py | sed -n '1,180p'
printf '%s\n' '--- Rust binding module ---'
cat -n switchyard_rust/libsy.py | sed -n '1,180p'
printf '%s\n' '--- public symbol references ---'
rg -n --hidden \
-g '!\\.git/**' \
-e 'quiet_dependency_loggers' \
-e 'SwitchyardRuntimeError' \
-e 'LibsyError' \
-e 'from switchyard import|import switchyard' \
-e '__all__' .Repository: NVIDIA-NeMo/Switchyard
Length of output: 23947
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- repository state ---'
git status --short
git log --oneline -5
printf '%s\n' '--- parent versions of the affected files ---'
for file in switchyard/__init__.py switchyard/cli/command_utils.py switchyard_rust/libsy.py; do
printf '\n--- %s at HEAD^ ---\n' "$file"
git show "HEAD^:$file" 2>&1 | sed -n '1,180p' || true
done
printf '%s\n' '--- package-level and submodule exports ---'
for file in switchyard/libsy/__init__.py switchyard/cli/launch_command.py pyproject.toml docs/reference/rust_api.md CONTRIBUTING.md; do
if test -f "$file"; then
printf '\n--- %s ---\n' "$file"
cat -n "$file" | sed -n '1,220p'
fi
done
printf '%s\n' '--- historical declarations of the named symbols ---'
git grep -n -E 'quiet_dependency_loggers|SwitchyardRuntimeError|LibsyError' HEAD^ -- \
switchyard switchyard_rust crates tests docs 2>/dev/null | sed -n '1,240p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 47483
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- commit metadata ---'
git show -s --format=fuller HEAD
printf '%s\n' '--- affected commit diff ---'
git diff --find-renames HEAD^ HEAD -- \
switchyard/__init__.py \
switchyard/cli/command_utils.py \
switchyard/cli/switchyard_cli.py \
switchyard_rust/libsy.py \
switchyard_rust/core.py \
switchyard_rust/__init__.py \
crates/switchyard-py/src/errors.rs \
docs \
CHANGELOG.md
printf '%s\n' '--- current Rust Python exports and exception declarations ---'
for file in switchyard_rust/__init__.py switchyard_rust/core.py crates/switchyard-py/src/errors.rs switchyard/cli/switchyard_cli.py; do
printf '\n--- %s ---\n' "$file"
cat -n "$file" | sed -n '1,220p'
done
printf '%s\n' '--- release and migration references ---'
rg -n --hidden -g '!\\.git/**' -i \
-e 'deprecated|deprecat|breaking|migration|compatib|server stack|0\\.2\\.0|quiet_dependency_loggers|SwitchyardRuntimeError' \
README.md CHANGELOG.md docs CONTRIBUTING.md pyproject.toml switchyard switchyard_rust crates .github 2>/dev/null | sed -n '1,260p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 50380
Add an explicit migration plan for the removed Python APIs.
This commit removes package-level exports, quiet_dependency_loggers, and the SwitchyardRuntimeError base of LibsyError. Existing imports and exception handlers can fail. Retain compatibility shims through a documented removal release, or document this as a breaking release with symbol-level replacement guidance in CHANGELOG.md.
📍 Affects 3 files
switchyard/__init__.py#L4-L14(this comment)switchyard/cli/command_utils.py#L16-L16switchyard_rust/libsy.py#L43-L43
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/__init__.py` around lines 4 - 14, Document this as a breaking
release in CHANGELOG.md, including symbol-level migration guidance for the
removed package-level exports, quiet_dependency_loggers, and
SwitchyardRuntimeError base. Cover switchyard/__init__.py,
switchyard/cli/command_utils.py, and switchyard_rust/libsy.py; no direct code
changes are required at these sites unless you choose to retain compatibility
shims through a documented removal release.
Source: Coding guidelines
Comment on lines
88
to
91
| def silence_launch_loggers(*, local_logger: logging.Logger) -> None: | ||
| """Keep dependency chatter out of a child process terminal UI.""" | ||
| for noisy in ( | ||
| "switchyard", | ||
| "httpx", | ||
| "httpcore", | ||
| "openai", | ||
| "anthropic", | ||
| ): | ||
| logging.getLogger(noisy).setLevel(logging.WARNING) | ||
| logging.getLogger("switchyard").setLevel(logging.WARNING) | ||
| local_logger.setLevel(logging.INFO) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Dependency log records can still reach stderr and corrupt the terminal UI.
silence_launch_loggers now sets the level only on the switchyard logger. Records from httpx, httpcore, openai, and anthropic still propagate to the root logger. configure_debug_file_logging removes every root handler at lines 67-69, so Python falls back to logging.lastResort, which writes to stderr at WARNING and above. A single dependency warning then prints over the PTY terminal UI. The removed dependency logger levels previously prevented this.
The docstring also states the function keeps dependency chatter out of the terminal UI, but the body no longer configures any dependency logger.
Attach a logging.NullHandler to the root logger, or raise the root level, so the last-resort handler never writes to stderr. This keeps the change small and does not restore per-dependency configuration.
🔧 Proposed fix to stop last-resort output
Apply in configure_debug_file_logging, after the root handlers are cleared:
root = logging.getLogger()
for handler in root.handlers[:]:
root.removeHandler(handler)
handler.close()
root.setLevel(logging.WARNING)
+ # Without a root handler, logging falls back to lastResort and writes
+ # dependency warnings to stderr, which corrupts the PTY terminal UI.
+ root.addHandler(logging.NullHandler())Then align the docstring with the new scope:
def silence_launch_loggers(*, local_logger: logging.Logger) -> None:
- """Keep dependency chatter out of a child process terminal UI."""
+ """Lower switchyard log output so it does not disturb the terminal UI."""
logging.getLogger("switchyard").setLevel(logging.WARNING)
local_logger.setLevel(logging.INFO)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/cli/launchers/launcher_runtime.py` around lines 88 - 91, Update
configure_debug_file_logging to attach a logging.NullHandler to the root logger
immediately after clearing its handlers, preventing logging.lastResort from
writing dependency warnings to stderr. Revise silence_launch_loggers’ docstring
to accurately describe the terminal-logging behavior and new scope.
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci.yml (1)
216-227: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winValidate the top-level command allowlist.
The current checks reject selected removed commands but do not prove that
switchyard --helpexposes onlylaunch. An additional stale top-level command can pass both checks.
.github/workflows/ci.yml#L216-L227: runswitchyard --helpand assert thatlaunchis the only available subcommand.tests/test_cli_reference_docs.py#L50-L56: assert that_subparsers(_build_parser())has exactly{"launch"}.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 216 - 227, The CLI validation must prove that the top-level command allowlist contains only launch. In .github/workflows/ci.yml lines 216-227, add a switchyard --help check that confirms launch is the sole available subcommand; in tests/test_cli_reference_docs.py lines 50-56, assert that _subparsers(_build_parser()) equals {"launch"}.
🧹 Nitpick comments (1)
crates/switchyard-py/src/libsy_bindings.rs (1)
31-39: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider naming the failing header in the error message.
The
httpcrate errors are generic, for example "invalid HTTP header name". A caller that passes many headers cannot tell which entry failed. Include the header name in the message. Do not include the header value, because header values often carry API keys.
try_appendis safe here. The input is aHashMap, so keys are unique, andtry_appendfails only when the map exceeds its maximum size.♻️ Proposed error-context improvement
for (name, value) in headers { - let name = HeaderName::from_bytes(name.as_bytes()) - .map_err(|error| PyValueError::new_err(error.to_string()))?; - let value = HeaderValue::from_str(value) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + let header_name = HeaderName::from_bytes(name.as_bytes()) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; + // Never include the header value in the error: values can carry credentials. + let value = HeaderValue::from_str(value) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; result - .try_append(name, value) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + .try_append(header_name, value) + .map_err(|error| PyValueError::new_err(format!("header {name}: {error}")))?; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/switchyard-py/src/libsy_bindings.rs` around lines 31 - 39, Update the header parsing errors in the loop over headers to include the failing header name alongside the underlying error message, for both HeaderName::from_bytes and HeaderValue::from_str failures. Do not include the header value, and preserve the existing try_append handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Line 142: Update the architecture and project-structure fenced blocks in
AGENTS.md to use the text language identifier on both opening fences, resolving
the markdownlint MD040 violations while leaving their contents unchanged.
In `@crates/switchyard-py/src/errors.rs`:
- Around line 10-19: Add a deprecation and migration record for the removed PyO3
exports, covering SwitchyardRuntimeError and the typed Switchyard*Error classes,
before finalizing the new LibsyError-only API in register and py_libsy_error.
Document the replacement behavior and migration path using the repository’s
established deprecation-plan location and format.
In `@crates/switchyard-server/README.md`:
- Around line 168-169: Update the README latency definition around
switchyard_total_latency_ms to state that timing ends after the wrapped stream
completes, not when the stream handle arrives. Keep the response-completion
wording consistent with stream completion and replace “connection accept” with
“connection acceptance.”
In `@docs/routing_algorithms/stage_router_routing.md`:
- Around line 154-157: Replace the universal lowest-threshold guidance in the
benchmark calibration section with picker-specific rules: for capable_first,
lower confidence_threshold only enough to rescue RESCUE without over-escalating
harmful LOSS downgrades; for efficient_first, calibrate it to increase
escalation for beneficial LOSS cases while avoiding unnecessary RESCUE changes.
Keep the corroborative scorer context and require selecting the best threshold
independently for each picker mode.
In `@switchyard_rust/__init__.py`:
- Line 4: Before removing the package-root exports from switchyard_rust, add an
explicit deprecation plan for the 34 previously public names. Preserve the
current exports during the deprecation period and document the planned removal
timeline and migration path for callers.
In `@switchyard/__init__.py`:
- Around line 4-14: Document this as a breaking release in CHANGELOG.md,
including symbol-level migration guidance for the removed package-level exports,
quiet_dependency_loggers, and SwitchyardRuntimeError base. Cover
switchyard/__init__.py, switchyard/cli/command_utils.py, and
switchyard_rust/libsy.py; no direct code changes are required at these sites
unless you choose to retain compatibility shims through a documented removal
release.
In `@switchyard/cli/launchers/launcher_runtime.py`:
- Around line 88-91: Update configure_debug_file_logging to attach a
logging.NullHandler to the root logger immediately after clearing its handlers,
preventing logging.lastResort from writing dependency warnings to stderr. Revise
silence_launch_loggers’ docstring to accurately describe the terminal-logging
behavior and new scope.
In `@switchyard/cli/launchers/shell_tui.py`:
- Around line 172-175: Snapshot a bounded footer height once per paint in the
shell TUI layout flow, and use that same value for footer rendering, shell row
calculation, scroll-region setup, and child PTY sizing. When the snapshot
differs from the previous layout, update TIOCSWINSZ and the scroll region
immediately rather than waiting for an outer-terminal resize. Add a regression
test that changes footer height between paints and verifies the child PTY and
rendered layout stay synchronized.
- Around line 388-392: Update the stdin forwarding loop around os.read and
master_fd so partial writes and BlockingIOError from os.write are handled
without losing bytes. Maintain a pending-output buffer, append newly read stdin
data, and monitor master_fd for writability until all buffered bytes are
written, while preserving the existing EOF break behavior.
- Around line 316-419: Refactor ShellTUI.run and its blocking event loop to an
async lifecycle, replacing the background footer thread and blocking
select/waits with asyncio-compatible tasks and awaits while preserving terminal
setup, child I/O, cleanup, and exit-code behavior. Update synchronous launcher
boundaries that invoke run to call the async lifecycle through asyncio.run(),
and adjust related lifecycle helpers to remain consistently awaitable.
---
Outside diff comments:
In @.github/workflows/ci.yml:
- Around line 216-227: The CLI validation must prove that the top-level command
allowlist contains only launch. In .github/workflows/ci.yml lines 216-227, add a
switchyard --help check that confirms launch is the sole available subcommand;
in tests/test_cli_reference_docs.py lines 50-56, assert that
_subparsers(_build_parser()) equals {"launch"}.
---
Nitpick comments:
In `@crates/switchyard-py/src/libsy_bindings.rs`:
- Around line 31-39: Update the header parsing errors in the loop over headers
to include the failing header name alongside the underlying error message, for
both HeaderName::from_bytes and HeaderValue::from_str failures. Do not include
the header value, and preserve the existing try_append handling.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8b113f39-f5b8-46c9-9c29-1081242baa7a
⛔ Files ignored due to path filters (2)
Cargo.lockis excluded by!**/*.lock,!Cargo.lockuv.lockis excluded by!**/*.lock,!uv.lock
📒 Files selected for processing (239)
.agents/skills/switchyard-stage-router-scorer/SKILL.md.github/ISSUE_TEMPLATE/bug_report.md.github/workflows/ci.yml.github/workflows/perf.ymlAGENTS.mdCHANGELOG.mdCargo.tomlINSTALLATION.mdREADME.mdbenchmark/README.mdbenchmark/score_staged_run.pycrates/libsy-llm-client/src/client.rscrates/libsy-llm-client/src/error.rscrates/switchyard-components/Cargo.tomlcrates/switchyard-components/src/backends/anthropic.rscrates/switchyard-components/src/backends/common.rscrates/switchyard-components/src/backends/context_overflow.rscrates/switchyard-components/src/backends/mod.rscrates/switchyard-components/src/backends/multi.rscrates/switchyard-components/src/backends/openai.rscrates/switchyard-components/src/backends/selection.rscrates/switchyard-components/src/backends/stats.rscrates/switchyard-components/src/contracts/backend.rscrates/switchyard-components/src/contracts/context.rscrates/switchyard-components/src/contracts/error.rscrates/switchyard-components/src/contracts/ids.rscrates/switchyard-components/src/contracts/mod.rscrates/switchyard-components/src/contracts/roles.rscrates/switchyard-components/src/contracts/types.rscrates/switchyard-components/src/dimension_collector/mod.rscrates/switchyard-components/src/dimension_collector/response/checks.rscrates/switchyard-components/src/dimension_collector/response/mod.rscrates/switchyard-components/src/dimension_collector/tool_signals.rscrates/switchyard-components/src/lib.rscrates/switchyard-components/src/request_processors/dimension_collector.rscrates/switchyard-components/src/request_processors/mod.rscrates/switchyard-components/src/request_processors/random_routing.rscrates/switchyard-components/src/request_processors/stats.rscrates/switchyard-components/src/response_processors/mod.rscrates/switchyard-components/src/response_processors/response_signals.rscrates/switchyard-components/src/response_processors/stats.rscrates/switchyard-components/src/stage_router.rscrates/switchyard-components/src/stats/accumulator.rscrates/switchyard-components/src/stats/cache_eligibility.rscrates/switchyard-components/src/stats/context.rscrates/switchyard-components/src/stats/cost.rscrates/switchyard-components/src/stats/mod.rscrates/switchyard-components/src/stats/usage.rscrates/switchyard-components/src/telemetry.rscrates/switchyard-components/tests/adversarial_multi_llm_backend.rscrates/switchyard-components/tests/adversarial_native_backends.rscrates/switchyard-components/tests/adversarial_random_routing.rscrates/switchyard-components/tests/contracts.rscrates/switchyard-components/tests/stats_accumulator.rscrates/switchyard-components/tests/stats_processors.rscrates/switchyard-components/tests/stats_usage_shapes.rscrates/switchyard-components/tests/support/config.rscrates/switchyard-components/tests/support/mod.rscrates/switchyard-py/Cargo.tomlcrates/switchyard-py/src/component_bindings.rscrates/switchyard-py/src/component_bindings/backends.rscrates/switchyard-py/src/component_bindings/config.rscrates/switchyard-py/src/component_bindings/dimension_collector.rscrates/switchyard-py/src/component_bindings/request_processors.rscrates/switchyard-py/src/component_bindings/response_processors.rscrates/switchyard-py/src/component_bindings/stage_router.rscrates/switchyard-py/src/component_bindings/stats.rscrates/switchyard-py/src/errors.rscrates/switchyard-py/src/interop.rscrates/switchyard-py/src/interop/context.rscrates/switchyard-py/src/interop/request.rscrates/switchyard-py/src/interop/response.rscrates/switchyard-py/src/interop/roles.rscrates/switchyard-py/src/interop/subagent.rscrates/switchyard-py/src/lib.rscrates/switchyard-py/src/libsy_bindings.rscrates/switchyard-py/src/py_serde.rscrates/switchyard-py/src/translation.rscrates/switchyard-server/README.mddocs/cli_reference.mddocs/getting_started.mddocs/internal/metrics_reference.mddocs/routing_algorithms/stage_router_routing.mdexamples/minimal.pyexamples/route.yamlexamples/utils.pypyproject.tomlswitchyard/__init__.pyswitchyard/cli/command_utils.pyswitchyard/cli/launchers/claude_code_launcher.pyswitchyard/cli/launchers/codex_cli_launcher.pyswitchyard/cli/launchers/cost_estimator.pyswitchyard/cli/launchers/launcher_runtime.pyswitchyard/cli/launchers/live_stats_footer.pyswitchyard/cli/launchers/openclaw_launcher.pyswitchyard/cli/launchers/session_summary.pyswitchyard/cli/launchers/shell_tui.pyswitchyard/cli/model_catalog/__init__.pyswitchyard/cli/model_catalog/model_discovery.pyswitchyard/cli/route_bundle.pyswitchyard/cli/switchyard_cli.pyswitchyard/lib/__init__.pyswitchyard/lib/backends/__init__.pyswitchyard/lib/backends/anthropic_native_llm_backend.pyswitchyard/lib/backends/backend_format_resolver.pyswitchyard/lib/backends/llm_target.pyswitchyard/lib/backends/multi_llm_backend.pyswitchyard/lib/backends/openai_llm_backend.pyswitchyard/lib/backends/openai_native_backend.pyswitchyard/lib/backends/stats_llm_backend.pyswitchyard/lib/chat_request/__init__.pyswitchyard/lib/chat_request/anthropic.pyswitchyard/lib/chat_request/base.pyswitchyard/lib/chat_request/openai_chat.pyswitchyard/lib/chat_request/openai_responses.pyswitchyard/lib/chat_response/__init__.pyswitchyard/lib/chat_response/anthropic.pyswitchyard/lib/chat_response/base.pyswitchyard/lib/chat_response/openai_chat.pyswitchyard/lib/chat_response/openai_responses.pyswitchyard/lib/chat_response/streaming_response_accumulator.pyswitchyard/lib/conversation_turn.pyswitchyard/lib/endpoints/__init__.pyswitchyard/lib/endpoints/anthropic_messages_endpoint.pyswitchyard/lib/endpoints/base.pyswitchyard/lib/endpoints/dispatch.pyswitchyard/lib/endpoints/error_envelope.pyswitchyard/lib/endpoints/models_endpoint.pyswitchyard/lib/endpoints/openai_chat_endpoint.pyswitchyard/lib/endpoints/outcome_metrics.pyswitchyard/lib/endpoints/prometheus_emitter.pyswitchyard/lib/endpoints/responses_endpoint.pyswitchyard/lib/endpoints/route_selection.pyswitchyard/lib/endpoints/routing_log_stats_endpoint.pyswitchyard/lib/endpoints/sse_helpers.pyswitchyard/lib/endpoints/stats_endpoint.pyswitchyard/lib/endpoints/upstream_error.pyswitchyard/lib/endpoints/upstream_error_log.pyswitchyard/lib/llm_client.pyswitchyard/lib/model_listing.pyswitchyard/lib/processors/__init__.pyswitchyard/lib/processors/format_translate.pyswitchyard/lib/processors/model_rewrite_request_processor.pyswitchyard/lib/processors/rl_logging_request_processor.pyswitchyard/lib/processors/rl_logging_response_processor.pyswitchyard/lib/processors/routing_log_response_processor.pyswitchyard/lib/processors/stats_request_processor.pyswitchyard/lib/processors/stats_response_processor_accumulator.pyswitchyard/lib/prometheus_exposition.pyswitchyard/lib/proxy_context.pyswitchyard/lib/request_metadata.pyswitchyard/lib/roles.pyswitchyard/lib/route_table.pyswitchyard/lib/startup_timing.pyswitchyard/lib/stats_accumulator.pyswitchyard/lib/switchyard.pyswitchyard/lib/tracing.pyswitchyard/server/__init__.pyswitchyard/server/server_util.pyswitchyard/server/switchyard_app.pyswitchyard/telemetry.pyswitchyard_rust/__init__.pyswitchyard_rust/_native.pyswitchyard_rust/components.pyswitchyard_rust/components.pyiswitchyard_rust/core.pyswitchyard_rust/libsy.pyswitchyard_rust/server.pyswitchyard_rust/translation.pytests/_chain_test_helpers.pytests/conftest.pytests/contract/__init__.pytests/contract/test_platform_imports.pytests/contract/test_proxy_context.pytests/contract/test_request_response_types.pytests/e2e/_helpers.pytests/e2e/conftest.pytests/e2e/test_passthrough_e2e.pytests/e2e/test_passthrough_responses_e2e.pytests/e2e_multiturn_responses.pytests/getting_started/test_getting_started.pytests/readme/test_readme.pytests/test_anthropic_native_llm_backend.pytests/test_anthropic_openai_translation.pytests/test_anthropic_output_config_strip.pytests/test_anthropic_probe.pytests/test_backend_format_resolver.pytests/test_build_and_serve.pytests/test_chat_request.pytests/test_chat_response.pytests/test_cli_reference_docs.pytests/test_codex_multiturn_traces.pytests/test_context_error_translation.pytests/test_context_window_exceeded_endpoint.pytests/test_cost_estimator_gemini.pytests/test_endpoint_state_contract.pytests/test_error_source_annotation.pytests/test_format_translate_processor.pytests/test_inference_e2e.pytests/test_infra.pytests/test_init_all_exports.pytests/test_launchers.pytests/test_live_stats_footer.pytests/test_llm_client.pytests/test_metrics_endpoint.pytests/test_no_stale_module_paths.pytests/test_outcome_metrics.pytests/test_prometheus_emitter.pytests/test_prometheus_exposition.pytests/test_python_server_passthrough.pytests/test_request_metadata.pytests/test_request_translation_engine.pytests/test_request_translation_engine_to_any_of.pytests/test_response_translation_engine.pytests/test_responses_openai_translation.pytests/test_rl_logging.pytests/test_rl_logging_e2e.pytests/test_route_bundle.pytests/test_route_selection_headers.pytests/test_route_table.pytests/test_routing_log_response_processor.pytests/test_shell_tui.pytests/test_sse_stream_close.pytests/test_stats_accumulator.pytests/test_stream_close_chain.pytests/test_stream_leak_repro.pytests/test_switchyard.pytests/test_switchyard_app_factory.pytests/test_switchyard_app_lifecycle.pytests/test_switchyard_rust_component_bindings.pytests/test_switchyard_rust_core_bindings.pytests/test_telemetry.pytests/test_tool_result_signal_collector.pytests/test_tracing.pytests/test_translation_engine_chaos.pytests/test_upstream_error_log.pytests/test_upstream_error_passthrough.pytests/translation/__init__.pytests/translation/test_format_fidelity_contract.py
💤 Files with no reviewable changes (106)
- Cargo.toml
- switchyard/lib/backends/openai_llm_backend.py
- crates/switchyard-components/src/response_processors/mod.rs
- switchyard/lib/init.py
- .agents/skills/switchyard-stage-router-scorer/SKILL.md
- switchyard/lib/chat_request/base.py
- switchyard/lib/chat_request/openai_responses.py
- switchyard/lib/chat_request/anthropic.py
- crates/switchyard-components/src/dimension_collector/mod.rs
- switchyard/lib/backends/anthropic_native_llm_backend.py
- switchyard/lib/chat_response/openai_chat.py
- switchyard/lib/endpoints/routing_log_stats_endpoint.py
- crates/switchyard-components/src/contracts/roles.rs
- switchyard/lib/chat_response/openai_responses.py
- switchyard/lib/chat_response/anthropic.py
- switchyard/cli/model_catalog/model_discovery.py
- switchyard/lib/backends/openai_native_backend.py
- benchmark/README.md
- switchyard/lib/backends/multi_llm_backend.py
- crates/switchyard-components/src/request_processors/dimension_collector.rs
- switchyard/lib/backends/stats_llm_backend.py
- crates/switchyard-py/src/component_bindings/request_processors.rs
- crates/switchyard-py/src/interop/roles.rs
- crates/switchyard-components/src/contracts/context.rs
- crates/switchyard-py/src/interop.rs
- crates/switchyard-components/src/backends/context_overflow.rs
- crates/switchyard-components/Cargo.toml
- crates/switchyard-py/src/component_bindings.rs
- examples/route.yaml
- crates/switchyard-components/src/stats/usage.rs
- crates/switchyard-components/src/backends/stats.rs
- crates/switchyard-py/src/component_bindings/stage_router.rs
- crates/switchyard-py/Cargo.toml
- crates/switchyard-py/src/component_bindings/config.rs
- switchyard/lib/chat_request/openai_chat.py
- crates/switchyard-components/tests/adversarial_native_backends.rs
- crates/switchyard-py/src/interop/request.rs
- crates/switchyard-py/src/lib.rs
- crates/switchyard-components/src/contracts/backend.rs
- switchyard/lib/endpoints/dispatch.py
- crates/switchyard-components/src/backends/multi.rs
- switchyard/cli/model_catalog/init.py
- switchyard/lib/endpoints/responses_endpoint.py
- switchyard/lib/chat_response/base.py
- switchyard/lib/endpoints/error_envelope.py
- crates/switchyard-components/tests/contracts.rs
- crates/switchyard-components/src/response_processors/stats.rs
- crates/switchyard-components/tests/adversarial_random_routing.rs
- crates/switchyard-py/src/interop/subagent.rs
- crates/switchyard-py/src/interop/context.rs
- crates/switchyard-components/src/contracts/error.rs
- crates/switchyard-components/src/dimension_collector/tool_signals.rs
- crates/switchyard-py/src/component_bindings/dimension_collector.rs
- crates/switchyard-components/src/contracts/types.rs
- switchyard/lib/conversation_turn.py
- crates/switchyard-components/src/dimension_collector/response/mod.rs
- switchyard/lib/endpoints/models_endpoint.py
- switchyard/lib/endpoints/route_selection.py
- crates/switchyard-components/src/stats/mod.rs
- switchyard/lib/endpoints/prometheus_emitter.py
- crates/switchyard-components/src/backends/mod.rs
- crates/switchyard-components/tests/support/config.rs
- crates/switchyard-components/src/lib.rs
- switchyard/cli/route_bundle.py
- crates/switchyard-components/src/backends/selection.rs
- switchyard/lib/endpoints/base.py
- crates/switchyard-components/src/stats/accumulator.rs
- examples/utils.py
- crates/switchyard-components/src/response_processors/response_signals.rs
- crates/switchyard-components/src/request_processors/random_routing.rs
- switchyard/lib/chat_request/init.py
- crates/switchyard-components/src/stats/cache_eligibility.rs
- switchyard/lib/endpoints/outcome_metrics.py
- crates/switchyard-py/src/translation.rs
- switchyard/lib/chat_response/init.py
- crates/switchyard-components/src/request_processors/stats.rs
- crates/switchyard-components/tests/adversarial_multi_llm_backend.rs
- benchmark/score_staged_run.py
- switchyard/lib/endpoints/anthropic_messages_endpoint.py
- switchyard/lib/endpoints/openai_chat_endpoint.py
- switchyard/lib/backends/backend_format_resolver.py
- crates/switchyard-py/src/component_bindings/response_processors.rs
- crates/switchyard-components/src/backends/anthropic.rs
- switchyard/lib/endpoints/init.py
- crates/switchyard-components/src/contracts/mod.rs
- crates/switchyard-py/src/component_bindings/stats.rs
- crates/switchyard-components/src/telemetry.rs
- crates/switchyard-components/tests/stats_accumulator.rs
- crates/switchyard-py/src/interop/response.rs
- switchyard/lib/backends/init.py
- crates/switchyard-components/src/stats/context.rs
- crates/switchyard-py/src/component_bindings/backends.rs
- crates/switchyard-components/src/contracts/ids.rs
- crates/switchyard-components/src/stage_router.rs
- switchyard/lib/chat_response/streaming_response_accumulator.py
- crates/switchyard-components/src/backends/common.rs
- examples/minimal.py
- crates/switchyard-components/tests/support/mod.rs
- crates/switchyard-components/src/backends/openai.rs
- crates/switchyard-py/src/py_serde.rs
- crates/switchyard-components/src/dimension_collector/response/checks.rs
- switchyard/lib/backends/llm_target.py
- crates/switchyard-components/src/stats/cost.rs
- crates/switchyard-components/tests/stats_processors.rs
- crates/switchyard-components/src/request_processors/mod.rs
- crates/switchyard-components/tests/stats_usage_shapes.rs
🛑 Comments failed to post (3)
switchyard/cli/launchers/shell_tui.py (3)
172-175: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Synchronize dynamic footer height with the child PTY.
footer_heightcan change between the reads on Lines 172-175. The child PTY size is set at startup on Line 337 and only updated during an outer-terminal resize on Lines 301-304.If the footer grows during a session, footer rendering reserves fewer shell rows but the child still writes for its old PTY height. The child can overwrite the footer.
Snapshot one bounded footer height per paint. When that height changes, update
TIOCSWINSZ, the scroll region, and footer rendering from the same layout value. Add a regression test that changes footer height between paints. Based on learnings: “Define verifiable success criteria, write regression tests for bugs and new validation, and verify each step.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@switchyard/cli/launchers/shell_tui.py` around lines 172 - 175, Snapshot a bounded footer height once per paint in the shell TUI layout flow, and use that same value for footer rendering, shell row calculation, scroll-region setup, and child PTY sizing. When the snapshot differs from the previous layout, update TIOCSWINSZ and the scroll region immediately rather than waiting for an outer-terminal resize. Add a regression test that changes footer height between paints and verifies the child PTY and rendered layout stay synchronized.Source: Learnings
316-419: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash set -euo pipefail ast-grep outline switchyard/cli/launchers/shell_tui.py --items all rg -n -C 3 'ShellTUI\(|\.run\(\)|asyncio\.run|async def' \ switchyard/cli testsRepository: NVIDIA-NeMo/Switchyard
Length of output: 9368
🏁 Script executed:
#!/bin/bash set -euo pipefail printf '%s\n' '--- shell_tui.py structure and run implementation ---' ast-grep outline switchyard/cli/launchers/shell_tui.py --items all sed -n '1,120p' switchyard/cli/launchers/shell_tui.py sed -n '300,430p' switchyard/cli/launchers/shell_tui.py printf '%s\n' '--- launcher call contexts ---' sed -n '130,215p' switchyard/cli/launchers/claude_code_launcher.py sed -n '165,215p' switchyard/cli/launchers/codex_cli_launcher.py sed -n '215,250p' switchyard/cli/launchers/openclaw_launcher.py printf '%s\n' '--- async and asyncio.run usage in launcher paths ---' rg -n -C 4 'def launch|def run|async def|asyncio\.run|ShellTUI' switchyard/cli testsRepository: NVIDIA-NeMo/Switchyard
Length of output: 30795
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' import ast from pathlib import Path target = Path("switchyard/cli/launchers/shell_tui.py") tree = ast.parse(target.read_text()) run = next( node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef) and node.name == "run" ) calls = { node.func.attr if isinstance(node.func, ast.Attribute) else getattr(node.func, "id", "") for node in ast.walk(run) if isinstance(node, ast.Call) } print(f"run_is_async={isinstance(run, ast.AsyncFunctionDef)}") print(f"run_calls_select={bool({'select'} & calls)}") print(f"run_constructs_thread={any(isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute) and n.func.attr == 'Thread' for n in ast.walk(run))}") for path in sorted(Path("switchyard/cli").rglob("*.py")): source = path.read_text() if "ShellTUI" not in source: continue module = ast.parse(source) matches = [] for node in ast.walk(module): if isinstance(node, ast.Call): text = ast.unparse(node) if ".run()" in text or "tui.run()" in text: matches.append((node.lineno, text)) if matches: print(path) for lineno, text in matches: print(f" {lineno}: {text}") print(f" asyncio.run_present={'asyncio.run' in source}") PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 905
Use an async lifecycle for
ShellTUI.
runis synchronous and owns a blockingselectloop plus a background thread. Refactor it to async APIs. Useasyncio.run()at synchronous launcher boundaries.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@switchyard/cli/launchers/shell_tui.py` around lines 316 - 419, Refactor ShellTUI.run and its blocking event loop to an async lifecycle, replacing the background footer thread and blocking select/waits with asyncio-compatible tasks and awaits while preserving terminal setup, child I/O, cleanup, and exit-code behavior. Update synchronous launcher boundaries that invoke run to call the async lifecycle through asyncio.run(), and adjust related lifecycle helpers to remain consistently awaitable.Source: Coding guidelines
388-392: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash set -euo pipefail python - <<'PY' import fcntl import os import pty master_fd, slave_fd = pty.openpty() try: flags = fcntl.fcntl(master_fd, fcntl.F_GETFL) fcntl.fcntl(master_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK) payload = b"x" * 1_000_000 try: written = os.write(master_fd, payload) print(f"requested={len(payload)} written={written}") except BlockingIOError: print("write raised BlockingIOError") finally: os.close(master_fd) os.close(slave_fd) PYRepository: NVIDIA-NeMo/Switchyard
Length of output: 516
🏁 Script executed:
#!/bin/bash set -euo pipefail file="switchyard/cli/launchers/shell_tui.py" printf '%s\n' '--- relevant source ---' sed -n '300,430p' "$file" printf '%s\n' '--- select/read/write references ---' rg -n -C 3 'select|os\.read|os\.write|master_fd|stdin_fd|BlockingIOError|pending' "$file" printf '%s\n' '--- callers and tests ---' rg -n -C 3 'ShellTUI|shell_tui|launch_shell|footer_height' switchyard tests 2>/dev/null || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 26536
🏁 Script executed:
#!/bin/bash set -euo pipefail python3 - <<'PY' import errno import fcntl import os read_fd, write_fd = os.pipe() try: flags = fcntl.fcntl(write_fd, fcntl.F_GETFL) fcntl.fcntl(write_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK) # Fill the nonblocking output queue, then issue the same kind of write # used by the launcher. while True: try: os.write(write_fd, b"x" * 65536) except BlockingIOError: break payload = b"y" * 65536 try: written = os.write(write_fd, payload) print(f"nonblocking_write=requested:{len(payload)} written:{written}") except BlockingIOError as exc: print(f"nonblocking_write=requested:{len(payload)} error:{exc.errno == errno.EAGAIN}") print("source_write_has_retry_or_exception_handling=false") finally: os.close(read_fd) os.close(write_fd) PY printf '%s\n' '--- helper implementation and focused tests ---' sed -n '55,80p' switchyard/cli/launchers/shell_tui.py sed -n '330,405p' tests/test_shell_tui.pyRepository: NVIDIA-NeMo/Switchyard
Length of output: 3815
Retain stdin bytes that the nonblocking PTY cannot accept.
os.write(master_fd, data)can raiseBlockingIOErroror write only part ofdata. Buffer unwritten bytes and monitormaster_fdfor writability until the buffer is empty.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@switchyard/cli/launchers/shell_tui.py` around lines 388 - 392, Update the stdin forwarding loop around os.read and master_fd so partial writes and BlockingIOError from os.write are handled without losing bytes. Maintain a pending-output buffer, append newly read stdin data, and monitor master_fd for writability until all buffered bytes are written, while preserving the existing EOF break behavior.
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
switchyard servecommand, YAML route bundles, FastAPI server, legacy chain, processors, backends, and server-only testsswitchyard-componentscrate and its compatibility PyO3 bindingslibsyandswitchyard-serverbindingsThis draft is the review and migration reference for the removed Python path. The implementation remains available in Git history if it needs to be consulted later.
Why
The native Rust server and libsy path now own serving, routing, translation, and observability. Keeping a second Python server duplicated behavior, dependencies, tests, and maintenance while exposing a deprecated interface.
How
switchyard launchthe only Python CLI commandswitchyard-pyto the native server and libsy bindingsswitchyard-serverand use a native TOML deployment in the performance workflowWhat to review
switchyard.libsy, andswitchyard_rust.serverremain supportedValidation
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspaceuv run maturin developuv run ruff check .uv run mypy switchyard switchyard_rustuv run pytest tests/ -v -m "not integration"(134 passed, 2 deselected)make publish(strict documentation build)switchyard --helpexposes onlylaunchSummary by CodeRabbit
Breaking Changes
switchyard servecommand and Python route-bundle server.New Features
Documentation