Conversation
Track how many eval rollouts failed due to sandbox/infra errors. Derived from the difference between total eval inputs and successful outputs. Also logged in the all-fail early return path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| decode_env_overrides: Annotated[ | ||
| dict[str, str], | ||
| Field(description="Extra environment variables exported only on decode nodes."), | ||
| ] = {} |
There was a problem hiding this comment.
New config fields added without CHANGELOG update
Low Severity
This PR adds four new config fields (num_prefill_replicas, num_decode_replicas, prefill_env_overrides, decode_env_overrides) and changes the semantics of num_prefill_nodes / num_decode_nodes (from "per replica" to "Total") in DisaggregatedInferenceDeploymentConfig, but CHANGELOG.md has not been updated. The project rule requires a changelog entry for any PR that modifies configuration structures or usage patterns.
Additional Locations (1)
Triggered by project rule: BugBot Instructions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Note
Temporary PR for testing purposes.
🤖 Generated with Claude Code
Note
High Risk
High risk because it changes distributed inference job topology/routing and monkey-patches vLLM DP engine pause/resume behavior, which can affect cluster stability and hang/timeout modes. Also updates SLURM templates and monitoring interfaces, which can break deployments if misconfigured.
Overview
Enables multi-replica disaggregated (prefill/decode) inference by adding
num_prefill_replicas/num_decode_replicas, per-role env overrides, and validation that replicas evenly divide node counts; SLURM templates now compute sub-replica ranks/heads accordingly.Switches
vllm-routerpolicy fromround_robintoconsistent_hash(both standalone and PD-disaggregated) and threads the new replica/env parameters through the inference and RL SLURM entrypoints.Extends the vLLM monkey-patch for DP pause/resume to ignore
START_DP_WAVEwhile paused and to force a global unfinished sync on resume to avoid weight-update deadlocks.Improves eval robustness/observability by tracking
failed_rollouts, warning on failed groups, loggingfailed_rolloutsmetrics even when all outputs fail, and addingMonitor.log_eval_samples()with a W&B implementation that logs eval completions to a separate table.Written by Cursor Bugbot for commit 986e65d. This will update automatically on new commits. Configure here.