Skip to content

fix(steering): port declarative override parity (compose+precedence) to v2 runner - #224

Merged
RhizoNymph merged 1 commit into
feat/dynamic-steeringfrom
fix/v2-declarative-override-parity
Jul 3, 2026
Merged

fix(steering): port declarative override parity (compose+precedence) to v2 runner#224
RhizoNymph merged 1 commit into
feat/dynamic-steeringfrom
fix/v2-declarative-override-parity

Conversation

@RhizoNymph

Copy link
Copy Markdown
Owner

What & why

_apply_request_override on the v2 GPU runner (vllm/v1/worker/gpu/steering_runner_mixin.py) had silently dropped three behaviors that v1's implementation (vllm/v1/worker/steering_model_runner_mixin.py) has, even though declarative per-request gates (PR #219) are live on v2 — the default runner for Qwen3/Llama. This is a real user-facing correctness bug.

The gap on v2:

  1. compose_admitted ignored. A declarative add gate emits RequestSteeringOverride(compose_admitted=True), meant to add on top of the request's admitted static decode steering. v2 registered action.vectors raw, so on v2 an add gate replaced the client's static decode steering instead of composing. Fixed by folding the admitted decode spec (_resolve_request_steering(sp, "decode"), admitted params sourced from v2's _steering_reqs) with the gate delta via merge_steering_specs, mirroring v1.

  2. Operator-wins precedence never enforced. v2 never ran the precedence check and never wrote _req_override_source, so the inherited _apply_scale_update / _apply_monitor_update precedence checks (which read _req_override_source) were vacuous on v2 — a client declarative gate could scale/monitor/override a request already owned by an operator consumer. Fixed by adding the same source == DECLARATIVE_SOURCE and owner != source rejection and stamping _req_override_source on every apply.

  3. Clear path leaked the source map. The vectors is None clear didn't pop _req_override_source. Fixed; request finish/preempt cleanup already routes through the inherited _drop_request_dynamic_override, which purges both maps.

The v2 docstring's "Identical routing semantics to v1" claim is now accurate.

Also hardened (v1 + v2): the compose_admitted fold calls _resolve_request_steering(..., "decode"), which can raise RuntimeError when a request's steering module is not registered on the worker. That exception escaped _apply_request_override's except clauses and would crash the engine, defeating observer isolation. Both runners now catch it and return a structured rejection, keeping prior state.

Files

  • vllm/v1/worker/gpu/steering_runner_mixin.py — v2 parity port (precedence check, _req_override_source bookkeeping, compose_admitted fold, guarded resolve, clear-path source pop).
  • vllm/v1/worker/steering_model_runner_mixin.py — guard the v1 compose fold's _resolve_request_steering against RuntimeError.
  • tests/v1/worker/test_gpu_v2_steering_glue.py — 8 new v2 tests mirroring v1 coverage (compose fold, no-compose raw, declarative-yields-to-operator + operator-takes-over, clear/finish purge _req_override_source, inherited scale + monitor precedence now effective on v2).
  • docs/design/dynamic_steering.md §8.2 — precedence paragraph now notes both runners are kept in lock-step and mentions the guarded fold.

Notes

Sibling PR fix/declarative-gate-fail-closed may touch steering_model_runner_mixin.py's action-apply path concurrently; expect a possible merge conflict there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant