fix(sglang-sidecar): advertise enable_eagle so KV routers bigram-align hashes by ishandhanani · Pull Request #14374 · ai-dynamo/dynamo

ishandhanani · 2026-09-05T02:46:28Z

Summary

Advertise SGLang EAGLE speculative decoding in the native sidecar's model card, matching the Python worker. This lets KV routers use the same bigram hashing as the engine's KV events.

Derive enable_eagle from the resolved speculative algorithm: EAGLE, EAGLE3, FROZEN_KV_MTP, and the legacy NEXTN alias.
Carry it through LlmRegistration into ModelRuntimeConfig.
Expose the boolean consistently in the Python constructor, getter, dataclass, and stub, defaulting to False and preserving existing positional arguments and missing-attribute compatibility.

Validation

Built the Python extension with debug symbols and incremental compilation disabled.
Python binding suite: 26 passed, including Eagle round-trip, dataclass defaults, and legacy positional arguments. Repeated after rebasing onto the latest PR head.
Sidecar Eagle tests: 2 passed. Backend model-card propagation test: 1 passed. These Rust tests ran before the main-merge rebase; the tested registration implementation is unchanged.
Rust formatting, Python lint/format checks, and diff checks passed.
No end-to-end inference validation in this update.

Related Issues

🚫 This PR is NOT linked to an issue:

Confirmed — no related issue

Summary by CodeRabbit

New Features
- Added support for identifying EAGLE-compatible speculative decoding modes, including EAGLE, EAGLE3, FROZEN_KV_MTP, and NEXTN.
- EAGLE settings are now consistently propagated into local model runtime configuration.
- Python-based model registrations can optionally specify whether EAGLE is enabled.
Bug Fixes
- Improved configuration handling so EAGLE behavior is preserved consistently across supported backends.
- Added coverage for supported, unsupported, and case-insensitive speculative decoding mode values.

…g EAGLE events With EAGLE speculative decoding, SGLang keys its radix cache by token bigrams and publishes KV events with bigram token_ids. The sidecar decodes those and hashes them as Eagle blocks, but it never told the router: LlmRegistration had no enable_eagle, so the model card said false and the router hashed prompts the normal way. Every event was accepted and none ever matched. - Add LlmRegistration::enable_eagle and pass it into ModelRuntimeConfig. - SGLang sidecar sets it from server_info.speculative_algorithm (EAGLE, EAGLE3, FROZEN_KV_MTP, NEXTN), same predicate as SGLang's SpeculativeAlgorithm.is_eagle() and the Python worker's register.py. - KvRouter: DYN_ROUTER_WAIT_FOR_MODEL_CARD=1 makes a stock router wait for the worker's card (bounded by DYN_ROUTER_MODEL_CARD_WAIT_SECS) instead of taking a one-shot snapshot. is_eagle is frozen from that lookup, so a router that starts before its workers otherwise stays non-Eagle for good. Logs the resolved card. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

coderabbitai · 2026-09-09T00:12:58Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e244277f-0ae2-46f4-a411-1b4e18926f41

📥 Commits

Reviewing files that changed from the base of the PR and between e0dedc5 and 0d19c82.

📒 Files selected for processing (5)

lib/backend-common/examples/mocker/src/engine.rs
lib/backend-common/src/engine.rs
lib/backend-common/src/worker.rs
lib/bindings/python/rust/backend.rs
lib/sidecar/sglang/src/engine.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Walkthrough

Changes

The change adds enable_eagle to engine registration metadata. Worker runtime configuration and Python bindings preserve the value. SGLang derives the value from supported speculative algorithms and tests the resulting registration.

EAGLE registration

Layer / File(s)	Summary
Registration contract and runtime propagation `lib/backend-common/src/engine.rs`, `lib/backend-common/src/worker.rs`, `lib/backend-common/examples/mocker/src/engine.rs`	`LlmRegistration` exposes `enable_eagle`. Local model runtime configuration preserves the value. Mocker registration defaults to `false`, and tests verify propagation.
Python registration compatibility `lib/bindings/python/rust/backend.rs`	Python registrations initialize or read the optional `enable_eagle` attribute and default to `false` when it is absent or unset.
SGLang EAGLE detection `lib/sidecar/sglang/src/engine.rs`	SGLang recognizes supported speculative algorithms case-insensitively, registers the derived EAGLE setting, and tests supported, unsupported, and malformed values.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 0d19c

This change enables EAGLE-compatible SGLang registrations to select matching KV routing behavior while preserving false defaults for other backends and older Python registrations. The implemented propagation and unit coverage indicate no remaining merge-blocking risk.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 46.15% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the main change: the SGLang sidecar advertises `enable_eagle` so KV routers can align bigram hashes.
Description check	✅ Passed	The description explains the implementation, validation, compatibility behavior, and issue status. It omits the template's explicit “Where should the reviewer start?” section, but the description is o…

Fix all pre-merge checks with AI

_{Comment @coderabbitai help to get the list of available commands.}

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

…gle-discovery Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

pull-request-size Bot added the size/L label Sep 5, 2026

github-actions Bot added the fix label Sep 5, 2026

ishandhanani force-pushed the idhanani/sglang-sidecar-enable-eagle branch from 2e39577 to ce4b251 Compare September 5, 2026 02:53

pull-request-size Bot added size/M and removed size/L labels Sep 5, 2026

ishandhanani force-pushed the idhanani/sglang-sidecar-enable-eagle branch from ce4b251 to d007300 Compare September 5, 2026 02:57

pull-request-size Bot added size/L and removed size/M labels Sep 5, 2026

fix(sglang-sidecar): keep Eagle fix scoped to registration

447937f

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

pull-request-size Bot added size/M and removed size/L labels Sep 9, 2026

Merge branch 'main' into idhanani/sglang-sidecar-enable-eagle

0d19c82

ishandhanani marked this pull request as ready for review September 9, 2026 00:09

ishandhanani requested review from a team as code owners September 9, 2026 00:09

fix(backend): expose Eagle registration in Python types

6742dc6

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

pull-request-size Bot added size/L and removed size/M labels Sep 9, 2026

ishandhanani enabled auto-merge (squash) September 9, 2026 00:36

PeaBrane approved these changes Sep 9, 2026

View reviewed changes

jain-ria approved these changes Sep 9, 2026

View reviewed changes

fix(sidecar): copy EPP workspace member into build image

2e9a9e1

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

ishandhanani requested a review from a team as a code owner September 9, 2026 00:39

This comment has been minimized.

Sign in to view

Merge remote-tracking branch 'origin/main' into idhanani/dyn-14374-ea…

08fe966

…gle-discovery Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

ishandhanani merged commit 10ecc28 into main Sep 9, 2026
121 checks passed

ishandhanani deleted the idhanani/sglang-sidecar-enable-eagle branch September 9, 2026 07:17

jain-ria mentioned this pull request Sep 15, 2026

fix(sglang): emit prefill handoff before completion in sidecar #14260

Merged

Navigation Menu

Repository navigation

fix(sglang-sidecar): advertise enable_eagle so KV routers bigram-align hashes - #14374

fix(sglang-sidecar): advertise enable_eagle so KV routers bigram-align hashes#14374
ishandhanani merged 6 commits into
mainfrom
idhanani/sglang-sidecar-enable-eagle

ishandhanani commented Sep 5, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

This comment has been minimized.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ishandhanani commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Related Issues

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

This comment has been minimized.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ishandhanani commented Sep 5, 2026 •

edited

Loading

coderabbitai Bot commented Sep 9, 2026 •

edited

Loading