feat(frontend): add exact MM routing for generate

[furionw]

Why

/inference/v1/generate accepts tokenized multimodal rollouts, but the frontend currently routes only on token IDs. Different images with identical placeholder layouts therefore appear equivalent to the KV router. Exact request hashing also requires the block size, LoRA identity, and worker capability from the same selected WorkerSet; a single engine-selection interface preserves that invariant without exposing irrelevant parsing options.

What Change

• Project image identity into routing-only token sequences.
• Gate exact projection on worker-advertised runtime capability.
• Return generate engines and routing metadata through one atomic getter.

Test Plan

• Run focused Generate, model discovery, and KV-router release tests.
• Run pre-commit on all changed Rust files.

Summary by CodeRabbit

• New Features

  • Improved multimodal request routing for image inputs, including support for exact token-based matching when available.
  • Added routing support that accounts for selected LoRA adapters and KV-cache configuration.
  • Added compatibility for vLLM multimodal identifiers and routing capabilities.

• Bug Fixes

  • Added validation and safe fallback handling for malformed, unsupported, or inconsistent multimodal metadata.
  • Improved worker selection consistency when available workers or their capabilities change.

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 45.57% (-4.14%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b30d438 | Docs | Datadog PR Page | Give us feedback!}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 86cdbf81-6491-4968-a509-d1b873b003d2

📥 Commits

Reviewing files that changed from the base of the PR and between b13288e and ed101f9.

📒 Files selected for processing (6)

• lib/kv-router/src/protocols.rs
• lib/llm/src/discovery.rs
• lib/llm/src/discovery/model.rs
• lib/llm/src/discovery/model_manager.rs
• lib/llm/src/http/service/generate.rs
• lib/llm/src/local_model/runtime_config.rs

---

Walkthrough

Adds multimodal identifier hashing, richer generate-engine selection metadata, exact multimodal routing preprocessing, LoRA routing hints, capability-based fallback, and comprehensive validation tests for Generate requests.

Changes

Generate exact multimodal routing

Layer / File(s)	Summary
Multimodal identifier hashing lib/kv-router/src/protocols.rs	Adds hash_mm_identifier with empty, canonical 64-character hex, and xxh3_64 fallback handling, covered by unit tests.
Worker-specific engine selection lib/llm/src/discovery/..., lib/llm/src/local_model/runtime_config.rs	Introduces GenerateEngineSelection, returns worker-set routing metadata atomically, exposes the exact-MM capability constant, and updates discovery APIs and tests.
Generate multimodal routing preprocessing lib/llm/src/http/service/generate.rs	Parses vLLM image metadata, projects block-aligned routing tokens when supported, propagates LoRA hints, and falls back to token-only routing for invalid or unsupported metadata with corresponding tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description is missing the required Overview, Details, Where should reviewer start?, and Related Issues sections from the template.	Rewrite it to match the template, add a reviewer-start file callout, and include the required issue linkage or confirmed no-issue checkbox.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly mentions exact MM routing for generate, which matches the main change, though the frontend scope is a bit misleading.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[biswapanda]

/ok to test 5df0396

[devin-ai-integration]

Devin Review found 1 new potential issue.

[devin-ai-integration]

🔍 Event-side pad normalization depends on a resolved image_token_id the capability flag does not cover

Exact matching only works when the KV-event normalizer was constructed with image_token_id (see KvEventSourceConfig::Zmq in lib/llm/src/kv_router/publisher/mod.rs:56-66); otherwise vLLM BlockStored events are hashed with block_mm_infos instead of pad substitution and can never match this projection. The chat path gates the same projection on self.routing_image_token_id (lib/llm/src/preprocessor.rs:1958-1962), whereas the /generate path relies solely on the worker-advertised vllm_exact_mm_routing flag. If a worker can advertise the capability while its event source lacks a resolved image token id, MM routing silently degrades rather than falling back. Worth confirming that whatever sets the capability also guarantees the normalizer's image token id.

---

Was this helpful? React with 👍 or 👎 to provide feedback.