feat(vllm): add aggregated text-embedding worker (DIS-2092)

[tzulingk]

Overview:

MVP for DIS-2092 — adds an aggregated text-embedding worker shape to Dynamo's vLLM backend. Users can move an existing Qwen3-Embedding-0.6B vLLM deployment onto Dynamo+vLLM without changing engine args. (Builds on the --runner pooling flag preservation fix in #9710, now merged.)

The Rust frontend's /v1/embeddings route and ModelType::Embedding plumbing have existed since the SGLang embedding backend shipped. This PR registers a vLLM worker on the same endpoint.

Details:

Architecture. A new --embedding-worker flag in backend_args.py selects a different worker shape: pooling-mode AsyncLLM fronted by a new EmbeddingWorkerHandler that calls AsyncLLM.encode(prompt, pooling_params, request_id) and converts the PoolingRequestOutput to an OpenAI-shaped response. Dispatch happens in WorkerFactory.create() — the embedding branch is checked first, before the existing disaggregation_mode switch, because it's a different worker shape rather than a variant of decode.

EmbeddingWorkerHandler is a standalone class — it does NOT inherit BaseWorkerHandler. That base does generation-only init (media loaders, KV-block lookup via get_dp_range_for_worker, embedding cache manager) that would either fail or be meaningless on a pooling engine. Pooling inference is a single forward pass with no KV cache, no multimodal, no streamed decode — a separate class is clearer than overriding most of the base's behavior.

What's intentionally skipped on the embedding path:

• KV-events publisher — no KV cache, nothing to publish.
• Forward-pass-metrics relay — relays decode-phase ZMQ metrics; no decode here.
• InstrumentedScheduler — hard-codes pooling_params=None (would silently disable the pooling pass) and emits decode-shaped metrics that don't apply. It's only installed when --benchmark-mode is set, which is rejected for embedding workers via _validate_embedding_worker_exclusivity(). See DIS-2092 appendix for the full discussion.
• P/D disaggregation — rejected at parse time. --embedding-worker combined with any non-agg --disaggregation-mode raises ValueError.

Validation. DynamoVllmConfig._validate_embedding_worker_exclusivity() rejects --embedding-worker combined with prefill/decode/encode disaggregation, with any multimodal flag, or with --enable-multimodal.

Launch script. examples/backends/vllm/launch/agg_embed.sh defaults to Qwen/Qwen3-Embedding-0.6B with the standard pooling args: --runner pooling --dtype float32 --pooler-config '{"pooling_type":"MEAN","use_activation":false}'.

Unit tests (test_vllm_unit.py → TestEmbeddingWorkerFlag):

• test_default_false
• test_flag_sets_true
• test_rejects_prefill_disagg
• test_rejects_decode_disagg
• test_rejects_multimodal_combo

A dispatch test (test_embedding_worker_takes_priority) was added to test_vllm_worker_factory.py to verify --embedding-worker is checked first and the other _create_*_worker methods are not invoked.

Note on benchmarking

This PR rejects --benchmark-mode for embedding workers, but that flag is Dynamo's in-process self-profiling — synthetic-request sweeps run at startup before serving traffic, used for capacity planning, engine-arg tuning, and planner input. It is separate from external HTTP load benchmarking via vllm bench serve (DIS-2095's methodology), which works against /v1/embeddings today and continues to work with this PR.

Decision: not adding --benchmark-mode embed now. Embedding workloads are roughly (batch_size × ISL → latency) — far simpler than decode workloads, with fewer interacting knobs. External HTTP load benchmarking (DIS-2095) covers the capacity-planning and engine-tuning needs. The one remaining wedge is Dynamo planner integration for auto-scaling; revisit when the planner needs embedding-aware metrics. See DIS-2092's appendix for the full reasoning.

Where should the reviewer start?

1. components/src/dynamo/vllm/worker_factory.py → _create_embedding_worker() (new) and the dispatch branch in create(). That's the structural change.
2. components/src/dynamo/vllm/handlers.py → EmbeddingWorkerHandler (new class at the end of the file). Note the standalone (non-BaseWorkerHandler) design — see the docstring for the reasoning.
3. components/src/dynamo/vllm/backend_args.py → --embedding-worker flag, embedding_worker config field, and _validate_embedding_worker_exclusivity().
4. examples/backends/vllm/launch/agg_embed.sh → reference launch config for the new worker shape.

Closes DIS-2092 (MVP scope).

[biswapanda]

overall lgtm.
one comment on opportunity to improve perf

[nnshah1]

LGTM