feat(vllm): gate chat-shaped Prometheus collectors on embedding worker

[tzulingk]

Overview:

Mirrors the SGLang publisher gating (PR #9830) for the vLLM backend.

vLLM's StatLoggerFactory previously constructed a DynamoStatLoggerPublisher (with its WorkerMetricsPublisher / NATS endpoint + chat-shaped LLMBackendMetrics gauges -- total_blocks, gpu_cache_usage, model_load_time) on every worker, including embedding workers running pooling engines that emit no SchedulerStats. The result on an embedding deployment was a /metrics scrape full of gauges stuck at zero, drowning out the embedding-shaped collectors that operators actually want to alert on.

Details:

components/src/dynamo/vllm/publisher.py -- StatLoggerFactory gains an embedding_worker: bool flag. When set, create_stat_logger returns a new _NoopStatLogger (record + log_engine_initialized are pass-through) and the chat-shaped DynamoStatLoggerPublisher is never constructed. The component_gauges is not None assert that guards the chat path is also skipped on the embedding branch, since the factory no longer needs gauges to wire up.

components/src/dynamo/vllm/main.py -- setup_vllm_engine inspects stat_logger.embedding_worker and skips the LLMBackendMetrics(registry=DYNAMO_COMPONENT_REGISTRY, ...) registration entirely on the embedding path, so no chat-shaped collectors land on the embedding worker's registry. Return-type element widens to Optional[LLMBackendMetrics].

components/src/dynamo/vllm/worker_factory.py -- _create_embedding_worker flips embedding_worker=True on the factory it constructs. EngineSetupResult widens the trailing element to match. The chat/decode and prefill paths pass through unchanged.

vLLM still calls the factory unconditionally during AsyncLLM init, so the no-op logger is necessary to keep that seam happy without leaking chat-shaped state into the embedding worker.

Embedding-shaped metrics (e.g. dynamo_embedding_batch_size, dynamo_embedding_input_tokens, dynamo_embedding_latency_seconds) are registered on the Rust frontend, not this worker -- so this change removes the irrelevant gauges without taking any embedding observability with it.

Where should the reviewer start?

1. components/src/dynamo/vllm/publisher.py -- the StatLoggerFactory.create_stat_logger short-circuit and the new _NoopStatLogger. This is the meat of the change.
2. components/src/dynamo/vllm/main.py -- the setup_vllm_engine branch that decides whether to register LLMBackendMetrics.
3. components/src/dynamo/vllm/worker_factory.py -- single call-site flip in _create_embedding_worker.
4. components/src/dynamo/vllm/tests/test_vllm_publisher.py -- four unit tests. The embedding-worker test patches DynamoStatLoggerPublisher to raise to prove the constructor doesn't run on that path; the chat-worker sibling test verifies the default path still wires up the publisher.

🤖 Generated with Claude Code

---

Summary by CodeRabbit

• Bug Fixes

  • Resolved initialization issues for embedding worker configurations by properly handling optional metrics collection.

• Tests

  • Added comprehensive unit tests verifying embedding worker behavior and no-op logging functionality.

• Refactor

  • Optimized metrics collection system to skip unnecessary gauge tracking for embedding workers, improving resource efficiency.

[coderabbitai]

Walkthrough

This PR adds embedding-worker support to the vLLM stat logger by introducing a no-op logger implementation and gating the StatLoggerFactory with an embedding_worker flag, allowing metrics collection to be skipped for pooling engines while maintaining compatibility with chat workloads.

Changes

Embedding Worker Stat Logger Gating

Layer / File(s)	Summary
No-op logger implementation and factory gating components/src/dynamo/vllm/publisher.py	Introduce _NoopStatLogger class that safely drops all record() calls and implements log_engine_initialized() as a no-op. Extend StatLoggerFactory.__init__() to accept embedding_worker: bool flag and update create_stat_logger() to return _NoopStatLogger when enabled, avoiding the full metrics publisher path.
Type contracts for optional metrics components/src/dynamo/vllm/worker_factory.py, components/src/dynamo/vllm/main.py	Update EngineSetupResult type alias to allow final tuple element LLMBackendMetrics to be Optional, and update setup_vllm_engine() return type signature to match, reflecting the embedding-worker pooling path where metrics are not constructed.
Conditional metrics initialization in engine setup components/src/dynamo/vllm/main.py	In setup_vllm_engine(), detect the embedding_worker flag from stat_logger and conditionally construct LLMBackendMetrics only for non-embedding workloads; set component_gauges to None and skip assignment to stat_logger.component_gauges for the embedding path.
Worker factory integration with embedding flag components/src/dynamo/vllm/worker_factory.py	Update embedding-worker initialization to instantiate StatLoggerFactory with embedding_worker=True, ensuring the factory gates to _NoopStatLogger for pooling-engine setup.
Test suite for embedding worker gating components/src/dynamo/vllm/tests/test_vllm_publisher.py	Add module setup, imports, and four tests verifying: _NoopStatLogger is returned when embedding_worker=True; noop logger safely accepts record() calls with None stats; component_gauges can remain None with embedding flag; and default behavior constructs DynamoStatLoggerPublisher with expected parameters.

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: gating (conditionally disabling) chat-shaped Prometheus collectors on embedding workers in the vLLM backend.
Description check	✅ Passed	The description follows the template structure with Overview, Details, and Where should the reviewer start sections. It provides comprehensive context about the change and includes all required information.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/src/dynamo/vllm/main.py`:
- Line 415: The code dereferences component_gauges even when it's None
(embedding path), causing a crash; wrap any calls like
component_gauges.set_model_load_time(load_time) (and any other
component_gauges.* calls in the same function/section around lines 440-458) with
a None check (e.g., if component_gauges is not None:
component_gauges.set_model_load_time(...)) so writes are skipped for embedding
workers; update the same guard pattern wherever component_gauges is used in this
function to avoid startup errors.

In `@components/src/dynamo/vllm/tests/test_vllm_publisher.py`:
- Around line 26-31: The test module's pytestmark list includes a framework
marker pytest.mark.vllm but lacks the required single component marker; update
the pytestmark list (in test_vllm_publisher.py where the pytestmark variable is
defined) to include exactly one of the component markers — e.g., add
pytest.mark.core alongside pytest.mark.vllm (keeping the other existing markers
pytest.mark.unit, pytest.mark.gpu_0, pytest.mark.pre_merge) so the module
satisfies the (backend × component) marker contract.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 21a54296-dfd1-4b59-9727-292faa002414

📥 Commits

Reviewing files that changed from the base of the PR and between efd2c0c and 956bef3.

📒 Files selected for processing (4)

• components/src/dynamo/vllm/main.py
• components/src/dynamo/vllm/publisher.py
• components/src/dynamo/vllm/tests/test_vllm_publisher.py
• components/src/dynamo/vllm/worker_factory.py