[Misc] Add Ray Prometheus logger to V1

[eicherseiji]

Implements the following item from #10582:

• [P1] Allow users to define their own prometheus client and other arbitrary loggers.

Tested with V1 as follows (with ray-project/ray#52719):

VLLM_USE_V1=1 python serve.py

#serve.py
import ray
from ray import serve
from ray.serve.llm import LLMConfig, build_openai_app

llm_config = LLMConfig(
    model_loading_config=dict(
        model_id="qwen-0.5b",
        model_source="Qwen/Qwen2.5-0.5B-Instruct",
    ),
    deployment_config=dict(
        autoscaling_config=dict(
            min_replicas=1, max_replicas=1,
        )
    ),
    log_engine_metrics=True
)

ray.init(num_cpus=3, num_gpus=1)
app = build_openai_app({"llm_configs": [llm_config]})
serve.run(app, blocking=True)

Ray dashboard output:

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[comaniac]

Overall LGTM. Mainly coding style and structure discussions.

Also cc @markmc

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @eicherseiji.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[markmc]

No fundamental objection to adding a vllm.v1.metrics.ray.RayPrometheusStatLogger custom logger

I'd prefer not to add the extra PrometheusMetrics indirection

And ~90% of the diff is unnecessary reformatting of code which makes it more difficult to review

[eicherseiji]

Thanks for the review @markmc! Apologies for reformatting changes. My pre-commit hook may have been configured incorrectly.

[kouroshHakha]

Hey @eicherseiji I made some changes to the interface of PrometheusStatLogger that I think will conflict your stuff, but at the same time it will make this PR easier to write and review if you incorporate those changes. for easier review https://github.com/njhill/vllm/pull/6/files but this one is actually against master (which includes other stuff) #18053

[eicherseiji]

Done. Thanks @kouroshHakha

[eicherseiji]

Starting with a rebase due to test failure:

[2025-05-14T01:44:26Z] �[36m�[1m=========================== short test summary info ============================�[0m
[2025-05-14T01:44:26Z] �[31mFAILED�[0m entrypoints/openai/test_audio.py::�[1mtest_chat_streaming_audio[https://vllm-public-assets.s3.us-west-2.amazonaws.com/multimodal_asset/mary_had_lamb.ogg-fixie-ai/ultravox-v0_5-llama-3_2-1b]�[0m - AssertionError: assert 'This audio a...t from a poem' == 'This audio a...a traditional'
[2025-05-14T01:44:26Z]   
[2025-05-14T01:44:26Z]   - This audio appears to be a snippet from a traditional
[2025-05-14T01:44:26Z]   ?                                           ^^^^^^^ ^^^
[2025-05-14T01:44:26Z]   + This audio appears to be a snippet from a poem
[2025-05-14T01:44:26Z]   ?                                           ^ ^^
[2025-05-14T01:44:26Z] �[31m====== �[31m�[1m1 failed�[0m, �[32m426 passed�[0m, �[33m2 skipped�[0m, �[33m36 warnings�[0m�[31m in 3825.08s (1:03:45)�[0m�[31m ======�[0m
�[0m^^^ +++
[2025-05-14T01:44:29Z] �[31m🚨 Error: The command exited with status 1�[0m]

[markmc]

Pretty happy to merge the ray stuff, but there's still a lot of unrelated changes I'd like to see removed

[eicherseiji]

Thanks again @markmc for the review! Going to remove mentions to multiprocess_mode and unnecessary type hint changes for now.

Let me know what the recommendation is for Ray wrapper class injection (should we prefer static class variables for now?)

[markmc]

Let me know what the recommendation is for Ray wrapper class injection (should we prefer static class variables for now?)

What you had the first time around with the static class variables was fine - I just asked to remove the extra Metrics class

[markmc]

Looks great, thanks for your patience. Just some questions on the test you added

[markmc]

You're not actually verifying metrics behavior?

[eicherseiji]

Just a smoke test for now; added a comment.

[eicherseiji]

In the future it would be helpful to verify output to avoid regressions. But setting up a ray cluster in the test environment could be non-trivial.

So I think that ROI is okay with this to start, given the current design (class injection with a wrapper that's verified in ray-project).

[eicherseiji]

Responded to comments @markmc! 🙏

[eicherseiji]

Just a smoke test for now; added a comment.

[eicherseiji]

In the future it would be helpful to verify output to avoid regressions. But setting up a ray cluster in the test environment could be non-trivial.

So I think that ROI is okay with this to start, given the current design (class injection with a wrapper that's verified in ray-project).

[markmc]

Thanks again for your patience!

[simon-mo]

stamp