[Feature]: The v1 engine does not support add_logger. #17702
Description
🚀 The feature, motivation and pitch
When I tried to add Prometheus metric monitoring with the following code：
from vllm.entrypoints.logger import RequestLogger
from vllm.engine.metrics import RayPrometheusStatLogger
self.engine = AsyncLLMEngine.from_engine_args(engine_args)
vllm_config = self.engine_args.create_engine_config(UsageContext.ENGINE_CONTEXT)
additional_metrics_logger: RayPrometheusStatLogger = RayPrometheusStatLogger(
local_interval=0.5,
labels=dict(model_name=self.config["model_name"],instance_name = self.config["log_instance_name"]),
vllm_config=vllm_config
)
self.engine.add_logger("ray", additional_metrics_logger)The following error occurs：
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/concurrent/futures/_base.py", line 449, in result
(ServeController pid=14828) return self.__get_result()
(ServeController pid=14828) ^^^^^^^^^^^^^^^^^^^
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
(ServeController pid=14828) raise self._exception
(ServeController pid=14828) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 965, in initialize_and_get_metadata
(ServeController pid=14828) await self._replica_impl.initialize(deployment_config)
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 694, in initialize
(ServeController pid=14828) raise RuntimeError(traceback.format_exc()) from None
(ServeController pid=14828) RuntimeError: Traceback (most recent call last):
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 671, in initialize
(ServeController pid=14828) self._user_callable_asgi_app = await asyncio.wrap_future(
(ServeController pid=14828) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 1363, in initialize_callable
(ServeController pid=14828) await self._call_func_or_gen(
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/_private/replica.py", line 1324, in _call_func_or_gen
(ServeController pid=14828) result = callable(*args, **kwargs)
(ServeController pid=14828) ^^^^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14828) File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/serve/api.py", line 221, in __init__
(ServeController pid=14828) cls.__init__(self, *args, **kwargs)
(ServeController pid=14828) File "/tmp/ray/session_2025-05-06_00-07-58_411387_1/runtime_resources/working_dir_files/_ray_pkg_64c58096c64555a2/serve_model_submit.py", line 95, in __init__
(ServeController pid=14828) self.engine.add_logger("ray", additional_metrics_logger)
(ServeController pid=14828) ^^^^^^^^^^^^^^^^^^^^^^
(ServeController pid=14828) AttributeError: 'AsyncLLM' object has no attribute 'add_logger'However, the same code does not have this issue with the V0 engine.
Does the V1 engine support metric collection? It seems that add_logger cannot be used?
If not supported, when will it be supported?
If supported, how should it be used?
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.