Repository navigation
Conversation
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Contributor
|
Documentation preview: https://vllm--55915.org.readthedocs.build/en/55915/ |
mergify
Bot
added
documentation
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Etelis
marked this pull request as ready for review
September 8, 2026 18:30
Etelis
requested review from
AndreasKaratzas,
DarkLight1337,
NickLucche,
aarnphm,
njhill and
robertgshaw2-redhat
as code owners
September 8, 2026 18:30
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
almogtavor
approved these changes
Oct 12, 2026
almogtavor
left a comment
almogtavor
left a comment
Contributor
There was a problem hiding this comment.
besides that one comment, lgtm
Co-authored-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com> Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #54963.
Expose
vllm:engine_healthy{engine="<rank>"}so routers can exclude unhealthy FT ranks while their API ports remain alive. Each scrape uses the existing cachedget_status()result and reports zero if the client has failed.The implementation stays in the metrics module. A separate registry keeps rank health out of shared Prometheus files. The remaining changes are documentation and tests.
Tests
Python source
0baf5b966d, 2× NVIDIA H200, DeepSeek-V2-Lite, DP/EP=2, TP=1, NIXL 1.3.2, PyTorch 2.13.0+cu130. Compiled/package assets came from the pinned0.28.1rc1.dev472+gd9105ea80image.MODEL_NAME=deepseek-ai/DeepSeek-V2-Lite pytest tests/v1/fault_tolerance/test_fault_tolerance_e2e.py -v -s: 2 passed. Exception/retry produced1 → 0 → 1and serving resumed. Worker SIGKILL produced zero for the dead rank and its unhealthy peer.pytest tests/entrypoints/serve/instrumentator/test_fault_tolerance_metrics.py -q: 6 passed, covering client failure, missing status, per-port isolation and metric filtering. Baseline metrics module: 4 failed, 2 passed in the same container.