fix(models): share the cold-rebuild budget across serial custom probes (#7481) - #17
Open
RajPabnani03 wants to merge 1 commit into
Open
RajPabnani03 wants to merge 1 commit into
RajPabnani03 wants to merge 1 commit into
Conversation
nesquena#7481) Custom /v1/models probes ran serially with a 5s per-endpoint cap against a 4s foreground rebuild budget, so one unreachable active endpoint starved every named custom provider behind it. Hand each serial custom probe min(cap, budget/N). Keep probe order, SSRF, auth, and out-of-band publication unchanged. Refs nesquena#7481 Amp-Thread-ID: https://ampcode.com/threads/T-01a0a025-57cd-772b-9b81-3f7007899e56 Co-authored-by: Raj Pabnani <rajpabnani01@gmail.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Serial custom
/v1/modelsprobes used a 5s per-endpoint cap against a 4s foreground rebuild budget, so one unreachable active endpoint starved later reachable named providers. Timeout is nowmin(cap, budget/N). Probe order, SSRF, auth, and out-of-band publication are unchanged.Refs nesquena#7481 (not Fixes: out-of-band generation/disk races from PR nesquena#7506 review are out of scope).
Canonical target
Intended against nesquena/hermes-webui:master. Cross-fork
createPullRequestreturned 403 Resource not accessible by integration for this token.Cherry-pick onto current upstream
master(94fd2da8):0bbb7e7e1b32abdd1bac5bd40515dc16b038af26(local upstream-based equivalent:
7eb27b4a/ originalf3b71a44)Compare (will look 18 ahead / 1922 behind if opened as-is):
nesquena/hermes-webui@master...RajPabnani03:fix/7481-custom-probe-budget-fairness
Overlaps / conflicts with open PR nesquena#7506 (larger scheduler + generation fence; still CHANGES_REQUESTED). This PR is the narrower timeout-share only.