Repository navigation
fix(bench): send bearer auth for keyed decode runs - #136
Conversation
plotarmordev
added
the
triage/P2
d7dd7de raised the shipped per-prompt image cap from 4 to 100 in start.sh, start-tp4.sh, .env.example and three README references, but the filer-facing hint block in .github/ISSUE_TEMPLATE/1_bug.md was missed. This is the only tracked site left saying 4 (git grep 'image:4' returns this line and nothing else) and it contradicts README:198/380/692, .env.example:162, start.sh:180 and start-tp4.sh:230. It sits in the text a reporter reads while deciding what to check, so it re-seeds the confusion the raised default was meant to remove: the cap is a copy-once .env value (start.sh:52), so an operator whose .env still pins 4 is told here that 4 is current and reads the mismatch as a server fault rather than a local config value. The boot log already prints the effective value (start.sh:1184). Comment-only change: no knob, script behaviour, default or measured number changes, and nothing is executed on either node.
plotarmordev
added
P2
|
Maintainer follow-up before merge: Retain the transport-only bearer-auth change, add loopback HTTP coverage for key precedence/empty values and keyless health/metrics, and document the precedence contract. Applying normal follow-up/prerequisite commits, preserving the original PR ancestry without a force push. Prepared exact head: |
plotarmordev
left a comment
plotarmordev
left a comment
There was a problem hiding this comment.
Approving exact corrected head 5f2db3d0e1d77b38e183eafa7b50e523d29b6a3f after independent HELP review and the focused checks recorded above. Retain the transport-only bearer-auth change, add loopback HTTP coverage for key precedence/empty values and keyless health/metrics, and document the precedence contract.
Proceeding with a normal, head-pinned merge. No hosted checks are configured/reported; local results are not being represented as hosted CI. No new GPU or performance qualification is claimed. Follow-up: run merged-branch hosted validation when CI becomes available.
plotarmordev
added
the
reviewed
Summary
Closes #117.
tests/bench_decode.pynow sendsAuthorization: Bearer <key>whenAPI_KEYor the documentedVLLM_API_KEYenvironment variable is set. This makes the decode benchmark usable against keyed/v1serves.The change is transport-only: prompts, timing, token calculations, and speculative-decoding accounting are unchanged.
/healthand/metricsremain unchanged because the README documents those endpoints as unguarded.Testing
python3 tests/test_bench_decode_auth.pypython3 -m py_compile tests/bench_decode.py tests/test_bench_decode_auth.pypython3 tests/test_bringup_robustness.pypython3 tests/test_numeric_config.pypython3 tests/test_start_overrides.pypython3 tests/test_suppress_stops.pypython3 tests/test_warm_restart_stdout.pygit diff --checkNo live 2-node DGX Spark run was performed because this workspace does not have the target hardware.
Checklist