fix(e2e): skip redundant inference verify in hermes-inference-switch - #4109
fix(e2e): skip redundant inference verify in hermes-inference-switch#4109hunglp6d wants to merge 1 commit into
Conversation
The nemohermes inference set call was timing out (60 s) during endpoint verification against z-ai/glm-5.1 on NVIDIA Endpoints. This step is redundant because Phase 4 of the test already validates live inference after the switch. Pass --no-verify to eliminate the flaky timeout without reducing coverage. Fixes nightly-e2e run 26318179268 job hermes-inference-switch-e2e. Signed-off-by: Hung Le <hple@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
PR Review AdvisorFindings: 0 needs attention, 0 worth checking, 0 nice ideas This is an automated advisory review. A human maintainer must make the final merge decision. |
E2E Advisor RecommendationRequired E2E: None Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
E2E Scenario Advisor RecommendationRequired scenario E2E: None Full scenario advisor summaryE2E Scenario AdvisorBase: Required scenario E2E
Optional scenario E2E
Relevant changed files
|
|
Closed by #4152, which retries the transient verify and falls back to --no-verify — exactly the failure mode this issue hit. Today's nightly hermes-inference-switch-e2e is green. |
Summary
The
hermes-inference-switch-e2enightly job failed becausenemohermes inference settimed out (60 s) verifying thez-ai/glm-5.1model athttps://integrate.api.nvidia.com/v1/chat/completions. The verification is redundant: Phase 4 of the test already validates live inference after the switch viacheck_inference_localandcheck_hermes_api_chat.This PR passes
--no-verifyto skip the endpoint verification timeout without reducing coverage.Related Issue
Fixes #4111
Changes
test/e2e/test-hermes-inference-switch.sh: Added--no-verifyto thenemohermes inference setcall, with a comment explaining why verification is redundant in this test.Validation
The
custom-e2evalidation branch could not be pushed because the PAT lacksworkflowscope (GitHub blocks pushes of.github/workflows/files without it). To validate manually:7c7f7a428624ad72082d7b11395e25d7ae43daadhermes-inference-switch-e2e (#77481693221)Type of Change
Verification
npx prek run --all-filespassesnpm testpassesAI Disclosure
Signed-off-by: Hung Le hple@nvidia.com