Skip to content

[CI] Temporarily Disable Llama4 MoE Refactor Test#35870

Merged
khluu merged 1 commit intomainfrom
robertgshaw2-redhat-patch-1
Mar 3, 2026
Merged

[CI] Temporarily Disable Llama4 MoE Refactor Test#35870
khluu merged 1 commit intomainfrom
robertgshaw2-redhat-patch-1

Conversation

@robertgshaw2-redhat
Copy link
Copy Markdown
Collaborator

@robertgshaw2-redhat robertgshaw2-redhat commented Mar 3, 2026

  • Removed Llama-4 Scout model configurations from the file.
  • unable to reproduce locally

- Removed Llama-4 Scout model configurations from the file.
- unable to reproduce locally

Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
@robertgshaw2-redhat robertgshaw2-redhat changed the title Remove Llama-4 Scout model configs from config-h100.txt [CI] Temporarily Disable Llama4 MoE Refactor Test Mar 3, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes three Llama-4 Scout model configurations from config-h100.txt. While this may resolve issues with failing tests, it also reduces test coverage. My review includes a comment on the removed lines suggesting a better approach to temporarily disable tests to ensure they are not forgotten, which is to comment them out and add a reference to a tracking issue. This improves maintainability and ensures test coverage can be restored.

I am having trouble creating individual review comments. Click here to see my feedback.

tests/evals/gsm8k/configs/moe-refactor/config-h100.txt (11-13)

high

Removing these test configurations reduces test coverage for the Llama-4 Scout models. The PR description mentions being 'unable to reproduce locally', which suggests these tests might be failing.

If this is a temporary measure to unblock development, it's crucial to ensure these tests are not permanently lost. Instead of deleting the entries, please consider commenting them out and adding a note with a link to a tracking issue. This makes the temporary nature of the change explicit and helps ensure the tests are re-enabled once the underlying issue is resolved.

For example:

# Temporarily disabled due to issue [LINK TO ISSUE].
# Llama-4-Scout-Fp8-ModelOpt-fi-cutlass.yaml
# Llama-4-Scout-Fp8-ModelOpt-marlin.yaml
# Llama-4-Scout-Fp8-ModelOpt-triton.yaml

If the intention is to permanently remove these tests, the corresponding YAML configuration files should also be deleted to avoid having orphaned files in the repository.

@mergify mergify Bot added the llama Related to Llama models label Mar 3, 2026
@robertgshaw2-redhat robertgshaw2-redhat added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 3, 2026
@khluu khluu merged commit 881a6b0 into main Mar 3, 2026
17 checks passed
@khluu khluu deleted the robertgshaw2-redhat-patch-1 branch March 3, 2026 18:36
Copilot AI pushed a commit to machov/vllm that referenced this pull request Mar 10, 2026
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
avinashsingh77 pushed a commit to avinashsingh77/vllm that referenced this pull request Mar 12, 2026
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
wendyliu235 pushed a commit to wendyliu235/vllm-public that referenced this pull request Mar 18, 2026
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

llama Related to Llama models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants