[CI] Temporarily Disable Llama4 MoE Refactor Test#35870
Conversation
- Removed Llama-4 Scout model configurations from the file. - unable to reproduce locally Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
There was a problem hiding this comment.
Code Review
This pull request removes three Llama-4 Scout model configurations from config-h100.txt. While this may resolve issues with failing tests, it also reduces test coverage. My review includes a comment on the removed lines suggesting a better approach to temporarily disable tests to ensure they are not forgotten, which is to comment them out and add a reference to a tracking issue. This improves maintainability and ensures test coverage can be restored.
I am having trouble creating individual review comments. Click here to see my feedback.
tests/evals/gsm8k/configs/moe-refactor/config-h100.txt (11-13)
Removing these test configurations reduces test coverage for the Llama-4 Scout models. The PR description mentions being 'unable to reproduce locally', which suggests these tests might be failing.
If this is a temporary measure to unblock development, it's crucial to ensure these tests are not permanently lost. Instead of deleting the entries, please consider commenting them out and adding a note with a link to a tracking issue. This makes the temporary nature of the change explicit and helps ensure the tests are re-enabled once the underlying issue is resolved.
For example:
# Temporarily disabled due to issue [LINK TO ISSUE].
# Llama-4-Scout-Fp8-ModelOpt-fi-cutlass.yaml
# Llama-4-Scout-Fp8-ModelOpt-marlin.yaml
# Llama-4-Scout-Fp8-ModelOpt-triton.yaml
If the intention is to permanently remove these tests, the corresponding YAML configuration files should also be deleted to avoid having orphaned files in the repository.
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.