[CI] And PPL test for Qwen3.5.#35853
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds a PPL test for the Qwen3.5 model and optimizes existing PPL tests for Gemma, GPT, and other Qwen models by pre-calculating and hardcoding the Hugging Face perplexity scores. This is a good approach to speed up CI. The changes are mostly correct, but I've found a minor style inconsistency in one of the test files that should be addressed.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: wang.yuqi <noooop@126.com>
|
Hi @noooop, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
Hi @noooop, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Purpose
And PPL test for Qwen3.5.
Test Plan
tests/models/language/generation_ppl_test/test_qwen.py::test_ppl[model_info2]
Test Result
Model: Qwen/Qwen3.5-0.8B
VLLM: dtype:torch.bfloat16 head_dtype:torch.bfloat16 19.51075553894043
Transformers: Constant 19.38858413696289
Difference (%): 0.630120286837389
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.