[CI] Pin DeepSeek-V4 Flash GPQA nightly to low reasoning effort

[GDzhu01]

Summary

• Pass the optional reasoning_effort benchmark setting through to AISBench chat request generation.
• Set reasoning_effort: low for the DeepSeek-V4 Flash W8A8 A3 nightly GPQA case. Keep thinking: true and the performance case unchanged.
• Add a focused test for the generated request config, including unchanged behavior when the setting is absent.

Context

The failing nightly job evaluated this case without an explicit reasoning effort. The DeepSeek-V4 default thinking strength changed upstream, so this pins the intended GPQA protocol rather than relying on that default.

An earlier manual GPQA Diamond run using reasoning_effort="low" and chat_template_kwargs={"thinking": True} scored 88.38% (175/198) on the e5118d1/vLLM 0.28.0 setup. That is supporting evidence, not a validation of this PR's current-main nightly run.

Validation

• Parsed the modified YAML and Python files; confirmed only the GPQA benchmark selects low.

• git diff --check passed.

• The new unit test and full A3 nightly GPQA run are pending CI/runner execution.

• vLLM main: vllm-project/vllm@84030bb

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the AISBench tool to support the reasoning_effort parameter, enabling more precise control over model inference configurations. It specifically applies this setting to the DeepSeek-V4 Flash nightly GPQA benchmark to stabilize performance evaluation against upstream default changes. The changes include updates to the runner logic and the addition of a comprehensive unit test to ensure configuration generation remains accurate.

Highlights

• Reasoning Effort Support: Added support for the optional reasoning_effort parameter in AISBench chat request generation to allow finer control over model behavior.
• GPQA Benchmark Configuration: Pinned reasoning_effort: low for the DeepSeek-V4 Flash W8A8 A3 nightly GPQA test case to ensure consistent evaluation results.
• New Unit Testing: Introduced a new unit test to verify that request configurations are correctly generated, including handling of the new reasoning_effort setting.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Code Review

Suggested PR Title:

[Benchmark][Feature] Add support for reasoning_effort configuration in aisbench

Suggested PR Summary:

### What this PR does / why we need it?
This PR introduces support for the `reasoning_effort` configuration in the `aisbench` benchmarking tool. This allows specifying the reasoning effort (e.g., "low") for models like DeepSeek-V4-Flash. The changes include:
- Retrieving and propagating `reasoning_effort` in `AisbenchRunner`.
- Updating the request configuration generator to inject `reasoning_effort` into the generated python configuration file.
- Adding a nightly E2E test configuration for `DeepSeek-V4-Flash-W8A8-A3` with `reasoning_effort: low`.
- Adding unit tests to verify the correct generation of the request configuration when `reasoning_effort` is specified.

### Does this PR introduce _any_ user-facing change?
Yes, users can now configure `reasoning_effort` in their `aisbench` configuration files.

### How was this patch tested?
- Added unit test `test_request_config_reasoning_effort` in `tests/ut/tools/test_aisbench.py`.
- Added nightly E2E configuration in `tests/e2e/nightly/single_node/models/configs/DeepSeek-V4-Flash-W8A8-A3.yaml`.

I have no feedback to provide as the implementation is correct and well-tested.