[CI] Add long and short prompt tests for DeepSeek-V3.2

[starmountain1997]

What this PR does / why we need it?

This PR enhances the test_deepseek3_2_w8a8_pruning_mtp_tp2_ep E2E test by adding both short and long prompt test cases:

• Short test: Validates basic functionality with minimal input ("Hello ")
• Long test: Validates the model can handle prompts near its maximum context length (~163K tokens, approaching the max_position_embeddings limit of 163,840)

Additionally, explicitly sets max_model_len=163840 to ensure the test properly exercises the model's full context window capability.

Does this PR introduce any user-facing change?

No. This change only affects internal E2E testing infrastructure.

How was this patch tested?

The modified test case will be executed as part of the E2E test suite and has been validated here.

• vLLM version: v0.15.0
• vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0

[gemini-code-assist]

Summary of Changes

Hello @starmountain1997, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances an existing distributed offline inference test for the DeepSeek-V3.2 model by introducing distinct prompt sets for short and long inputs. The primary goal is to improve the test coverage for varying input lengths, particularly to validate the model's behavior and performance when processing very long sequences, which is crucial for robust model deployment.

Highlights

• Test Prompt Refactoring: The existing example_prompts in the test_deepseek3_2_w8a8_pruning_mtp_tp2_ep function have been refactored into two distinct lists: short_example_prompts and long_example_prompts.
• Introduction of Long Input Test Case: An extremely long prompt string, created by repeating 'Hello ' 163,840 times, has been added to short_example_prompts to specifically test the model's handling of very large inputs.
• Dual Test Execution: The vllm_model.generate_greedy method is now called twice within the test, once for short_example_prompts and once for long_example_prompts, allowing for separate evaluation of different input lengths.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• tests/e2e/multicard/2-cards/test_offline_inference_distributed.py
  • Replaced the single example_prompts list with two separate lists: short_example_prompts and long_example_prompts.
  • Added a very long string (163,840 repetitions of 'Hello ') to short_example_prompts to simulate a large input.
  • Modified the vllm_model.generate_greedy call to be executed twice, once for short_example_prompts and once for long_example_prompts.
  • Included a TODO comment to re-evaluate the max_tokens setting for these new test cases.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/scripts/config.yaml

Activity

• No human activity (comments, reviews, approvals) has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Code Review

This pull request attempts to add a test for long and short prompts for the DeepSeek v3.2 model. However, the implementation has a couple of critical issues. The list for long prompts is empty, making that part of the test a no-op. Additionally, the VllmRunner is not configured with a max_model_len large enough to handle the very long prompt being added, which will cause the test to fail. I've added comments with suggestions to fix these issues. I've also suggested a more descriptive PR title and a summary following the repository's style guide.

Suggested PR Title:

[Test][Feature] Add long and short prompt test for DeepSeek v3.2

Suggested PR Summary:

### What this PR does / why we need it?

This PR adds a test case to `test_offline_inference_distributed.py` to verify the handling of both short and very long prompts with the `DeepSeek-V3.2-W8A8-Pruning` model. This is necessary to ensure the model's stability and correctness when processing inputs of varying lengths, especially those approaching the maximum context window.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

This PR consists of a new test case. It will be validated by the CI pipeline.

[gemini-code-assist]

This long prompt will likely cause the test to fail because VllmRunner is initialized with the default max_model_len=1024. Please set max_model_len in the VllmRunner constructor to a value large enough to handle this prompt, for example max_model_len=163840.

[gemini-code-assist]

The long_example_prompts list is empty. As a result, the call to vllm_model.generate_greedy(long_example_prompts, max_tokens) on line 274 will be a no-op. To make the test meaningful, this list should be populated with long prompts.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.