Skip to content

[Doc][CPU binding] Add user/developer guide for CPU binding#7045

Merged
wangxiyuan merged 7 commits intovllm-project:mainfrom
chenchuw886:cpu_bind_docs
Mar 10, 2026
Merged

[Doc][CPU binding] Add user/developer guide for CPU binding#7045
wangxiyuan merged 7 commits intovllm-project:mainfrom
chenchuw886:cpu_bind_docs

Conversation

@chenchuw886
Copy link
Copy Markdown
Contributor

@chenchuw886 chenchuw886 commented Mar 6, 2026

What this PR does / why we need it?

This PR adds comprehensive documentation for the CPU binding feature on Ascend NPUs. It includes:

  • A detailed developer guide (docs/source/developer_guide/feature_guide/cpu_binding.md) covering the design, internal logic, allocation examples, and troubleshooting for the CPU binding mechanism.
  • A concise user guide (docs/source/user_guide/feature_guide/cpu_binding.md) explaining the core concepts, usage, and common issues for end-users.
  • An update to additional_config.md to use consistent terminology for binding strategies (global-slicing and topo-affinity).

This documentation is needed to help both developers and users understand, use, and debug the CPU binding feature, which is critical for performance on ARM+Ascend platforms.

Does this PR introduce any user-facing change?

No. This is a documentation-only update.

How was this patch tested?

The documentation has been reviewed for clarity and technical accuracy. The examples and descriptions align with the implementation in vllm_ascend/cpu_binding.py.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces comprehensive documentation for the CPU binding feature in vLLM Ascend. This feature is designed to optimize performance on ARM servers with Ascend NPUs by pinning worker processes and threads to specific CPU cores, thereby reducing cross-NUMA traffic and stabilizing inference latency. The documentation, split into developer and user guides, covers the design, allocation strategies, usage, and troubleshooting, providing users and developers with a clear guide to leverage this performance enhancement.

Highlights

  • New CPU Binding Documentation (Developer Guide): A comprehensive developer guide for CPU binding has been added, detailing its design, execution flow, allocation examples, and technical references for vLLM Ascend on ARM systems.
  • New CPU Binding Documentation (User Guide): A user-focused guide for CPU binding has been added, explaining core concepts, usage examples, troubleshooting, and limitations, including a Mermaid diagram for visual understanding.
  • Updated Configuration Description: The description for the enable_cpu_binding configuration option has been updated to accurately reflect the CPU allocation strategies (global-slicing for A3 and topo-affinity for other device types).

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • docs/source/developer_guide/feature_guide/cpu_binding.md
    • Added a detailed developer guide for CPU binding, covering its design, execution flow, allocation examples, and technical references.
  • docs/source/user_guide/configuration/additional_config.md
    • Updated the description of the enable_cpu_binding configuration option to accurately reflect the CPU allocation strategies.
  • docs/source/user_guide/feature_guide/cpu_binding.md
    • Added a user-focused guide for CPU binding, including core concepts, usage examples, troubleshooting, and limitations, along with a Mermaid diagram for the execution flow.
Activity
  • No specific activity has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Mar 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 6, 2026

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive documentation for the CPU binding feature on Ascend NPUs, adding both a detailed developer guide and a concise user guide. The changes also include updating configuration documentation for consistency. The documentation is well-structured and accurately reflects the feature's implementation. As per the repository's style guide, I have provided a suggested PR title and summary below to improve the pull request's presentation.

Suggested PR Title:

[Doc][Misc] Add user and developer guides for CPU binding

Suggested PR Summary:

### What this PR does / why we need it?
This PR adds comprehensive documentation for the CPU binding feature on Ascend NPUs. It includes:

- A detailed developer guide (`docs/source/developer_guide/feature_guide/cpu_binding.md`) covering the design, internal logic, allocation examples, and troubleshooting for the CPU binding mechanism.
- A concise user guide (`docs/source/user_guide/feature_guide/cpu_binding.md`) explaining the core concepts, usage, and common issues for end-users.
- An update to `additional_config.md` to use consistent terminology for binding strategies (`global-slicing` and `topo-affinity`).

This documentation is needed to help both developers and users understand, use, and debug the CPU binding feature, which is critical for performance on ARM+Ascend platforms.

### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only update.

### How was this patch tested?
The documentation has been reviewed for clarity and technical accuracy. The examples and descriptions align with the implementation in `vllm_ascend/cpu_binding.py`.

Note: Security Review has been skipped due to the limited scope of the PR.

@chenchuw886 chenchuw886 changed the title [Doc] Add user/developer guide for CPU binding [Doc][CPU binding] Add user/developer guide for CPU binding Mar 6, 2026
chenchuw886 and others added 3 commits March 7, 2026 01:18
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Comment thread docs/source/user_guide/feature_guide/cpu_binding.md Outdated
chenchuw886 added 3 commits March 10, 2026 09:49
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
@wangxiyuan wangxiyuan merged commit 14c71b1 into vllm-project:main Mar 10, 2026
22 checks passed
Nagisa125 pushed a commit to starmountain1997/vllm-ascend that referenced this pull request Mar 17, 2026
…ject#7045)

### What this PR does / why we need it?
This PR adds comprehensive documentation for the CPU binding feature on
Ascend NPUs. It includes:

- A detailed developer guide
(`docs/source/developer_guide/feature_guide/cpu_binding.md`) covering
the design, internal logic, allocation examples, and troubleshooting for
the CPU binding mechanism.
- A concise user guide
(`docs/source/user_guide/feature_guide/cpu_binding.md`) explaining the
core concepts, usage, and common issues for end-users.
- An update to `additional_config.md` to use consistent terminology for
binding strategies (`global-slicing` and `topo-affinity`).

This documentation is needed to help both developers and users
understand, use, and debug the CPU binding feature, which is critical
for performance on ARM+Ascend platforms.

### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only update.

### How was this patch tested?
The documentation has been reviewed for clarity and technical accuracy.
The examples and descriptions align with the implementation in
`vllm_ascend/cpu_binding.py`.

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@4034c3d

---------

Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Co-authored-by: chenchuw886 <chenchuw@huawei.com>
@chenchuw886 chenchuw886 deleted the cpu_bind_docs branch April 15, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants