[Doc][CPU binding] Add user/developer guide for CPU binding#7045
[Doc][CPU binding] Add user/developer guide for CPU binding#7045wangxiyuan merged 7 commits intovllm-project:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces comprehensive documentation for the CPU binding feature in vLLM Ascend. This feature is designed to optimize performance on ARM servers with Ascend NPUs by pinning worker processes and threads to specific CPU cores, thereby reducing cross-NUMA traffic and stabilizing inference latency. The documentation, split into developer and user guides, covers the design, allocation strategies, usage, and troubleshooting, providing users and developers with a clear guide to leverage this performance enhancement. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
bc21835 to
f65db50
Compare
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive documentation for the CPU binding feature on Ascend NPUs, adding both a detailed developer guide and a concise user guide. The changes also include updating configuration documentation for consistency. The documentation is well-structured and accurately reflects the feature's implementation. As per the repository's style guide, I have provided a suggested PR title and summary below to improve the pull request's presentation.
Suggested PR Title:
[Doc][Misc] Add user and developer guides for CPU bindingSuggested PR Summary:
### What this PR does / why we need it?
This PR adds comprehensive documentation for the CPU binding feature on Ascend NPUs. It includes:
- A detailed developer guide (`docs/source/developer_guide/feature_guide/cpu_binding.md`) covering the design, internal logic, allocation examples, and troubleshooting for the CPU binding mechanism.
- A concise user guide (`docs/source/user_guide/feature_guide/cpu_binding.md`) explaining the core concepts, usage, and common issues for end-users.
- An update to `additional_config.md` to use consistent terminology for binding strategies (`global-slicing` and `topo-affinity`).
This documentation is needed to help both developers and users understand, use, and debug the CPU binding feature, which is critical for performance on ARM+Ascend platforms.
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only update.
### How was this patch tested?
The documentation has been reviewed for clarity and technical accuracy. The examples and descriptions align with the implementation in `vllm_ascend/cpu_binding.py`.Note: Security Review has been skipped due to the limited scope of the PR.
996e641 to
f7d564e
Compare
Signed-off-by: chenchuw886 <chenchuw@huawei.com> Signed-off-by: c00818886 <chenchuwei@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
Signed-off-by: c00818886 <chenchuwei@huawei.com>
c875837 to
b5965b4
Compare
Signed-off-by: c00818886 <chenchuwei@huawei.com>
60d52b8 to
ae6e8b8
Compare
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Signed-off-by: chenchuw886 <chenchuw@huawei.com>
9d9b6a8 to
da5ff10
Compare
…ject#7045) ### What this PR does / why we need it? This PR adds comprehensive documentation for the CPU binding feature on Ascend NPUs. It includes: - A detailed developer guide (`docs/source/developer_guide/feature_guide/cpu_binding.md`) covering the design, internal logic, allocation examples, and troubleshooting for the CPU binding mechanism. - A concise user guide (`docs/source/user_guide/feature_guide/cpu_binding.md`) explaining the core concepts, usage, and common issues for end-users. - An update to `additional_config.md` to use consistent terminology for binding strategies (`global-slicing` and `topo-affinity`). This documentation is needed to help both developers and users understand, use, and debug the CPU binding feature, which is critical for performance on ARM+Ascend platforms. ### Does this PR introduce _any_ user-facing change? No. This is a documentation-only update. ### How was this patch tested? The documentation has been reviewed for clarity and technical accuracy. The examples and descriptions align with the implementation in `vllm_ascend/cpu_binding.py`. - vLLM version: v0.16.0 - vLLM main: vllm-project/vllm@4034c3d --------- Signed-off-by: chenchuw886 <chenchuw@huawei.com> Signed-off-by: c00818886 <chenchuwei@huawei.com> Co-authored-by: chenchuw886 <chenchuw@huawei.com>
What this PR does / why we need it?
This PR adds comprehensive documentation for the CPU binding feature on Ascend NPUs. It includes:
docs/source/developer_guide/feature_guide/cpu_binding.md) covering the design, internal logic, allocation examples, and troubleshooting for the CPU binding mechanism.docs/source/user_guide/feature_guide/cpu_binding.md) explaining the core concepts, usage, and common issues for end-users.additional_config.mdto use consistent terminology for binding strategies (global-slicingandtopo-affinity).This documentation is needed to help both developers and users understand, use, and debug the CPU binding feature, which is critical for performance on ARM+Ascend platforms.
Does this PR introduce any user-facing change?
No. This is a documentation-only update.
How was this patch tested?
The documentation has been reviewed for clarity and technical accuracy. The examples and descriptions align with the implementation in
vllm_ascend/cpu_binding.py.