Skip to content

Comments

[perf] fix: infer server profiler args fix#5121

Merged
wuxibin89 merged 2 commits intoverl-project:mainfrom
mengchengTang:profiler_fix
Feb 4, 2026
Merged

[perf] fix: infer server profiler args fix#5121
wuxibin89 merged 2 commits intoverl-project:mainfrom
mengchengTang:profiler_fix

Conversation

@mengchengTang
Copy link
Contributor

@mengchengTang mengchengTang commented Jan 30, 2026

What does this PR do?

Description

This PR addresses issues with profiler configuration for inference engines (vLLM and SGLang) and refactors the profiler code for better cohesion and maintainability.

Key Changes:

  • vLLM Profiler Configuration Fix:
    Implemented automatic setup of profiler-related environment variables for vLLM engine before process launch. Users no longer need to manually set vLLM environment variables.
    Added support for configuring the profiler via profiler_config for newer vLLM versions, ensuring compatibility.

  • SGLang Profiler Bug Fix:
    Fixed a bug introduced in @PR 5025 regarding SGLang profiler data collection.
    Corrected the argument construction logic to ensure accurate profiling parameters are passed to the SGLang backend.

  • Code Refactoring & Cohesion:
    Centralized profiler logic by introducing adapter functions (build_vllm_profiler_args, build_sglang_profiler_args) within the profiler component.
    These adapters encapsulate the logic for constructing engine-specific profiler arguments, improving code organization and readability.

  • Testing:
    Added comprehensive unit tests to verify the correctness of the new adapter functions and ensure regression testing for the fixed bugs.

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

@mengchengTang mengchengTang force-pushed the profiler_fix branch 2 times, most recently from 0a29dad to 33825ca Compare January 30, 2026 07:13
@mengchengTang mengchengTang changed the title server profiler optimz [perf] fix: infer server profiler args fix Jan 30, 2026
@mengchengTang mengchengTang force-pushed the profiler_fix branch 2 times, most recently from a93b92d to 0d78963 Compare January 30, 2026 09:03
@mengchengTang mengchengTang force-pushed the profiler_fix branch 3 times, most recently from fb79232 to 8b901af Compare January 31, 2026 06:55
@mengchengTang mengchengTang force-pushed the profiler_fix branch 8 times, most recently from 7372d88 to b5bf0c1 Compare February 2, 2026 06:42
@wuxibin89
Copy link
Collaborator

@mengchengTang Please fix sanity check.

@mengchengTang
Copy link
Contributor Author

mengchengTang commented Feb 2, 2026

@mengchengTang Please fix sanity check.

Got it, I'll fix it soon and add GPU/NPU verification results.


2026/02/04 update gpu and npu test result:

Scene Discrete Mode With Stack with_memory Record Shapes Steps Ranks Status
vLLM 0.11.0 + NPU TRUE FALSE FALSE FALSE [2] [1] pass
vLLM 0.11.0 + NPU TRUE TRUE TRUE TRUE [2] All pass
vLLM 0.11.0 + GPU TRUE FALSE FALSE FALSE [2] [1] pass
vLLM 0.11.0 + GPU TRUE TRUE TRUE TRUE [2] All pass
vLLM 0.13.0 + NPU TRUE FALSE FALSE FALSE [2] [1] pass
vLLM 0.13.0 + NPU TRUE TRUE TRUE TRUE [2] All pass
vLLM 0.13.0 + GPU TRUE FALSE FALSE FALSE [2] [1] pass
vLLM 0.13.0 + GPU TRUE TRUE TRUE TRUE [2] All pass
SGLang + NPU TRUE FALSE FALSE FALSE [2] [1] pass
SGLang + NPU TRUE TRUE TRUE TRUE [2] All pass
SGLang + GPU TRUE FALSE FALSE FALSE [2] [1] pass
SGLang + GPU TRUE TRUE TRUE TRUE [2] All pass

@mengchengTang mengchengTang force-pushed the profiler_fix branch 2 times, most recently from 9b2e1dd to fac853e Compare February 3, 2026 06:18
@mengchengTang mengchengTang force-pushed the profiler_fix branch 3 times, most recently from fd593ce to 431e5c3 Compare February 3, 2026 13:23

async def start_profile(self, **kwargs):
# TODO: Persist global_step to engine server-created file/path
kwargs.pop("global_step")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

wucong25
wucong25 previously approved these changes Feb 4, 2026
@wuxibin89 wuxibin89 merged commit e4915bd into verl-project:main Feb 4, 2026
83 of 89 checks passed
Tjh-UKN pushed a commit to Tjh-UKN/verl that referenced this pull request Feb 13, 2026
### What does this PR do?

**Description**

This PR addresses issues with profiler configuration for inference
engines (vLLM and SGLang) and refactors the profiler code for better
cohesion and maintainability.

**Key Changes:**

- vLLM Profiler Configuration Fix:
Implemented automatic setup of profiler-related environment variables
for vLLM engine before process launch. Users no longer need to manually
set vLLM environment variables.
Added support for configuring the profiler via profiler_config for newer
vLLM versions, ensuring compatibility.

- SGLang Profiler Bug Fix:
Fixed a bug introduced in [@pr
5025](verl-project#5025) regarding SGLang
profiler data collection.
Corrected the argument construction logic to ensure accurate profiling
parameters are passed to the SGLang backend.

- Code Refactoring & Cohesion:
Centralized profiler logic by introducing adapter functions
(build_vllm_profiler_args, build_sglang_profiler_args) within the
profiler component.
These adapters encapsulate the logic for constructing engine-specific
profiler arguments, improving code organization and readability.

- Testing:
Added comprehensive unit tests to verify the correctness of the new
adapter functions and ensure regression testing for the fixed bugs.

### Checklist Before Starting

- [x] Search for similar PRs. Paste at least one query link here: ...
- [x] Format the PR title as `[{modules}] {type}: {description}` (This
will be checked by the CI)
- `{modules}` include `fsdp`, `megatron`, `veomni`, `sglang`, `vllm`,
`rollout`, `trainer`, `ci`, `training_utils`, `recipe`, `hardware`,
`deployment`, `ray`, `worker`, `single_controller`, `misc`, `perf`,
`model`, `algo`, `env`, `tool`, `ckpt`, `doc`, `data`, `cfg`, `reward`
- If this PR involves multiple modules, separate them with `,` like
`[megatron, fsdp, doc]`
  - `{type}` is in `feat`, `fix`, `refactor`, `chore`, `test`
- If this PR breaks any API (CLI arguments, config, function signature,
etc.), add `[BREAKING]` to the beginning of the title.
  - Example: `[BREAKING][fsdp, megatron] feat: dynamic batching`

### Test

> For changes that can not be tested by CI (e.g., algorithm
implementation, new model support), validate by experiment(s) and show
results like training curve plots, evaluation results, etc.

### API and Usage Example

> Demonstrate how the API changes if any, and provide usage example(s)
if possible.

```python
# Add code snippet or script demonstrating how to use this
```

### Design & Code Changes

> Demonstrate the high-level design if this PR is complex, and list the
specific changes.

### Checklist Before Submitting

> [!IMPORTANT]
> Please check all the following items before requesting a review,
otherwise the reviewer might deprioritize this PR for review.

- [x] Read the [Contribute
Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md).
- [x] Apply [pre-commit
checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting):
`pre-commit install && pre-commit run --all-files --show-diff-on-failure
--color=always`
- [x] Add / Update [the
documentation](https://github.com/volcengine/verl/tree/main/docs).
- [x] Add unit or end-to-end test(s) to [the CI
workflow](https://github.com/volcengine/verl/tree/main/.github/workflows)
to cover all the code. If not feasible, explain why: ...
- [ ] Once your PR is ready for CI, send a message in [the `ci-request`
channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the
`verl` Slack
workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ).
(If not accessible, please try [the Feishu group
(飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).)
- [ ] If your PR is related to the `recipe` submodule, please also
update the reference to the submodule commit via `git submodule update
--remote` or `cd recipe && git pull origin main`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants