[Benchmark] add benchmark for custom activation op#23908
[Benchmark] add benchmark for custom activation op#23908vllm-bot merged 13 commits intovllm-project:mainfrom
Conversation
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
@ProExpertProg Hi, I add some benchmarks for custom op. Could you please take a look? |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new benchmark script for custom activation operations. The script is well-structured, but I've found a couple of issues that should be addressed. There's a minor logic error in a conditional block that could lead to unexpected behavior, and some unreachable code due to a misunderstanding of the argparse library's error handling. Addressing these points will improve the script's correctness and maintainability.
ProExpertProg
left a comment
ProExpertProg
left a comment
There was a problem hiding this comment.
Could you also add the ability to compare to the torch.compiled forward_native numbers?
Sorry, I'm not sure what that means. I compiled the |
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
@mgoin Hi, could you please review this when you get a chance? |
ProExpertProg
left a comment
ProExpertProg
left a comment
There was a problem hiding this comment.
Instead of manually specifying the dimensions, I would just always do a sweep of popular sizes (and you can let users override via a comma-separated CLI flag).
|
I would set up the sweep similar to https://github.com/vllm-project/vllm/blob/main/benchmarks/kernels/bench_per_token_quant_fp8.py |
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
github-actions
bot
added
the
ready
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
3 participants
Purpose
Add benchmark for custom activation op.
#19817
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.