add dfx for operator FusedDeepMoe#317
Conversation
Signed-off-by: Wangyibo1005 <2633333316@qq.com>
Summary of ChangesHello @wangyibo1005, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request implements a new, centralized DFX (Diagnostics, Fault management, and eXperience) logging and error handling mechanism. It introduces a set of utility headers ( Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new DFX (logging/error handling) framework and refactors the FusedDeepMoe operator to use it. This is a good step towards standardized logging and error reporting, providing more context in logs and structured error handling. I've identified a few areas for improvement: a potential stack overflow due to a large stack allocation in a logging macro, a generic exception handler that could be more specific, a logical error in a validation check, and some code redundancy in the new macros. Addressing these points will make the new DFX framework more robust and maintainable.
* 'main' of https://github.com/1329009851/sgl-kernel-npu: Optimize the performance of the Combine Ant Moving function and the use of HCCL buffer (sgl-project#314) deepep adapt custom cann installation path (sgl-project#327) [Chore] CANN version bump to 8.5.0 (sgl-project#326) add dfx for operator FusedDeepMoe (sgl-project#317)
* 'main' of https://github.com/sgl-project/sgl-kernel-npu: (24 commits) [Doc] Improved README.md content and English grammar and integrated the DeepWiki badge for Ask AI (sgl-project#345) (test) add solve_tril from upstream (sgl-project#339) Add AscendC triangular inverse (sgl-project#332) support the situation that topk maybe -1 on machine A3 (sgl-project#313) chunk_gated_delta_rule_npu output final state (sgl-project#341) The environment variable DEEPEP_HCCL_BUFFSIZE is added, and the priority of DEEPEP_HCCL_BUFFSIZE is higher than that of HCCL_BUFFSIZE. (sgl-project#329) Added the low_latency operator API documentation. (sgl-project#337) Added the verification of num_max_dispatch_tokens_per_rank to the decode operator adaptation layer. (sgl-project#330) Document get_dispatch_layout API (sgl-project#338) 【Doc】add fused deep moe doc (sgl-project#335) add deepep normal api doc (sgl-project#336) remove the limit that A2 internode only support topk 8 (sgl-project#323) Optimize the performance of the Combine Ant Moving function and the use of HCCL buffer (sgl-project#314) deepep adapt custom cann installation path (sgl-project#327) [Chore] CANN version bump to 8.5.0 (sgl-project#326) add dfx for operator FusedDeepMoe (sgl-project#317) Integrate ccache for faster compilation (sgl-project#318) Modify contribution guide (sgl-project#315) fix bmm transpose in cann 8.5 (sgl-project#316) fix little batchsize and int8 quant on ci (sgl-project#302) ...
* upstream/main: add function for deep-ep tests (sgl-project#301) [Doc] Improved README.md content and English grammar and integrated the DeepWiki badge for Ask AI (sgl-project#345) (test) add solve_tril from upstream (sgl-project#339) Add AscendC triangular inverse (sgl-project#332) support the situation that topk maybe -1 on machine A3 (sgl-project#313) chunk_gated_delta_rule_npu output final state (sgl-project#341) The environment variable DEEPEP_HCCL_BUFFSIZE is added, and the priority of DEEPEP_HCCL_BUFFSIZE is higher than that of HCCL_BUFFSIZE. (sgl-project#329) Added the low_latency operator API documentation. (sgl-project#337) Added the verification of num_max_dispatch_tokens_per_rank to the decode operator adaptation layer. (sgl-project#330) Document get_dispatch_layout API (sgl-project#338) 【Doc】add fused deep moe doc (sgl-project#335) add deepep normal api doc (sgl-project#336) remove the limit that A2 internode only support topk 8 (sgl-project#323) Optimize the performance of the Combine Ant Moving function and the use of HCCL buffer (sgl-project#314) deepep adapt custom cann installation path (sgl-project#327) [Chore] CANN version bump to 8.5.0 (sgl-project#326) add dfx for operator FusedDeepMoe (sgl-project#317) Integrate ccache for faster compilation (sgl-project#318)
* support small bs = 1 or 2 * support small bs = 1 or 2 * support small bs = 1 or 2 * support small bs = 1 or 2 * shallow fused topk=-1 * add dfx for operator FusedDeepMoe Signed-off-by: Wangyibo1005 <2633333316@qq.com> --------- Signed-off-by: Wangyibo1005 <2633333316@qq.com> Co-authored-by: EnigmaTHU <wang-yb22@mails.tsinghua.edu.cn>
Motivation
Debugging and triaging issues in the FusedDeepMoe operator was hindered by fragmented, ad-hoc logging and error checks. This PR introduces a centralized DFX (Diagnostics, Fault management, and eXperience) mechanism so that failures are reported in a consistent, information-rich way, improving observability and long-term maintainability.
Modification
Add a lightweight DFX utility layer via new headers (e.g., dfx_base.h, ops_log.h, ops_error.h) to provide standardized logging and structured error-reporting macros.
Refactor FusedDeepMoe host-side logic (infer/tiling paths) to adopt the new macros, replacing legacy checks/logging patterns (e.g., migrating from OP_TILING_CHECK / OP_LOGE to OPS_ERR_IF / OPS_LOG_E).
Improve log context and robustness, including consistent module/package naming and more detailed diagnostic fields (e.g., richer contextual logging and structured inner-error reporting).
Result and Report
More actionable diagnostics: errors are now reported through a unified framework with standardized formatting and richer context, making operator issues easier to reproduce and root-cause.
Cleaner and more maintainable code: logging/error handling is centralized, reducing duplicated patterns and improving consistency across operator components.
Scope: changes are primarily DFX/instrumentation and refactoring (5 files changed; net +379/−103), with no intended change to the operator’s functional behavior. Detailed debug infomation can be viewed in plog now.
Before:
Without debug detail
After:
DEEPEP_OPS[FusedDeepMoeTilingFuncImpl][48856] OpName:[FusedDeepMoe] "HCCL_BUFFSIZE is too SMALL, epRankSize = 16, maxBs = 256, moeExpertNumPerRank = 16, " " tokenLength = 7168, " " NEEDED_HCCL_BUFFSIZE(epRankSize * maxBs * moeExpertNumPerRank * tokenLength * ** TOKEN_DTYPE_BYTE_SIZE * DOUBLE_BUFFER) = 1793MB, HCCL_BUFFSIZE=200MB" [ERROR] [048856,python]: