[Model] Integrate Bailing V3 and repair KDA/MXFP4 contracts (#468) - #551
Merged
yangzhuxinyzx merged 4 commits intoSep 7, 2026
Merged
Conversation
Route store_dtype == 'mxfp4' to make_deepseek_v4_mxfp4_moe_method() in Fp8Config.get_quant_method so BailingMoeV3's routed experts use the SM70-aware MXFP4 kernel (Mxfp4SM70MoEMethod on V100, Mxfp4MoEMethod otherwise) instead of the generic Fp8SM70MoEMethod/Fp8MoEMethod. Changes: - Fp8Config.__init__: add store_dtype param + self.store_dtype - Fp8Config: add ignored_layers = ignored_layers or [] (was missing, get_quant_method called is_layer_skipped(ignored_layers=self.ignored_layers)) - Fp8Config.from_config: read store_dtype from config and pass through - Fp8Config.get_quant_method: insert store_dtype == 'mxfp4' guard before the SM70 FP8-dequant fallback in the RoutedExperts branch - Add BailingMoeV3ForCausalLM to model registry - Add bailing_moe_v3.py (1Cat-adapted: FLA kda import, FusedMoE, MXFP4 config) Co-authored-by: Claude
Merge #468 on current main. Correct safe-gate forwarding, merged two-state cache metadata and copy functions, suffix precision exclusions, and fused MLA mapping. Add source-level CPU/GPU and graph regressions without model E2E. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-assisted maintainer-requested integration of #468 with original ancestry, based on latest main 378a93a (#548/#549/#550 preserved).
Audit repairs: forward kda_safe_gate into the actual kernel; make layer/model/cache-copy contracts consistently use merged QKV convolution plus FP32 recurrent state; implement segment-bounded suffix exclusions through the real FP8 dispatch while preserving exact default for existing models; register fused MLA weight mapping and reject incompatible TP head counts.
Verification: 23 focused CPU/GPU tests passed, including both gate formulas against FP64, SD/DS changing-input CUDA Graph state updates with unrelated-slot canaries, TP/PP checkpoint shard mapping, mixed MXFP4 dispatch and unquantized exclusions. Shared quantization regression selection: 54 passed. Scoped pre-commit passed. Python 3.12.13 / Torch 2.10.0+cu128 / owned V100. No full-checkpoint loading or model E2E/performance claim. Compatibility and remaining validation boundaries in docs/models/bailing_moe_v3.md. Draft until hosted gates and final source SHA review pass.