[mickqian]

Summary

• add tuned Triton MoE configurations for Qwen3-VL and Kimi-VL on H100, plus Qwen3-VL on H200
• reuse a tuned up-projection configuration when a separate down-projection configuration is unavailable
• add Kimi-VL architecture recognition to the MoE tuning helper and configurable tuner batch sizes/search spaces
• align the FlashInfer all-reduce microbenchmark precision with the serving path

Root cause

The VLM MoE shapes were missing current-Triton hardware configurations, causing heuristic or older-config fallback. Kimi-VL was also not recognized by the generic tuning helper.

Performance data

Isolated H100 Kimi-VL MoE (E=16, N=1408) tuning improves the selected Triton configurations by 19.8–47% across the measured batch-size sweep. End-to-end Kimi-VL H100 TP=4 at request-rate=8 improves from 41.59 ms to 40.95 ms mean E2E (1.6%); this deliberately reports the smaller application-level effect rather than extrapolating the microbenchmark gain.

The added Qwen configurations cover H100 TP=4 (E=32, N=768) and H200 TP=1 (E=128, N=768).

Validation

• H100 CUDA MoE configuration tests (2 passed)
• ruff format --check, ruff check, git diff --check, and project pre-commit hooks

Impact

Uses reproducible tuned configuration selection instead of fallback paths for the measured VLM MoE workloads.

---

CI States

Latest PR Test (Base): ✅ Run #29152897359
Latest PR Test (Extra): ❌ Run #29152897303

[gemini-code-assist]

Code Review

This pull request introduces several enhancements to the benchmarking and configuration of Triton MoE and FlashInfer kernels, including support for the KimiVL model, the ability to specify custom Triton search space files and batch sizes, and a fallback mechanism to reuse tuned up-projection configs for down-projection when separate configs are missing. It also adds Triton 3.6.0 configuration files for NVIDIA H200 and H100 GPUs alongside corresponding unit tests. Feedback on the changes suggests improving the robustness of the custom search space file loading by explicitly specifying UTF-8 encoding and validating that the loaded configurations contain all required Triton keys.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

When loading a custom search space file via --search-space-file, there are two potential issues:

1. Opening the file without an explicit encoding can lead to UnicodeDecodeError on platforms where the default encoding is not UTF-8 (e.g., Windows).
2. If the JSON file contains configurations that are missing required Triton keys (such as BLOCK_SIZE_M, BLOCK_SIZE_N, BLOCK_SIZE_K, etc.), it will cause cryptic KeyErrors later during filtering or sorting.

We should open the file with encoding="utf-8" and validate that each configuration dictionary contains all the required keys.

        if args.search_space_file:
            with open(args.search_space_file, encoding="utf-8") as f:
                search_space = json.load(f)
            if not isinstance(search_space, list) or not all(
                isinstance(config, dict) for config in search_space
            ):
                raise ValueError(
                    "--search-space-file must contain a JSON list of configs"
                )
            required_keys = {"BLOCK_SIZE_M", "BLOCK_SIZE_N", "BLOCK_SIZE_K", "GROUP_SIZE_M", "num_warps", "num_stages"}
            for i, config in enumerate(search_space):
                if not required_keys.issubset(config.keys()):
                    raise ValueError(
                        f"Config at index {i} in --search-space-file is missing required keys: {required_keys - config.keys()}"
                    )

[mickqian]

/tag-and-rerun-ci