[CI] Add support for testing dependency commits before release#2353
[CI] Add support for testing dependency commits before release#2353yzh119 merged 1 commit intoflashinfer-ai:mainfrom
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughThe changes establish a CI environment configuration system by introducing a new setup script that sources a Python package override configuration file, allowing conditional dependency version pinning during tests. This setup script is integrated into eleven test task scripts to ensure environment preparation before package installation. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @yongwww, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the Continuous Integration (CI) pipeline by introducing a flexible system for overriding Python package dependencies. This allows developers to test specific versions or commits of upstream packages, such as TVM-FFI, directly within CI without the overhead of rebuilding Docker images. The feature is designed to be optional and is disabled by default, providing a targeted way to validate dependency changes in test branches before they are officially released, thereby improving the robustness of the development workflow. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a useful mechanism for testing specific commits of dependencies in CI by allowing overrides of Python package versions. The changes are well-structured, with a new environment file for configuration and a setup script to apply the overrides. The modifications to the existing test scripts to incorporate this new mechanism are straightforward.
I have one suggestion for the new setup_test_env.sh script to improve its robustness and conciseness. Overall, this is a good addition to the CI workflow.
yongwww
requested review from
cyx-6,
jimmyzho,
kahyunnam,
nvmbreughe and
yzh119
as code owners
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@scripts/setup_test_env.sh`:
- Around line 18-24: The pip install for the TVM-FFI override can fail silently;
update the block that checks TVM_FFI_REF and runs pip install (the lines
referencing TVM_FFI_REF and the pip install
"git+https://github.com/apache/tvm-ffi.git@${TVM_FFI_REF}") to detect failure
and stop the script: after running pip install, check its exit status and if
non-zero log a clear error including the TVM_FFI_REF value and exit with a
non-zero code (or enable errexit at the top of the script) so tests won't
proceed with an unexpected/missing TVM-FFI version.
🧹 Nitpick comments (1)
ci/setup_python.env (1)
1-17: Well-documented configuration file.The documentation is clear and the examples are helpful. The warning on line 9 about not merging to main is important.
Consider adding a CI check to prevent accidental merges with active overrides.
A simple CI step could grep for uncommented
_REF=assignments and fail if targeting the main branch, providing a safety net against human error.
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
ci/setup_python.envscripts/setup_test_env.shscripts/task_jit_run_tests_part1.shscripts/task_jit_run_tests_part2.shscripts/task_jit_run_tests_part3.shscripts/task_jit_run_tests_part4.shscripts/task_jit_run_tests_part5.shscripts/task_test_blackwell_kernels.shscripts/task_test_jit_cache_package_build_import.shscripts/task_test_multi_node_comm_kernels.shscripts/task_test_nightly_build.shscripts/task_test_single_node_comm_kernels.sh
🧰 Additional context used
🧠 Learnings (4)
📚 Learning: 2025-12-30T09:34:39.900Z
Learnt from: CR
Repo: flashinfer-ai/flashinfer PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-30T09:34:39.900Z
Learning: Clear JIT cache with `rm -rf ~/.cache/flashinfer/` when debugging compilation issues or changing compilation flags
Applied to files:
scripts/task_test_jit_cache_package_build_import.sh
📚 Learning: 2025-12-30T09:34:39.900Z
Learnt from: CR
Repo: flashinfer-ai/flashinfer PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-30T09:34:39.900Z
Learning: Applies to include/**/*.cuh : Kernel code in `include/flashinfer/` is automatically picked up by JIT compilation on changes - no pip reinstall needed
Applied to files:
scripts/task_test_jit_cache_package_build_import.sh
📚 Learning: 2025-12-30T09:34:39.900Z
Learnt from: CR
Repo: flashinfer-ai/flashinfer PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-30T09:34:39.900Z
Learning: Applies to flashinfer/jit/**/*.py : JIT module generators in `flashinfer/jit/` must follow the pattern: compute URI → create directory → (optional) render Jinja template → copy sources → return JitSpec
Applied to files:
scripts/task_test_jit_cache_package_build_import.sh
📚 Learning: 2025-12-30T09:34:39.900Z
Learnt from: CR
Repo: flashinfer-ai/flashinfer PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-30T09:34:39.900Z
Learning: Applies to csrc/**/*_jit_binding.cu : Create TVM-FFI bindings in files matching the pattern `csrc/*_jit_binding.cu` using the `TVM_FFI_DLL_EXPORT_TYPED_FUNC(name, func)` macro to expose C++ functions
Applied to files:
scripts/task_jit_run_tests_part2.sh
🔇 Additional comments (11)
scripts/task_jit_run_tests_part2.sh (1)
9-10: LGTM!The environment setup is correctly sourced before package installation, and the use of
${BASH_SOURCE[0]}ensures reliable path resolution regardless of how the script is invoked.scripts/task_test_multi_node_comm_kernels.sh (1)
8-9: LGTM!Consistent with the pattern used across other test scripts.
scripts/task_test_single_node_comm_kernels.sh (1)
8-9: LGTM!Consistent placement before package installation.
scripts/task_jit_run_tests_part3.sh (1)
9-10: LGTM!Consistent with other
task_jit_run_tests_part*.shscripts.scripts/task_test_nightly_build.sh (1)
6-7: No issue with double-sourcing;setup_test_env.shis idempotent.The script is sourced at line 7 and again indirectly at line 58 via
task_jit_run_tests_part${TEST_SHARD}.sh. However,setup_test_env.shperforms only idempotent operations: it sources an environment file if present and conditionally installs a package, both of which are safe to repeat. TheSKIP_INSTALL=1environment variable set before the child script further mitigates any redundancy concerns.scripts/task_jit_run_tests_part5.sh (1)
9-10: LGTM!The environment setup is correctly placed before the
SKIP_INSTALLcheck, ensuring package overrides are applied regardless of whether the main package installation is skipped. The path resolution usingBASH_SOURCEis robust.scripts/task_test_blackwell_kernels.sh (1)
5-6: LGTM!The environment setup is correctly placed at the beginning of the script, before any variable definitions or test logic. This ensures package overrides are applied consistently.
scripts/setup_test_env.sh (1)
1-16: LGTM on path resolution and env file sourcing.The script correctly determines the repository root and conditionally sources the override file. Note: A previous review suggested adding
set -euo pipefail, but since this is a sourced script, strict error settings would affect the parent scripts. The current approach is acceptable, though you may consider addingset -eif you want failures to halt execution.scripts/task_test_jit_cache_package_build_import.sh (1)
6-7: LGTM!The environment setup is correctly integrated at the beginning of the script, ensuring package overrides are applied before the flashinfer package installation and JIT cache build steps.
scripts/task_jit_run_tests_part4.sh (1)
9-11: LGTM!The placement before
pip installis correct, ensuring package overrides are applied before installation. The path resolution using$(dirname "${BASH_SOURCE[0]}")is robust and handles execution from any working directory.scripts/task_jit_run_tests_part1.sh (1)
9-11: LGTM!Consistent with the other test scripts in this PR. The environment setup is correctly placed before package installation.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
|
/bot run |
3 participants
📌 Description
Add mechanism to override Python package versions (e.g., TVM-FFI) in CI without rebuilding Docker images. This enables testing specific commits of dependencies before they are officially released.
This is optional and disabled by default. No packages are overridden in the normal CI workflow. The override is intended for use in test PRs to validate dependency changes before they are released upstream.
🔍 Related Issues
🚀 Pull Request Checklist
Thank you for contributing to FlashInfer! Before we review your pull request, please make sure the following items are complete.
✅ Pre-commit Checks
pre-commitby runningpip install pre-commit(or used your preferred method).pre-commit install.pre-commit run --all-filesand fixed any reported issues.🧪 Tests
unittest, etc.).Reviewer Notes
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.