feat: add collect-env environment report tool + issue templates (bug, feature request) - #4007
Conversation
Issue reporters often cannot state their environment precisely, and the most common unreproducible-issue root cause is version confusion: multiple copies of cuDNN/cuBLAS/CUDA runtime installed where the loaded one is not the one the user assumes. Add flashinfer/collect_env.py (python -m flashinfer.collect_env / flashinfer collect-env [--json]) that dumps a paste-ready report: - flashinfer / flashinfer-cubin / flashinfer-jit-cache versions with an explicit mismatch flag, JIT cache and local cubin store stats, resolved target CUDA archs - GPU/driver properties in CUDA enumeration order (nvidia-smi order as no-torch fallback), CUDA toolkit as resolved by flashinfer's JIT vs env - loaded-vs-installed GPU library disambiguation: force-load, then parse /proc/self/maps for what is actually mapped and scan LD_LIBRARY_PATH / site-packages / CUDA_HOME / ldconfig for other on-disk installs, aggregated per directory with pip provenance and conflict flags - torch build info, cuDNN frontend + loaded backend version, relevant package versions incl. serving frameworks' declared flashinfer pins, FLASHINFER_*/CUDA*/... environment variables, GPU topology The module is standalone-runnable (stdlib-only at import; every probe individually guarded) so it still produces a report when import flashinfer or torch is broken, and is strictly offline/read-only. Also add the repo's first GitHub issue template, wiring the command in as a required Environment field. AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds a standalone environment collector covering Python, CUDA, GPU, package, library, and topology details; exposes text and JSON CLI commands; adds tests, documentation, and structured bug and feature request forms. ChangesEnvironment Reporting
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Reporter
participant collect-env CLI
participant collect_env_info
participant GPU probes
participant format_report
Reporter->>collect-env CLI: Run collect-env
collect-env CLI->>collect_env_info: Gather environment report
collect_env_info->>GPU probes: Inspect CUDA, GPUs, and libraries
GPU probes-->>collect_env_info: Return guarded probe results
collect_env_info-->>collect-env CLI: Return ordered report
collect-env CLI->>format_report: Format text output
format_report-->>Reporter: Display report
Reporter->>Reporter: Paste report into bug form
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new environment diagnostic tool, flashinfer.collect_env, to assist in debugging by gathering system, GPU, and library information. It includes a new CLI command, an updated issue template, and comprehensive unit tests. The reviewer provided a suggestion to improve the robustness of the version-parsing regular expression in flashinfer/collect_env.py to better handle various versioning schemes.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| tags = [] | ||
| versions = set() | ||
| for p in paths: | ||
| m = re.search(r"\.so\.(\d+(?:\.\d+)+)$", p) |
There was a problem hiding this comment.
The regular expression to find the library version from the filename might not handle all versioning schemes. Specifically, it requires at least one dot in the version number (e.g., 1.2), and won't match single-number versions like libfoo.so.8.
To make it more robust, I suggest changing the regex to handle versions with zero or more dots.
| m = re.search(r"\.so\.(\d+(?:\.\d+)+)$", p) | |
| m = re.search(r"\.so\.(\d+(\.\d+)*)$", p) |
There was a problem hiding this comment.
Done in 33ebdfd — the regex now accepts single-number suffixes ((\d+(?:\.\d+)*)).
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/ISSUE_TEMPLATE/bug_report.yml:
- Around line 18-26: Update the environment-output textarea in the bug report
template to remove the prefilled “PASTE OUTPUT HERE” value and configure it as
placeholder text instead, while preserving the existing required validation so
contributors must provide actual output.
In `@flashinfer/collect_env.py`:
- Around line 129-138: Update _installed_distributions to canonicalize each
distribution name with the same hyphen-normalization used by _check_pin(),
converting underscores and dots as well as casing before storing the mapping.
Preserve the existing behavior of skipping distributions without a metadata name
and returning their versions.
- Around line 541-545: Update _get_env_vars to sanitize collected environment
data before returning it: redact values for credential-like variable names,
including secrets matched by broad prefixes such as NVIDIA_, VLLM_, and SGLANG_,
and replace the user home-directory prefix in path values with a non-sensitive
placeholder. Preserve the existing sorted collection and exact/prefix matching
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 29abd899-4fc3-4df2-a09f-e5585b23e51c
📒 Files selected for processing (5)
.github/ISSUE_TEMPLATE/bug_report.ymlCLAUDE.mdflashinfer/__main__.pyflashinfer/collect_env.pytests/utils/test_collect_env.py
Companion to the bug-report template in this PR, so the two forms split cleanly (Jingfan): broken/wrong/slower-than-before goes to Bug Report, new capability/speedup goes here — each template's description states the routing rule to avoid categorization confusion (Kaustubh). Changes vs the circulated draft, per Slack feedback: - Required fields cut from 10 to 4 (problem, requested outcome, target hardware, inference engine) + a searched-existing-issues checkbox on both templates (Jingfan); everything else demoted to clearly-marked optional with a note that filling more helps the team act (Brian, Yang) - Hardware dropdown gains Ampere (SM80/86) and Ada (SM89) entries - Inference engine becomes multi-select with a Not-applicable option - Evidence dropdown removed; trace/repro invited in the Problem field - Impact label/description contradiction fixed (multi-select 'Impact') - Workload skeleton's version bullets replaced by collect_env reference so both templates share one environment tool (Brian, Yang) - Bug template: explicit perf-regression wording, and reproducer guidance asks for a short standalone script rather than a vllm serve run (Brian) AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
YangXu1990uiuc
changed the title
| - SM80 / SM86 - Ampere (A100, A30, RTX 30) | ||
| - SM89 - Ada (L4, L40S, RTX 40) | ||
| - SM90 - Hopper (H100, H200, or H20) | ||
| - SM100 - Blackwell datacenter (B200 or GB200) |
There was a problem hiding this comment.
this does not align with product nomenclature
There was a problem hiding this comment.
This comment is anchored on the previous revision — 4e77e03 (pushed after Brian raised the same point on Slack) dropped the architecture marketing names and segment qualifiers entirely. The dropdown now uses the SM number as the key with product examples only: SM100 (B200, GB200), SM103 (B300, GB300), SM120 (RTX PRO 6000, RTX 50-series), SM121 (GB10 — DGX Spark, RTX Spark). Please take a look at the current revision — if a specific product name is still off, happy to fix.
| attributes: | ||
| label: Before submitting | ||
| options: | ||
| - label: I have searched existing issues and did not find a solution or an existing report. |
There was a problem hiding this comment.
for future, i believe it's common practice that a bot flags dup issues and "threatens" to close if not justifying against the duplication. e.g. claude code repo
There was a problem hiding this comment.
Agreed — a dedup bot that flags likely duplicates and auto-closes unless the reporter justifies (like anthropics/claude-code) is a good follow-up. Out of scope here; this checkbox is the manual placeholder until then.
| id: description | ||
| attributes: | ||
| label: Bug description | ||
| description: What happened, and what did you expect to happen instead? |
There was a problem hiding this comment.
we can separate this into two fields
There was a problem hiding this comment.
Done in 33ebdfd — split a dedicated optional "Error messages / logs" textarea (render: text) out of the reproduction field, following the claude-code structure. The repro field now asks only for the script.
| description: Select every engine or integration path this applies to. | ||
| multiple: true | ||
| options: | ||
| - vLLM |
There was a problem hiding this comment.
for future, this info can be used to auto-label stuff with a git action
There was a problem hiding this comment.
Agreed — that's part of why these are dropdowns rather than free text: the selections render machine-parseably in the issue body, so a labeler action can map hardware/engine choices to labels. Good follow-up once the templates land.
aleozlx
left a comment
aleozlx
left a comment
There was a problem hiding this comment.
reviewed. one blocking comment on product names
'Blackwell' spans SM100/103/120/121 and market-segment qualifiers like 'datacenter' are already stale (GB300 DGX Station, RTX PRO 6000 Server Edition), so architecture marketing names and segment labels are dropped; every row is now 'SMxxx (product examples)'. Per Brian Ryu's feedback. AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| @@ -0,0 +1,64 @@ | |||
| name: 🐛 Bug Report | |||
There was a problem hiding this comment.
are the emojis in the "name" necessary? i don't get it
- bug template: Environment field switches from prefilled value to placeholder + render:text — prefilled content satisfied required:true, letting reports through without any environment output (CodeRabbit) - bug template: split error messages/logs out of the reproduction field into a dedicated optional textarea, following the claude-code template structure (aleozlx) - collect_env: canonicalize installed-distribution names ([-_.]+ -> -) to match _check_pin, so underscore metadata names like flashinfer_python resolve in pin diagnostics (CodeRabbit) - collect_env: redact env values whose name contains KEY/TOKEN/SECRET/ PASSWORD/CREDENTIAL — the broad prefixes can match e.g. NVIDIA_API_KEY and reports are pasted publicly (CodeRabbit). Home-dir paths are kept as-is: path identity is the report's forensic payload. - collect_env: version-from-filename regex accepts single-number suffixes like libfoo.so.8 (Gemini) AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| description: Select every confirmed target. | ||
| multiple: true | ||
| options: | ||
| - SM80 / SM86 (A100, A30, RTX 30-series) |
There was a problem hiding this comment.
SM87 Orin? (avoid getting filed into unknown)
There was a problem hiding this comment.
Added in edd2607 — folded into the Ampere row: SM80 / SM86 / SM87 (A100, A30, RTX 30-series, Jetson Orin).
| - SM90 (H100, H200, H20) | ||
| - SM100 (B200, GB200) | ||
| - SM103 (B300, GB300) | ||
| - SM120 (RTX PRO 6000, RTX 50-series) |
There was a problem hiding this comment.
SM110 Thor? (avoid getting filed into unknown)
There was a problem hiding this comment.
Added in edd2607 as SM110 (Jetson AGX Thor, DRIVE AGX Thor). Double-checked the number since Thor was renumbered: CC 10.1 (sm_101) in the CUDA 12.x era → CC 11.0 (sm_110) since CUDA 13.0 — consistent with our is_sm110a_supported (major == 11 and CUDA ≥ 13.0) and nvcc 13.x's arch list (compute_110 present, compute_101 gone).
Per review: embedded targets should not have to file under 'Other or unknown'. Thor is SM110 on CUDA 13+ (renumbered from CC 10.1 in the CUDA 12.x era) — matches is_sm110a_supported (major == 11, CUDA >= 13.0) and nvcc 13.x's arch list (compute_110 present, compute_101 gone). AI-assisted (Claude Code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on over length limit) (#4100) ## 📌 Description Follow-up to #4007: the 🐛 Bug Report template was silently missing from the issue template chooser (only 🚀 Kernel / Feature Request appeared), even though the file is valid YAML and passes the published issue-forms JSON schema. **Root cause:** GitHub enforces an *undocumented* length limit on an issue form's top-level `description` and silently drops templates that exceed it. Ours was **218 characters**; the feature template's was 180. **Evidence (bisected on a fork, one variable per variant):** | description | length | renders in chooser | |---|---|---| | T7 (with `—` and `/`) | 137 | ✅ | | feature template | 180 | ✅ | | bug template (byte-exact copy) | 218 | ❌ | | T6 (all non-ASCII punctuation removed) | 241 | ❌ | Removing `render:` / `placeholder:` did not help; special characters and emoji are innocent (feature template has both). The limit sits in (180, 218] — presumably 200, matching the legacy markdown templates' `about` field. **Fix:** tighten both templates' descriptions to a single sentence (bug: 61 chars, feature: 94) — the chooser shows the two templates side by side, so the cross-referencing routing sentences were redundant anyway. Verified rendering on a fork. ## 🔍 Related Issues Follow-up to #4007. ## 🚀 Pull Request Checklist ### ✅ Pre-commit Checks - [x] I have installed `pre-commit` by running `pip install pre-commit` (or used your preferred method). - [x] I have installed the hooks with `pre-commit install`. - [x] I have run the hooks manually with `pre-commit run --all-files` and fixed any reported issues. ## 🧪 Tests - [x] Tests have been added or updated as needed. - [x] All tests are passing (`unittest`, etc.). (Template-only change; YAML validated, rendering verified on a fork with identical content.) ## Reviewer Notes Neither GitHub's issue-forms docs nor the SchemaStore JSON schema encode this limit — worth remembering when editing template descriptions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated the bug report template to use shorter, clearer guidance for reporting items that are broken, wrong, or slower than before (including correctness issues, crashes, and performance regressions). * Updated the feature request template description to use a single, streamlined prompt covering capability improvements and related requests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Yang Xu <yanxu@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2 participants
📌 Description
Issue reporters often can't state their environment precisely, and the most common unreproducible-issue root cause is version confusion — multiple copies of cuDNN/cuBLAS/CUDA runtime installed where the loaded one is not the one the user assumes.
This PR adds an environment-forensics tool and wires it into a new (first) GitHub issue template:
python -m flashinfer.collect_env # or: flashinfer collect-env [--json]What the report covers
CUDA_HOME/proc/self/mapsfor what is truly mapped, scansLD_LIBRARY_PATH/ site-packages /CUDA_HOME/ ldconfig for other on-disk installs; aggregated per directory with pip provenance, ⚠ on real conflictsFLASHINFER_*/CUDA*/… env vars, GPU topologyDesign constraints (documented in the module docstring)
import flashinferor torch is broken; the file can becurl-ed and run standalone with bare Pythonshow-config's artifact status), ~7 s wall clockExample of the headline section on a box with a pip-wheel CUDA stack plus a local toolkit:
🔍 Related Issues
Improves triage for env-dependent reports in general (e.g. the recurring "works on my machine" class).
🚀 Pull Request Checklist
✅ Pre-commit Checks
pre-commitby runningpip install pre-commit(or used your preferred method).pre-commit install.pre-commit run --all-filesand fixed any reported issues.🧪 Tests
unittest, etc.).tests/utils/test_collect_env.pyasserts the core contract (collection never raises, sections present, JSON-serializable) and runs GPU-less. Manually verified on a multi-arch box (A100/L40S/H100/B200-class, driver 595.71.05): full run in a venv, standalone run with bare/usr/bin/python3(no torch/flashinfer — degrades gracefully),--jsonparses.Reviewer Notes
/proc/self/mapsis Linux-only; on other platforms that section degrades to the on-disk scan.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
collect-envCLI command to generate detailed environment reports (runtime, CUDA/GPU details, loaded libraries, and version info).Documentation
--jsonmode.Tests
Update (2026-07-17): also adds
🚀 Kernel / Feature Request(feature_request.yml), a slimmed revision of the template draft circulated on Slack, incorporating the thread feedback (thanks @kaustubh, Brian, Jingfan):python -m flashinfer.collect_env; bug-template reproducer guidance now asks for a short standalone script rather than a fullvllm serverun