Add collect_env environment report tool for bug reports - #400
Merged
Conversation
Issue reporters often can't state their environment precisely, and the most common unreproducible-issue root cause is version confusion: multiple cuDNN/CUDA copies installed where the loaded one is not the one the user assumes. python -m cudnn.collect_env produces an offline, read-only report: frontend/backend versions with mismatch flags (stale pip metadata, torch's libcudnn vs the frontend's dlopen'ed backend), the frontend's libcudnn search-order resolution, GPUs in CUDA enumeration order, loaded-vs-on-disk GPU libraries via /proc/self/maps with pip provenance, relevant packages incl. torch's declared cuDNN pin, and CUDNN_*/CUDA_* env vars. Stdlib-only at module level with every probe individually guarded, so the file also runs standalone with bare Python when import cudnn is broken. Referenced from the bug-report issue template and README. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
example |
Contributor
📝 WalkthroughWalkthroughAdds a standalone cuDNN environment-reporting CLI that collects guarded system, GPU, CUDA, framework, package, and library-linkage details, supports Markdown or JSON output, and is documented and tested. ChangesEnvironment report collection
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant main
participant collect_env_info
participant nvidia_smi as nvidia-smi
participant torch
participant proc_maps as /proc/self/maps
main->>collect_env_info: collect environment sections
collect_env_info->>nvidia_smi: query GPU and topology details
collect_env_info->>torch: inspect CUDA and cuDNN linkage when available
collect_env_info->>proc_maps: inspect loaded GPU library paths
collect_env_info-->>main: return ordered report
main->>main: format Markdown or emit JSON
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Anerudhan
self-requested a review
July 16, 2026 20:33
Anerudhan
added
mod-frontend
Anerudhan
approved these changes
Jul 16, 2026
Collaborator
|
@cudnn-ci-bot run |
|
🚀 Running mirror pipeline Branch: cudnn-gh/pr-400-2d82817 |
Anerudhan
approved these changes
Jul 16, 2026
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
test/python/test_collect_env.py (2)
17-28: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover the
GPU Topologysection too.
collect_env_info()also emitsGPU Topology(python/cudnn/collect_env.py:502-519), but this whitelist omits it, so a missing section could regress without failing the test."Environment Variables", + "GPU Topology",🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/python/test_collect_env.py` around lines 17 - 28, Update the section whitelist in the collect_env report test to include “GPU Topology” alongside the existing expected sections. Keep the existing presence and dictionary-type assertions unchanged so collect_env_info() regressions for that section are detected.
52-54: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTest the CLI
--jsonpath, not just dictionary serialization.
json.dumps(report)can pass even ifmain()mishandles--jsonor prints invalid output. Invoke the CLI (ormain()with patched arguments), capture stdout, and validate it withjson.loads().🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/python/test_collect_env.py` around lines 52 - 54, Update test_json_serializable to exercise the CLI --json path by invoking main() or the CLI with patched arguments, capturing stdout, and parsing the emitted output with json.loads(). Preserve the report fixture while validating the actual command output rather than only calling json.dumps(report).README.md (1)
158-167: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument both output modes and the standalone command.
The collector supports
python -m cudnn.collect_env --json, but the README only documents Markdown output. Also show the fallback command explicitly, e.g.python collect_env.py, so the recovery path is directly copy-pastable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` around lines 158 - 167, Update the “Environment report” section to document both default Markdown output and the JSON mode using the --json option. Make the broken-import fallback explicitly copy-pastable by showing the standalone command as python collect_env.py after downloading collect_env.py, while preserving the existing module invocation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@python/cudnn/collect_env.py`:
- Around line 451-460: Update the library-report construction around by_dir and
_describe_lib_dir so loaded paths and disk-only paths are rendered as separate
entries. Iterate over loaded[family] and on_disk[family] - loaded[family]
independently, preserving each exact loaded copy and labeling only loaded paths
as LOADED, including when both sets share a directory.
- Around line 112-121: Use a shared distribution-name normalization helper for
both _installed_distributions and the requirement-name handling around lines
287–290. The helper must strip whitespace, lowercase names, and canonicalize
dots, underscores, and hyphens consistently so equivalent names resolve to the
same key and existing package/pin diagnostics remain complete.
- Around line 486-490: Update _get_env_vars to exclude or redact environment
entries whose names indicate credentials, such as API keys, tokens, passwords,
or secrets, even when they match _ENV_PREFIXES or _ENV_EXACT. Preserve the
existing collection of non-sensitive diagnostic variables and ensure the
returned data is safe to print in public bug reports.
---
Nitpick comments:
In `@README.md`:
- Around line 158-167: Update the “Environment report” section to document both
default Markdown output and the JSON mode using the --json option. Make the
broken-import fallback explicitly copy-pastable by showing the standalone
command as python collect_env.py after downloading collect_env.py, while
preserving the existing module invocation.
In `@test/python/test_collect_env.py`:
- Around line 17-28: Update the section whitelist in the collect_env report test
to include “GPU Topology” alongside the existing expected sections. Keep the
existing presence and dictionary-type assertions unchanged so collect_env_info()
regressions for that section are detected.
- Around line 52-54: Update test_json_serializable to exercise the CLI --json
path by invoking main() or the CLI with patched arguments, capturing stdout, and
parsing the emitted output with json.loads(). Preserve the report fixture while
validating the actual command output rather than only calling
json.dumps(report).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2e5402a3-a09d-4a44-bb23-72b2db3c759c
📒 Files selected for processing (4)
.github/ISSUE_TEMPLATE/bug_report.ymlREADME.mdpython/cudnn/collect_env.pytest/python/test_collect_env.py
Comment on lines
+112
to
+121
| def _installed_distributions(): | ||
| """{normalized_name: version} for every installed distribution.""" | ||
| import importlib.metadata | ||
|
|
||
| dists = {} | ||
| for dist in importlib.metadata.distributions(): | ||
| name = (dist.metadata.get("Name") or "").strip() | ||
| if name: | ||
| dists[name.lower()] = dist.version | ||
| return dists |
Contributor
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
Canonicalize distribution names consistently.
Lines 118–120 only lowercase metadata names, while Lines 287–290 normalize requirement names by replacing ., _, and -. Equivalent names such as transformer_engine and transformer-engine therefore fail lookup, producing incomplete package and pin diagnostics.
Use one normalization helper for both paths.
Also applies to: 287-290
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/cudnn/collect_env.py` around lines 112 - 121, Use a shared
distribution-name normalization helper for both _installed_distributions and the
requirement-name handling around lines 287–290. The helper must strip
whitespace, lowercase names, and canonicalize dots, underscores, and hyphens
consistently so equivalent names resolve to the same key and existing
package/pin diagnostics remain complete.
Comment on lines
+451
to
+460
| by_dir = {} | ||
| for p in loaded[family] | on_disk[family]: | ||
| by_dir.setdefault(os.path.dirname(p), set()).add(p) | ||
| if not by_dir: | ||
| continue | ||
| loaded_dirs = {os.path.dirname(p) for p in loaded[family]} | ||
| lines = [] | ||
| for d in sorted(by_dir, key=lambda d: (d not in loaded_dirs, d)): | ||
| state = "LOADED " if d in loaded_dirs else "on disk" | ||
| lines.append(f"{state} {_describe_lib_dir(family, d, by_dir[d], dists)}") |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Preserve which exact library copy is loaded.
The union on Line 452 merges loaded and disk-only versions by directory, then Line 459 labels the entire directory LOADED. If one directory contains multiple versions, the report shows all versions as loaded and cannot identify the mapped copy—the primary diagnostic this section promises.
Render loaded[family] and on_disk[family] - loaded[family] separately, even when they share a directory.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/cudnn/collect_env.py` around lines 451 - 460, Update the
library-report construction around by_dir and _describe_lib_dir so loaded paths
and disk-only paths are rendered as separate entries. Iterate over
loaded[family] and on_disk[family] - loaded[family] independently, preserving
each exact loaded copy and labeling only loaded paths as LOADED, including when
both sets share a directory.
Comment on lines
+486
to
+490
| def _get_env_vars(): | ||
| info = {} | ||
| for k in sorted(os.environ): | ||
| if k.startswith(_ENV_PREFIXES) or k in _ENV_EXACT: | ||
| info[k] = os.environ[k] |
Contributor
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Redact credentials before printing environment variables.
Broad prefixes such as NVIDIA_, CUDA_, and TORCH_ can match API keys, tokens, or passwords. Because this output is intended for public bug reports, redact sensitive names while retaining diagnostic variables.
Proposed fix
def _get_env_vars():
info = {}
for k in sorted(os.environ):
if k.startswith(_ENV_PREFIXES) or k in _ENV_EXACT:
- info[k] = os.environ[k]
+ if re.search(r"(?:TOKEN|SECRET|PASSWORD|API_KEY|CREDENTIAL|AUTH)", k, re.I):
+ info[k] = "<redacted>"
+ else:
+ info[k] = os.environ[k]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def _get_env_vars(): | |
| info = {} | |
| for k in sorted(os.environ): | |
| if k.startswith(_ENV_PREFIXES) or k in _ENV_EXACT: | |
| info[k] = os.environ[k] | |
| def _get_env_vars(): | |
| info = {} | |
| for k in sorted(os.environ): | |
| if k.startswith(_ENV_PREFIXES) or k in _ENV_EXACT: | |
| if re.search(r"(?:TOKEN|SECRET|PASSWORD|API_KEY|CREDENTIAL|AUTH)", k, re.I): | |
| info[k] = "<redacted>" | |
| else: | |
| info[k] = os.environ[k] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@python/cudnn/collect_env.py` around lines 486 - 490, Update _get_env_vars to
exclude or redact environment entries whose names indicate credentials, such as
API keys, tokens, passwords, or secrets, even when they match _ENV_PREFIXES or
_ENV_EXACT. Preserve the existing collection of non-sensitive diagnostic
variables and ensure the returned data is safe to print in public bug reports.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Issue reporters often can't state their environment precisely, and the most common unreproducible-issue root cause is version confusion — multiple cuDNN/CUDA copies installed where the loaded one is not the one the user assumes. This is especially acute for cudnn-frontend: the Python bindings
dlopenlibcudnnvia a search order (LD_LIBRARY_PATH→ pipnvidia/cudnn/lib→ soname fallback) that can resolve to a different copy than the one torch linked.This PR adds an environment-forensics tool and wires it into the bug-report issue template and README:
python -m cudnn.collect_env # or --jsonWhat the report covers
cudnn.__version__vs pipnvidia-cudnn-frontendmetadata (stale build / shadowing checkout), andtorch.backends.cudnn.version()vs the frontend's loaded backend (two libcudnn copies in one process)_dlopen_cudnn()search-order resolution, traced read-onlylibcudnnincl. sublibraries,libcublas,libcudart,libnvrtc, driver): parses/proc/self/mapsfor what is truly mapped, scansLD_LIBRARY_PATH/ site-packages /CUDA_HOME/ ldconfig for other on-disk installs; aggregated per directory with pip provenance, ⚠ on real conflicts including mixed sublibrary versions in one directoryCUDNN_*/CUDA_*/… env vars, GPU topologyDesign constraints (documented in the module docstring)
import cudnnor torch is broken; the file can becurl-ed and run standalone with bare Python (useful triage even for C++-only users: it inventories every libcudnn on disk)Example headline section from a box with a local backend build plus pip wheels:
Tests
test/python/test_collect_env.py(L0) asserts the core contract: collection never raises, sections present, frontend/backend versions reported, JSON-serializable. Verified manually: full run in a venv on a multi-arch box (SM80/89/90/100), standalone run with bare/usr/bin/python3(no torch — degrades gracefully),--jsonparses, black clean.Reviewer notes
/proc/self/mapsis Linux-only; elsewhere that section degrades to the on-disk scan. The dlopen search trace mirrorscudnn/__init__.py:_dlopen_cudnn()(Linux); on Windows it reports<not traced>.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Tests