perfxpert: docs audit + ship-readiness scanners (lint / link-check / sample-exec) by ammarwa · Pull Request #5180 · ROCm/rocm-systems

ammarwa · 2026-04-19T03:55:45Z

Summary

Ships a comprehensive docs audit for perfxpert, fills coverage gaps for newly-landed features (including the Phase 8 public Python API and the fast-install wrapper), relocates all perfxpert artifacts under experimental/python/perfxpert/, and gates CI on three scanners that must run clean against the shipping docs tree. Stacked on #5179.

What lands

Docs audit tooling

experimental/python/perfxpert/scripts/lint.sh — banned API strings + historical-anchor exception
experimental/python/perfxpert/scripts/link-checker.py — internal Markdown link validation, --strict mode
experimental/python/perfxpert/scripts/test-samples.py — fenced code-sample extractor + executor, --strict mode
experimental/python/perfxpert/tests/test_docs_tooling/test_ship_readiness.py — CI gate: all three scanners must exit 0 against the real docs tree

Per-doc violation fixes

One commit per affected document: CHANGELOG, README, architecture, migration-to-agentic, getting-started, CONTRIBUTING, fixtures, mcp_tools, providers, agents, walkthroughs.

New feature coverage

experimental/python/perfxpert/docs/architecture/agent-hierarchy.md
experimental/python/perfxpert/docs/integration/mcp-server.md — 43 READ_ONLY tools + 5-client integration guide (Claude Desktop / Claude Code / Codex / Gemini / opencode)
experimental/python/perfxpert/docs/contributing/external-tools.md
experimental/python/perfxpert/docs/architecture/gate-cascade.md
experimental/python/perfxpert/docs/guides/agentic-mode.md
experimental/python/perfxpert/docs/guides/python-api.md — covers the Phase 8 public perfxpert.api surface (the 1:1 mirror of the 7 agent MCP tools added in perfxpert: bundled opencode fork + multi-backend launcher (claude / codex / gemini) #5179)

Multi-backend launcher docs (covers #5179 features)

experimental/python/perfxpert/docs/guides/backends.md — per-backend install / uninstall recipes, Codex trust-gate workflow, AC-9a gate-probe coverage table naming the specific small models (haiku-4-5, gemini-2.5-flash, opencode-default; Codex not probed — prompt-layer-only)
experimental/python/perfxpert/docs/architecture/backend-adapter.md — adapter protocol + lifecycle contract for contributors

Getting-started guide refresh

Multi-backend install sections (claude / codex / gemini)
Codex trust-gate workflow + PERFXPERT_AUTO_TRUST security contract
PERFXPERT_AIRGAP=1 mode
MCP server option (43 READ_ONLY tools)
Fallback chain (PERFXPERT_LLM_FALLBACK_CHAIN)
Fast-install wrapper (scripts/pip-install-from-git.sh) — new § documents the submodule.active scoping that drops install time from ~141 s to ~30 ms
All env vars + commands verified against current code

Brand + legacy scrub

Zero prior-codename references in shipping docs
scripts/lint.sh enforces the scrub going forward (banned-string list + historical-anchor exception)

Other cleanups in this stack

All perfxpert artifacts migrated under experimental/python/perfxpert/ — nothing perfxpert-specific remains at the rocm-systems monorepo root
12 orphan GIFs (64 MB) removed from docs/guides/assets/ — they were not referenced by any markdown and several represented removed features
Phase-scoped filenames renamed for post-merge readability (e.g. phase5_runbook.md → audit_gate_runbook.md, mi300x_phase7.md → mi300x_baseline.md)
Dead local-only scratch directories removed (brainstorm + decision records are local-only artifacts)
Dead canonical_facts.json / inventory.json / inventory-baseline.json + their aggregator script removed (data was stale; live property is already enforced by test_ship_readiness.py)

Docs covering the Phase 8 late-landing features

The following shipping docs in this PR now cover the features that
landed late on #5179 — trace_diff.diff_runs, severity-colored
hotspot classes, source_locations schema-0.3.1 additive,
agent_diff_specialist (8th agent), perfxpert diff / perfxpert ci:

experimental/python/perfxpert/docs/guides/python-api.md —
perfxpert.api.agent_diff_specialist + diff_runs usage.
experimental/python/perfxpert/docs/integration/mcp-server.md —
perfxpert_agent_diff_specialist + trace_diff_diff_runs wire
names + schema-0.3.1 diff block example.
experimental/python/perfxpert/docs/contributing/output_formats.md —
severity-class table (CRITICAL / HOT / WARM / COOL thresholds +
hex palette + per-format rendering).
experimental/python/perfxpert/docs/contributing/schemas.md —
additive fields hotspots[i].source_locations + trace_diff
under schema 0.3.1.
experimental/python/perfxpert/docs/guides/getting-started.md §7 —
Compare runs + CI gating (\perfxpert diff` / `perfxpert ci`)`
walkthrough, including a rc=1-on-regression CI snippet.
experimental/python/perfxpert/docs/guides/backends.md — updated
backend LLM contract now references all 43 tools + perfxpert_agent_diff_specialist.

Scanner outputs (strict mode)

bash experimental/python/perfxpert/scripts/lint.sh --strict → exit 0
python experimental/python/perfxpert/scripts/link-checker.py --strict → exit 0
python experimental/python/perfxpert/scripts/test-samples.py --strict → exit 0
pytest experimental/python/perfxpert/tests/test_docs_tooling/ → 3 pass

Test plan

All three scanners green
test_ship_readiness.py validates all three scanners run clean against the shipping docs tree
1379 pass / 7 skip / 0 fail from experimental/python/perfxpert/

Stacked on #5179.

ammarwa · 2026-04-23T00:23:18Z

Rebuilt PR 5180 on top of the current #5179 base in 4a9c3a4f80.

What changed:

replaced the stale pre-rebase ancestry with a clean phase-9 docs/docs-tooling diff on top of 2d63402453
kept scope to docs, docs-tooling scripts, docs-gate tests, and one SKIP-SAMPLE fix in experimental/python/perfxpert/mcp_server/README.md
removed stale PR-status / "soon Codex" wording, fixed the Claude/Codex MCP command examples, and dropped references to local-only decision records
fixed the reviewer-found blockers:
- scripts/test-samples.py now points PYTHONPATH at the real perfxpert app root
- docs/guides/agentic-mode.md now matches the current air-gap root behavior (recommendations=[])
- removed dead GIF references from getting-started.md and marked the perfxpert-mcp README sample as SKIP-SAMPLE

Validation:

bash experimental/python/perfxpert/scripts/lint.sh --strict
python3 experimental/python/perfxpert/scripts/link-checker.py --strict experimental/python/perfxpert
python3 experimental/python/perfxpert/scripts/test-samples.py --strict experimental/python/perfxpert
env PYTHONPATH=experimental/python/perfxpert python3 -m pytest -q experimental/python/perfxpert/tests/test_docs_tooling/test_ship_readiness.py experimental/python/perfxpert/tests/test_integration/test_docs_links.py -> 6 passed

Review:

two reviewer passes on the cleaned tree; both final rechecks reported no blockers.

…ATT flamegraph + RCCL + pragma advice (#5221) ## Summary Nine advanced specialist features + supporting infrastructure on top of the docs-audit branch. Every feature is an additive MCP surface extension — no existing behaviour changes, airgap parity preserved, webview invariants preserved. ### Features landed - **Live Roofline SVG** — interactive inline `<svg>` chart in the webview: log-log axes, peak-compute horizontals per-dtype, HBM-BW diagonal, ridge annotation, per-kernel dots coloured by bottleneck class, click-to-rec-anchor, wheel-zoom. New MCP tool: `roofline.plot_points`. - **ATT Flamegraph** — dependency-free inline-SVG flame graph beside the existing ATT stall table. Zero new MCP tools (renders from existing `thread_trace` payload). - **Communication analysis (RCCL)** — per-collective bus-bandwidth vs arch peak, comm/compute overlap ratio, Ring-AllReduce factor `2(N-1)/N`, efficiency classification (poor/fair/good). New MCP tools: `rccl_analysis.analyze_collectives`, `interconnect.lookup_peaks`. New webview `.scard` "Communication" section. - **Change Impact Prediction** — roofline-model speedup prediction per recommendation technique. 5 seeded techniques from `proven_optimizations.yaml`. Amdahl + Tier-2 + unknown-technique gates. Always-on (no CLI flag). `Predicted: 1.15-1.45× (conf 70%)` rendered on every rec card. Tracking-side (prediction-vs-observed pairing) follows in the next branch. - **Kernel Fusion Finder** — detect adjacent short kernels on same tensor shape; estimate fusion speedup. New MCP tool: `kernel_fusion.find_fusion_candidates`. Bound into Compute Specialist allowlist. - **GPU Runtime Monitor** — ingest pre-captured `amd-smi monitor --json` / `rocm-smi --json` logs (no live shell-out, no sudo). Thermal envelope + throttle detection. MCP tools: `gpu_runtime_monitor.parse_amd_smi_json`, `parse_rocm_smi_json`, `analyze_thermal`. New knowledge YAML: `thermal_specs.yaml`. - **Unified Memory Analysis** — MI300X paging detection + cross-die penalty estimation from rocprofiler events. MCP tool: `unified_memory.analyze_paging`. Bound into Memory Specialist allowlist. - **Dependency Graph Analysis** — DAG reconstruction from `hipEvent` / `hipStreamWaitEvent` / barrier events; critical-path extraction; GPU-bubble detection. MCP tool: `dependency_graph.reconstruct_dag`. Bound into Latency Specialist allowlist. - **LLVM loop-hint pragma advice** — new knowledge YAML `compiler_pragmas.yaml` with 3 allowlisted entries (`unroll(full)`, `unroll_count(N)`, `unroll(disable)`) + 7 explicitly rejected entries (`vectorize*`, `interleave*`, `distribute`, `pipeline*`) visible to the fence. `--advanced` CLI gate + `PERFXPERT_ADVANCED_RECS=1` env. Mandatory Tier-0 source anchor; auto-skip on Triton-generated kernels. MCP tools: `pragma.lookup_pragmas`, `pragma.explain_pragma`, `pragma.suggest_pragmas_for_kernel`. ### Fence template consolidation Root + Recommendation fences adopt the `diff_specialist.md` 6-section template (Inputs / Outputs / Deterministic procedure / Verdict rules / Escalation / Never). Explicit-rejection pattern for compiler hints (rejected entries kept in YAML with `gpu_applicable: false` so the fence can actively say "no"). 11 phantom tool references cleaned across all 8 fences. ### Schema additions - `CommunicationBlock` + `CollectiveEntry` Pydantic models (typed RCCL output). - `matrix_meter.yaml` + `attention_patterns.yaml` now carry required `applies_to_gfx` (CDNA2+ gate). - `fusion_patterns.yaml` entries now carry required `units: {speedup_multiplier | time_saved_fraction}`. - `memory_patterns.yaml:15` detector path corrected (`unified_memory.analyze_paging`). - `schema_version` bump chain: 0.3.0 → 0.3.1 (source_locations) → 0.3.2 (communication) → 0.3.3 (predicted_impact) → 0.3.4 (roofline) → 0.4.0 (ATT). ### Ship metrics - **MCP READ_ONLY tools:** 56 (8 agent + 47 classifier/knowledge + 1 trace_diff). `arch.lookup_peaks` canonical; `sol._lookup_peaks` / `roofline._lookup_peaks` are intentionally private aliases (underscore-prefixed) to avoid duplicate registration. - **Agents:** 8 (unchanged — no new agents introduced). Compute / Memory / Latency specialists each at 5/5 allowlist cap after feature rebinding. - **Tests:** 1633 passed / 0 failed / 3 env-skipped (OpenAI 429 quota × 2, gemini CLI not installed × 1). - **Strict scanners:** `scripts/{lint.sh, link-checker.py, test-samples.py} --strict` rc=0. - **Airgap parity:** every new feature deterministic under `PERFXPERT_AIRGAP=1`; unit tests exercise airgap branch. - **Webview invariants preserved:** 1 DOCTYPE / 1 `</body>` / 1 `</html>`; only `.scard/.shdr/.sbody` (no `.card`); `%` units on bar widths; Summary precedes Execution Breakdown. ### CI + workflow hygiene - Nightly benchmarks pinned to `linux-gfx942-1gpu-ossci-rocm`; triggers reduced to `schedule: cron "0 2 * * *"` + `workflow_dispatch` only (no per-PR). - Dead `experimental/python/perfxpert/.github/workflows/` tree relocated to repo-root `.github/workflows/` with proper `paths:` filters. ## Test plan - [x] `pytest tests/ --tb=no -q` → 1633 / 0 / 3 on this branch - [x] `scripts/lint.sh --strict` rc=0 - [x] `scripts/link-checker.py --strict .` rc=0 - [x] `scripts/test-samples.py --strict` rc=0 - [x] Airgap smoke on all 4 formats (text/json/markdown/webview) against fixtures - [x] `perfxpert diff` + `perfxpert ci` + `--advanced` + `--baseline` CLI paths verified - [x] `perfxpert.api.*` 1:1 identity with MCP-registered callables (8 agent tools + `trace_diff_diff_runs`) - [x] Fence phantom-tool audit: 0 references to unregistered MCP tools across all 8 fences - [x] Three reviewers (spec / practical / regression) + AI-usability reviewer green before opening - [ ] Clean-Docker end-to-end with `rocprof-trace-decoder` + every Tier — runs post-merge-prep ## Documentation - `docs/guides/getting-started.md` — new subsections for every feature - `docs/guides/python-api.md` — every new MCP tool + `predict_impact.*` surface - `docs/integration/mcp-server.md` — 56-tool inventory - `docs/architecture/agent-hierarchy.md` — 8 agents + post-Phase-10 allowlists - `docs/contributing/schemas.md` — `CommunicationBlock` + `CollectiveEntry` + schema-version chain - `docs/contributing/output_formats.md` — Communication scard + Live Roofline + ATT flamegraph layouts - `docs/contributing/agents.md` — diff_specialist fence as canonical template - `README.md` — tool count + feature matrix + install quickstart 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

ammarwa marked this pull request as ready for review April 19, 2026 03:55

ammarwa requested a review from a team as a April 19, 2026 03:55

github-actions Bot added documentation Improvements or additions to documentation github actions Pull requests that update GitHub Actions code labels Apr 19, 2026

systems-assistant Bot added the organization: ROCm label Apr 19, 2026

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from 3f4fcc1 to f0fbaf3 Compare April 19, 2026 16:49

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from e767580 to 4480a73 Compare April 19, 2026 16:55

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from f0fbaf3 to 100f5dd Compare April 19, 2026 17:12

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from b89d2fc to 0f89115 Compare April 19, 2026 17:15

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from 100f5dd to 1d1b1a0 Compare April 19, 2026 17:45

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from 0f89115 to a48dfb2 Compare April 19, 2026 17:58

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from 1d1b1a0 to ffdf2f5 Compare April 19, 2026 18:25

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from a48dfb2 to 10346d8 Compare April 19, 2026 18:26

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from ffdf2f5 to ac56aba Compare April 19, 2026 18:29

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from 10346d8 to bbc16fe Compare April 19, 2026 18:29

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from ac56aba to 74ecec1 Compare April 19, 2026 19:02

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from a5414a6 to 6deb460 Compare April 19, 2026 19:02

ammarwa changed the title ~~perfxpert: Phase 9 (docs audit + corrections + zero-violation baseline) [stacked 9/9]~~ perfxpert: docs audit + ship-readiness scanners (lint / link-check / sample-exec) Apr 19, 2026

ammarwa force-pushed the users/aelwazir/perfxpert-phase8-opencode-fork branch from 74ecec1 to 273b58f Compare April 19, 2026 20:12

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch 8 times, most recently from 7d3fedb to a5cdbe4 Compare April 20, 2026 01:00

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch 6 times, most recently from fcce0f8 to 0be393f Compare April 21, 2026 00:34

ammarwa requested review from a team April 21, 2026 01:15

github-actions Bot added project: rocprofiler-register project: rocprofiler-sdk project: rocprofiler-compute labels Apr 21, 2026

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch 6 times, most recently from 60b60ba to eeed57a Compare April 21, 2026 04:41

ammarwa mentioned this pull request Apr 21, 2026

perfxpert: CLI entry points + MCP server + opencode launcher + execution-tool safety #5171

Closed

6 of 7 tasks

ammarwa force-pushed the users/aelwazir/perfxpert-phase9-docs-audit branch from d76fa98 to fbe5a50 Compare April 21, 2026 05:17

ammarwa removed request for a team April 21, 2026 05:22

ammarwa removed project: rocprofiler-register project: rocprofiler-sdk project: rocprofiler-compute labels Apr 21, 2026

docs(perfxpert): rebuild PR 5180 on latest phase8 base

4a9c3a4

Navigation Menu

Repository navigation

perfxpert: docs audit + ship-readiness scanners (lint / link-check / sample-exec) - #5180

perfxpert: docs audit + ship-readiness scanners (lint / link-check / sample-exec)#5180
ammarwa merged 2 commits into
users/aelwazir/perfxpert-phase8-opencode-forkfrom
users/aelwazir/perfxpert-phase9-docs-audit

ammarwa commented Apr 19, 2026 •

edited

Loading

Uh oh!

ammarwa commented Apr 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ammarwa commented Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What lands

Docs audit tooling

Per-doc violation fixes

New feature coverage

Multi-backend launcher docs (covers #5179 features)

Getting-started guide refresh

Brand + legacy scrub

Other cleanups in this stack

Docs covering the Phase 8 late-landing features

Scanner outputs (strict mode)

Test plan

Uh oh!

ammarwa commented Apr 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ammarwa commented Apr 19, 2026 •

edited

Loading