feat(rocprofiler-compute): store the memory chart layout file in the analysis db by xuchen-amd · Pull Request #12582 · ROCm/rocm-systems

xuchen-amd · 2026-09-30T20:58:17Z

Motivation

ROCm Optiq can draw the memory chart from the analysis database alone. Each workload stores its memory chart layout file unchanged, so the layout file (#12548) is the only memory chart format.

Bugfix: a GPU whose layout file was skipped as broken no longer stops the database output.

Technical Details

New JSON column compute_workload.memory_chart_layout_extdata: the workload's layout file, or null when its GPU has no layout. Schema version 2.4.0.
Layout metrics are panel 3 metric names, unique within panel 3. Readers find each value and unit in the existing metric tables; src/memory_chart/README.md says how.
No conversion, metric ids, or format version are stored.

Issue Tracking

JIRA ID: AIPROFCOMP-683

Test Plan

python -m pytest tests/unit tools/test_memory_chart_*.py and project pre-commit.
On real gfx950 workloads (one and two in a database, and with -b 2): analyze --output-format db, then draw the chart from the database with a reader written only from the README.

Test Result

1606 tests pass at the tip; pre-commit passes.
The stored column equals the layout file, and every layout metric finds its value.
The chart drawn from the database matches the terminal analyze -b 3 chart byte for byte.

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

🤖 Generated with Claude Code

Copilot

🟡 Changes recommended

The generated database schema diagram must be regenerated to include the new workload column.

1 open finding

Regenerate schema diagram for the new column · New

What changed in this PR

Stores architecture-specific memory-chart layouts in analysis databases so ROCm Optiq can render charts using stable workload metric IDs.

Changes:

Adds layout serialization and database persistence.
Centralizes metric-ID and memory-chart panel helpers.
Adds documentation and comprehensive unit-test coverage.

File	Description
`CMakeLists.txt`	Registers the extdata tests.
`docs/design/hld-memory-chart-db-structure.md`	Documents the database-layout design.
`docs/how-to/analyze/cli.rst`	Documents the new JSON column.
`docs/how-to/analyze/optiq.rst`	Links Optiq users to the layout documentation.
`src/memory_chart/extdata.py`	Converts resolved layouts into stored form.
`src/rocprof_compute_analyze/analysis_db.py`	Persists layouts with workloads.
`src/utils/analysis_orm.py`	Adds the column and updates schema version.
`src/utils/parser.py`	Uses centralized metric-ID formatting.
`src/utils/tty.py`	Shares panel detection and table-ID formatting.
`src/utils/utils_common.py`	Adds metric-ID and panel helpers.
`tests/test_categories.yaml`	Categorizes the new tests.
`tests/unit/memory_chart/test_extdata.py`	Tests layout serialization.
`tests/unit/rocprof_compute_analyze/test_analysis_db.py`	Tests database persistence and fallback cases.
`tests/unit/utils/test_utils_common.py`	Tests metric-name-to-ID mapping.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

vedithal-amd

will review

vedithal-amd

Design

The layout file should be the only memory chart format and the contract with Optiq.
Store it in the column as-is, with no conversion.
Optiq should draw the chart from the file and the database alone. Anything it needs must be in the file or in the memory chart README.

PR description

Please sync it with the branch. It names a commit and a helper that aren't here.
Please replace the per-commit list with a short summary of the final change.

- The HLD describes the design: the layout file stored as-is, names looked up within panel 3, explicit arrow direction and drawing hints, and no format version. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

- Store each workload's layout file unchanged in the new memory_chart_layout_extdata column, or null when its GPU has no layout, so the layout file is the only memory chart format readers need. - Bugfix: a GPU whose layout file was skipped as broken no longer stops the database output. - Readers find layout metrics by name within panel 3; no metric ids or format version are stored. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…ayout - The memory chart README names the analysis database column that holds each workload's layout file, and says it is null without a layout. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

- Drop the drawing steps, which repeat the field tables. - Drop notes that only matter to the terminal renderer or the tests. - Shorten the contract intro to one sentence. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

vedithal-amd

Rebase

I rebased this branch onto #12548 without the merge commits, so the stack lands cleanly. The code is unchanged.

README

I trimmed the memory chart README to a layout guide: dropped the drawing steps, which repeat the field tables, and the terminal-only notes.

Review

The stored layout file is now the single memory chart format, and every earlier comment is addressed.

## Motivation The memory chart for every architecture is now drawn by one renderer from a JSON layout file, one per architecture family. Adding an architecture means adding a layout file; no Python changes. The layout file is also the memory chart contract with ROCm Optiq. It states everything needed to draw the chart, so Optiq reads it directly. #12582 stores it in the analysis database. Bugfix: the Memory Chart tables keep their titles when the chart is not drawn (baseline comparison with several workloads). ## Technical Details - **Layouts** (`src/memory_chart/layouts/*.json`): every block states where it goes (`column`, `order`, `position`, `parent` or `host`), every arrow states its `direction`, and each part states how it is drawn (block `style`, metric `bar`, arrow `style`). - **Loader** (`loader.py`) checks a layout against the format rules. A broken file is skipped with a warning, so only its architectures lose the chart. - **Renderer** (`render.py`) replaces the per-architecture `mem_chart_*.py` files and reads the layout fields instead of deriving them. - **README** (`src/memory_chart/README.md`) is the contract: every field, how to draw a layout, the rules a reader can rely on, and where to find each metric's value in the analysis database. - **Tools:** `tools/memory_chart_layout_format.py` (pre-commit formatter) and `tools/memory_chart_preview.py <arch>` (draws a layout without a GPU). Chart changes users will see: - The boxes in the columns share one width, sized to the widest box; arrows are sized to fit their labels and values. - gfx1250 adds instruction cache, scalar data cache, and GL1 latency metrics; gfx115x shows instruction and data cache bandwidth separately. - Percent values get a progress bar on every architecture. - Labels such as the memory type are part of the block title, for example `DRAM (DDR5/LPDDR5)`. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - `tools/memory_chart_preview.py <arch>` for every architecture, with full values, no values, and memory bandwidth stalls. - `rocprof-compute analyze -p <gfx950 workload> -b 3`, with one workload and with two. ## Test Result - 1604 tests pass at the tip; pre-commit passes. - The chart draws for every architecture, and each layout shows every metric of its memory chart panel and no other. - With two workloads, the Memory Chart tables keep their titles. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

…analysis db (#12582) ## Motivation ROCm Optiq can draw the memory chart from the analysis database alone. Each workload stores its memory chart layout file unchanged, so the layout file (#12548) is the only memory chart format. Bugfix: a GPU whose layout file was skipped as broken no longer stops the database output. ## Technical Details - New JSON column `compute_workload.memory_chart_layout_extdata`: the workload's layout file, or null when its GPU has no layout. Schema version 2.4.0. - Layout metrics are panel 3 metric names, unique within panel 3. Readers find each value and unit in the existing metric tables; `src/memory_chart/README.md` says how. - No conversion, metric ids, or format version are stored. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - On real gfx950 workloads (one and two in a database, and with `-b 2`): `analyze --output-format db`, then draw the chart from the database with a reader written only from the README. ## Test Result - 1606 tests pass at the tip; pre-commit passes. - The stored column equals the layout file, and every layout metric finds its value. - The chart drawn from the database matches the terminal `analyze -b 3` chart byte for byte. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

## Motivation The memory chart for every architecture is now drawn by one renderer from a JSON layout file, one per architecture family. Adding an architecture means adding a layout file; no Python changes. The layout file is also the memory chart contract with ROCm Optiq. It states everything needed to draw the chart, so Optiq reads it directly. #12582 stores it in the analysis database. Bugfix: the Memory Chart tables keep their titles when the chart is not drawn (baseline comparison with several workloads). ## Technical Details - **Layouts** (`src/memory_chart/layouts/*.json`): every block states where it goes (`column`, `order`, `position`, `parent` or `host`), every arrow states its `direction`, and each part states how it is drawn (block `style`, metric `bar`, arrow `style`). - **Loader** (`loader.py`) checks a layout against the format rules. A broken file is skipped with a warning, so only its architectures lose the chart. - **Renderer** (`render.py`) replaces the per-architecture `mem_chart_*.py` files and reads the layout fields instead of deriving them. - **README** (`src/memory_chart/README.md`) is the contract: every field, how to draw a layout, the rules a reader can rely on, and where to find each metric's value in the analysis database. - **Tools:** `tools/memory_chart_layout_format.py` (pre-commit formatter) and `tools/memory_chart_preview.py <arch>` (draws a layout without a GPU). Chart changes users will see: - The boxes in the columns share one width, sized to the widest box; arrows are sized to fit their labels and values. - gfx1250 adds instruction cache, scalar data cache, and GL1 latency metrics; gfx115x shows instruction and data cache bandwidth separately. - Percent values get a progress bar on every architecture. - Labels such as the memory type are part of the block title, for example `DRAM (DDR5/LPDDR5)`. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - `tools/memory_chart_preview.py <arch>` for every architecture, with full values, no values, and memory bandwidth stalls. - `rocprof-compute analyze -p <gfx950 workload> -b 3`, with one workload and with two. ## Test Result - 1604 tests pass at the tip; pre-commit passes. - The chart draws for every architecture, and each layout shows every metric of its memory chart panel and no other. - With two workloads, the Memory Chart tables keep their titles. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

…analysis db (#12582) ## Motivation ROCm Optiq can draw the memory chart from the analysis database alone. Each workload stores its memory chart layout file unchanged, so the layout file (#12548) is the only memory chart format. Bugfix: a GPU whose layout file was skipped as broken no longer stops the database output. ## Technical Details - New JSON column `compute_workload.memory_chart_layout_extdata`: the workload's layout file, or null when its GPU has no layout. Schema version 2.4.0. - Layout metrics are panel 3 metric names, unique within panel 3. Readers find each value and unit in the existing metric tables; `src/memory_chart/README.md` says how. - No conversion, metric ids, or format version are stored. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - On real gfx950 workloads (one and two in a database, and with `-b 2`): `analyze --output-format db`, then draw the chart from the database with a reader written only from the README. ## Test Result - 1606 tests pass at the tip; pre-commit passes. - The stored column equals the layout file, and every layout metric finds its value. - The chart drawn from the database matches the terminal `analyze -b 3` chart byte for byte. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

## Motivation The memory chart for every architecture is now drawn by one renderer from a JSON layout file, one per architecture family. Adding an architecture means adding a layout file; no Python changes. The layout file is also the memory chart contract with ROCm Optiq. It states everything needed to draw the chart, so Optiq reads it directly. #12582 stores it in the analysis database. Bugfix: the Memory Chart tables keep their titles when the chart is not drawn (baseline comparison with several workloads). ## Technical Details - **Layouts** (`src/memory_chart/layouts/*.json`): every block states where it goes (`column`, `order`, `position`, `parent` or `host`), every arrow states its `direction`, and each part states how it is drawn (block `style`, metric `bar`, arrow `style`). - **Loader** (`loader.py`) checks a layout against the format rules. A broken file is skipped with a warning, so only its architectures lose the chart. - **Renderer** (`render.py`) replaces the per-architecture `mem_chart_*.py` files and reads the layout fields instead of deriving them. - **README** (`src/memory_chart/README.md`) is the contract: every field, how to draw a layout, the rules a reader can rely on, and where to find each metric's value in the analysis database. - **Tools:** `tools/memory_chart_layout_format.py` (pre-commit formatter) and `tools/memory_chart_preview.py <arch>` (draws a layout without a GPU). Chart changes users will see: - The boxes in the columns share one width, sized to the widest box; arrows are sized to fit their labels and values. - gfx1250 adds instruction cache, scalar data cache, and GL1 latency metrics; gfx115x shows instruction and data cache bandwidth separately. - Percent values get a progress bar on every architecture. - Labels such as the memory type are part of the block title, for example `DRAM (DDR5/LPDDR5)`. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - `tools/memory_chart_preview.py <arch>` for every architecture, with full values, no values, and memory bandwidth stalls. - `rocprof-compute analyze -p <gfx950 workload> -b 3`, with one workload and with two. ## Test Result - 1604 tests pass at the tip; pre-commit passes. - The chart draws for every architecture, and each layout shows every metric of its memory chart panel and no other. - With two workloads, the Memory Chart tables keep their titles. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

…analysis db (#12582) ## Motivation ROCm Optiq can draw the memory chart from the analysis database alone. Each workload stores its memory chart layout file unchanged, so the layout file (#12548) is the only memory chart format. Bugfix: a GPU whose layout file was skipped as broken no longer stops the database output. ## Technical Details - New JSON column `compute_workload.memory_chart_layout_extdata`: the workload's layout file, or null when its GPU has no layout. Schema version 2.4.0. - Layout metrics are panel 3 metric names, unique within panel 3. Readers find each value and unit in the existing metric tables; `src/memory_chart/README.md` says how. - No conversion, metric ids, or format version are stored. ## Issue Tracking JIRA ID: AIPROFCOMP-683 ## Test Plan - `python -m pytest tests/unit tools/test_memory_chart_*.py` and project pre-commit. - On real gfx950 workloads (one and two in a database, and with `-b 2`): `analyze --output-format db`, then draw the chart from the database with a reader written only from the README. ## Test Result - 1606 tests pass at the tip; pre-commit passes. - The stored column equals the layout file, and every layout metric finds its value. - The chart drawn from the database matches the terminal `analyze -b 3` chart byte for byte. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Vignesh Edithal <Vignesh.Edithal@amd.com>

xuchen-amd added this pull request to stack #12562 September 30, 2026 20:58

github-actions Bot added documentation Improvements or additions to documentation project: rocprofiler-compute labels Sep 30, 2026

systems-assistant Bot added the organization: ROCm label Sep 30, 2026

xuchen-amd removed this pull request from stack #12562 October 2, 2026 15:03

xuchen-amd mentioned this pull request Oct 7, 2026

feat(rocprofiler-compute): memory chart using json layouts #12548

Merged

1 task done

xuchen-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch 2 times, most recently from d3067ad to 006f865 Compare October 7, 2026 22:59

xuchen-amd changed the title ~~feat(rocprofiler-compute): store memory chart structure in analysis db~~ feat(rocprofiler-compute): store memory chart layouts in analysis db Oct 7, 2026

xuchen-amd requested a balanced review from Copilot October 7, 2026 23:02

Copilot started reviewing on behalf of xuchen-amd October 7, 2026 23:03

View session

xuchen-amd added this pull request to stack #13066 October 7, 2026 23:04

Copilot AI reviewed Oct 7, 2026

View reviewed changes

Comment thread projects/rocprofiler-compute/src/utils/analysis_orm.py Resolved

xuchen-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch from 006f865 to 9060d10 Compare October 7, 2026 23:07

xuchen-amd marked this pull request as ready for review October 7, 2026 23:08

xuchen-amd requested review from a team and prbasyal-amd as code owners October 7, 2026 23:08

xuchen-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch from 9060d10 to 1071b88 Compare October 8, 2026 16:29

xuchen-amd changed the title ~~feat(rocprofiler-compute): store memory chart layouts in analysis db~~ feat(rocprofiler-compute): store memory chart rendering spec in analysis db Oct 8, 2026

xuchen-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch from 1071b88 to 6d9f980 Compare October 8, 2026 16:35

abchoudh-amd approved these changes Oct 9, 2026

View reviewed changes

vedithal-amd requested changes Oct 9, 2026

View reviewed changes

vedithal-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch from 818904f to 8b875cc Compare October 13, 2026 17:55

vedithal-amd requested changes Oct 14, 2026

View reviewed changes

xuchen-amd marked this pull request as draft October 14, 2026 21:23

xuchen-amd changed the title ~~feat(rocprofiler-compute): store memory chart rendering spec in analysis db~~ feat(rocprofiler-compute): store the memory chart layout file in the analysis db Oct 15, 2026

xuchen-amd marked this pull request as ready for review October 15, 2026 21:06

xuchen-amd mentioned this pull request Oct 15, 2026

refactor(rocprofiler-compute): share the metric id rule #13594

Draft

1 task done

xuchen-amd requested a review from vedithal-amd October 16, 2026 03:45

xuchen-amd and others added 3 commits October 16, 2026 22:26

vedithal-amd force-pushed the users/xuchen-amd/memory_chart_db_structure branch from 00fbfc1 to 16d1bc3 Compare October 16, 2026 22:30

vedithal-amd approved these changes Oct 16, 2026

View reviewed changes

xuchen-amd merged commit 46d77d4 into rocprofiler-compute-develop Oct 17, 2026
75 checks passed

xuchen-amd deleted the users/xuchen-amd/memory_chart_db_structure branch October 17, 2026 02:24

Navigation Menu

Repository navigation

feat(rocprofiler-compute): store the memory chart layout file in the analysis db - #12582

feat(rocprofiler-compute): store the memory chart layout file in the analysis db#12582
xuchen-amd merged 4 commits into
users/xuchen-amd/memory_chart_json_layoutsfrom
users/xuchen-amd/memory_chart_db_structure

xuchen-amd commented Sep 30, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

vedithal-amd left a comment

Uh oh!

vedithal-amd left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vedithal-amd left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

xuchen-amd commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Issue Tracking

Test Plan

Test Result

Submission Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

vedithal-amd left a comment

Choose a reason for hiding this comment

Uh oh!

vedithal-amd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vedithal-amd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xuchen-amd commented Sep 30, 2026 •

edited

Loading