Skip to content

[MIOpen] Remove stale Naive solver systemDB entries gfx942 (ALMIOPEN-1776) - #7028

Merged
amd-bartgips merged 6 commits into
developfrom
users/bartgips/remove_stale_naive_systemdb_entries
Jun 6, 2026
Merged

amd-bartgips merged 6 commits into
developfrom
users/bartgips/remove_stale_naive_systemdb_entries

Conversation

@amd-bartgips

@amd-bartgips amd-bartgips commented May 4, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

The gfx942 system find-DB contains entries where the naive convolution solver is listed as
fastest. For a subset of these shapes, GPU benchmarking shows a non-naive heuristic solver
is significantly faster in practice. This PR removes those entries so MIOpen can select the
correct solver at runtime.

Risk Assessment

Low risk. The change is a targeted removal of selected stale system-DB entries for gfx942 where the naive convolution solver was previously recorded as fastest but is empirically outperformed by heuristic-selected solvers. No changes to public APIs, ABI, kernel implementations, or build system are introduced; this PR only updates static DB contents. At runtime, affected cases will gracefully fall back to existing heuristic selection or full search, both of which are already well-tested execution paths. The removal criteria are conservative (≥10% measured speedup and validation against unconstrained heuristics), minimizing the risk of regressions while improving solver selection accuracy.

Technical Details

To make this somewhat feasible, we have relied on the heuristics model to select an alternative kernel to compare against (rather than run exhaustive tuning).
TLDR: we extracted all system-DB entries where naive was deemed fastest and ran benchmarks to compare the naive kernel time for these shapes to the kernel time of the kernel suggested by heuristics.

In more detail:

  • Candidate extraction parsed DB lines directly (not grep-only), classifying each as
    sole-naive (naive is the only entry) or naive-fastest (naive has the lowest embedded
    kernel time on the line)
  • Benchmark compared two modes on all 5,737 shapes:
    • Baseline: system DB present (naive entries active)
    • Experimental: MIOPEN_DEBUG_CONV_DIRECT=0 so heuristics must find a non-naive solver
  • Removal rule: remove the entire DB line if speedup ≥ 10% AND the heuristic solver is non-naive
    • speedup = (baseline_time − experimental_time) / baseline_time × 100
  • Additional validation: After identifying 249 candidate keys by the benchmark rule above, each
    was checked against the unconstrained TunaNet heuristic (no MIOPEN_DEBUG_CONV_DIRECT=0).
    13 keys were excluded because TunaNet also selects naive for those shapes — removing their
    DB entry would offer no benefit (MIOpen still picks naive, but via the slower heuristic
    path rather than the fast DB lookup)
  • Whole-line removal is intentional: the remaining non-naive entries in each line are equally
    stale; removing the line lets MIOpen's find phase pick fresh with current heuristics

Test Plan

  1. Parse both gfx942 DB files; classify naive-fastest keys (5,737 unique)
  2. Run interleaved baseline/experimental benchmark across all 5,737 shapes
  3. Apply conservative removal rule (≥10% speedup, non-naive experimental solver) → 249 candidates
  4. For each candidate, run the unconstrained heuristic (no MIOPEN_DEBUG_CONV_DIRECT=0) and
    exclude any key where TunaNet would also select naive → 236 final removals
  5. Apply removal, verify no removed keys remain in either DB file
  6. Recompress .fdb.txt → .fdb.txt.bz2

Test Result

Full benchmark completed (5,737 shapes, gfx942 MI300X):

Metric Value
Comparable pairs 5,710
Initial candidates (≥10% speedup, non-naive) 249
Excluded (heuristic also picks naive) 13
Final removals 236
Keep (naive genuinely fastest) 5,474
Avg speedup on removed shapes +86.0%

The 5,474 "keep" shapes confirm naive is legitimately the fastest solver for the vast majority
of these DB entries. The 13 excluded shapes exceed the speedup threshold by benchmark but
TunaNet independently corroborates naive as best in production — their DB entries are preserved.
In the scatter plots, points above the remove-boundary line that are not marked "removed" are
these 13 excluded cases.

See a graphic overview of the benchmark results for all 5,474 shapes under consideration:
image

(note that there are some red dots below the removal threshold/boundary, these correspond to shapes where the TunaNet heuristics model would also select the Naive solver, hence removal would not help us here.)

Submission Checklist

…fx942

Of the initial 249 candidate keys (naive fastest by embedded DB time and
>=10% faster heuristic alternative), 13 were excluded because the
unconstrained TunaNet heuristic also selects naive for those shapes.
Removing those entries would offer no benefit: MIOpen would still pick
naive, but via the slower heuristic path rather than the fast DB lookup.

The remaining 236 keys are safe to remove: after removal, TunaNet
selects a genuinely non-naive solver (e.g. ConvHipImplicitGemm3DGroupFwdXdlops,
ConvAsmImplicitGemmGTCDynamicWrwXdlopsNHWC, GemmFwd1x1_0_1) for each shape.

Co-Authored-By: Claude Sonnet 4 (1M context) <noreply@anthropic.com>
@amd-ahyttine amd-ahyttine changed the title [MIOpen] Remove stale Naive solver systemDB entries gfx942 [MIOpen] Remove stale Naive solver systemDB entries gfx942 (ALMIOPEN-1776) May 15, 2026

@JonathanLichtnerAMD JonathanLichtnerAMD left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but we need to add a "Risk Assessment" section to the PR description.

Also, this is still a "draft" PR.

@amd-bartgips
amd-bartgips marked this pull request as ready for review May 26, 2026 11:28
@amd-bartgips
amd-bartgips requested a review from a team as a code owner May 26, 2026 11:28
@amd-bartgips

Copy link
Copy Markdown
Contributor Author

LGTM, but we need to add a "Risk Assessment" section to the PR description.

Also, this is still a "draft" PR.

Added such a section and moved it to "ready for review" 👍

@amd-bartgips
amd-bartgips enabled auto-merge (squash) May 29, 2026 06:52
@JonathanLichtnerAMD

Copy link
Copy Markdown
Contributor

LGTM. Should we add an item to the changelog for this?

@amd-bartgips
amd-bartgips merged commit f42668f into develop Jun 6, 2026
50 checks passed
@amd-bartgips
amd-bartgips deleted the users/bartgips/remove_stale_naive_systemdb_entries branch June 6, 2026 12:49
JonathanLichtnerAMD added a commit that referenced this pull request Jul 15, 2026
JIRA ID: ALMIOPEN-2312

## Motivation

`projects/miopen/CHANGELOG.md`'s "MIOpen 3.5.2 for ROCm 7.14.0" section
only had 4 entries, but 68 commits touched `projects/miopen` between the
`therock-7.13` tag and `release/therock-7.14`. Auditing that commit
range (via `git log --oneline
therock-7.13..remotes/origin/release/therock-7.14 -- projects/miopen`)
found 18 additional user-facing changes — new gfx950/gfx1250
architecture support, SystemDB refreshes, AI-heuristic improvements,
several correctness/crash bug fixes, and OpenCL solver removals — that
were never documented. This PR backfills those entries so the 7.14.0
changelog accurately reflects what shipped.

## Technical Details

Added 24 lines to the existing "MIOpen 3.5.2 for ROCm 7.14.0" section of
`projects/miopen/CHANGELOG.md`, merging new bullets into the existing
`Added`/`Changed`/`Resolved Issues` sub-sections (no new entries were
duplicated or reworded) and adding a new `Removed` sub-section for
solver removals. No existing content was altered.

Each new bullet was traced to a specific commit + PR:

| Entry | Commit | PR |
|---|---|---|
| Added gfx950 (MI350X/MI355X) SystemDB | `60d8e1e5175` | #8081 |
| Added initial gfx1250 architecture support | `d3f057b2f22` | #7587 |
| Added JSON-formatted performance logs (`MIOPEN_PERFORMANCE_LOGS`) |
`ffbde871260` | #5736 |
| Refreshed gfx942 SystemDB | `faf754a9ef6` | #8174 |
| Refined 2D AI heuristics (Tunanet/KTN) for gfx942 & gfx950 |
`c24ad415884` | #7109 |
| Fixed thread-safety issue in AI-heuristic model caches | `cd957402309`
| #9149 (cherry-pick of #9131) |
| Fixed CK grouped-conv solvers not always registered in host |
`c971d54d7ae` | #9105 |
| Fixed legacy CK buffer resource descriptor (gfx1250) | `1dee011b8b0` |
#8758 |
| Removed stale gfx942 SystemDB naive-solver entries | `f42668f6316` |
#7028 |
| Fixed softmax NaN when `beta=0` | `fb8157c8512` | #8426 |
| Fixed BN backward spatial variant 1 vectorized memory access bug |
`1351a7e8b10` | #6765 |
| Fixed use-after-free of `kernel_id` in `InitInvokerFactoryNCHW` |
`394ebffb50d` | #7922 |
| Fixed SoftMarginLoss forward reduce-kernel kernarg corruption |
`1e65a31570f` | #8065, #8135 |
| Fixed build with `MIOPEN_ENABLE_AI_KERNEL_TUNING=OFF` | `e6288c3ca61`
| #6800 |
| Fixed install RPATH for MIOpenDriver / CK backend libs | `ef8a4cf2f93`
| #7485 |
| Fixed dangling lambda causing log-level 6 segfault | `190f9791737` |
#6953 |
| Enabled Windows long-path support for `addkernels` | `1f9d69724cf` |
#7387 |
| Removed OCL solver `ConvOclDirectFwd1x1` | `126c720cadd` | #6491 |
| Removed OCL solver `conv_ocl_dir2Dfwdgen` | `2d499ddf709` | #6633 |
| Removed OCL solvers `ConvOclBwdWrW1x1`, `ConvOclBwdWrW2` |
`b5d43a33dc6` | #6859 |
| Removed OCL solvers `OpTensorLeadingOnes`, `Op5dTensorGeneric` |
`8642c83b7de` | #7347 |

The remaining ~40 commits in the 68-commit range were deliberately
excluded (tests/CI/build/docs/internal refactors/1:1 OCL→HIP swaps with
no behavior change, per each commit's own stated scope) and are not
represented here.

## Risk Assessment

**Risk Level: 🟢 Low**

This is a documentation-only change to a single Markdown file. No code,
build configuration, tests, or public API surface is touched.

### Impacted Components

- `projects/miopen/CHANGELOG.md` only.

### Potential Side Effects

None expected. This does not change any runtime behavior; it only
documents changes that already shipped on `release/therock-7.14`.

### Mitigation Steps

None required beyond normal review — a reviewer familiar with the 7.14
release can spot-check a few entries against the linked commit hashes/PR
numbers for accuracy.

## Test Plan

Not applicable (no code changes). Verification was done by
cross-referencing every added bullet against a specific commit hash and
PR number pulled from `git log`, and by diffing the edited section
against `git show
remotes/origin/release/therock-7.14:projects/miopen/CHANGELOG.md` to
confirm no existing entry was duplicated or altered.

## Test Result

All 21 new bullets map 1:1 to a real commit/PR in the
`therock-7.13..release/therock-7.14` range for `projects/miopen`. No
pre-existing changelog text was modified — `git diff` shows a pure
addition (24 insertions, 0 deletions). Two borderline commits
(`a67834413e8` "Standalone build", `7c7cd5774a9` "enable warnings about
deprecated API/features") were deliberately left out pending author
judgment on whether they warrant an entry.
pschang-phy pushed a commit that referenced this pull request Jul 17, 2026
JIRA ID: ALMIOPEN-2312

## Motivation

`projects/miopen/CHANGELOG.md`'s "MIOpen 3.5.2 for ROCm 7.14.0" section
only had 4 entries, but 68 commits touched `projects/miopen` between the
`therock-7.13` tag and `release/therock-7.14`. Auditing that commit
range (via `git log --oneline
therock-7.13..remotes/origin/release/therock-7.14 -- projects/miopen`)
found 18 additional user-facing changes — new gfx950/gfx1250
architecture support, SystemDB refreshes, AI-heuristic improvements,
several correctness/crash bug fixes, and OpenCL solver removals — that
were never documented. This PR backfills those entries so the 7.14.0
changelog accurately reflects what shipped.

## Technical Details

Added 24 lines to the existing "MIOpen 3.5.2 for ROCm 7.14.0" section of
`projects/miopen/CHANGELOG.md`, merging new bullets into the existing
`Added`/`Changed`/`Resolved Issues` sub-sections (no new entries were
duplicated or reworded) and adding a new `Removed` sub-section for
solver removals. No existing content was altered.

Each new bullet was traced to a specific commit + PR:

| Entry | Commit | PR |
|---|---|---|
| Added gfx950 (MI350X/MI355X) SystemDB | `60d8e1e5175` | #8081 |
| Added initial gfx1250 architecture support | `d3f057b2f22` | #7587 |
| Added JSON-formatted performance logs (`MIOPEN_PERFORMANCE_LOGS`) |
`ffbde871260` | #5736 |
| Refreshed gfx942 SystemDB | `faf754a9ef6` | #8174 |
| Refined 2D AI heuristics (Tunanet/KTN) for gfx942 & gfx950 |
`c24ad415884` | #7109 |
| Fixed thread-safety issue in AI-heuristic model caches | `cd957402309`
| #9149 (cherry-pick of #9131) |
| Fixed CK grouped-conv solvers not always registered in host |
`c971d54d7ae` | #9105 |
| Fixed legacy CK buffer resource descriptor (gfx1250) | `1dee011b8b0` |
#8758 |
| Removed stale gfx942 SystemDB naive-solver entries | `f42668f6316` |
#7028 |
| Fixed softmax NaN when `beta=0` | `fb8157c8512` | #8426 |
| Fixed BN backward spatial variant 1 vectorized memory access bug |
`1351a7e8b10` | #6765 |
| Fixed use-after-free of `kernel_id` in `InitInvokerFactoryNCHW` |
`394ebffb50d` | #7922 |
| Fixed SoftMarginLoss forward reduce-kernel kernarg corruption |
`1e65a31570f` | #8065, #8135 |
| Fixed build with `MIOPEN_ENABLE_AI_KERNEL_TUNING=OFF` | `e6288c3ca61`
| #6800 |
| Fixed install RPATH for MIOpenDriver / CK backend libs | `ef8a4cf2f93`
| #7485 |
| Fixed dangling lambda causing log-level 6 segfault | `190f9791737` |
#6953 |
| Enabled Windows long-path support for `addkernels` | `1f9d69724cf` |
#7387 |
| Removed OCL solver `ConvOclDirectFwd1x1` | `126c720cadd` | #6491 |
| Removed OCL solver `conv_ocl_dir2Dfwdgen` | `2d499ddf709` | #6633 |
| Removed OCL solvers `ConvOclBwdWrW1x1`, `ConvOclBwdWrW2` |
`b5d43a33dc6` | #6859 |
| Removed OCL solvers `OpTensorLeadingOnes`, `Op5dTensorGeneric` |
`8642c83b7de` | #7347 |

The remaining ~40 commits in the 68-commit range were deliberately
excluded (tests/CI/build/docs/internal refactors/1:1 OCL→HIP swaps with
no behavior change, per each commit's own stated scope) and are not
represented here.

## Risk Assessment

**Risk Level: 🟢 Low**

This is a documentation-only change to a single Markdown file. No code,
build configuration, tests, or public API surface is touched.

### Impacted Components

- `projects/miopen/CHANGELOG.md` only.

### Potential Side Effects

None expected. This does not change any runtime behavior; it only
documents changes that already shipped on `release/therock-7.14`.

### Mitigation Steps

None required beyond normal review — a reviewer familiar with the 7.14
release can spot-check a few entries against the linked commit hashes/PR
numbers for accuracy.

## Test Plan

Not applicable (no code changes). Verification was done by
cross-referencing every added bullet against a specific commit hash and
PR number pulled from `git log`, and by diffing the edited section
against `git show
remotes/origin/release/therock-7.14:projects/miopen/CHANGELOG.md` to
confirm no existing entry was duplicated or altered.

## Test Result

All 21 new bullets map 1:1 to a real commit/PR in the
`therock-7.13..release/therock-7.14` range for `projects/miopen`. No
pre-existing changelog text was modified — `git diff` shows a pure
addition (24 insertions, 0 deletions). Two borderline commits
(`a67834413e8` "Standalone build", `7c7cd5774a9` "enable warnings about
deprecated API/features") were deliberately left out pending author
judgment on whether they warrant an entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants