[yassinsolim]

JIRA ID: ROCM-29026

Motivation

Smoke/CPU_DBSync_NONE.StaticFDBSync/10 fails on gfx1200 (#13394). The gfx1200 system
database was added by #10050 and has not been touched since. 20 of its find-db entries name a
CK instance that CK now refuses to run, so TestPerfCfgParams rejects the stored perf config,
FindSolutionImpl logs Invalid config loaded from Perf Db, and the shape falls back to an
untuned solver.

#12088 did this cleanup for gfx1201. gfx1200 never got one.

Unlike gfx1201, gfx1200 has a single cause. All 20 are ConvHipImplicitGemmGroupWrwXdlops
FP16 grouped backward-weights entries whose largest tensor holds between 2^30 and 2^31
elements, which puts them in the gap between MIOpen's large-tensor gate and CK's own overflow
guard:

• MIOpen's RequiresLargeTensorCKInstance (src/ck_impl/implicitgemm_ck_util.hpp, as restored
  by revert(miopen): partial revert of CK byte span gate for large instances #12396) counts elements and fires at 2^31. None of the 20 reaches it, so MIOpen keeps
  offering the regular int32-indexing instances and the tuner recorded one of them.
• CK's tensor_exceeds_2gb (tensor_size_check.hpp, added by fix(ck): convolution edge cases large tensors #11754 on 2026-09-30) fires at
  2^31 elements or 2^31 bytes. For a 2-byte type the byte arm trips at 2^30 elements, so
  all 20 do reach it. It sets stride_overflow on the argument and the non-large-tensor
  instance bails out of IsSupportedArgument.

The database predates #11754, which added this check. Before #11754, CK accepted these configs without testing for the overflow, so the tuner recorded them unflagged. That does not mean the earlier runs computed correct results for these shapes; #11754 exists precisely because unguarded int32 indexing can silently compute wrong results, which is the same risk #11585 and #12396 are about for the other directions.

The source-side gap is tracked separately in #13419; this PR only removes the rows that
can no longer resolve.

Technical Details

20 find-db entries and 20 perf-db rows removed. Both numbers are produced by
MIOPEN_DBSYNC_CLEAN=1, which is MIOpen's own predicate (solv.IsApplicable then
solv.TestPerfCfgParams), not a hand-built list.

group	find-db entries	perf-db rows	solver
CK byte-span overflow	20	20	ConvHipImplicitGemmGroupWrwXdlops

Nothing outside that solver is touched. By CK instance family:

CK instance	entries	split_k values
DeviceGroupedConvBwdWeightTwoStage_Wmma_CShuffleV3	12	64, 128
DeviceGroupedConvBwdWeight_Wmma_CShuffleV3	8	16, 32, 64, 128

File deltas: the find database keeps all 90688 keys and changes 20 lines, each losing only the
unusable solver. The perf database goes from 77756 to 77736 lines, because for all 20 the CK
entry was the key's only perf-db row.

Why each entry fails, measured rather than inferred

An instrumented build was used to record, for every IsCKArgsSupported call the test makes,
whether MIOpen's gate fired, whether the stored instance was found, and which CK line rejected
it. Across all 40 failing checks (20 entries, each checked twice: once by TestPerfCfgParams
and once by FindSolution) the answer is identical:

CKSIZE tensor_exceeds_2gb elems=1233125376 bytes=2466250752 dtsz=2
Stride overflow! In .../device_grouped_conv_bwd_weight_two_stage_wmma_cshuffle_v3.hpp:1177, in function: IsSupportedArgument
DBG13394 key=96-224-224-7x7-3-224-224-256-3x3-1x1-1x1-0-NCHW-FP16-W|rlt=0|found=1|large_ok=1|supported=0|split_k=64|
         in_es=1233125376|wei_es=14112|out_es=38535168|in_b=2466250752|wei_b=28224|out_b=77070336|
         kid=DeviceGroupedConvBwdWeightTwoStage_Wmma_CShuffleV3<...>+64

• rlt=0: RequiresLargeTensorCKInstance returned false on all 40.
• found=1: the recorded instance still exists in the registry on all 40. No entry fails
  because an instance was renamed or dropped.
• large_ok=1: the large-tensor filter rejected nothing.
• supported=0: CK's IsSupportedArgument refused, in all 40 at the arg.stride_overflow
  branch, in device_grouped_conv_bwd_weight_two_stage_wmma_cshuffle_v3.hpp:1180 (24 checks)
  or device_grouped_conv_bwd_weight_wmma_cshuffle_v3.hpp:1054 (16 checks).
• All 40 have max element span below 2^31 and max byte span above 2^31.

So nothing here is a split-K problem, an LDS-size or KPerBlock % KPerXdl rejection from
#13079, or a PrefetchStages problem (#13373). The whole set is the byte/element threshold
gap. Element spans run 1.10e9 to 2.11e9 and byte spans 2.21e9 to 4.12e9; every entry is FP16.

On the issue's count of 31

#13394 reports 31 failures. This PR removes 20. The other 11 are not removed here.

The 20 fail in every build tested: the from-source build above and TheRock's CI artifacts. The 11 (DeviceGroupedConvBwdWeight_Explicit_Xdl x4, DeviceGroupedConvBwdWeightTwoStage_Xdl_CShuffle x4, DeviceGroupedConvBwdWeight_Xdl_CShuffleV3_WmmaPorted x3) fail only in TheRock's build. On the same gfx1200 GPU with byte-identical database files, TheRock artifacts from run 37986471185 (rocm-libraries 3ecaa6512fb, no per-target CK changes) and from run 37986476054 (per-target cutover) both fail the same 31 with a byte-identical gfx1200 CK plugin, while the source build fails 20. So the 11 come from the TheRock-built binaries. The cause is not isolated yet; #13394 has the details.

An earlier version of this description attributed the 11 to the per-target CK branch. That was wrong.

Consequence: after this PR, TheRock's gfx1200 StaticFDBSync/10 still fails on those 11 until they are resolved.

Performance consequence

16 of the 20 removed entries were the fastest recorded winner for their key. No key loses its
last entry, so there is no fallback to a runtime find.

For those 16 the next best entry is a median 1.34x slower and at worst 1.84x. The worst case is
64-222-222-3x3-3-224-224-512-0x0-1x1-1x1-0-NCHW-FP16-W, which goes from
ConvHipImplicitGemmGroupWrwXdlops at 50.6900 ms to GemmWrwUniversal at 93.5055 ms. The new
fastest solver is GemmWrwUniversal in all 16 cases.

None of this is recoverable by re-tuning as the code stands: CK refuses the regular instances
for these shapes, and MIOpen's gate does not route them to a large-tensor instance either, so a
fresh tuning run on gfx1200 would record the same unusable configs again. Closing that is what
#13419 is for.

Test Plan

Build CK and MIOpen from one checkout for gfx1200 and run the DBSync test on gfx1200 hardware.

test_db_sync --gtest_filter='Smoke/CPU_DBSync_NONE.StaticFDBSync/10'

Test Result

node       Slurm job 68055153, AMD Radeon RX 9060 XT, gfx1200
           rocminfo Compute Unit: 32, hipDeviceProp multiProcessorCount 16, warpSize 32
           test prints "Handle CU count: 16", selects the gfx120020 database
container  rocm/dev-ubuntu-22.04@sha256:80de159c03b84ef2551d6230a06a59de5cdad1cde288c136bc2042a93aeeda75
source     ROCm/rocm-libraries e56d6561886b79500d814a3cc039ee477bf45f01, MIOpen and CK built
           from the same checkout, CK installed over /opt/rocm so MIOpen binds the CK it was
           built against
cmake      MIOPEN_TEST_DBSYNC=ON MIOPEN_TEST_DISCRETE=ON MIOPEN_BUILD_CK=OFF
           MIOPEN_USE_COMPOSABLEKERNEL=ON MIOPEN_USE_HIPCONV=ON BUILD_DEV=OFF

Before, against the exact blobs of the commit under test:

FDB: "/work/db/active/gfx120020.HIP.fdb.txt", PDB: "/work/db/active/gfx120020.db.txt"
[  FAILED  ] Smoke/CPU_DBSync_NONE.StaticFDBSync/10, where GetParam() = ("gfx1200", 0) (2773 ms)
20 "Invalid perf config found", 0 "Solver is not applicable", 0 "PDB entry does not exist",
0 "Invalid solution"

Oracle pass, MIOPEN_DBSYNC_CLEAN=1:

[       OK ] Smoke/CPU_DBSync_NONE.StaticFDBSync/10 (2774 ms)
20 "Removal complete for fdb-key:"

After, with the cleaned files bz2-compressed and decompressed again, exactly as committed:

roundtrip OK gfx120020.HIP.fdb.txt 4d7c3116a6e2d4beaf8c5315b0d0f5919da7e407b3279d44cc73c4ae68961bf6
roundtrip OK gfx120020.db.txt      08e62d0fd96438d52b350a4bc288a4ff6cf9c8bb5cf8ab826b7ab28904a0b644
[       OK ] Smoke/CPU_DBSync_NONE.StaticFDBSync/10 (2356 ms)
0 residual failures

Content checks on the committed files:

• pre-change tree content byte identical to develop: decompressed sha256 ab5e514b728496d9
  (find-db) and 2c94d6a19418a233 (perf-db), the same blobs as at 3ecaa6512fb and at the
  current develop head
• removal list generated by MIOPEN_DBSYNC_CLEAN=1, so it is MIOpen's own predicate
• removals confined to ConvHipImplicitGemmGroupWrwXdlops, zero outside it
• the post-change run reports no PDB entry does not exist for tunable failure, so no
  surviving tunable find-db entry was left without its perf-db row
• bz2 round trip verified against the cleaned files

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/rocm-libraries/blob/develop/CONTRIBUTING.md#pull-requests

[Copilot]

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

[therock-pr-bot]

⏳ Waiting on Required Checks

⏳ Required checks have not all concluded yet. This bot run is exiting without failing so those checks can report independently.

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	PR does not contain code files — Unit Test auto-passed
🔎 pre-commit	⏳ Pending	⏳ Still running…
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

⏳ Required checks are still running or have not started yet.
This PR is not ready for review until they complete.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[therock-pr-bot]

⏳ Waiting for required checks to start or finish:

• ⏳ pre-commit

[SreecharanGundaboluAMD]

LGTM

[assistant-librarian]

TheRock Submodule Bump Activity

Newest first

• 2026-10-16 — Included in TheRock via ROCm/TheRock#9032.