Add partially mapped resources tests #476

bob80905 · 2025-10-17T22:23:12Z

This PR is the first step in bringing support to testing with partially mapped and unmapped resources.
It lays out the infrastructure necessary to initialize resources with only a certain number of tiles mapped.
The Load overload that takes the 2nd status argument is implemented in DXC, so we take advantage of that and test that when accessing data in an unmapped resource, we get 0'd data and the status integer is also 0.

The test is currently unsupported in Clang and Vulkan, but will be supported in the future.
Fixes #182

bogner

This is great. We may want to revisit and see if we can do some code deduplication in a nice way at some point, but I think this makes sense as is for now.

A few of minor comments.

bogner · 2025-10-18T01:02:08Z

include/Support/Pipeline.h

  std::optional<VulkanBinding> VKBinding;
  Buffer *BufferPtr = nullptr;
  bool HasCounter;
+  int TilesMapped = -1;


std::optional<int> would be clearer than using -1 as a sigil here, and isn't really that much more expensive.

bogner · 2025-10-18T01:03:56Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+
+

Some extra whitespace here

bogner · 2025-10-18T01:05:05Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+  // 32 S structs inside X or Y occupy 64KB of data. (32 * 512 ints * 4 bytes per int)
+  // So, any index into the buffer >= [32] will access a new "tile"


Looks like this might be over 80-column. It's a bit annoying to clang-format within these .test files (easiest is to paste in another file and clang-format that...), but please try to follow LLVM style regardless.

bogner · 2025-10-18T01:07:28Z

lib/API/DX/Device.cpp

-        memset(ExtraData, 0, CBVSize - R.size() - 1);
+        memset(ExtraData, 0, CBVSize - R.size());


Is this an unrelated bug fix? I'm nervous about a change like this without accompanying tests.

Yes, it is, but I won't be in the business of writing a test for this change here in this PR.
I'll undo this for now.
#477

Are you still intending to undo this change, or did you find it is actually required?

I am intending to undo this change, it should already be undone right now.

bogner · 2025-10-18T01:09:39Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+    Format: Int32
+    Stride: 2048 # S is 512 ints, 512*4 = 2048.
+    FillSize: 131072
+    FillValue: 1


It's easier to see what parts of the data are read if we use something more noticeable here (like 42, or 9001 or whatever)

bogner · 2025-10-18T01:11:05Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+# XFAIL: Clang
+# XFAIL: Vulkan


These need comments like Unimplemented <link to issue> or Bug <link to issue> so that we can track why they're XFAIL'd and loop back appropriately.

lib/API/DX/Device.cpp

test/Feature/HLSLLib/PartiallyMappedResources.test

lib/API/DX/Device.cpp

bogner · 2025-10-23T21:13:05Z

lib/API/DX/Device.cpp

+      // Tile mapping setup (optional if NumTiles > 0)
+      const UINT NumTiles = getNumTiles(R.TilesMapped, ResDesc.Width);
+
+      ComPtr<ID3D12Heap> Heap; // optional, only created if NumTiles > 0
+
+      if (NumTiles > 0) {


This whole block (up to the updateTileMappings call) looks identical to the SRV version. Can we pull out a helper function to call in both cases?

Turns out I'll have to leave unmapped UAVs for another time, so I'll pull out a helper function in a future PR.

bogner · 2025-10-23T21:15:11Z

lib/API/DX/Device.cpp

+      // Tile mapping setup (optional if NumTiles > 0)
+      const UINT NumTiles = getNumTiles(R.TilesMapped, ResDesc.Width);


This comment feels a bit confusing - NumTiles being zero is the edge case given that it can only happen if we explicitly set zero tiles. "optional if NumTiles > 0" sounds to me like NumTiles will usually be zero.

lib/API/DX/Device.cpp

…80905/offload-test-suite into add_unmapped_resources_tests

spall · 2025-10-29T18:32:40Z

lib/API/DX/Device.cpp

    addUploadEndBarrier(IS, Destination, R.isReadWrite());
  }

+  UINT getNumTiles(std::optional<int> NumTiles, UINT64 Width) {


should this just contain a uint instead of an int? I assume the number of tiles can never be negative

spall · 2025-10-29T18:36:32Z

lib/API/DX/Device.cpp

+    else
+      // Map the entire buffer by computing how many 64KB tiles cover it
+      Ret = static_cast<UINT>(
+          (Width + D3D12_TILED_RESOURCE_TILE_SIZE_IN_BYTES - 1) /


could this operation overflow? Why is width a uint64?

Currently the arg that feeds into the width parameter is an int datatype. So it doesn't need to be a uint64, at least not yet. I'll change it.

could width or d3d12_tiled_resource_tile_size_in_bytes be uint32 max or half that amount; could this operation overflow after you change width to be a uint32?

I don't believe D3D12_TILED_RESOURCE_TILE_SIZE_IN_BYTES can be any larger than 65536. However, width could be any arbitrary size, depending on how the developer specified it. So yes, I think this operation could overflow, though in exceedingly specific and pathological circumstances.

FWIW: in D3D12, Width is a uint64 because it needs to be able to support describing buffers that are larger than 4gb.

Yes, though for this API:
UpdateTileMappings
The pRangeTileCounts param is a UINT. Its value in my code comes directly from counting the number of tiles. So I don't think this should ever be > uint32_max.

spall · 2025-10-29T19:33:55Z

lib/API/DX/Device.cpp

      if (R.size() < CBVSize) {
        void *ExtraData = static_cast<char *>(ResDataPtr) + R.size();
-        memset(ExtraData, 0, CBVSize - R.size() - 1);
+        memset(ExtraData, 0, CBVSize - R.size());


should this line be unmodified? I see a long unresolvd comment thread about it and the conclusion is the change should be undone, but I still see a modification?

spall · 2025-10-29T20:13:14Z

lib/API/DX/Device.cpp

-                ComPtr<ID3D12Resource> Readback)
-        : Upload(Upload), Buffer(Buffer), Readback(Readback) {}
+                ComPtr<ID3D12Resource> Readback,
+                ComPtr<ID3D12Heap> Heap = nullptr)


why is this defaulted to null but the others are not?

Both CBV and UAV don't need to pass the Heap to the Resource bundle (yet) because only SRVs are using reserved resources by default, and that's the only case where the lifetime of the Heap need to be extended. The other cases will need non-null values, especially for functions like Bind* or CreateComputeCommands.

lib/API/DX/Device.cpp

damyanp · 2025-11-10T23:34:31Z

Code LGTM, on assumption that all these tests failing aren't related to this change.

bob80905 · 2025-11-11T04:58:52Z

Code LGTM, on assumption that all these tests failing aren't related to this change.

llvm/llvm-project#166449
Actually, the above PR is a prerequisite for this PR, otherwise the new file will fail.

bogner · 2025-11-11T17:34:42Z

Code LGTM, on assumption that all these tests failing aren't related to this change.

llvm/llvm-project#166449 Actually, the above PR is a prerequisite for this PR, otherwise the new file will fail.

The test will only fail with clang without that change, it should pass with dxc - we generally mark such tests as XFAIL: Clang waiting on the unimplemented feature.

bogner

API generally looks good to me. A few comments on improving / simplifying the test.

bogner · 2025-11-11T17:44:48Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+  if (CAFMResult){
+    CAFM[idx] = CAFMResult;
+    Out[idx] = Out0.x;
+    Out[idx + 4] = status;


In practice using status directly is the same as using the result of CheckAccessFullyMapped, but at least according to the docs it's illegal to access status directly. We should probably use CAFMResult here if it's useful (or just omit this - the CAFM buffer records the information already).

bogner · 2025-11-11T17:49:43Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+
+  uint status;
+  int4 Out0 = X.Load(0, status);
+  int CAFMResult = CheckAccessFullyMapped(status) ? 1 : 0;


Why not bool CAFMResult = CheckAccessFullyMapped(status);? Also why not make CAFM an RWStructuredBuffer<bool> instead of casting this to int at all?

bogner · 2025-11-11T17:54:39Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+  int4 Out0 = X.Load(0, status);
+  int CAFMResult = CheckAccessFullyMapped(status) ? 1 : 0;
+  if (CAFMResult){
+    CAFM[idx] = CAFMResult;


It would be clearer to assign to CAFM[idx] unconditionally - that way we're not relying on the initialization of the CAFM buffer at all. It might also make sense to explicitly assign a known constant (like 0x5a5a5a5a) to Out[idx] when CAFMResult is false

bogner · 2025-11-11T19:32:55Z

test/Feature/HLSLLib/PartiallyMappedResources.test

+  if (CAFMResult)
+    Out[idx] = Result.x;
+  else {
+    if (all(Result == int4(0,0,0,0)))


Is Result even defined if CheckAccessFullyMapped returns false? We've read from an unmapped tile in that case...

There are some references from d3d11 that show that we expect 0 when loading from an unmapped tile:
https://learn.microsoft.com/en-us/windows/win32/direct3d11/operations-available-on-tiled-resources
https://microsoft.github.io/DirectX-Specs/d3d/archive/D3D11_3_FunctionalSpec.htm

I think reading from unmapped depends on the tiled resource tier, and is undefined in some cases: https://learn.microsoft.com/en-us/windows/win32/api/d3d12/ne-d3d12-d3d12_tiled_resources_tier

bob80905 added 9 commits October 16, 2025 14:01

first attempt

f43844d

Merge branch 'main' into add_unmapped_resources_tests

38fde3c

Merge branch 'main' into add_unmapped_resources_tests

d7908c7

first attempt

a3b4ce1

self review

2d371ed

add check access fully mapped

284d8fe

make all 3 new functions more consistent between each other

9c6664b

rename test file name

eda2715

address variable casing errors

b58d924

bogner reviewed Oct 18, 2025

View reviewed changes

damyanp reviewed Oct 18, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

test/Feature/HLSLLib/PartiallyMappedResources.test Show resolved Hide resolved

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

bob80905 added 3 commits October 20, 2025 18:51

address Justin and Damyan

59a0b7c

fix build errors

d3cbf91

attempt to unify SRV

8d9cbbb

bob80905 force-pushed the add_unmapped_resources_tests branch from 8d9cbbb to d3cbf91 Compare October 22, 2025 20:12

bob80905 added 2 commits October 22, 2025 15:03

fix texturelayout flag and another flag

fb4a631

add metal XFAIL

1c9752f

bogner reviewed Oct 23, 2025

View reviewed changes

improve, but keep UAV the same

3e78f39

bob80905 force-pushed the add_unmapped_resources_tests branch from afcbf1e to 3e78f39 Compare October 24, 2025 17:46

bob80905 added 3 commits October 24, 2025 10:53

remove unwanted changes

fb86727

address confusing comment Justin pointed out

b9b202b

clang-format

361225b

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

damyanp reviewed Oct 24, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

bob80905 added 6 commits October 24, 2025 12:51

address damyan, xfail intel

eb89f86

remove reserved CBV function, and capitalize var names

c59e9e2

Merge branch 'main' into add_unmapped_resources_tests

8579b37

add some missing constts

6a7878d

Merge branch 'add_unmapped_resources_tests' of https://github.com/bob…

26f1045

…80905/offload-test-suite into add_unmapped_resources_tests

self review

06bf89c

damyanp assigned damyanp and spall Oct 29, 2025

spall reviewed Oct 29, 2025

View reviewed changes

bob80905 added 2 commits November 6, 2025 13:11

address Sarah

e7f9a91

add overflow assert

2b2b9aa

damyanp reviewed Nov 10, 2025

View reviewed changes

lib/API/DX/Device.cpp Outdated Show resolved Hide resolved

fix assert condition, and update resource elty

7b8fa4c

damyanp approved these changes Nov 10, 2025

View reviewed changes

add back accidentally removed XFAIL

b21a1c6

bogner reviewed Nov 11, 2025

View reviewed changes

bob80905 added 3 commits November 11, 2025 10:43

update test

b6effce

validate 0's return behavior

ffff4eb

use bool vals instead of ints

0dd409e

bogner reviewed Nov 11, 2025

View reviewed changes

remove 0's check, just assign if cafm returns false

84e9d82

bogner approved these changes Nov 11, 2025

View reviewed changes

use 1 and 0 rather than true and false to resolve error

a861854

		// 32 S structs inside X or Y occupy 64KB of data. (32 * 512 ints * 4 bytes per int)
		// So, any index into the buffer >= [32] will access a new "tile"

		memset(ExtraData, 0, CBVSize - R.size() - 1);
		memset(ExtraData, 0, CBVSize - R.size());

		// Tile mapping setup (optional if NumTiles > 0)
		const UINT NumTiles = getNumTiles(R.TilesMapped, ResDesc.Width);

Add partially mapped resources tests #476

Are you sure you want to change the base?

Add partially mapped resources tests #476

Uh oh!

Conversation

bob80905 commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bogner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spall Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bob80905 Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

damyanp commented Nov 10, 2025

Uh oh!

bob80905 commented Nov 11, 2025

Uh oh!

bogner commented Nov 11, 2025

Uh oh!

bogner left a comment

Choose a reason for hiding this comment

bob80905 commented Oct 17, 2025 •

edited

Loading

spall Nov 7, 2025 •

edited

Loading

bob80905 Nov 7, 2025 •

edited

Loading