Loading from Buffer Addresses

[Tobski]

Is your feature request related to a problem? Please describe.
ResourceDescriptorHeaps are an incredibly flexible way of accessing descriptors, but they imply that all descriptors are the same size, which is untrue more or less universally across vendors.
As a simple example, an unformatted buffer is generally going to look like an address and a size, whereas an image has significantly more data associated with it.

Vulkan Working Group members have looked at preliminary data suggesting that having all descriptors in a homogenous array results in notable slowdown (high single to low double digits) for some apps, to varying degrees on different vendors, compared to using separate parallel arrays. This has been verified by experiments with VK_EXT_mutable_descriptor_type, though we have no publishable data at present.

Some mitigation strategies exist for drivers, but we'd like to see developers given tools to manage this better themselves as we move to increasingly "bindless" ways of managing descriptors (and thus less scope for driver intervention).

A key idea we've identified is that if we could separate buffers (and acceleration structures) out from the main resource heap, and provide these in a separate array, it would allow developers the option to pull these from a more tightly packed array with limited shader changes, and we believe this can recover most or possibly all of the performance compared to just using a single flat array.

In addition, giving developers such tools will unlock more flexibility in managing data, particularly unlocking the ability to create recursive data structures without resorting to local offsets.

Describe the solution you'd like
Ideally, we'd like to see this addressed in a way that works for both Vulkan and DirectX, without requiring any API changes.
One potential path we see is to use buffer addresses for this purpose.
In both Vulkan and DX12 it's possible to obtain a GPU address for a buffer resource, and in Vulkan it's possible to further use this in the shader in lieu of a buffer descriptor, though it's exposed in HLSL in the vk namespace as a simple load from address, and doesn't fit very neatly with the rest of HLSL.
We also want to avoid doing something where we add general pointer support to HLSL, as this is both an enormous task and may be undesirable as HLSL is a largely "robust" language to which adding significant pointer support could compromise.

An option we'd like to entertain is providing a way to create a buffer from a base address and size:

struct SizedAddress {
    uint64_t  address;
    uint64_t  size;
};

// Root buffer acting as a heap
StructuredBuffer<SizedAddress> BufferHeap;

[numthreads(32,1,1)]
void CSMain()
{
    // Resource Heap syntax
    ByteAddressBuffer myBuffer = ResourceDescriptorHeap[0];

    // Rough proposed new syntax
    ByteAddressBuffer myBuffer2 = ByteAddressBuffer(BufferHeap[0].address,BufferHeap[0].size);
}

{RW}ByteAddressBuffer and {RW}StructuredBuffer would now be allowed to be created from a 64-bit address and 64-bit size anywhere in code. This would allow users to define recursive structures in memory with little effort.

Describe alternatives you've considered
Other options/extensions considered include:

• extending the existing templated raw load method in the vk namespace to DX12 as well, but as mentioned this doesn't fit very well.
• Allowing {RW}Structured/ByteAddress buffers to be created as a sized-type directly from memory, to avoid the constructor and simplify management, though this can be achieved in user code.
• Adding "SizedAddress" (or similar) as a built-in type to avoid the constructor and potentially add other helper functions.
• Foregoing the size altogether; the size adds a level of robustness that is likely desirable by many users; though applications can always bypass it by setting it to UINT64_MAX (which hopefully compilers will detect and omit excess range checks, if it can't be done by DXC).

Generally there are probably a wide range of syntax options here that would work, and I'm raising this as an issue rather than a proposal in large part because I believe others in the HLSL team would be better suited to determine the best syntax.

Additional context
The main question I have is whether this can be added without any runtime changes. Both APIs and their respective IR/IL support pointer operations, so in theory this can be done; however looking at DXIL it seems to use a 32-bit pointer model, which may make this non-viable without runtime changes for a hypothetical SM6.X.

If this can't be done within the context of the existing runtime, it would be nice to consider this for inclusion in future Shader Models, though I understand that there is currently no escalation process from here for that, and we may need to push this through other channels. If runtime changes are needed, please let me know and we'll pursue this via other means.

As additional background, this was originally raised against the DXC repo here: microsoft/DirectXShaderCompiler#4732

[llvm-beanz]

There are a couple of different challenges in solving this as described which make it require a shade model revision.

The code sample demonstrates reading addresses and sizes from a UAV and converting those into new buffers. CPU and GPU virtual addresses aren't guaranteed to match on all hardware platforms or runtimes, so at a minimum we would need an interface to convert a CPU address to a GPU address, or (even if the virtual addresses are the same) an interface to ensure that the CPU address is mapped to the GPU.

We also probably want to create some sort of span object rather than creating a new UAV since UAV objects have a lot of special to them...

This does represent a request that we frequently get, so I do think it is interesting and valuable to track. Unfortunately since it is going to require a shader model change we'll need to develop it under NDA with hardware vendors.

[Tobski]

Thanks for the response! I suppose a question which is worth asking is whether it would make sense to do the shading language design separately from the NDA runtime changes? Particularly if there's desire to do something like a "span" which is primarily a language construct rather than something an IHV needs to expose. I don't know if you're set up to do this sort of separation but I figure it's worth asking at least!

[llvm-beanz]

I don’t have a firm answer here. On a very personal level (not speaking for my team or employer), I want our team to be as open about this as we possibly can. We don’t know yet exactly where those lines will fall. This process is truly an experiment for us, just like open sourcing DXC was.

Part of the challenge with navigating our longstanding process with hardware vendors is that often even the existence of a language feature implies features of the underlying hardware that supports it.

In this case, I suspect we can be more open since the capabilities of GPU IO MMU units is pretty well understood, but I don’t want to publicly promise anything that gets people yelling at me Monday morning.

I suspect we might even be able to publicly try to define spans as an object constructible from existing buffer types, which would allow the description of spans to be public, while other aspects of your request here might need to be under NDA. I’m going to try and be as creative as I can about slicing things up to maximize our public visibility.

Given the wave of holidays coming up way too quickly here in the US I don’t know that I’ll have good answers for you before the end of the year, but please be patient. I want to address your need and engage you in the process to get there. <insert unoriginal and inspirational phrase about long journeys… We’ll get there> 😄 .

[Tobski]

Thanks! Totally understand, I had roughly the same dilemma facing me to get agreement from the Vulkan WG in posting this 😅 - looking forward to seeing what comes out of this, however it happens. Thanks for being as transparent as you have been!

Edit: Also yes I had the same thought about spans, defining them in terms of existing language features seems like a good starting point if that's a viable option.

[devshgraphicsprogramming]

I'd actually want scoped pointers.

Ideally we could have qualifiers/attributes on pointers like one has in C++ i.e. casting a volatile or const pointer to another type is an error.

Similarly you could annotate a pointer that it can only point at a specific RWBuffer, array variable or shared memory (we could use a single type for all smem variable declarations if smem aliasing is implemented). You couldnt convert or cast from one pointer to another with different attribute/qualifier.

If HLSL gains support for something like SPIR-V/GLSL's buffer device address, then the qualifier/attribute for all your global RWBuffer pointers could be identical.

The person to talk about this to would be @nanokatze or @Hugobros3 as they're dealing with that in their own compiler projects.

[StarsX]

If we bind a dummy SRV/UAV buffer to the root with address 0:
ID3D12GraphicsCommandList::SetComputeRootUnorderedAccessView(RootParameterIndex, (D3D12_GPU_VIRTUAL_ADDRESS)0);
Then logically, we can index the content of this buffer in shader using ByteAddressBuffer. However, there is lack of one step: we need a uint64 index.

[devshgraphicsprogramming]

It looks like if this gets accepted: microsoft/DirectXShaderCompiler#5089

Then it would eventually have to be refactored into using references, and then you'd have to scope them, i.e.

[const] T& [cbuffer,image,global,shared,private] [volatile] [restrict]

basically borrowing from CUDA/OpenCL/Metal.

Not sure if there's any point in differentiating between host and device as opposed to having just a global, as currently neither GLSL or HLSL do, even SPIR-V memory model lumps them into a single global region.

I'd suggest cbuffer or uniform instead of constant to avoid confusion with const, constexpr and consteval. Or just skipping it entirely in the first draft as SSBOs have same perf as UBOs and on some architectures are literally the same thing.

Also not sure that image makes any sense as a scope as you should know that the reference is a reference to an image by just looking at T itself, and forbid any memory region qualifier other than global.

P.S. No clue about object and ray data memory regions, but IMHO they make no sense in light of SPIR-V only having 4 regions and actual GPUs only caring about 4 or 3 (with UMA).