Compiler infinite loop issue

[hannes-vernooij]

I seem to have found an infinite loop during shader compilation and I was only able to reproduce this infinite loop when the following three conditions were met:

• When the handle to index in the ByteAddressBufferArray was returned from a struct
• When the ByteAddressBuffer is accessed in the function (it doesn't need to be used)
• When the function returns an array of values, I tested it with float2,float3 and int2.

When any of these steps are missing it will not result in an infinite loop. I have attached the source code of the shader below.

struct RenderResourceHandle {
    uint handle;

    uint readIndex() { return this.handle; }
};

ByteAddressBuffer g_byteAddressBuffer[] : register(t0, space3);

struct Test{
    RenderResourceHandle h;

    float3 infLoop()[2] {
        ByteAddressBuffer b = g_byteAddressBuffer[this.h.readIndex()];
        float3 v[2] = {
            0.xxx,0.xxx,
        };
        return v;
    }
};

[numthreads(8, 8, 1)] void main(uint2 dispatchIdx
                                : SV_DispatchThreadID, uint3 pxInTile
                                : SV_GroupThreadID) {

    RenderResourceHandle resourceHandle;
    resourceHandle.handle = 0;

     Test t;
     t.h = resourceHandle;
     float3 w[2] = t.infLoop();
}

I have tried to compile this shader with the latest version from master.
shader playground link: http://shader-playground.timjones.io/8db5b73aef2c0f7eb76810f4afcdb49a

As workaround I wrapped the data in a struct and returned the struct instead.

[damyanp]

Setting to Dormant as there's a workaround for this.