What SPIR-V dialect does Unified Runtime consume?

[FranklandJack]

The urProgramCreate API seems to mandate that ur_module_handle_ts are SPIR-V modules:

Create Program from input SPIR-V modules. 

However we don't specify which SPIR-V dialect we can consume. It should probably either reference the OpenCL environment spec or a new Unified Runtime dialect.

[kbenzie]

For adapters which consume SPIR-V, it is Kernel capability SPIR-V as defined for OpenCL. Not all adapters will consume SPIR-V though.

[aarongreig]

so we do have the device IL_VERSION property, maybe it would be enough to spec that out a bit more and leave the specifics to individual plugins as an implementation detail?

[alycm]

Really any SPIR-V consumer should specify an environment for how SPIR-V is interpreted, e.g. as with OpenCL, Vulkan, or Level Zero. Often the SPIR-V specification will defer definitions of behaviour to a "client API", so without an environment specification to pair with the SPIR-V specification is incomplete.

For unified runtime it may be sufficient to just state that either the OpenCL or Level Zero one applies. But is that true? Level Zero and OpenCL SPIR-V requirements are certainly broadly similar at a glance, but are they exactly the same?

That is an argument for letting the adapter specify the SPIR-V environment, but could that mean that N SPIR-V using adapters could need to pack N SPIR-V variants in per binary? N is currently 2, but going to at least 3 is plausible. It could be that Unified Runtime has a single SPIR-V environment for the front-end, and if an adapter needs very different SPIR-V in the back-end then it needs to translate it. Like how clvk can consume kernel SPIR-V from OpenCL but then generates shader SPIR-V for Vulkan.

[aarongreig]

My knee-jerk is that "let the adapter specify" is what we should go with for now, if only because adding "build SPIR-V translation infrastructure" to our objectives for this release cycle doesn't seem wise at first blush. Clearly there is more thinking to do though. I'll look into exactly how similar the zer and cl env specs are, if the differences are truly minimal (or even non-existent?) the unified SPIR-V front end option wont sound so ambitious for the short term. If the differences are purely capability and/or extension based maybe all we need is an extension struct that lets you query a device for that sort of thing.

[kbenzie]

This leads me to thinking we should have a place for adapter specific documentation to reside, potentially as part of the spec?

[aarongreig]

Yeah at the very least it will probably be useful if we document "this adapter was written to use version x.x of the underlying api and can take advantage of extensions x y and z" somewhere

[aarongreig]

The good news is that the differences between the l0 and cl env specs are almost non-existent, to the point where I do think we could declare one of the two to be our spir-v spec and pass the same spir-v binary to both backends without any translation needed. The only material difference between the two I could find is that l0 explicitly allows fp16 values as kernel arguments and CL doesn't. As tempting as that sounds (no need for multiple binaries, no need for much additional work from us for the time being) I do have my concerns with this:

• Elements of either env spec will not entirely make sense in a ur context as they are tightly coupled to their respective APIs. It's self explanatory but there's nothing actually saying you need to check a UR device's ATOMIC_64 property before using 64 bit atomics in your SPIR-V, rather than the l0 device property or cl extension either of those specs tell you to look at. I'm not sure how comfortable we should be with "basically makes sense but not well defined".
• I still worry that committing to this route means creating some serious work down the line if we do end up with a vulkan plugin for instance. There's a bunch of prior art out there for that kind of spir-v transformation so I don't think it's a difficult problem per se, but it would be overhead and another part of the layer that could go wrong.

That said the obvious alternatives of copying and adapting one of the existing specs, or writing our own small (as small as possible anyway) spec are not very attractive to me either.

[kbenzie]

That said the obvious alternatives of copying and adapting one of the existing specs, or writing our own small (as small as possible anyway) spec are not very attractive to me either.

Yeah, I'd like to avoid that if we can!

[aarongreig]

^ might need to mess with ur_device_binary_t

[alycm]

Discussion call 2023-05-17:

• Should UR have an opinion on SPIR-V environment or is it per-adapter?
• One response: Probably per adapter, but can a high-level language query what SPIR-V environment is in use?
• One answer: Can query which adapter is in use, and then can document environment.
• No programmatic way to query SPIR-V environment, version of SPIR-V supported.
• We could mandate required SPIR-V features. Not currently the plan, but is pragmatically true, likely needs addressed.