[Backend] Add a shared layout for padding

[antiagainst]

This commit adds a new shared memory layout for padding.
Padding cannot be represented with linear layout, so we need
to define it at a parallel level with the swizzled shared layout.

Intermediate lowering steps don't need to concern about
the exact padding actually; only when we are making the
1-D physical allocation and creating pointers for indexing
we then need to factor in the padding. It means we can
leverage existing linear layout facilities for reasoning the
element mapping.

[ThomasRaoux]

can we move those builder to helper functions instead? We have that for the other shared layouts and it's super annoying (and we have been meaning to clean it up)

[ThomasRaoux]

for context it's annoying because builders are not expected to handle logic to decide on the layout

[antiagainst]

Yeah good point. Done with a66fa0d.

[ThomasRaoux]

sorry my comment should have been more clear, what I meant is that adding a builder that take dotKWidth and other such parameters is confusing as it contains logic on how to avoid bank conflicts based on the reg layout. Those make it very hard to read the code as builder are usually expected to be simple and not contain logic related to bank conflicts or other such considerations. My suggestion was to make this an explicit function that would call into the default builder.

[antiagainst]

Yeah makes sense. I don't need this builder right away. (It was added because I also enabled pipeliner on AMD side to emit this layout just to try out correctness with b622870 but I reverted it to make this pull request focusing on core changes.) So I just dropped it with 25221f4 and can do it properly later when needed.

[Jokeren]

In which cases should we still use PaddedSharedEncoding but not the swizzled layout?

[antiagainst]

In which cases should we still use PaddedSharedEncoding but not the swizzled layout?

One example is for CDNA4, we have global load direct to LDS (i.e. shared memory) instruction. However, that instruction does not support scattered write when writing to LDS--the whole warp uses one m0 register to describe the base offset (see 3.7. M0 Memory Descriptor in https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf) and the whole warp can only write consecutive banks. So we cannot do swizzled write like the normal way. Instead we need to "reverse" swizzle the global pointers when load from global memory. Those "reverse" swizzle introduces overhead because we need to do warp shuffle with ds_permute to exchange global pointers etc. which is a source of performance issues. Using padded layout could avoid the cost there.

[lezcano]

Amazing! Just a minor point (and see Thomas' point) but otherwise looks great!

[lezcano]

It might be better to do it inside getAllocationShapePerCTA

[antiagainst]

I actually was trying to do that. Then I realized it's not that compatible--getAllocationShapePerCTA assumes the original ranked shape, while after factoring in padding fundamentally we only have a 1-D size. Also getAllocationShapePerCTA is used quite a few places that assumes original rank. So ends up I'm doing it this way given only when doing allocation or the final pointer indexing we care about the exact physical memory.

[lezcano]

actually makes sense.

[lezcano]

LGTM but let's wait for Thomas' ok

[lezcano]

actually makes sense.

[ThomasRaoux]

LGTM