Enable plr min

[adityalj]

Motivation

For High rate MFMAs in the newer architecture because of insufficient vgprs we can not unroll the loop. This leads to exposed memory operations reducing efficiency. Here we break A and B into two equal parts and divide loop into 4 subiters by reordering MFMAs such as A0B0 ,A0B1,A1B0,A1B1. This helps us to separate dependencies and hide exposed reads.

Technical Details

Rigid schedule is imposed on the instructions in order to get following desired code.

Prefetch phase : read A0,B0
MainLoop:
sub-iter0: MFMA (A0,B0) read(A1,B1)
sub-iter1: MFMA (A0,B1) write(A,B) Global Read(A,B)
sub-iter2: MFMA (A1,B0)
sub-iter3:MFMA (A1,B1) read(A0,B0)

NoLoadLoop:
sub-iter0: MFMA (A0,B0) read(A1,B1)
sub-iter1: MFMA (A0,B1) write(A,B)
sub-iter2: MFMA (A1,B0)
sub-iter3:MFMA (A1,B1) read(A0,B0)

Test Plan

Added Two .yaml files to test the changes.
Currently supported only for even wave tile sizes>2 f8 and f16 using double LDS buffer.

Test Result

For supported configurations , .s is generated with 4 sub-iters along with prefetch. 3-5% improvement in the performance

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[hcman2]

Could you paste an example of before/after comparison of asm code?

[msujon-AMD]

We need test cases with tf32+plr (maybe, with complex datatypes later on).

[b-shi]

We need test cases with tf32+plr (maybe, with complex datatypes later on).

I updated TF32 tox tests with PLR test cases.

[b-shi]

Could you paste an example of before/after comparison of asm code?

Sure, is there a particular MT case you want to see?

Roughly speaking this change breaks up a single iteration to A = A0 + A1 and B = B0 + B1. Then we have

----- Preloop
LRA0
LRB0
----- Main loop
MFMA A0, B0, LRA1, LRB1
MFMA A1, B0
MFMA A0, B1
MFMA A1, B1, LRA0, LRB0

Previously we had something like:

----- Preloop
----- Main loop
MFMA A, B, LRA, LRB 

but the LRA, LRB are interleaved using some dependency checks between LR outputs and MFMA inputs.

[hcman2]

Could you paste an example of before/after comparison of asm code?

Sure, is there a particular MT case you want to see?

I am OK about the changes. The example is to make it easier to understand what this PR do in the future.

Moreover, do you plan to extend this feature?
such as supporting DU > miK, or tile % 2 !=0?
Some of kernels should get benefit with this feature.

On the other hand, do we have any performance regression with this feature always on?

[b-shi]

Could you paste an example of before/after comparison of asm code?

Sure, is there a particular MT case you want to see?

I am OK about the changes. The example is to make it easier to understand what this PR do in the future.

Moreover, do you plan to extend this feature? such as supporting DU > miK, or tile % 2 !=0? Some of kernels should get benefit with this feature.

On the other hand, do we have any performance regression with this feature always on?

Thanks @hcman2 for taking a look again! No perf regressions were observed these changes.

Yes, the case tile % 2 !=0 should be generalized and also possibly DU > miK. We will need to investigate the perf benefits for these since they would involve quite a bit of changes in the current scheduling code.