[hipblaslt] added problem sizes for PTS#2560
Conversation
…nv3d (#2560) Compiler fix: iree-org/iree#22320 Issue: iree-org/iree#22312
solaslin
force-pushed
the
users/solaslin/adding-pts-problems
branch
3 times, most recently
from
9843dbd to
7d13175
Compare
There was a problem hiding this comment.
Put some requested changes and checking:
- make sure all four sets have
rotating: 512 - Confirm api_method
- be aware of the naming, better not to show any client's information.
- Put your new suite in all() in the correct position
You can test locally:
under ./hipblaslt/, type cmd:
./clients/scripts/performance/hipblaslt-perf --suite matmul_set_3 --samples 1
to see if you can see an output csv.
projects/hipblaslt/clients/scripts/performance/problems/matmul_probset3_bench.yaml
Outdated
Show resolved
Hide resolved
projects/hipblaslt/clients/scripts/performance/problems/matmul_probset3_bench.yaml
Show resolved
Hide resolved
projects/hipblaslt/clients/scripts/performance/problems/matmul_probset3_bench.yaml
Show resolved
Hide resolved
projects/hipblaslt/clients/scripts/performance/problems/matmul_probset3_bench.yaml
Show resolved
Hide resolved
projects/hipblaslt/clients/scripts/performance/problems/matmul_probset3_bench.yaml
Show resolved
Hide resolved
solaslin
force-pushed
the
users/solaslin/adding-pts-problems
branch
2 times, most recently
from
083cdd6 to
a59cfb4
Compare
|
I just randomly pick one problem in RVS (https://github.com/ROCm/ROCmValidationSuite/blob/master/rvs/conf/gfx1200/gst_single.conf#L80-L82) for Navi4, do we already have problems for RVS? Maybe I just didn't find it... |
solaslin
force-pushed
the
users/solaslin/adding-pts-problems
branch
from
a59cfb4 to
a958faa
Compare
solaslin
force-pushed
the
users/solaslin/adding-pts-problems
branch
2 times, most recently
from
a99c0ed to
62f0e94
Compare
testcases are from Math\perf_hipBLAS and Math\hipBLAS
- add rotating: 512 for each block in probset3 - update suites.py to align with corrected problem sizes
1.Select 100 representative test cases from the mad-internal tests and add them to pts 2.Collate the tests in RVS and AGFHC and add them to pts
solaslin
force-pushed
the
users/solaslin/adding-pts-problems
branch
from
502b2c7 to
4e21273
Compare
perfci run on commit 8617098 |
[hipblaslt] added problem sizes for PTS ## Motivation <!-- Explain the purpose of this PR and the goals it aims to achieve. --> Open this PR for team members to add their own focused test sizes. (PTS) ## Technical Details <!-- Explain the changes along with any relevant GitHub links. --> ## Test Plan <!-- Explain any relevant testing done to verify this PR. --> ## Test Result <!-- Briefly summarize test outcomes. --> ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
| --- | ||
| include: ../../../tests/data/hipblaslt_common.yaml | ||
| include: ../../../tests/data/matmul_common.yaml | ||
|
|
||
| Definitions: | ||
|
|
||
| test precision: &hpc_gemm_shapes | ||
| - *single_precision | ||
| - *hpa_bf16_precision | ||
| - *hpa_half_precision | ||
| - *f8_fnuz_precision_dst_f8_fnuz | ||
| - *f8_fnuz_precision_dst_fp16 | ||
|
|
There was a problem hiding this comment.
Message for Mahmood, Wasiq:
include: ../../../tests/data/hipblaslt_common.yaml
include: ../../../tests/data/matmul_common.yaml
The two includes already define almost all commonly used datatypes, transA,B...etc. definitions which are also used for gtest. So adding the two line can avoid copying-pasting.
But you still can add your own definition (usually is a customized lists) such as:
Definitions:
test precision: &hpc_gemm_shapes
- *single_precision
- *hpa_bf16_precision
- *hpa_half_precision
- *f8_fnuz_precision_dst_f8_fnuz
- *f8_fnuz_precision_dst_fp16
the definition from line#8 to line#12 can be found in the two included yamls, however, you can still make a new group for your own test cases.
The rest of the content simply look like any one in matmul_gtest.yaml
Be sure to add rotating: 512 which makes the perf more accurate.
| def matmul_set_4(): | ||
| """gemm benchset 4""" | ||
|
|
||
| problemlist = [Problem(args={"--log_function_name" : "" , "--yaml" : "matmul_probset4_bench.yaml"})] | ||
| yield ProblemSet(benchType="matmul", name="benchset_4", problems=problemlist) | ||
|
|
||
| def matmul_set_5(): | ||
| """gemm benchset 5""" | ||
|
|
||
| problemlist = [Problem(args={"--log_function_name" : "" , "--yaml" : "matmul_probset5_bench.yaml"})] | ||
| yield ProblemSet(benchType="matmul", name="benchset_5", problems=problemlist) | ||
|
|
||
| def matmul_set_6(): | ||
| """gemm benchset 6""" | ||
|
|
||
| problemlist = [Problem(args={"--log_function_name" : "" , "--yaml" : "matmul_probset6_bench.yaml"})] | ||
| yield ProblemSet(benchType="matmul", name="benchset_6", problems=problemlist) | ||
|
|
There was a problem hiding this comment.
Message for Mahmood, Wasiq:
When you added a new bench.yaml, be sure to make it "a suite", you can just follow the coding convention.
A basic suite = a problem set , but a suite can also yield "multiple suites" just like putting building blocks together.
| if "942" in target_arch: | ||
| if "gfx942" in target_arch: | ||
| # Put focused tests set for gfx942 | ||
| yield from matmul_set_1() # this problemset is an initial test example for gfx942 | ||
| yield from matmul_set_2() | ||
| # Can put any other interested set for gfx942 | ||
| elif "950" in target_arch: | ||
| yield from matmul_set_3() | ||
| yield from matmul_set_4() | ||
| yield from matmul_set_5() | ||
| yield from matmul_set_6() | ||
| elif "gfx950" in target_arch: | ||
| # Put focused tests set for gfx950 | ||
| yield from matmul_set_2() | ||
| # Can put any other interested set for gfx950 | ||
| elif "gfx90a" in target_arch: | ||
| # FIXME- please replace the examples with the real interested problems for gfx90a | ||
| yield from matmul_set_3() | ||
| yield from matmul_set_4() | ||
| yield from matmul_set_5() | ||
| elif "gfx110" in target_arch: | ||
| # FIXME- please replace the examples with the real interested problems for gfx110? | ||
| yield from matmul_set_3() | ||
| elif "gfx120" in target_arch: | ||
| # FIXME- please replace the examples with the real interested problems for gfx120? | ||
| yield from matmul_set_3() | ||
|
|
There was a problem hiding this comment.
Message for Mahmood, Wasiq:
For Jenkins PTS, full perf ci job must use --suite all , you can see this all() suite yield multiple basic suites, according to which the architecture running the CI job. (This is done by rocJenkins, you don't have to do it)
In summary, if you have a new problem-set need to be traced in PTS:
- Prepare the yaml file
- Add a in suite.py which yields the ProblemSet from the yaml
- According to your testing gfxXXX, put in suite all(), for the correct architecture.
- In your local test, of course you can only do "--suite " to see if you can run the bench and output a csv file.
5 participants
Motivation
Open this PR for team members to add their own focused test sizes. (PTS)
Technical Details
Test Plan
Test Result
Submission Checklist