Repository navigation
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure about that comment, see #5529 (comment)
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, @marbre since the submodule bumps (rocm-libraries) started running the gfx950 tests recently, removing
hip-pythonwill silently skip 12 GPU tests.Example tests that will be skipped but required on gfx950, passing now: https://github.com/ROCm/TheRock/actions/runs/29815512767/job/88684173821#step:14:4809
Do you have guidance on the right mechanism to add
hip-pythonin TheRock workflow?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I have not. There isn't a
hip-pythonrelease for the current ROCm 7.14 production release. Maybe @domcharrier or @psamfass can give guidance.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marbre @archana-ramalingam
What APIs are called? If this subset is stable between ROCm 7.2.2 and ROCm LATEST then it is very likely safe to use
hip-python~=7.2.2. All libraries and function symbols are loaded lazily. There would only be an issue if you use newly added HIP APIs or a signature had changed (type/order/number of args/return values).Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TensileLite gfx950 GPU subtile tests use hip-python for HIP runtime calls to assemble and execute generated subtile kernels —
hipModuleLoad,hipModuleLaunchKernel,hipMalloc,hipMemcpy,hipDeviceSynchronize, etc.hip-python 7.2.2usesdlopen("libamdhip64.so")at runtime — so it calls into whichever ROCm is installed (10.x in TheRock). It works today because the APIs we use haven't changed, but newer ROCm versions could add/change struct fields that the 7.2 wrapper might not know about.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should move this requirement to be local to only the tests that need it.
The unconditional download of 64MB hip-python packages is actually contributing pretty significantly to test time on some gfx942 runners with slow network, and pip/uv caching isn't enabled (? see also #3674).
For example: https://github.com/ROCm/TheRock/actions/runs/32165005832/job/96055002747?pr=7449#step:8:160
Aggregate statistics from that run show all 51 test jobs downloaded the 64MB wheel but none of them used it as TensileLite is the only user with relevant test imports and the "quick" category ran with
not gpu and not skip-gfx94X. Timing was pretty variable due to differing network speeds:Out of curiosity, why is hip-python so large, at 64MB compressed? Looking at https://pypi.org/project/hip-python/7.2.2.562.43/#files, I see:
hipblas.cpython-314-x86_64-linux-gnu.sohip.cpython-314-x86_64-linux-gnu.sohipsparse.cpython-314-x86_64-linux-gnu.sohipsolver.cpython-314-x86_64-linux-gnu.sorccl.cpython-314-x86_64-linux-gnu.sochipblas.cpython-314-x86_64-linux-gnu.sohiprand.cpython-314-x86_64-linux-gnu.soThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
correction: it might be used at "quick", but only for tensilelite and no other components
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While we're at it, we can move at least sccache and numpy to component-specific test requirements too:
TheRock/requirements-test.txt
Lines 16 to 29 in 04a2474
botocore from boto3 is what we use to download artifacts, so I don't think we can easily slice that out
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For " # TODO: Keep in sync with rocm-libraries/projects/hipblaslt/tensilelite/requirements.txt ", what if we copy that requirements.txt into the test artifact for tensilelite?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filed #7491 to follow-up.