feat: add support for cache-hinted load and store operations

[kaeun97]

Follow up of #51 requested by @gmarkall.

• Add documentation.
• Test cases for erroneous arguments. It would be good to check for accidental use on shared or local arrays, but this may not be easy to do.
• Add additional validations as described in comments in ld_cache_operator and st_cache_operator.
• Add test for bitwidth validation
• Refactor the implementation - the load and store implementations contain a lot of common code.
• Decide on whether to support complex, and work out why it presently doesn't work.
• Support CPointer()
• Test for supportCPointer()

Updated items from feedback.

• Small nits on the documentation
• Small nits on the tests
• Add tests with 2D arrays
• Fix the simulator, either by stub implementations and test skips

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[gmarkall]

Some relevant discussion also in https://github.com/NVIDIA/numba-cuda/pull/51/files#r2513450704

[gmarkall]

Decide on whether to support complex, and work out why it presently doesn't work.

I should mention, I think it's fine if we don't support complex types for this PR, and could add it in a follow-up PR.

[gmarkall]

A quick follow up on a couple of items:

Add documentation.

Do you need any assistance here? I know that adding documentation is often a bit hard to find the right place and wording, so I'm happy to add something here if you want.

Test cases for erroneous arguments. It would be good to check for accidental use on shared or local arrays, but this may not be easy to do.

I see you have test cases for various user errors already. I think checking for accidental use on shared or local arrays is probably hard / impossible without something like #236, so maybe this item can be considered already done?

[gmarkall]

/ok to test

[copy-pr-bot]

/ok to test

@gmarkall, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

[gmarkall]

/ok to test 393554a

[kaeun97]

Thanks for the follow up! @gmarkall

I see you have test cases for various user errors already. I think checking for accidental use on shared or local arrays is probably hard / impossible without something like #236, so maybe this item can be considered already done?

I agree. If needed, I can follow up on that later (ideally after #236 gets merged)

Do you need any assistance here? I know that adding documentation is often a bit hard to find the right place and wording, so I'm happy to add something here if you want.

I've added a rough draft that needs refinement (in terms of the content, where to put it, etc). Would you be able to take a quick look? Would be happy to iterate on that. If needed, we can hop on a call or feel free to directly make changes to this branch.

[gmarkall]

Many thanks for all the efforts so far - this is looking very good!

I have a few comments on the diff - I know that some of the code I've commented on in the tests comes from my original PR, so I apologise for the review requesting changes on it now.

I think the only gap in testing is a test with arrays with more than one dimension - at the moment the _get_element_pointer function looks correct, but I do think we could do with the else branch of it getting exercised by a test.

The CI reveals that the simulator also needs some stubs for these additions, so that the test code can be imported on the simulator. These could just look like:

ldca = None
...

in somewhere like numba_cuda/numba/cuda/simulator/api.py. Then the tests could be skipped on the simulator by adding a @skip_on_cudasim decorator to the test case.

Alternatively, you could implement these operations in the simulator (they needn't simulate the cache behaviour, they only need to behave as they would on the device without respect to the performance difference the cache policy might make), but I would not bother as the simulator is already quite limited and on the road to being superseded by better availability of GPUs and better on-device debugging support.

So to summarise, I think the remaining items to finish up are:

• Small nits on the documentation
• Small nits on the tests
• Add tests with 2D arrays
• Fix the simulator, either by stub implementations and test skips (likely preferable), or by implementing the operations in the simulator (more adventurous, not necessarily more interesting, up to your choice 🙂).

Many thanks again!

[kaeun97]

@gmarkall Thank you for the thorough feedback. I believe I have addressed all items you have mentioned. Would be great if you can re-visit the PR!

[gmarkall]

/ok to test

[copy-pr-bot]

/ok to test

@gmarkall, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

[gmarkall]

/ok to test fe9e8ac

[gmarkall]

This is looking great - many thanks!

[gmarkall]

I'm just waiting to merge this until https://github.com/NVIDIA/numba-cuda/actions/runs/19534738630 completes, as it's a run of a new CI workflow for main - I'd like to make sure we have one complete successful run of that before continuing with merging other PRs.

[gmarkall]

Many thanks @kaeun97!