Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add macros KVIKIO_EXPECT and KVIKIO_FAIL to improve exception handling #653

Merged

Conversation

kingcrimsontianyu
Copy link
Contributor

@kingcrimsontianyu kingcrimsontianyu commented Mar 4, 2025

This PR follows cuDF's enhanced exception handling approach and adds the macros KVIKIO_EXPECT and KVIKIO_FAIL to KvikIO.

The benefit is that the file name and line number where the exception is thrown are automatically included in the exception message. This also leads to cleaner code.

This PR does not add the stacktrace capability as in cuDF's exception handling. This feature should be added in the future.

Closes #654

@kingcrimsontianyu kingcrimsontianyu added improvement Improves an existing functionality non-breaking Introduces a non-breaking change c++ Affects the C++ API of KvikIO labels Mar 4, 2025
Copy link

copy-pr-bot bot commented Mar 4, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

2 similar comments
@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

@kingcrimsontianyu kingcrimsontianyu force-pushed the add-expect-and-fail-macros branch from 538890e to c841e8b Compare March 6, 2025 19:19
@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

@kingcrimsontianyu
Copy link
Contributor Author

/ok to test

@kingcrimsontianyu
Copy link
Contributor Author

Doxygen documentation on macros has been added in this PR:
image

@kingcrimsontianyu kingcrimsontianyu marked this pull request as ready for review March 6, 2025 21:17
@kingcrimsontianyu kingcrimsontianyu requested review from a team as code owners March 6, 2025 21:17
@kingcrimsontianyu
Copy link
Contributor Author

kingcrimsontianyu commented Mar 7, 2025

@madsbk @bdice I'm wondering if you have any thought about this CI failure: https://github.com/rapidsai/kvikio/actions/runs/13708284889/job/38339415663?pr=653

Additional observations

KvikIO_CUDA_SUPPORT=ON (default)

The tests in test_cufile_driver.py all passed on my PC with the following setup:

  • build-all -DBUILD_TESTS=ON -j 16
  • dev container CUDA 11.8-pip or 12.8-pip

KvikIO_CUDA_SUPPORT=OFF

However, if I add -DKvikIO_CUDA_SUPPORT=OFF to the build-all command, the build will fail under CUDA 11.8-pip or 12.8-pip due to an nvcomp issue:

[1/27] Transpiling cufile_driver.pyx to cufile_driver.cxx
      [2/27] Transpiling future.pyx to future.cxx
      [3/27] Transpiling defaults.pyx to defaults.cxx
      [4/27] Transpiling libnvcomp_ll.pyx to libnvcomp_ll.cxx
      [5/27] Building CXX object kvikio/_lib/CMakeFiles/libnvcomp_ll.dir/libnvcomp_ll.cxx.o
      FAILED: kvikio/_lib/CMakeFiles/libnvcomp_ll.dir/libnvcomp_ll.cxx.o
      /usr/bin/sccache /usr/bin/g++ -DBS_THREAD_POOL_ENABLE_PAUSE=1 -DKVIKIO_LIBCURL_FOUND -Dlibnvcomp_ll_EXPORTS -isystem /usr/include/python3.10 -isystem /home/coder/kvikio/cpp/include -isystem /home/coder/kvikio/cpp/build/pip/cuda-11.8/release/_deps/bs_thread_pool-src/include -isystem /home/coder/kvikio/python/kvikio/build/pip/cuda-11.8/release/_deps/nvcomp_proprietary_binary-src/include -O3 -DNDEBUG -fPIC -MD -MT kvikio/_lib/CMakeFiles/libnvcomp_ll.dir/libnvcomp_ll.cxx.o -MF kvikio/_lib/CMakeFiles/libnvcomp_ll.dir/libnvcomp_ll.cxx.o.d -o kvikio/_lib/CMakeFiles/libnvcomp_ll.dir/libnvcomp_ll.cxx.o -c /home/coder/kvikio/python/kvikio/build/pip/cuda-11.8/release/kvikio/_lib/libnvcomp_ll.cxx
      /home/coder/kvikio/python/kvikio/build/pip/cuda-11.8/release/kvikio/_lib/libnvcomp_ll.cxx:1053:10: fatal error: cuda_runtime.h: No such file or directory
       1053 | #include "cuda_runtime.h"
            |          ^~~~~~~~~~~~~~~~
      compilation terminated.
      [6/27] Transpiling file_handle.pyx to file_handle.cxx
      [7/27] Transpiling arr.pyx to arr.cxx
      [8/27] Transpiling libnvcomp.pyx to libnvcomp.cxx
      [9/27] Transpiling buffer.pyx to buffer.cxx
      [10/27] Transpiling remote_handle.pyx to remote_handle.cxx
      [11/27] Building CXX object kvikio/_lib/CMakeFiles/cufile_driver.dir/cufile_driver.cxx.o
      [12/27] Building CXX object kvikio/_lib/CMakeFiles/future.dir/future.cxx.o
      [13/27] Building CXX object kvikio/_lib/CMakeFiles/defaults.dir/defaults.cxx.o

@@ -52,7 +52,7 @@ using CUstream = struct CUstream_st*;
#define CU_POINTER_ATTRIBUTE_CONTEXT 0
#define CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL 0
#define CU_POINTER_ATTRIBUTE_DEVICE_POINTER 0
#define CU_MEMHOSTREGISTER_PORTABLE 0
#define CU_MEMHOSTALLOC_PORTABLE 0
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fix ensures a successful build with -DKvikIO_CUDA_SUPPORT=OFF option.
Related to the merged PR: #637

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so the cuda_runtime.h issue is all sorted out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the confusion. I updated my comment above. The cuda_runtime.h issue still exists.

#define CU_MEMHOSTALLOC_PORTABLE here is a fix for the C++-part of the build:

# This is successful now
build-kvikio-cpp  -DKvikIO_CUDA_SUPPORT=OFF -DBUILD_TESTS=ON -j 16

build-all -DKvikIO_CUDA_SUPPORT=OFF -DBUILD_TESTS=ON -j 16 still fails with the cuda_runtime.h issue.

}
return {};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This return {} and ditto below is to avoid a compile warning (error):

no return statement in function returning...

Copy link
Member

@madsbk madsbk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks @kingcrimsontianyu

@kingcrimsontianyu
Copy link
Contributor Author

kingcrimsontianyu commented Mar 7, 2025

The CI issue on CUDA 11.8 is now fixed.

According to test_wheel.sh

# If running CUDA 11.8 on arm64, we skip tests marked "cufile" since
# cuFile didn't support arm until 12.4

So the failed test should have been decorated by the marker @pytest.mark.cufile.

PS: The local build fail with build-all -DKvikIO_CUDA_SUPPORT=OFF -DBUILD_TESTS=ON -j 16 needs further investigation.
PPS: The local build fail also occurs to branch-25.04.

@kingcrimsontianyu
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit ab867bd into rapidsai:branch-25.04 Mar 7, 2025
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Affects the C++ API of KvikIO improvement Improves an existing functionality non-breaking Introduces a non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add utility macros that automatically attach line number and file name to the exception messages
3 participants