-
Notifications
You must be signed in to change notification settings - Fork 306
Skip IPC mempool tests on WSL #1045
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 33 commits
65b944e
67d5f45
47a8d89
7f7be24
d88d627
aab97af
fe47883
46cd33c
4366efe
2b49a77
57ead02
0a9be40
1da2c82
f7deec9
57fc851
56a2381
5e21482
c95d29b
7edd190
8ce40b8
2168cd1
b40487d
d66d4da
1f9bb51
4b9c417
8d291a6
55546cb
8664327
7af149b
5977cab
90590ec
034125f
75c4d25
cd6b714
baac405
d8b0a46
bbe82c8
7726e05
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we please not create yet another folder at the repo root? If we are hacking
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's avoid path hacking. It really is possible to use the tools as-is, without resorting to hacks on the first attempt. Just move the WSL detection into
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think path hacking is necessary and can be justified in this case. We are defining common test utilities that can be reused in both cuda.core and cuda.bindings (and potentially cuda.pathfinder) test suites. Without hacking the path we'd end up copying/pasting the same code around.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In that case, moving to
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, that exactly why this folder exists. Its holding common checks so we don't have to copy/paste variants of them in every module.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Complete agreement for production code. In pytest code, I've seen this many times. Unfortunately pytest is missing a facility for making this nicer. I don't think duplicating our test code is a healthy response. Of course, one little thing doesn't matter, but having such a hindrance tends to push tired or rushed developers (and who isn't one) into non-ideal compromises, although they may not be as obvious as a
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think the helpers are written by only calling I do appreciate having a common module we can reuse. In the future I'd like to move some of the NVRTC/NVVM/PTX code snippets to this new module so that we only hard-code them in one place and reuse as much as possible.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree having common helpers is ideal, but we shouldn't need to hack
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Additionally, some of the functionality being built here is already being duplicated here: NVIDIA/numba-cuda#488 So maybe we do want to ship some of these as something like testing utilities? 😅
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Having a package with our test utils sounds great to me. Would you want to publish that to PyPI, too? If not, I'm not sure how easy or tricky it'll be for numba-cuda to integrate the package into the dev workflow, but it still seems like a good goal. @cpcloud have you done something like that before?
rparolin marked this conversation as resolved.
Outdated
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import functools | ||
| import os | ||
| from contextlib import suppress | ||
| from typing import Union | ||
|
|
||
| import pytest | ||
| from cuda.core.experimental._utils.cuda_utils import handle_return | ||
|
|
||
|
|
||
| def _detect_wsl() -> bool: | ||
| data = "" | ||
| with suppress(Exception), open("/proc/sys/kernel/osrelease") as f: | ||
| data = f.read().lower() | ||
| if "microsoft" in data or "wsl" in data: | ||
| return True | ||
| return any(os.environ.get(k) for k in ("WSL_DISTRO_NAME", "WSL_INTEROP")) | ||
|
|
||
|
|
||
| IS_WSL: bool = _detect_wsl() | ||
|
|
||
|
|
||
|
rparolin marked this conversation as resolved.
Outdated
|
||
| @functools.cache | ||
| def supports_ipc_mempool(device_id: Union[int, object]) -> bool: | ||
| """Return True if mempool IPC via POSIX file descriptor is supported. | ||
|
|
||
| Uses cuDeviceGetAttribute(CU_DEVICE_ATTRIBUTE_MEMPOOL_SUPPORTED_HANDLE_TYPES) | ||
| to check for CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR support. Does not | ||
| require an active CUDA context. | ||
| """ | ||
| if _detect_wsl(): | ||
| return False | ||
|
|
||
| try: | ||
| # Lazy import to avoid hard dependency when not running GPU tests | ||
| try: | ||
| from cuda.bindings import driver # type: ignore | ||
| except Exception: | ||
| from cuda import cuda as driver # type: ignore | ||
|
|
||
| # Initialize CUDA | ||
| handle_return(driver.cuInit(0)) | ||
|
|
||
| # Resolve device id from int or Device-like object | ||
| dev_id = int(getattr(device_id, "device_id", device_id)) | ||
|
|
||
| # Query supported mempool handle types bitmask | ||
| attr = driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_MEMPOOL_SUPPORTED_HANDLE_TYPES | ||
| mask = handle_return(driver.cuDeviceGetAttribute(attr, dev_id)) | ||
|
|
||
| # Check POSIX FD handle type support via bitmask | ||
| posix_fd = driver.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR | ||
| return (int(mask) & int(posix_fd)) != 0 | ||
| except Exception: | ||
| return False | ||
|
|
||
|
|
||
| __all__ = [ | ||
| "IS_WSL", | ||
| "supports_ipc_mempool", | ||
| ] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import functools | ||
| import os | ||
| from contextlib import suppress | ||
| from typing import Union | ||
|
|
||
| from cuda.core.experimental._utils.cuda_utils import handle_return | ||
|
|
||
|
|
||
| def _detect_wsl() -> bool: | ||
| data = "" | ||
| with suppress(Exception), open("/proc/sys/kernel/osrelease") as f: | ||
| data = f.read().lower() | ||
| if "microsoft" in data or "wsl" in data: | ||
| return True | ||
| return any(os.environ.get(k) for k in ("WSL_DISTRO_NAME", "WSL_INTEROP")) | ||
|
|
||
|
|
||
| IS_WSL: bool = _detect_wsl() | ||
|
|
||
|
|
||
| @functools.cache | ||
| def supports_ipc_mempool(device_id: Union[int, object]) -> bool: | ||
| """Return True if mempool IPC via POSIX file descriptor is supported. | ||
|
|
||
| Uses cuDeviceGetAttribute(CU_DEVICE_ATTRIBUTE_MEMPOOL_SUPPORTED_HANDLE_TYPES) | ||
| to check for CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR support. Does not | ||
| require an active CUDA context. | ||
| """ | ||
| if _detect_wsl(): | ||
| return False | ||
|
|
||
| try: | ||
| # Lazy import to avoid hard dependency when not running GPU tests | ||
| try: | ||
| from cuda.bindings import driver # type: ignore | ||
| except Exception: | ||
| from cuda import cuda as driver # type: ignore | ||
|
|
||
| # Initialize CUDA | ||
| handle_return(driver.cuInit(0)) | ||
|
|
||
| # Resolve device id from int or Device-like object | ||
| dev_id = int(getattr(device_id, "device_id", device_id)) | ||
|
|
||
| # Query supported mempool handle types bitmask | ||
| attr = driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_MEMPOOL_SUPPORTED_HANDLE_TYPES | ||
| mask = handle_return(driver.cuDeviceGetAttribute(attr, dev_id)) | ||
|
|
||
| # Check POSIX FD handle type support via bitmask | ||
| posix_fd = driver.CUmemAllocationHandleType.CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR | ||
| return (int(mask) & int(posix_fd)) != 0 | ||
| except Exception: | ||
| return False | ||
|
|
||
|
|
||
| __all__ = [ | ||
| "IS_WSL", | ||
| "supports_ipc_mempool", | ||
| ] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| [build-system] | ||
| requires = ["setuptools>=77.0.0"] | ||
| build-backend = "setuptools.build_meta" | ||
|
|
||
| [project] | ||
| name = "cuda-python-test-helpers" | ||
| version = "0.1.0" | ||
| description = "Shared test helpers for CUDA Python projects" | ||
| readme = {file = "README.md", content-type = "text/markdown"} | ||
| authors = [{ name = "NVIDIA Corporation" }] | ||
| license = "Apache-2.0" | ||
| requires-python = ">=3.9" | ||
| classifiers = [ | ||
| "Programming Language :: Python :: 3 :: Only", | ||
| "Operating System :: POSIX :: Linux", | ||
| ] | ||
|
|
||
| [tool.setuptools] | ||
| packages = ["cuda_python_test_helpers"] | ||
|
|
||
| [project.urls] | ||
| repository = "https://github.com/NVIDIA/cuda-python" |
Uh oh!
There was an error while loading. Please reload this page.