python3Packages.gpuctypes: init at unstable-2023-11-26 by MatthewCroughan · Pull Request #278187 · NixOS/nixpkgs

MatthewCroughan · 2024-01-02T04:20:32Z

Description of changes

~~Awaiting tinygrad/gpuctypes#18 to add a license.~~

Things done

Add a 👍 reaction to pull requests you find important.

pkgs/development/python-modules/gpuctypes/default.nix

MatthewCroughan · 2024-01-12T13:21:47Z

After some testing, these paths are still required it seems, the final application that's using this library just needs to set HIP_PATH, so this should be in a mergeable state now.

wozeparrot · 2024-01-12T16:00:43Z

LGTM, have tested it with HIP and it does work, just that HIP_PATH needs to be set.

GaetanLepage · 2024-01-13T09:22:54Z

Result of nixpkgs-review pr 278187 run on x86_64-linux 1

4 packages built:

python311Packages.gpuctypes
python311Packages.gpuctypes.dist
python312Packages.gpuctypes
python312Packages.gpuctypes.dist

natsukium · 2024-01-13T09:44:36Z

pkgs/development/python-modules/gpuctypes/default.nix

+}:
+buildPythonPackage {
+  pname = "gpuctypes";
+  version = "unstable-2023-11-26";


Can we adopt 0.2.0?
There is nothing new in the diff with this commit.
tinygrad/gpuctypes@c969506
tinygrad/gpuctypes@c969506...3d402bb

natsukium · 2024-01-13T09:45:33Z

pkgs/development/python-modules/gpuctypes/default.nix

+buildPythonPackage {
+  pname = "gpuctypes";
+  version = "unstable-2023-11-26";
+


Please add pyproject = true to avoid legacy build.

I did ^ and I also needed to include

nativeBuildInputs = [ setuptools ];

@natsukium There is no pyproject.toml in the root of the repo, but only a setup.py. I'm not a python packaging expert, can you explain to me why the function argument is named pyproject and what the effect of enabling it is, when there is no pyproject.toml in the root of the repo?

As I said before, please read the document first.
#236913 (comment)

natsukium · 2024-01-13T09:53:04Z

pkgs/development/python-modules/gpuctypes/default.nix

+  };
+
+  postPatch = ''
+    # patch correct path to opencl


This comment seems redundant since this is a common operation.
I would like to avoid inline comments to prevent unintended changes to the derivation due to comment modification.

natsukium · 2024-01-13T09:53:17Z

pkgs/development/python-modules/gpuctypes/default.nix

+    # patch correct path to opencl
+    substituteInPlace gpuctypes/opencl.py --replace "ctypes.util.find_library('OpenCL')" "'${ocl-icd}/lib/libOpenCL.so'"
+
+    # patch correct path to hip


natsukium · 2024-01-13T10:01:32Z

pkgs/development/python-modules/gpuctypes/default.nix

+
+  postPatch = ''
+    # patch correct path to opencl
+    substituteInPlace gpuctypes/opencl.py --replace "ctypes.util.find_library('OpenCL')" "'${ocl-icd}/lib/libOpenCL.so'"


opencl should also be available in darwin, we need to use darwin.apple_sdk.frameworks.OpenCL.

We can use stdenv.hostPlatform.extensions.sharedLibrary to get the extension of the shared object if necessary.

natsukium · 2024-01-13T10:03:17Z

pkgs/development/python-modules/gpuctypes/default.nix

+  '';
+
+  pythonImportsCheck = [ "gpuctypes" ];
+


Can we enable these tests with pytestCheckHook?
https://github.com/tinygrad/gpuctypes/tree/master/test

SomeoneSerge · 2024-01-13T10:14:27Z

pkgs/development/python-modules/gpuctypes/default.nix

Can I bother you with adding optional cuda support? 🙃

I think you could just patch into https://github.com/tinygrad/gpuctypes/blob/77b94adaf8e27fa5ef7caa0bdd949bd528576d4b/gpuctypes/cuda.py#L146-L148

_libraries["libnvrtc.so"] = "${cudaPackages.cuda_nvrtc}/lib/libnvrtc.so"

and libcuda.so is "${addDriverRunpath.driverLink}/lib/libcuda.so" but you need to first test if it os.path.exists in case we're not running under NixOS

Also would be nice to display a nixpkgs-specific warning about cudaSupport in the cudaSupport = false branch, because people might expect gpuctypes to just work but since they rely on the broken-by-design ctypes and find_library they can't

I'd keep the CDLL bit but make it if "libnvrtc.so" not in ...

Damn I'm not even sure what to suggest upstream. The thing is, CDLL("libcuda.so") is much less broken than CDLL(find_library("libcuda.so"))

Adding this to a hacked up nixified version of tinygrad allows me to run on my gpu

(lib.optionalString cudaSupport '' substituteInPlace gpuctypes/cuda.py --replace "ctypes.util.find_library('cuda')" "\"${addDriverRunpath.driverLink}/lib/libcuda.so\"" substituteInPlace gpuctypes/cuda.py --replace "ctypes.util.find_library('nvrtc')" "\"${cudaPackages.cuda_nvrtc}/lib/libnvrtc.so\"" '');

I haven't tested it too hard, but the following works as expected

CUDA=1 DEBUG=4 python3 -c "import tinygrad; print(tinygrad.Tensor([1,3,-5]).dot(tinygrad.Tensor([4,-2,-1])).realize().numpy())"

*** CUDA 1 copy 3, CUDA <- CPU arg 2 mem 0.00 GB tm 86.45us/ 0.09ms ( 0.00 GFLOPS, 0.00 GB/s) *** CUDA 2 copy 3, CUDA <- CPU arg 2 mem 0.00 GB tm 26.82us/ 0.11ms ( 0.00 GFLOPS, 0.00 GB/s) 0 ━┳ STORE MemBuffer(idx=0, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(1,), strides=(0,), offset=0, mask=None, contiguous=True),))) 1 ┗━┳ SUM (1,) 2 ┗━┳ MUL 3 ┣━━ LOAD MemBuffer(idx=1, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),))) 4 ┗━━ LOAD MemBuffer(idx=2, dtype=dtypes.int, st=ShapeTracker(views=(View(shape=(3,), strides=(1,), offset=0, mask=None, contiguous=True),))) #define INFINITY (__int_as_float(0x7f800000)) #define NAN (__int_as_float(0x7fffffff)) extern "C" __global__ void r_3(int* data0, const int* data1, const int* data2) { int acc0 = 0; int val0 = data1[0]; int val1 = data1[1]; int val2 = data1[2]; int val3 = data2[0]; int val4 = data2[1]; int val5 = data2[2]; float alu0 = (((float)(val2)*(float)(val5))+(((float)(val1)*(float)(val4))+(((float)(val0)*(float)(val3))+(float)(acc0)))); acc0 = (int)(alu0); data0[0] = acc0; } *** CUDA 3 r_3 arg 3 mem 0.00 GB tm 17.60us/ 0.13ms ( 0.00 GFLOPS, 0.00 GB/s) *** CPU 4 copy 1, CPU <- CUDA arg 2 mem 0.00 GB tm 36.58us/ 0.17ms ( 0.00 GFLOPS, 0.00 GB/s) 3 avg: 0.00 GFLOPS 0.00 GB/s total: 4 kernels 0.00 GOPS 0.00 GB 0.17 ms

jeremygrant · 2024-01-13T22:52:58Z

pkgs/development/python-modules/gpuctypes/default.nix

+    substituteInPlace gpuctypes/opencl.py --replace "ctypes.util.find_library('OpenCL')" "'${ocl-icd}/lib/libOpenCL.so'"
+
+    # patch correct path to hip
+    substituteInPlace gpuctypes/hip.py --replace "/opt/rocm/lib/libamdhip64.so" "${rocmPackages.clr}/lib/libamdhip64.so"


Can we wrap these in a rocmSupport check to avoid having to build rocm?

(lib.optionalString rocmSupport '' substituteInPlace gpuctypes/hip.py --replace "/opt/rocm/lib/libamdhip64.so" "${rocmPackages.clr}/lib/libamdhip64.so" substituteInPlace gpuctypes/hip.py --replace "/opt/rocm/lib/libhiprtc.so" "${rocmPackages.clr}/lib/libhiprtc.so" '') +

GaetanLepage · 2024-02-11T02:29:50Z

Followed up in #287914

MatthewCroughan · 2024-02-11T03:04:33Z

Obsoleted by #287914

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 2, 2024

MatthewCroughan mentioned this pull request Jan 2, 2024

Add a Nix flake for development and usage tinygrad/tinygrad#2970

Closed

wozeparrot reviewed Jan 2, 2024

View reviewed changes

pkgs/development/python-modules/gpuctypes/default.nix Outdated Show resolved Hide resolved

MatthewCroughan force-pushed the mc/gpuctypes branch from a151dfe to 57f2d60 Compare January 12, 2024 13:19

python3Packages.gpuctypes: init at unstable-2023-11-26

17fd055

MatthewCroughan force-pushed the mc/gpuctypes branch from 57f2d60 to 17fd055 Compare January 12, 2024 16:06

wozeparrot approved these changes Jan 12, 2024

View reviewed changes

delroth added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jan 12, 2024

ofborg bot requested a review from wozeparrot January 12, 2024 17:19

GaetanLepage requested review from SomeoneSerge and natsukium January 13, 2024 09:20

natsukium requested changes Jan 13, 2024

View reviewed changes

SomeoneSerge reviewed Jan 13, 2024

View reviewed changes

jeremygrant reviewed Jan 13, 2024

View reviewed changes

MatthewCroughan closed this Feb 11, 2024

MatthewCroughan deleted the mc/gpuctypes branch February 11, 2024 03:04

Uh oh!

Conversation

MatthewCroughan commented Jan 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Uh oh!

MatthewCroughan commented Jan 12, 2024

Uh oh!

wozeparrot commented Jan 12, 2024

Uh oh!

GaetanLepage commented Jan 13, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SomeoneSerge Jan 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeremygrant Jan 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GaetanLepage commented Feb 11, 2024

Uh oh!

MatthewCroughan commented Feb 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

MatthewCroughan commented Jan 2, 2024 •

edited

Loading

SomeoneSerge Jan 13, 2024 •

edited

Loading

jeremygrant Jan 13, 2024 •

edited

Loading