OpenGL interoperability

[cajoek]

🚀 The feature

Zero-copy transfer of data between PyTorch and OpenGL on GPU by including "OpenGL interoperability" from CUDA in torchvision.

Motivation, pitch

I am working on a real-time machine learning graphics project which uses OpenGL both as an intermediate processing step in the model and to visualize the output. Right now transfer of data between PyTorch and OpenGL is a problem for both training and inference.
Without any additional packages i can copy data from PyTorch CUDA to CPU and then back to OpenGL on GPU, this is very simple but slow.
I can instead use some cuda bindings for python and a separate CUDA Toolkit installation to avoid the data transfer but this is quite complex and there are many competing ways and tools for doing this which makes it hard to navigate.

Alternatives

No response

Additional context

The 2 main ways I have been using OpenGL from python are with the packages moderngl and PyOpenGL.