implement Tensor into_vec #125
Conversation
|
|
||
| // check NO copy | ||
| assert_eq!(result_vec.capacity(), original_vec_capacity); | ||
| assert!(std::ptr::eq(result_vec.as_ptr(), original_vec_ptr)); |
There was a problem hiding this comment.
@decahedron1 this test seems to fail, which means that we enter in the Err(buf) case
There was a problem hiding this comment.
maybe there's something wrong with the TensorStorage::from_vec which underneath uses Buffer::from_custom_allocation
There was a problem hiding this comment.
UPDATE: i think i fixed ... but I'm missing something around how to tie with the provided allocator. My idea here was that the user can provider custom allocators. eg. CudaAllocator to hold CudaImages to interop easily with other frameworks like ort, vpi, etc to not move memory forth and back
There was a problem hiding this comment.
Hmm, what was causing the test failure? The fix in 1642803 doesn't seem obvious to me, but again, I know nothing about arrow_buffer 🤔
Integration with CUDA buffers would have to go through a different API, TensorRefMut::from_raw. It just needs a pointer and shape, so Kornia's existing APIs suffice. TensorRefMuts can be used zero-copy in session inputs just like a Value, but since the memory is user-managed, it does have a lifetime bound. See ort's cudarc example: https://github.com/pykeio/ort/blob/60f6ecae9064a75a474678998af42ea87067b1b1/examples/cudarc/src/main.rs#L35-L43
There was a problem hiding this comment.
Hmm, what was causing the test failure?
I might need to investigate a bit more the Buffer::from_custom_allocation, maybe is doing copy because later the test checking ptrs is failing.
Integration with CUDA buffers would have to go through a different API,
I'll play with it , thanks for the pointers.
Tensor/TensorStorage--into_vec()to accomplish zero-copies between other libraries e.gort@decahedron1