Has anyone experimented with direct transfer using nvidias GPUDirect?
It would be great to have a direct HDF5 to GPU pipeline.
2 Likes
some talk about it on gpu channel in slack. heards it’s only for windows and design for games for now
I found that thread, there are just a bunch of sources saying its a lot more than that:
Hi, reviving this thread since I’m also interested in this. Some of my ML workflows with big datasets are I/O limited. I found this presentation https://www.hdfgroup.org/wp-content/uploads/2020/10/GPU_Direct_IO_with_HDF5-_John_Ravi.pdf but couldn’t find an implementation.
The relevant CUDA.jl issue is Support for GPUDirectStorage · Issue #1630 · JuliaGPU/CUDA.jl · GitHub. Seems like it’s PRs welcome.