Do we have some API/traits for out-of core arrays (e.g. on-disk arrays), especially if access to the array data is most efficient in a certain block/chunk size?
Thanks for the link!
In addition to chunking information, I guess it would be useful if we had a standard API to determine where data (in an array, or maybe just data in general) is located (CPU-RAM, disk, GPU-RAM, etc.) so user code can make informed decisions on when and how to convert (e.g. resulting in a GPU to CPU transfer, …).
There’s also https://github.com/meggart/DiskArrays.jl, of course, in a more specific context.
Yes I was about to mention DiskArrays as well. However, in the future it would be nice if the package’s functionality could be integrated in whatever Array trait system people will come up with in the future so that DiskArrays integrate more nicely with other special array types.