View into HDF5 dataset


#1

Is it possible, for writing, to get a “view” into part of a HDF5 dataset containing an array of some dimension?

I have a HDF5 file with some dataset, let’s say, data, where data is an array of some dimension. I want to increase the size of data over time. Specifically I increase the last dimension of data. So far I’m doing

    set_dims!(f["data"],(elsize...,new_count))
    f["data"][colons...,end-count+1:end] = something_here

where elsize is the size of data left out the last dimension and count/new_count is the size/new size of data in the last dimension.

However, f["data"][colons...,end-count+1:end] alone has many allocations because I guess it reads the (uninitialized) values in that part of data before writing the new data to it. For a regular array I would use a view here on the l.h.s. to avoid that copy. So, is it possible to get a “view” into part of a HDF5 dataset containing an array of some dimension to avoid those allocations?


#2

Some relevant previous discussions here: Appending an element to a JLD2 file and here: HDF5 speed? but alas, neither has an easy fix for this situation.