Is there an array type, that allows me to create a virtual (or lazy) representation of the following array x
?
sz = (64, 64, 2, 10589)
tmp = reshape(range(-1,1,64),(:,1,1,1));
x = repeat(tmp,1,sz[2],sz[3],sz[4]);
It would be ok to materialize in memory the 64 elements of the range. But I would like to avoid having all 64*64*2*10589
elements.
I tried LazyArrays
, but the result does not seem to be an AbstractArray
and I cannot use it with Flux.DataLoader
for example:
using Flux
data = randn(64, 64, 2, 10589)
tmp = reshape(range(-1,1,64),(:,1,1,1));
x = repeat(tmp,1,sz[2],sz[3],sz[4]);
dataloader = Flux.DataLoader((data,x); batchsize=128)
first(dataloader)
# ok
using LazyArrays
x_lazy = @~ repeat(tmp,1,sz[2],sz[3],sz[4]);
dataloader = Flux.DataLoader((data,x_lazy); batchsize=128)
I get the error:
ERROR: MethodError: no method matching length(::Applied{LazyArrays.DefaultApplyStyle, typeof(repeat), Tuple{Base.ReshapedArray{…}, Vararg{…}}})
The function `length` exists, but no method is defined for this combination of argument types.
Any ideas?