Hi,

I need to work with a very big Array

`z=zeros(Int64,2^K,2^K,K,T)`

What is the proper way of pre-allocating this Array in such a way that it uses the least memory possible when K is large?

Hi,

I need to work with a very big Array

`z=zeros(Int64,2^K,2^K,K,T)`

What is the proper way of pre-allocating this Array in such a way that it uses the least memory possible when K is large?

How big is `K`

? Will most of the elements be `0`

? If so, using a sparse data structure will be helpful. Also, are you sure you need to store the array? If you can get the same effect from looping over a small portion of it, that should be much more efficient.

2 Likes

K is around 15! Yes, most elements will be 0. However, most of this are 3-dimensional arrays and I could not get sparse structures to work in this case. A MWE of my problem can be found here Performance Tips for Combinatorial Problem

Is there any structure to it? Blocks, for instance?

Yes, you can think of `z=zeros(Int64,2^K,2^K,K,T)`

as 2^K*2^K matrices of size K*T

If you donâ€™t need all of the matrices at the same time, store them as an array of arrays, with possibly â€śnothingâ€ť entries.

1 Like