# Size of Multi-dimensional SharedArray

**URL:** <https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099>\
**Category:** New to Julia\
**Tags:** bug\
**Created:** [January 1, 2018, 10:21pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099 "2018-01-01T22:21:53Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 1, 2018, 10:21pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/1 "2018-01-01T22:21:53Z")

</div>

Is it a bug with `Base.summarysize` or something very inefficient about SharedArray? A 1D SharedArray only have a small overhead but a 2D SharedArray uses 2x more memory?

```julia
julia> Base.summarysize(rand(1000000))
8000000

julia> Base.summarysize(SharedArray{Float64}(rand(1000000)))
8000213

julia> Base.summarysize(rand(1000000,2))
16000000

julia> Base.summarysize(SharedArray{Float64}(rand(1000000,2)))
32000221

```

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 2, 2018, 12:46am UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/2 "2018-01-02T00:46:30Z")

</div>

I’d take anything that `summarysize` says with a large grain of salt.  
It doesn’t count the header (40 bytes on a 64-bit platform), nor the actual amount allocated, just the sizeof the vector (what is used).

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 2, 2018, 6:52pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/3 "2018-01-02T18:52:12Z")

</div>

Thanks for the insight. Do you know of any good way to accurately measure how much memory it is using? I have a use case that I could load up to 100 GiB into shared memory and it would be a bummer if it fails miserably for my lack of understanding 🙂

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [January 2, 2018, 7:19pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/4 "2018-01-02T19:19:53Z")

</div>

I am getting the following on v0.6.2:

```julia
julia> Base.summarysize(rand(1000000))
8000000

julia> Base.summarysize(SharedArray{Float64}(rand(1000000)))
8000341

julia> Base.summarysize(rand(1000000,2))
16000000

julia> Base.summarysize(SharedArray{Float64}(rand(1000000,2)))
16000349

```

There is also `Base.shmem_rand` btw.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [January 2, 2018, 7:23pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/5 "2018-01-02T19:23:12Z")

</div>

> [@mohamed82008](#):
>
> Base.summarysize(SharedArray{Float64}(rand(1000000,2)))

Interesting, what OS/platform are you using? I also have v0.6.2:

```julia
$ julia
               _
   _ _ _(_)_ | A fresh approach to technical computing
  (_) | (_) (_) | Documentation: https://docs.julialang.org
   _ _ _| |_ __ _ | Type "?help" for help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\ __'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-apple-darwin14.5.0

julia> Base.summarysize(SharedArray{Float64}(rand(1000000,2)))
32000221

```

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [January 2, 2018, 7:24pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/6 "2018-01-02T19:24:36Z")

</div>

Good old Windows.

```julia
julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17* (2017-12-13 18:08 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Prescott)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

```

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [January 2, 2018, 7:30pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/7 "2018-01-02T19:30:13Z")

</div>

On Linux with v0.6.1, I am getting the same result as you. But on nightly, getting the right result.

On Windows with latest nightly (downloaded today), I am getting:

```julia
julia> using SharedArrays

julia> Base.summarysize(SharedArray{Float64}(rand(1000_000,2)))
32000205

```

This really seems like a bug.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [January 2, 2018, 7:42pm UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/8 "2018-01-02T19:42:02Z")

</div>

I think it’s just double counting, because of the behavior of `summarysize`. It basically recursively counts how much memory is used by all objects reachable. A `SharedArray` has a field `loc_subarr_1d` and another `s`. The latter holds the whole array and the former holds a 1d view of the array. Mutating one mutates the other, so it’s just double counting. It’s probably doing something like:

```julia
mysizeof(f) = sum((sizeof(f), (mysizeof(getfield(f,i)) for i in 1:nfields(f))...))

```

But it’s weird that it’s not double counting for the 1D case 😕. Anyways, I think it deserves some attention from devs.

Maybe adding a bug keyword or something will get more attention more quickly. In the meantime, I think the equivalent of task manager in Windows or system monitor in Linux will give you a rough idea of memory used by large enough data.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 3, 2018, 8:01am UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/9 "2018-01-03T08:01:15Z")

</div>

> [@mohamed82008](#):
>
> Anyways, I think it deserves some attention from devs.
> 
> Maybe adding a bug keyword or something will get more attention more quickly.

If you haven’t already, you should consider opening an issue so that it does not get lost.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [January 3, 2018, 10:29am UTC](https://discourse.julialang.org/t/size-of-multi-dimensional-sharedarray/8099/10 "2018-01-03T10:29:24Z")

</div>

Done [https://github.com/JuliaLang/julia/issues/25367](https://github.com/JuliaLang/julia/issues/25367).