# Help understanding Dict allocation

**URL:** <https://discourse.julialang.org/t/help-understanding-dict-allocation/15809>\
**Category:** General Usage\
**Created:** [October 3, 2018, 3:45am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809 "2018-10-03T03:45:02Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![LeoK987](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leok987/32/4880_2.png) [@LeoK987](https://discourse.julialang.org/u/LeoK987)\
**Post date:** [October 3, 2018, 3:45am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/1 "2018-10-03T03:45:02Z")

</div>

I amd trying to find out about a huge allocation in my code (with --track-allocation=user):

```julia
      608 ValueDict = Dict{Function, Array{Float64,1}}()
        0 for atom in Atoms
        - values = zeros(Float64, iEnd)
        0 AtomValues(values, atom, iStart, iEnd)
8347801470 merge!(ValueDict, Dict((atom)=>values))
       - end

```

So I made a simple test:

```julia
        - function f1(i=50) # testing
        - 1
        - end
        - 
        - function f2(i=50) # testing
        - 1
        - end
        - 
        - function testDict()
   800080 ar = [1.:100000.;]
      736 fDict = Dict{Function, Array{Float64,1}}()
        - 	
        0 dicf = Dict(f1=>ar)
  2153071 merge!(fDict, dicf)
        0 dicf = Dict(f2=>ar)
  2152399 merge!(fDict, dicf)
        - 	
  7435020 println(typeof(dicf))
        - 	
    18512 aF = [f1, f2]
  2411338 println(typeof(f1), " ", typeof(aF[1]))
        - 	
   463126 dicf1 = Dict(aF[1]=>ar)
      336 merge!(fDict, dicf1)
   306246 dicf1 = Dict(aF[2]=>ar)
      336 merge!(fDict, dicf1)
        - 
     2144 println(typeof(dicf1))
        - 	
        0 for f in aF
      992 dicf2 = Dict(f=>ar)
        0 merge!(fDict, dicf2)
     3888 println(typeof(dicf2))
        - end
        - end

```

It makes me more confused. Can anyone explain about the different allocations in the above simple code?

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [October 3, 2018, 6:01am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/2 "2018-10-03T06:01:56Z")

</div>

Presumably you are not interested in the memory cost of compilation:

```julia
        - function f1(i=50) # testing
        - 1
        - end
        - 
        - function f2(i=50) # testing
        - 1
        - end
        - 
        - function testDict()
   802160 ar = [1.:100000.;]
      608 fDict = Dict{Function, Array{Float64,1}}()
        - 
        0 dicf = Dict(f1=>ar)
      480 merge!(fDict, dicf)
        0 dicf = Dict(f2=>ar)
      480 merge!(fDict, dicf)
        - 
     1744 println(typeof(dicf))
        - 
      112 aF = [f1, f2]
     1936 println(typeof(f1), " ", typeof(aF[1]))
        - 
      496 dicf1 = Dict(aF[1]=>ar)
        0 merge!(fDict, dicf1)
      496 dicf1 = Dict(aF[2]=>ar)
        0 merge!(fDict, dicf1)
        - 
     1744 println(typeof(dicf1))
        - 
        0 for f in aF
      992 dicf2 = Dict(f=>ar)
        0 merge!(fDict, dicf2)
     3888 println(typeof(dicf2))
        - end
        - end
        - 
        - using Profile
        - testDict()
        - Profile.clear_malloc_data()
        - 
        - testDict()
        - 

```

although a few places are still a bit odd.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 3, 2018, 10:40am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/3 "2018-10-03T10:40:37Z")

</div>

Use `Valuedict[atom]=values` instead of `merge!(ValueDict, Dict((atom)=>values))`. The latter allocates an entire new `Dict` for every guy you want to put in.

A merge-combine that uses only a single hash eval is unfortunately missing in `Base`, but your `combine` function is “overwrite” anyway, so you get away with `setindex!`.

Also, are you sure that you can’t make your keys concretely typed? A `Dict{Function, Foo_T}` is barely better than a `Dict{Any, Foo_T}`.

---

<div class="post-metadata">

**Author:** ![LeoK987](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leok987/32/4880_2.png) [@LeoK987](https://discourse.julialang.org/u/LeoK987)\
**Post date:** [October 3, 2018, 11:11am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/4 "2018-10-03T11:11:14Z")

</div>

Thanks to all for the tips.

@foobar_lv2, I want to use the functions like f1, f2 etc. How can I be more concrete than using “Function”?

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 3, 2018, 11:20am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/5 "2018-10-03T11:20:37Z")

</div>

> [@LeoK987](#):
>
> How can I be more concrete than using “Function”?

Are your functions really “function functions” or are they closures (a single parametrized function)?

If they are closures, then you can define

```julia
struct Foo_func{T1,T2,T3}<:Function
param1::T1
param2::T2 
param3::T3 
end 

function (foo::Foo_func)(x)
#do something with x and foo.param1, foo.param2, foo.param3
end

valdict=Dict{Foo_func{Float64, Bool, Int32}, Array{Float64, 1}}()

```

PS: Regarding functions as arguments: Think very well on whether you want to dispatch on

```julia
function bar(foo::Function, x)
#....
end

```

or

```julia
function bar(foo:T, x) where {T<:Function}
#....
end

```

Both are semantically the same, but the second version tells julia to always specialize on the specific function, and the first allows julia to use heuristics to decide whether to specialize or not. That can save compile time, but can cost runtime. `@code_native`, `@code_llvm`, `@code_warntype` etc will lie to you if you use the first definition: They will tell you the code for the second definition while julia will in reality compile the first version (they will show you the fully realized method instead of the really existing method). This makes for very unfun profiling sessions.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [October 3, 2018, 11:46am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/6 "2018-10-03T11:46:55Z")

</div>

> Both are semantically the same, but the second version tells julia to always specialize on the specific function, and the first allows julia to use heuristics to decide whether to specialize or not. That can save compile time, but can cost runtime. `@code_native` , `@code_llvm` , `@code_warntype` etc will lie to you if you use the first definition: They will tell you the code for the second definition while julia will in reality compile the first version (they will show you the fully realized method instead of the really existing method). This makes for very unfun profiling sessions.

Do you have an example of that? Or a reference?

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [October 3, 2018, 11:54am UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/7 "2018-10-03T11:54:53Z")

</div>

[https://github.com/JuliaLang/julia/blob/master/doc/src/devdocs/functions.md](https://github.com/JuliaLang/julia/blob/master/doc/src/devdocs/functions.md)

I ran into this issue several times on 0.6, and will test more on 1.0 later. The simplest inlined cases apparently get correctly inferred in 1.0.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [October 3, 2018, 12:08pm UTC](https://discourse.julialang.org/t/help-understanding-dict-allocation/15809/8 "2018-10-03T12:08:28Z")

</div>

Interesting read, thanks!