# Deepcopy against the grain of Julia?

**URL:** <https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580>\
**Category:** New to Julia\
**Tags:** agents\
**Created:** [November 16, 2021, 11:51am UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580 "2021-11-16T11:51:33Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![idontgetoutmuch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/idontgetoutmuch/32/321_2.png) [@idontgetoutmuch](https://discourse.julialang.org/u/idontgetoutmuch)\
**Post date:** [November 16, 2021, 11:51am UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/1 "2021-11-16T11:51:33Z")

</div>

I have implemented a particle filter for an agent-based model using JuliaDynamics but I seem to have ended up using `deepcopy` a lot. Also when I run the simple particle filter over a very simple model I don’t get very good performance

```julia
julia> @time runPf(inits, init_log_weights);
 56.329763 seconds (28.13 M allocations: 3.170 GiB, 1.81% gc time, 0.07% compilation time)

```

I feel I am “working against the grain” of Julia. Here’s my code: [https://gist.github.com/idontgetoutmuch/54b2482efea120566290b594507679ec](https://gist.github.com/idontgetoutmuch/54b2482efea120566290b594507679ec). How can I make it more Julia-esque? And how can I improve performance?

PS I realise I can parallelize the particle filter but I don’t want to do this until I am working with the grain of Julia

PPS I am a functional programmer by background and keep forgetting Julia does call by reference.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 16, 2021, 12:21pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/2 "2021-11-16T12:21:57Z")

</div>

The first thing you should do is to remove all global variables. For instance, here `actuals`, `Q` and `R` are globals:

```julia
function runPf(inits, init_log_weights)
    for i in 2:l
        (end_states2, logWeights2, inits2, a2) = pf(deepcopy(inits), init_log_weights, P, map(x -> convert(Float64,x), actuals[i]), Q, R);
        predicted[i] = mean(end_states2);
        inits = inits2;
        init_log_weights = logWeights2;
    end
end

```

(or at least declare them as `const`).

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [November 16, 2021, 12:32pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/3 "2021-11-16T12:32:27Z")

</div>

Looking at a profile of your program it seems 80 % of runtime is spent at

```julia
        Agents.step!(jnits[i], agent_step!, 1)

```

Maybe you should isolate a M(inimal;)WE around that to get the discussion going.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 16, 2021, 12:42pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/4 "2021-11-16T12:42:39Z")

</div>

> [@idontgetoutmuch](#):
>
> keep forgetting Julia does call by reference

Not really, functions pass around values, which of course may be containers.

Keep in mind that Julia does not have pointers in the C/C++/… sense.

---

<div class="post-metadata">

**Author:** ![idontgetoutmuch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/idontgetoutmuch/32/321_2.png) [@idontgetoutmuch](https://discourse.julialang.org/u/idontgetoutmuch)\
**Post date:** [November 16, 2021, 3:23pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/5 "2021-11-16T15:23:55Z")

</div>

Thank you - `Q` and `R` are now parameters but I see no improvement in performance.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 16, 2021, 3:50pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/6 "2021-11-16T15:50:07Z")

</div>

And `actuals`? The point is that while you have global variables and type instabilities, speculation about other sources of cost is a waste of time.

---

<div class="post-metadata">

**Author:** ![idontgetoutmuch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/idontgetoutmuch/32/321_2.png) [@idontgetoutmuch](https://discourse.julialang.org/u/idontgetoutmuch)\
**Post date:** [November 16, 2021, 4:24pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/7 "2021-11-16T16:24:08Z")

</div>

I think I have done that but with no improvement. But perhaps I have erred: [https://gist.github.com/idontgetoutmuch/54b2482efea120566290b594507679ec](https://gist.github.com/idontgetoutmuch/54b2482efea120566290b594507679ec) (updated).

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [November 16, 2021, 4:32pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/8 "2021-11-16T16:32:05Z")

</div>

You should do a basic profiling first. Help us help you by letting us know which lines take the most time. (also, try to make your code as small as possible, it increases likelyhood that someone else will read it 🙂 )

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 16, 2021, 4:32pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/9 "2021-11-16T16:32:43Z")

</div>

Ok, now that looks ok on that first call.

From my side, what prevents me to running your code is that as it is I would need to install many packages which I do not want to. You don’t seem to be using all of them in that code. The simplest the example gets, the better the chances of someone actually working on it to show where it can be improved.

In general, after type instabilities are removed, you should chase allocations. Any copy of something mutable of course allocates. But there are other places where you seem to be allocating new arrays. For example, here:

```julia
    wn = map(x -> exp(x), log_w .- maximum(log_w))
    wn = wn / sum(wn)

```

that is allocating 3 arrays (one in ` og_w .- maximum(log_w)`), one in the output of the `map`, and a third in the normalization:

```julia
julia> function f(log_w)
          wn = map(x -> exp(x), log_w .- maximum(log_w));
           wn = wn / sum(wn);
           return wn
       end   
f (generic function with 1 method)

julia> x = rand(10);

julia> @btime f($x);
  147.233 ns (3 allocations: 480 bytes)

```

If you rewrite that with some care:

```julia
julia> function f(log_w)
          wn = exp.(log_w .- maximum(log_w));
           swn = sum(wn)
           wn .= wn ./ swn;
           return wn
       end
f (generic function with 1 method)

julia> @btime f($x);
  97.897 ns (1 allocation: 160 bytes)

```

you can reduce that to one allocation and accelerate it quite a bit. This is one example. But profiling is important to know where to spend time optimizing. But only _after_ you removed the type instabilities. otherwise the time spent on each function may not be representative of anything related to the algorithms used.

---

<div class="post-metadata">

**Author:** ![idontgetoutmuch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/idontgetoutmuch/32/321_2.png) [@idontgetoutmuch](https://discourse.julialang.org/u/idontgetoutmuch)\
**Post date:** [November 16, 2021, 4:41pm UTC](https://discourse.julialang.org/t/deepcopy-against-the-grain-of-julia/71580/10 "2021-11-16T16:41:03Z")

</div>

Thank you very much for taking the time - I am going through and removing unneeded imports. I hope to return with an easier-to-run example.