# Integrating a long(ish) running simulation into Pluto

**URL:** <https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371>\
**Category:** Pluto\
**Created:** [October 25, 2023, 11:21am UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371 "2023-10-25T11:21:55Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![grahamstark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/grahamstark/32/5200_2.png) [@grahamstark](https://discourse.julialang.org/u/grahamstark)\
**Post date:** [October 25, 2023, 11:21am UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/1 "2023-10-25T11:21:55Z")

</div>

Hi,

quick question: I have a simulation model that takes in the order of 30-60 seconds to run. ([This one](https://pppc.virtual-worlds.scot/) if you’re interested).

If I wanted to embed it in a Pluto notebook, what’s the best practice? Can I detach it somehow from Pluto’s normal reactive behaviour? Is there something I can use as a progress indicator?

Graham

---

<div class="post-metadata">

**Author:** ![filchristou](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/filchristou/32/26760_2.png) [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Post date:** [October 25, 2023, 2:05pm UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/2 "2023-10-25T14:05:12Z")

</div>

I very often write big simulations in Pluto.

The basic tip is that generally your long simulation should be called from a function and not a script (or a `begin` block or consecutive cells). Now you can have this long function defined in Pluto, but I usually do it differently.

What I commonly do is start Pluto in a local directory with local package manager using  
[`Pkg.activate`](https://github.com/fonsp/Pluto.jl/wiki/%F0%9F%8E%81-Package-management#advanced-set-up-an-environment-with-pkgactivate). I define a module in this directory and I put my non-Pluto bulky code (e.g. your simulation) in there. Because of Revise, I can also update it easily.

Then when I start Pluto the first cells do `using Revise` and `Pkg.activate(".")`.  
I import my local module `using MyModule` and I access the functions from there.

> Can I detach it somehow from Pluto’s normal reactive behaviour?

No. If you change any variables that are used as arguments to call the simulation function in your notebook, then the simulation will rerun. But that’s desirable. If you structure your notebook nicely I don’t see why would that be a problem.

Regarding the progress bar, you can definitely use some inside you simulation function.  
See [ProgressMeter in Pluto](https://discourse.julialang.org/t/progressmeter-in-pluto/69190)  
Pluto will then visualize it.

btw your link doesn’t work.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [October 25, 2023, 2:11pm UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/3 "2023-10-25T14:11:55Z")

</div>

Using the temporary execution barrier feature in Pluto may help with the reactivity piece: [Temporary Execution barrier by lungben · Pull Request #985 · fonsp/Pluto.jl · GitHub](https://github.com/fonsp/Pluto.jl/pull/985)

---

<div class="post-metadata">

**Author:** ![filchristou](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/filchristou/32/26760_2.png) [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Post date:** [October 25, 2023, 2:17pm UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/4 "2023-10-25T14:17:05Z")

</div>

is this the “Disable cell” button ?

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [October 25, 2023, 5:33pm UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/5 "2023-10-25T17:33:11Z")

</div>

See also `Base.@async` and `Threads.@spawn`. Those create tasks that are detached from the interative thread. You can then call `fetch` on the task, which will wait until a result is available, and give you that.

---

<div class="post-metadata">

**Author:** ![grahamstark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/grahamstark/32/5200_2.png) [@grahamstark](https://discourse.julialang.org/u/grahamstark)\
**Post date:** [October 25, 2023, 7:12pm UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/6 "2023-10-25T19:12:28Z")

</div>

Thanks. That’s interesting and helpful.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [October 26, 2023, 6:17am UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/7 "2023-10-26T06:17:30Z")

</div>

FWIW, here is a macro I use if I deliberately want to save some output, in case I need to restart the notebook but don’t want to the simulation to recompute again. Note that this deliberately circumvents `Pluto.jl` consistency by using external state. So if you change something you need to consider that the values are not recomputed!  
The snippet requires `JLD2.jl` for saving/loading and creates a folder `~/home/results/notebooks/<Notebookname>/` and saves the content of variables there. To save something you do:

```julia
@load_or_generate myvar = some_long_ass_computation(p1,p2,p3,p4)

```

Then each time the cell is run, the macro checks whether there is a save file `myvar.jld2` in the corresponding folder and if yes just loads the contents. **If you change the computation, you need to delete the file for the computation to refresh!**

> **Code**
>
> ```julia
> begin
> NOTEBOOKNAME = basename(replace(@ __FILE__ , r"\.jl#==#.*" => ""))
> function load_or_generate(f, name)
> path = joinpath(homedir(), "results/notebooks", NOTEBOOKNAME)
> name = endswith(name, ".jld2") ? name : name*".jld2"
> isdir(path) || mkpath(path)
> fullpath = joinpath(path, name)
> isfile(fullpath) && return jldopen(fullpath, "r")["data"]
> data = f()
> jldsave(fullpath; data)
> return data
> end
> macro load_or_generate(expr)
> expr.head != :(=) && error("@load_or_generate is only applicable to simple assignments of the form `a = b`")
> var, rhs = expr.args
> #eqsign == := 
> return quote
> $(esc(var)) = $(esc(:load_or_generate))($(string(var))) do
> return $(esc(rhs))
> end
> end
> end
> md"""Setup save to $(joinpath(homedir(), "results/notebooks", NOTEBOOKNAME))"""
> end
> 
> ```

---

<div class="post-metadata">

**Author:** ![filchristou](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/filchristou/32/26760_2.png) [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Post date:** [October 26, 2023, 6:46am UTC](https://discourse.julialang.org/t/integrating-a-long-ish-running-simulation-into-pluto/105371/8 "2023-10-26T06:46:56Z")

</div>

While we are at it, and you really need to change something that pointlessly reruns the simulation, you can use memoization tools, to avoid that, e.g., **[Memoization.jl](https://github.com/marius311/Memoization.jl)**. But be really careful with the consistency of the results. That’s similar to @abraemer solution, but doesn’t work between new sessions.