# How Can I Create Permanently-Stored Lookup Tables?

**URL:** <https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755>\
**Category:** General Usage\
**Tags:** question\
**Created:** [April 21, 2021, 8:55pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755 "2021-04-21T20:55:53Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 21, 2021, 8:55pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/1 "2021-04-21T20:55:53Z")

</div>

Hello, everyone. How can I store computed values permanently, i.e. so they never have to be recomputed?

Specifically, I have generated 2 lookup tables via lengthy Monte Carlo simulations and stored them in CSV files. I would like to import them only once, and then store their values permanently in the Julia function that will use them. The tables both have 300 entries, so typing them in by hand is not an option.

Alternatively, I can run the Monte Carlo simulations again, if that leads to a way to store their contents permanently.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 21, 2021, 9:20pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/2 "2021-04-21T21:20:24Z")

</div>

Use a functor:

> [@Define data-dependent function, various ways](https://discourse.julialang.org/t/define-data-dependent-function-various-ways/48988/4):
>
> I think [function-like objects](https://docs.julialang.org/en/v1/manual/methods/#Function-like-objects) (sometimes called “functors”) would be yet another option to achieve the same kind of things. (AFAIU, this is what’s internally used to implement closures, and I think it requires a bit less work for the compiler to optimize. Maybe someone more knowledgeable will chime in and confirm or correct this) struct Fun data :: Vector{Int} end function (f::Fun)(x) s = 0. for i in 1:length(x) s += (x[i]-f.data[i])^2 end s end println(" Functor: ") functor …

Ps: of course you can read the tables into arrays, using CSV.jl

---

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 21, 2021, 9:24pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/3 "2021-04-21T21:24:55Z")

</div>

Thanks, I’ll look into it.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [April 21, 2021, 9:49pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/4 "2021-04-21T21:49:55Z")

</div>

Also note there are the [Memoization](https://github.com/marius311/Memoization.jl) and [Memoize](https://github.com/JuliaCollections/Memoize.jl) packages (and I think there is even a third or fourth package in this same subject) that allow you to simple annotate a function so it is never recomputed for the same arguments **in the same session**. I think that unfortunately, they do not support persistent storage (i.e., saving to a CSV) but maybe if you can relax your constraints they are good options (and maybe they could be interested on extending their use cases to support persistent storage after hearing you).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 21, 2021, 10:20pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/5 "2021-04-21T22:20:02Z")

</div>

Example using `DelimitedFiles`:

```julia
julia> using DelimitedFiles

julia> struct MyFunc
         data::Matrix{Float64}
       end

julia> (f::MyFunc)(x) = f.data*x

julia> data, header = readdlm("csv.dat",',',header=true)
([1.0 0.0 0.0; 0.0 1.0 0.0; 0.0 0.0 1.0], AbstractString["A" " B" " C"])

julia> f = MyFunc(data)
MyFunc([1.0 0.0 0.0; 0.0 1.0 0.0; 0.0 0.0 1.0])

julia> x = rand(3)
3-element Vector{Float64}:
 0.07056682527318503
 0.7581530064054021
 0.4889526606997521

julia> f(x)
3-element Vector{Float64}:
 0.07056682527318503
 0.7581530064054021
 0.4889526606997521

```

where `csv.dat` is:

```julia
A, B, C
1, 0, 0
0, 1, 0
0, 0, 1

```

---

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 21, 2021, 10:26pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/6 "2021-04-21T22:26:10Z")

</div>

Thanks guys, but I’m looking for permanent persistence, not just persistence for one session. In other words, I can close Julia, and when I reopen it the values will still be there, without the CSV files needing to be read in again or the simulations run again.

The idea is that I can eventually share this program with people who do not have access to the CSV files or the file that runs the Monte Carlo simulation.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 21, 2021, 10:29pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/7 "2021-04-21T22:29:17Z")

</div>

Then you have to write the data directly into the function, as an array. Just copy and paste it into an array constructor. But the code won’t be pretty.

---

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 21, 2021, 10:32pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/8 "2021-04-21T22:32:33Z")

</div>

That seems to work, thank you very much.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [April 21, 2021, 11:08pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/9 "2021-04-21T23:08:02Z")

</div>

By the way, if the arrays were really huge and it wasn’t practical to copy them into the source (or you needed some other kind of data that wasn’t amenable to that) you could use Pkg’s [artifacts system](https://pkgdocs.julialang.org/v1/artifacts/). It would be a bit more setup, but the end result would be that the artifacts could be automatically downloaded when your package is installed, and you could read in the arrays from the artifacts inside an [` __init__ ` function](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation) which would run at module-load time.

---

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 21, 2021, 11:49pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/10 "2021-04-21T23:49:27Z")

</div>

Thanks Eric. That sounds great, I will definitely look into it eventually.

---

<div class="post-metadata">

**Author:** ![adolgert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adolgert/32/20286_2.png) [@adolgert](https://discourse.julialang.org/u/adolgert)\
**Post date:** [April 21, 2021, 11:54pm UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/11 "2021-04-21T23:54:25Z")

</div>

The artifacts system does seem right for this. It doesn’t solve the problem of where to put the file, but it uses libcurl to get the file, so it can be on a website, in a web service, on FTP, or in a local file. Heck, it does GOPHER protocol. I wonder whether figshare or Zenodo would be appropriate storage places? If it were OK to generate the files once, then [Scratch.jl](https://github.com/JuliaPackaging/Scratch.jl) would be a good option, but it seems you want to make the file available beforehand.

---

<div class="post-metadata">

**Author:** ![RalphH](https://avatars.discourse-cdn.com/v4/letter/r/db5fbb/32.png) [@RalphH](https://discourse.julialang.org/u/RalphH)\
**Post date:** [April 22, 2021, 4:35am UTC](https://discourse.julialang.org/t/how-can-i-create-permanently-stored-lookup-tables/59755/12 "2021-04-22T04:35:03Z")

</div>

I took a cursory look at the page you linked, and it says the data containers are mutable. I don’t think that is a good idea for my application, but thanks anyway, I appreciate the info.

This is turning out to be quite a learning experience.