# How to make JuliaDB writes thread-safe

**URL:** <https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755>\
**Category:** New to Julia\
**Created:** [August 11, 2020, 6:31pm UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755 "2020-08-11T18:31:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [August 11, 2020, 6:31pm UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/1 "2020-08-11T18:31:16Z")

</div>

The documentation for JuliaDB doesn’t provide a thread-safe interface for writes. Thus, the code below gives unexpected results. Is there a recommended way to do this?

I have a hunch there may be an implementation utilizing some kind of explicit lock on the table, but I’m at a loss regarding these ‘First Steps’ ;).

```julia
using JuliaDB, Random

struct ModelState
  iter::Int # this name is meaningless
  x::Array{Float32,2}
end

myjdb2 = JuliaDB.table((modelname=String[],
                        state=ModelState[]);
                        pkey=[:modelname])

Threads.@threads for i in 1:10
  Random.seed!(i)
  mod = ModelState(i,rand(2,2))
  push!(rows(myjdb2),(modelname=string("foo",i),state=mod))
end

```

This is missing the seventh entry:

```julia
julia> myjdb2
Table with 9 rows, 2 columns:
modelname state
─────────────────────────────────────────────────────────────────────────
"foo1" ModelState(1, Float32[0.236033 0.312707; 0.346517 0.00790928])
"foo3" ModelState(3, Float32[0.811698 0.807622; 0.988432 0.970091])
"foo8" ModelState(8, Float32[0.0123577 0.880851; 0.894287 0.769253])
"foo2" ModelState(2, Float32[0.366796 0.210256; 0.523879 0.819338])
"foo10" ModelState(10, Float32[0.112582 0.344454; 0.368314 0.0566454])
"foo5" ModelState(5, Float32[0.530365 0.30131; 0.777009 0.881349])
"foo6" ModelState(6, Float32[0.511477 0.830376; 0.121374 0.0263919])
"foo9" ModelState(9, Float32[0.707078 0.258221; 0.26533 0.219948])
"foo4" ModelState(4, Float32[0.680079 0.92407; 0.874437 0.929336])

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 12, 2020, 3:36am UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/2 "2020-08-12T03:36:36Z")

</div>

I think that might be inherently thread unsafe since all threads will have to mutate the same array. If it’s not an operation happening really fast, you could just use some locks to make sure it’s safe though.

---

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [August 13, 2020, 9:43pm UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/3 "2020-08-13T21:43:35Z")

</div>

Thanks, I’ll maybe just resort to a basic array. `JuliaDB` just came on my radar while blindly searching for thread-safe IO solutions. I was probably browsing the docs and took their description of the parallelized query support out of context.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 13, 2020, 11:55pm UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/4 "2020-08-13T23:55:37Z")

</div>

`push!` is not thread-safe for a JuliaDB table _or_ a regular `Array`, so neither will work by default. You can do some manual locking, or you could make a wrapper around the table/array that allows thread-safe `push!`. I’m sure there’s also packages out there that make this easier in some way, but I’m not aware of any.

---

<div class="post-metadata">

**Author:** ![mkarikom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkarikom/32/7096_2.png) [@mkarikom](https://discourse.julialang.org/u/mkarikom)\
**Post date:** [August 14, 2020, 12:01am UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/5 "2020-08-14T00:01:06Z")

</div>

So would this hold true even if there were no write-conflicts within threads? Such a case seems thread-safe since each thread could write to a 2-d array where the top level index is over Threads.threadid().

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 14, 2020, 2:06am UTC](https://discourse.julialang.org/t/how-to-make-juliadb-writes-thread-safe/44755/6 "2020-08-14T02:06:18Z")

</div>

Even if the underlying array was large enough to accommodate all combined writes (which, if not, would most likely require a reallocation and move of the array data), the threads would race to increment the end-of-array index stored in the array object, so you’d end up with threads writing into the same array index, or various other kinds of undefined behavior.