Help with crash in Oracle with threads

General Usage
1

Hello, I need some help with this crash.

I’m using Julia 1.5.3

It appears that when using threading, this crashes Julia. The crash disappears when using a simple for loop.

Here is a simple MWE

using Oracle
conn = Oracle.Connection(user, password, connstring)

Oracle.execute(conn, "CREATE TABLE TESTING_CRASH (TESTID NUMBER)")

This for loop works fine

for i  = 1:100
    Oracle.execute(conn, "INSERT INTO testing_crash (TESTID) VALUES ($i)")
end

This threaded for loop causes Julia to crash

Threads.@threads for i  = 1:100
    Oracle.execute(conn, "INSERT INTO testing_crash (TESTID) VALUES ($i)")
end

Anyone can help me with this?

2

I do not believe accessing a connection is thread safe (for any database). Each thread should have it’s own connection. Connection Pooling Tutorial · Oracle.jl is normally the way you handle this. You create a pool size of say 20 connections and each thread takes a connection when needed and returns it when done. If there are no available connections then the thread waits for one to be returned.

The key is to have each thread use the connection for the minimum amount of time, so query for the information, return the connection, THEN process the data.

The size of the connection pool really depends on your queries and database server (so there is no single magic number for all). Too many and you are wasting server resources or worse causing your db server to thrash. Too few and you are not fully utilizing your database.

2 Likes
3

OK, thanks, that kind of makes sense. I guess I didn’t see any issues when running LibPQ like this so I thought Oracle should run the same way.

But in any case, it shouldn’t crash Julia, should it?

4

Oracle.jl calls C code. If the C code is not thread safe, then corruption is likely to occur. This can overwrite pointers, then accessing corrupt pointers often results in a process crash.

5

The Oracle.jl code is really just a bridge into the C library so I suspect it’s the C library that is causing memory corruption or something. So i don’t think there is much that can be done in Julia. The author of Oracle.jl could provide locking around the execute calls however that would be an overhead they might not want to incur.

6

Oracle.jl uses OPDI-C, which isn’t thread safe when DPI_MODE_CREATE_THREADED isn’t set. Oracle.jl seems not to set it.