MySQL Connection closes when writing to database from multiple workers

Specific Domains Data
1

Heya,

I am trying to write data generated by a compute intensive function into a database for storage and later analysis.

I have been trying to use MySQL.jl to manage the connection. I am running into an issue where when using MySQl.jl with distributed workers the connection closes and I don’t understand the error stack.

I need to use a database as the outputs are quite large, but not attached to MySQL.jl if there is a more up-to-date or better way of doing this.

using Distributed 
# addprocs(4)

@everywhere begin
    using MySQL
end

println("Opening DB Connection")

@everywhere dbName = "model"
@everywhere connection = DBInterface.connect(MySQL.Connection, "localhost","username", "password", db = dbName)

# on worker 2
@sync @distributed for i in 1:10
    query = string("INSERT INTO dummy (number) VALUES(", myid()+10+i,")")
    DBInterface.execute(connection, query)
end

thanks for the help!

2

It would probably still be very helpful if you posted it :slight_smile:

1 Like