I am trying to write data generated by a compute intensive function into a database for storage and later analysis.
I have been trying to use MySQL.jl to manage the connection. I am running into an issue where when using MySQl.jl with distributed workers the connection closes and I don’t understand the error stack.
I need to use a database as the outputs are quite large, but not attached to MySQL.jl if there is a more up-to-date or better way of doing this.
using Distributed # addprocs(4) @everywhere begin using MySQL end println("Opening DB Connection") @everywhere dbName = "model" @everywhere connection = DBInterface.connect(MySQL.Connection, "localhost","username", "password", db = dbName) # on worker 2 @sync @distributed for i in 1:10 query = string("INSERT INTO dummy (number) VALUES(", myid()+10+i,")") DBInterface.execute(connection, query) end
thanks for the help!