I heard that in Julia 1.3 I/O will be thread safe. Is it so? In a multi-threaded for loop, if I’m using @printf to write to DIFFERENT files (one file per thread id), should I still use lock/unlock to guard the file writing commands?
I’m using 1.3.0-rc4.1, and I’m doing exactly that without using a lock/unlock, which led to problems. I took great care to make sure all variables that are changed in the loop have only local scope inside the loop. But I still found out that in the files produced, some rows are missing many numbers; some rows started from the middle of the array to be printed, etc.
To complicate the problem, I cannot reproduce the issue using a simpler piece of code. I think what I’m doing is essentially the same as below (I even added sleep to make the function call during each print take longer), but the code below does not show the same issue. All rows are printed out completely and in order. Since this example code does NOT reproduce the issue, I can only ask if this is a valid use of multi-threading, or a bad practice (from the perspective of thread safety and correctness, not efficiency, as this is just an example)?
4 threads were started with Julia and all of them are used in these examples.
using Printf using Base.Threads function todB(x) sleep(0.001) return 10*log10(x) end function main() println("Test 1: Write to different files, with no lock") println("Running on $(nthreads()) threads...") for n = 1:nthreads() fn = "temp$n.txt" if isfile(fn) rm(fn) end end @threads for n = 1:10 fp = open("temp$(threadid()).txt", "a") for m = 1:1000 @printf(fp, "%.2f,", todB(m)) end @printf(fp, "\n") close(fp) end println("All done.") end @time main();
Now with a lock/unlock, even when I print to the same file in multiple threads, there is no issue at all whether in the other example below or in my actual application. However, I suspect performance is degraded due to lock regardless if one or multiple files are used, since when one thread is writing a file another thread cannot write, regardless if it’s the same file or different file it wants to write to.
using Printf using Base.Threads function todB(x) sleep(0.001) return 10*log10(x) end function main() println("Test 2: Write to the same file, with lock") println("Running on $(nthreads()) threads...") fp = open("temp0.txt", "w") rlock = ReentrantLock() @threads for n = 1:10 lock(rlock) for m = 1:1000 @printf(fp, "%.2f,", todB(m)) end @printf(fp, "\n") unlock(rlock) end close(fp) println("All done.") end @time main();