@printf thread safety in Julia 1.3?

New to Julia
1

I heard that in Julia 1.3 I/O will be thread safe. Is it so? In a multi-threaded for loop, if I’m using @printf to write to DIFFERENT files (one file per thread id), should I still use lock/unlock to guard the file writing commands?

I’m using 1.3.0-rc4.1, and I’m doing exactly that without using a lock/unlock, which led to problems. I took great care to make sure all variables that are changed in the loop have only local scope inside the loop. But I still found out that in the files produced, some rows are missing many numbers; some rows started from the middle of the array to be printed, etc.

To complicate the problem, I cannot reproduce the issue using a simpler piece of code. I think what I’m doing is essentially the same as below (I even added sleep to make the function call during each print take longer), but the code below does not show the same issue. All rows are printed out completely and in order. Since this example code does NOT reproduce the issue, I can only ask if this is a valid use of multi-threading, or a bad practice (from the perspective of thread safety and correctness, not efficiency, as this is just an example)?

4 threads were started with Julia and all of them are used in these examples.

using Printf
using Base.Threads

function todB(x)
	sleep(0.001)
	return 10*log10(x)
end

function main()
	println("Test 1: Write to different files, with no lock")
	println("Running on $(nthreads()) threads...")
	
	for n = 1:nthreads()
		fn = "temp$n.txt"
		if isfile(fn)
			rm(fn)
		end
	end

	@threads for n = 1:10
		fp = open("temp$(threadid()).txt", "a")
		for m = 1:1000
			@printf(fp, "%.2f,", todB(m))
		end
		@printf(fp, "\n")
		close(fp)
	end

	println("All done.")
end

@time main();

Now with a lock/unlock, even when I print to the same file in multiple threads, there is no issue at all whether in the other example below or in my actual application. However, I suspect performance is degraded due to lock regardless if one or multiple files are used, since when one thread is writing a file another thread cannot write, regardless if it’s the same file or different file it wants to write to.

using Printf
using Base.Threads

function todB(x)
	sleep(0.001)
	return 10*log10(x)
end

function main()
	println("Test 2: Write to the same file, with lock")
	println("Running on $(nthreads()) threads...")
	
	fp = open("temp0.txt", "w")

	rlock = ReentrantLock()
	@threads for n = 1:10
		lock(rlock)
		for m = 1:1000
			@printf(fp, "%.2f,", todB(m))
		end
		@printf(fp, "\n")
		unlock(rlock)
	end

	close(fp)

	println("All done.")
end

@time main();
2

I ran into the similar problem when multiple threads read different .jld2 files (using JLD2 package). I am using two functions to get around, basically the same solution as yours. I wish there is a @reentrantlock macro to lock critical resources, and it is possible if the macro declares a new ReentrantLock() variable somewhere, for example, in the current module.

function readcsvfile_notthreadsafe(args)::DataFrame 
      # read files
      # preprocess the data
      # return DataFrame
end 

rlock = ReentrantLock();
function readcsvfile(args)::DataFrame
     lock(rlock)
     df = DataFrame()
     try 
          df = readcsvfile_notthreadsafe(args)
     finally 
          unlock(rlock)
     end 
     return df
end