Atomic write(filename) append?

General Usage
1

dear julia wizards—is it possible to procure an atomic write-append that briefly locks an append-output file in order to prevent other processes/threads from interleaving output? the natural way would be safety in

write("/tmp/myatomic.txt", "hello\n"; append=true, lockme=true)

the julia history file probably needs file-locking support from the OS to prevent multiple julia master processes interleaving. alas, I am just asking whether slaves from the same julia master process can safely write short snippets to the same file without stepping on one another.

2

This might be a bit overkill, but it probably should work

z = [IOBuffer() for i in 1:10] ##@everywhere?
write(z[thread_id?], "somedata");

You could interleave by newlines or use another sort of marker with split(), or just stack them whole

1 Like
3

In general, I would suggest that the worker processes send messages to a RemoteChannel on the the master process, and let the master process do I/O (in an asynchronous Task). This serializes the output without involving file locking. It is also advantageous because in some cluster situations, worker processes may not even have access to the file system, and even if you have a distributed filesystem, locking is very tricky. It also allows the “file” to be something else, like a pipe or a terminal.

2 Likes