Running Julia with slurm, output not being printed until after completion

New to Julia
1

I’m running Julia on a computing cluster and submitting jobs through slurm. The submission file is relatively simple:

#!/bin/bash

#SBATCH -o test.out

module purge
module load modules/2.1.1-20230405 gcc/11.3.0
module load julia/1.8.3
julia ./test.jl

with the code:

println("Test 1")
sleep(600)
println("Test 2")

When I do this, the output of the code only gets written to the outfile after the code completes (I’ve let the code sleep for over an hour to confirm that this is the case, as opposed to there just being some finite delay). This seems to be at least somewhat Julia related, as non-Julia output such as information about module loading gets written to the outfile immediately. Is anyone familiar with this, or does anyone have an idea as to what might be causing this?

2

Julia’s standard output is buffered. This means the stuff is written out in large chunks. You can try to flush the stream with flush(stdout) to force it to output stuff. I think slurm does not buffer (at least on the cluster I use sometimes) so you should then see the message in the logfile. In principle you can flush everytime after printing something but bear in mind that this has a small performance penalty that can become noticeable if you log ALOT (but in that case you probably fill up the buffer rather quickly and will see the output with some delay).

5 Likes