Null bytes in repl_history on shared home directory

General Usage
1

I’m currently using julia on a network of computers with shared home directories. I’ve noticed that when I open REPLs on two machines simultaneously, the repl_history.jl file gets corrupted and I get this error message the next time I start a REPL

Click for full error message 
ERROR: Invalid history file (/home/lhupe/.julia/logs/repl_history.jl) format:
If you have a history file left over from an older version of Julia,
try renaming or deleting it.
Invalid character: '\0' at line 7
Stacktrace:
 [1] error(::String, ::String, ::String, ::Int64) at ./error.jl:42
 [2] hist_from_file(::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:434
 [3] setup_interface(::REPL.LineEditREPL, ::Bool, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:874
 [4] setup_interface(::REPL.LineEditREPL; hascolor::Bool, extra_repl_keymap::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:779
 [5] setup_interface at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:779 [inlined]
 [6] (::Pkg.var"#1#2")(::REPL.LineEditREPL) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Pkg.jl:519
 [7] __atreplinit(::REPL.LineEditREPL) at ./client.jl:339
 [8] #invokelatest#1 at ./essentials.jl:712 [inlined]
 [9] invokelatest at ./essentials.jl:711 [inlined]
 [10] _atreplinit at ./client.jl:346 [inlined]
 [11] (::Base.var"#764#766"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:382
 [12] #invokelatest#1 at ./essentials.jl:712 [inlined]
 [13] invokelatest at ./essentials.jl:711 [inlined]
 [14] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:367
 [15] exec_options(::Base.JLOptions) at ./client.jl:305
 [16] _start() at ./client.jl:484

[ Info: Disabling history file for this session

This is apparently independent of what I do during the simulatenous REPL sessions; I can reproduce the bug by immediately closing the REPLs after opening them. As soon as there are two sessions running at the same time on different hosts, the history file gets corrupted.

Does anybody else have this problem or know how to fix it?

2 Likes
2

For a detailed example: https://github.com/JuliaLang/julia/issues/36707

3

here is a simple script to fix:

egrep  "^(#|\t)" ~/.julia/logs/repl_history.jl > ~/.julia/logs/repl_history_fix.jl
mv ~/.julia/logs/repl_history.jl ~/.julia/logs/repl_history_old.jl
mv ~/.julia/logs/repl_history_fix.jl ~/.julia/logs/repl_history.jl
1 Like
4

Thanks for your response!
Your workaround has one tiny issue: If grep (and thus also egrep) finds null bytes within the first few bytes of a file, it will interpret it as a binary file (and thus not function as expected). In order to prevent this, you can force interpreting the file as text using the -a flag.
Also – on my system – for some reason, egrep doesn’t match the tab characters (ASCII 0x09) (which is probably due to an error on my side, I don’t have time to check now). If anyone else comes accross this problem, I’d suggest changing \t to a more generic \s to match all whitespace (which strangely enough works for me)

Additionally, I have since come up with my own workaround, which I forgot to post here. Using sed, one can remove the null bytes in place

sed -i 's/\x0//g' ~/.julia/logs/repl_history.jl

Still, both our solutions are only crude workarounds. I often have REPLs running on multiple machines using the same NFS home directory, and cleaning up the REPL history every few hours slightly impairs my workflow.
However, there is hope – another issue on this problem was recently opened, and it seems to be less dead than mine.