Error EBUSY on saving JLD2 file using Julia-1.3.0-rc2

alkoedl · September 27, 2019, 11:56am

I’m getting the following error when trying to save a file in JLD2 format on Julia-1.3.0-rc2:

$ C:/Users/blabla/AppData/Local/Julia-1.3.0-rc2/bin/julia.exe
[ Info: loading C:\Users\blabla\.julia\config\startup.jl
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.3.0-rc2.0 (2019-09-12)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.3) pkg> st
    Status `C:\Data\blabla\proj1\Project.toml`
  [5789e2e9] FileIO v1.0.7
  [033835bb] JLD2 v0.1.3

julia> using FileIO

julia> using JLD2

julia> jldopen("example.jld2", "w") do file
           file["bigdata"] = randn(5)
       end
ERROR: IOError: realpath: resource busy or locked (EBUSY)
Stacktrace:
 [1] uv_error at .\libuv.jl:97 [inlined]
 [2] realpath(::String) at .\path.jl:374
 [3] #jldopen#9(::Bool, ::Bool, ::typeof(jldopen), ::String, ::Bool, ::Bool, ::Bool, ::Type{JLD2.MmapIO}) at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\JLD2.jl:205
 [4] jldopen at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\JLD2.jl:203 [inlined] (repeats 2 times)
 [5] #jldopen#10(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(jldopen), ::String, ::String) at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\JLD2.jl:293
 [6] jldopen(::String, ::String) at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\JLD2.jl:288
 [7] #jldopen#33(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(jldopen), ::var"##7#8", ::String, ::Vararg{String,N} where N) at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\loadsave.jl:2
 [8] jldopen(::Function, ::String, ::String) at C:\Users\blabla\.julia\packages\JLD2\w2vgv\src\loadsave.jl:2
 [9] top-level scope at REPL[5]:1

The error does not occur on previous Julia versions up to (including) Julia-1.3.0-rc1. Any idea what has changed in rc2 that may cause this? Platform is Win7.

al

xiaodai · September 27, 2019, 12:48pm

Is that a DataFrame you are trying to save? If it is then try JDF.jl

alkoedl · September 27, 2019, 12:57pm

No, it’s not a DataFrame, it’s just a regular Array:

julia> typeof(randn(5))
Array{Float64,1}

nalimilan · September 27, 2019, 4:07pm

Does the problem happen with a fresh Julia session the first time you write to that file? Are you sure the file isn’t in use? It could be that Julia or another app fails to release a lock.

alkoedl · September 30, 2019, 9:57am

Yes, exactly. The problem happens with a fresh Julia session the first time I write to that file. The file is not in use otherwise. Due to the error, however, the file stays open and I cannot delete it from Explorer until the Julia session is closed. Moreover, the file is much larger than normally (512 kB vs 1 kB) and cannot be read by load("example.jld2").

FWIW, saving in text files such as

open("myfile.txt", "w") do io
          write(io, "Hello world!")
end

works as expected. Hence, I’m thinking whether something in JLD2/FileIO packages is not compatible with Julia-1.3.0-rc2 any longer.

Would be interesting if someone else can replicate this error.

xiaodai · September 30, 2019, 12:16pm

maybe you didn’t close the io using close(io)

alkoedl · September 30, 2019, 1:13pm

Thanks for the suggestion. But as far as I understand, the jldopen(...) do ... end construct should close the file automatically. Also note, that the code snippet works like a charm with all previous Julia versions. Only with 1.3.0-rc2 I’m getting the error. Unfortunately, I don’t see whether the problem lies most likely with JLD2 or FileIO or Julia-1.3.0-rc2 itself.

alkoedl · October 4, 2019, 9:55am

Some additional information:

The problem is still there with Julia-1.3.0-rc3. (The latest Julia version where the error does not occur is 1.3.0-rc1.)
It seems to be specific to Windows. Code snippet above works fine on macOS with any Julia version.
Package FileIO does not seem to be the problem. The error already occurs with only JLD2:

using JLD2
f = jldopen("example.jld2", "w")

produces the error message as in my initial post.

Paul_Soderlind · October 4, 2019, 1:32pm

using JLD2
f = jldopen("example.jld2", "w")

works fine on my Win 10 (64) computer with Julia-1.3.0-rc3 (it worked with rc2 too), provided I am in a folder where I have writing rights. In contrast, if I change over to eg. C:\ (which my user account is not allowed to write to) and try the same, then I get an error message.

alkoedl · October 4, 2019, 1:45pm

Thanks for checking. So the problem seems to be specific to Windows 7. The error I get when I don’t have permission to access the file (permission denied) is different from the error reported above ( resource busy or locked (EBUSY)).

Any ideas what changed in 1.3.0-rc2 with respect to file access that could make this error surface?

MikeErd · October 7, 2019, 7:20pm

Just to confirm the already posted behavior by a third person. The test has been done on two machines, to verify dependency on OS type.

Each with a completely fresh install of Julia-1.3.0-rc3.

-    Starting a new shell and activating local directory ( ]  + activate .).
-    ] + add JLD2
-    using JLD2
-    f = jldopen("example.jld2", "w")
     JLDFile D:\tmp\julia_test\example.jld2 (read/write)
     (no datasets)

On Windows 10 Home 64bit everything works as expected.

On Windows 7 Professional 64bit, the problem occurs as described already above.

ERROR: IOError: realpath: resource busy or locked (EBUSY)

alkoedl · October 18, 2019, 3:57pm

Thanks for the confirmation @MikeErd. For the sake of completeness, the problem is still there with Julia-1.3.0-rc4.

I opened an issue at JLD2.

I digged a little deeper into this issue and found that within jldopen("example.jld2", "w") a first call realpath("example.jld2") (JLD2.jl line 205) works just fine, but an identical second call realpath("example.jld2") (JLD2.jl line 232) yields Error EBUSY. In between the two calls an MmapIO object is initialized via io = openfile(JLD2.MmapIO, "example.jld2", true, true, true) which seems to succeed. Does anyone have an idea how the call to openfile() could change the way how realpath() works???

I can get the same behavior by running the three commands above directly in the REPL, i.e.

julia> using JLD2

julia> realpath("example.jld2")
"C:\\Data\\example.jld2"

julia> io = JLD2.openfile(JLD2.MmapIO, "example.jld2", true, true, true)
MmapIO

julia> realpath("example.jld2")
ERROR: IOError: realpath: resource busy or locked (EBUSY)
Stacktrace:
 [1] uv_error at .\libuv.jl:97 [inlined]
 [2] realpath(::String) at .\path.jl:374
 [3] top-level scope at none:0

The fact that the error does not occur with Julia versions < 1.3.0-rc2 seems to be somehow connected to (#33116) where the implementation of realpath changed considerably. Perhaps @musm or @fredrikekre know the details better and have an idea what’s going on here?

jcook · November 2, 2019, 7:32pm

I’m experiencing this problem too on v1.1 on my mac, having just updated to Catalina 10.15.1 (19B88).

julia> using JLD2

julia> @save "tmp.jld2" 0
ERROR: SystemError: realpath: No such file or directory
Stacktrace:
 [1] #systemerror#43(::Nothing, ::Function, ::Symbol, ::Bool) at ./error.jl:134
 [2] systemerror at ./error.jl:134 [inlined]
 [3] realpath(::String) at ./path.jl:367
 [4] #jldopen#9(::Bool, ::Bool, ::Function, ::String, ::Bool, ::Bool, ::Bool, ::Type{JLD2.MmapIO}) at /Users/james/.julia/packages/JLD2/KjBIK/src/JLD2.jl:205
 [5] jldopen at /Users/james/.julia/packages/JLD2/KjBIK/src/JLD2.jl:203 [inlined] (repeats 2 times)
 [6] #jldopen#10(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/james/.julia/packages/JLD2/KjBIK/src/JLD2.jl:293
 [7] jldopen(::String, ::String) at /Users/james/.julia/packages/JLD2/KjBIK/src/JLD2.jl:288
 [8] #jldopen#31(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(Main, Symbol("##3#4")), ::String, ::Vararg{String,N} where N) at /Users/james/.julia/packages/JLD2/KjBIK/src/loadsave.jl:2
 [9] jldopen(::Function, ::String, ::String) at /Users/james/.julia/packages/JLD2/KjBIK/src/loadsave.jl:2
 [10] top-level scope at /Users/james/.julia/packages/JLD2/KjBIK/src/loadsave.jl:48

But I can’t reproduce @alkoedl’s behaviour

julia> realpath("example.jld2")
ERROR: SystemError: realpath: No such file or directory
Stacktrace:
 [1] #systemerror#43(::Nothing, ::Function, ::Symbol, ::Bool) at ./error.jl:134
 [2] systemerror at ./error.jl:134 [inlined]
 [3] realpath(::String) at ./path.jl:367
 [4] top-level scope at none:0

shell> touch example.jld2

julia> realpath("example.jld2")
"<path>/example.jld2"

julia> io = JLD2.openfile(JLD2.MmapIO, "example.jld2", true, true, true)
MmapIO

julia> realpath("example.jld2")
"<path>/example.jld2"

Version info

Version 1.1.2-pre.0 (2019-05-17)
release-1.1/2aee2debbb (fork: 117 commits, 330 days)

thompsonmj · March 3, 2022, 7:07pm

I’m getting a similar issue:

save(joinpath("..","path","to","folder","dataframes.jld2"), "df_list", df_list)

ERROR: LoadError: IOError: realpath("..\\path\\to\\folder\\dataframes.jld2"): resource busy or locked (EBUSY)

when trying to save to a Box-synced directory.

The error does not occur writing to a non-cloud-synced location.

I also tried using mv() with Julia to move it from the non-Box location into the Box location and get

ERROR: LoadError: IOError: unlink("..\\data\\dataframes.jld2"): permission denied (EACCES)

Windows 10 Education version 20H2 OS build 19042.1526 and Julia 1.7.1

Topic		Replies	Views
IOError: unlink: resource busy or locked (EBUSY) in Windows General Usage	1	2878	December 9, 2019
Signal (7): Bus error New to Julia	0	846	January 21, 2021
Need data-storage package for 0.7 (JLD no longer working) General Usage	18	2360	August 8, 2018
JLD.jl vs JLD2.jl General Usage	23	8538	October 30, 2018
Error saving a .jld file New to Julia	2	891	February 6, 2020

Error EBUSY on saving JLD2 file using Julia-1.3.0-rc2

Related topics