Debugging saving error from tagsave DrWatson.jl

Michaela_Maskova
Tue, 16 Mar 2021 13:30:06 +0000
General Usage

Hi, I’m having trouble understanding an error happening when running a script. It is part of a larger project, therefore it’s kinda hard to add an example. But what happens is that I run a model, train it and try to save a lot of information about it using the function tagsave from DrWatson.jl package.

The error happens when running tagsave(savef, result, safe = true) where savef is the path of the file being saved, result is dictionary with results.

The error message looks like this:

┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID
└ @ DrWatson C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\saving_tools.jl:60
┌ Warning: `git` was not found in the current PATH, returning `nothing` instead of a patch.
└ @ DrWatson C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\saving_tools.jl:100
Error encountered while saving "C:\\Users\\masen\\Desktop\\GroupAD.jl\\data\\experiments/contamination-0.0\\vae_instance/MNIST/leave-one-out/class_index=10/seed=1\\activation=swish_aggregation=mean_batchsize=32_hdim=128_init_seed=76725879_lr=0.001_nlayers=4_score=reconstruction-mean_mean_zdim=8.bson".

Fatal error:
ERROR: LoadError: SystemError: opening file "C:\\Users\\masen\\Desktop\\GroupAD.jl\\data\\experiments/contamination-0.0\\vae_instance/MNIST/leave-one-out/class_index=10/seed=1\\activation=swish_aggregation=mean_batchsize=32_hdim=128_init_seed=76725879_lr=0.001_nlayers=4_score=reconstruction-mean_mean_zdim=8.bson": No such file or directory
Stacktrace:
 [1] handle_error(::SystemError, ::FileIO.File{FileIO.DataFormat{:BSON}}) at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\error_handling.jl:82
 [2] handle_exceptions(::Array{Any,1}, ::String) at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\error_handling.jl:77
 [3] save(::FileIO.Formatted, ::Any; options::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\loadsave.jl:210
 [4] save at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\loadsave.jl:193 [inlined]
 [5] #save#15 at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\loadsave.jl:118 [inlined]
 [6] save at C:\Users\masen\.julia\packages\FileIO\ZknoK\src\loadsave.jl:118 [inlined]
 [7] #_wsave#27 at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\DrWatson.jl:24 [inlined]
 [8] _wsave at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\DrWatson.jl:24 [inlined]
 [9] #wsave#28 at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\DrWatson.jl:35 [inlined]
 [10] wsave at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\DrWatson.jl:34 [inlined]
 [11] safesave(::String, ::Dict{Symbol,Any}) at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\saving_files.jl:142
 [12] tagsave(::String, ::Dict{Symbol,Any}; safe::Bool, gitpath::String, storepatch::Bool, force::Bool, source::Nothing) at C:\Users\masen\.julia\packages\DrWatson\u4mCx\src\saving_files.jl:88
 [13] experiment(::var"#7#11", ::NamedTuple{(:zdim, :hdim, :lr, :batchsize, :activation, :nlayers, :init_seed, :aggregation, :score),Tuple{Int64,Int64,Float32,Int64,String,Int64,Int64,String,String}}, ::Tuple{Tuple{Mill.BagNode{Mill.ArrayNode{Array{Float64,2},Nothing},Mill.AlignedBags,Nothing},Array{Float32,1}},Tuple{Mill.BagNode{Mill.ArrayNode{Array{Float64,2},Nothing},Mill.AlignedBags,Nothing},Array{Float32,1}},Tuple{Mill.BagNode{Mill.ArrayNode{Array{Float64,2},Nothing},Mill.AlignedBags,Nothing},Array{Float32,1}}}, ::String; verb::Bool, save_result::Bool, save_entries::Base.Iterators.Pairs{Symbol,Any,NTuple{7,Symbol},NamedTuple{(:fit_t, :history, :npars, :model, :modelname, :seed, :dataset),Tuple{Float64,Nothing,Int64,Nothing,String,Int64,String}}}) at C:\Users\masen\Desktop\GroupAD.jl\src\experiments.jl:33
 [14] point_cloud_experimental_loop(::typeof(sample_params), ::typeof(fit), ::typeof(edit_params), ::Int64, ::String, ::String, ::Float64, ::String, ::Int64, ::String) at C:\Users\masen\Desktop\GroupAD.jl\src\experiments_point_cloud.jl:77
 [15] top-level scope at C:\Users\masen\Desktop\GroupAD.jl\scripts\experiments_point_cloud\vae_instance.jl:145
 [16] include(::Function, ::Module, ::String) at .\Base.jl:380
 [17] include(::Module, ::String) at .\Base.jl:368
 [18] exec_options(::Base.JLOptions) at .\client.jl:296
 [19] _start() at .\client.jl:506
in expression starting at C:\Users\masen\Desktop\GroupAD.jl\scripts\experiments_point_cloud\vae_instance.jl:144

The dictionary I’m saving has 18 entries with a lot of data, so it’s hard to figure out where the error happens and what I should check. Do you have any ideas?

1 post - 1 participant