Testing in Docker: Base.runtests("file") failed 🐳

ImreSamu · December 14, 2021, 7:33am

Question:

How can I run : Base.runtests("file") inside docker environments?
Any best practice?

My test command is failed - with the docker official Julia image.
docker run -it --rm --privileged julia:1.7 julia -e 'Base.runtests("file")'

log:

...
Test Failed at /usr/local/julia/share/julia/test/file.jl:252
  Expression: isfile(t)
Test Failed at /usr/local/julia/share/julia/test/file.jl:253
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Test Failed at /usr/local/julia/share/julia/test/file.jl:262
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Test Failed at /usr/local/julia/share/julia/test/file.jl:276
  Expression: isdir(t)
...
Test Summary: | Pass  Fail  Error  Total
  Overall     | 1038    10      2   1050
    file      | 1038    10      2   1050
    FAILURE

Full-error-log-julia:1.7

$ docker run -it --rm  --privileged julia:1.7 julia -e 'Base.runtests("file")' 
Test  (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
file       (1) |        started at 2021-12-14T07:07:36.641
file       (1) |         failed at 2021-12-14T07:07:49.484
Test Failed at /usr/local/julia/share/julia/test/file.jl:252
  Expression: isfile(t)
Test Failed at /usr/local/julia/share/julia/test/file.jl:253
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Test Failed at /usr/local/julia/share/julia/test/file.jl:262
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Test Failed at /usr/local/julia/share/julia/test/file.jl:276
  Expression: isdir(t)
Test Failed at /usr/local/julia/share/julia/test/file.jl:277
  Expression: length(TEMP_CLEANUP) == 2
   Evaluated: 0 == 2
Test Failed at /usr/local/julia/share/julia/test/file.jl:283
  Expression: length(TEMP_CLEANUP) == 3
   Evaluated: 1 == 3
Test Failed at /usr/local/julia/share/julia/test/file.jl:287
  Expression: ispath(t)
Test Failed at /usr/local/julia/share/julia/test/file.jl:287
  Expression: ispath(t)
Test Failed at /usr/local/julia/share/julia/test/file.jl:293
  Expression: TEMP_CLEANUP_MAX[] == 4
   Evaluated: 3 == 4
Error During Test at /usr/local/julia/share/julia/test/file.jl:399
  Got exception outside of a @test
  TypeError: in typeassert, expected Base.IOError, got a value of type Bool
  Stacktrace:
    [1] test_stat_error(stat::typeof(stat), pth::String)
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:393
    [2] (::Main.Test32Main_file.var"#33#35"{String})()
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:417
    [3] cd(f::Main.Test32Main_file.var"#33#35"{String}, dir::String)
      @ Base.Filesystem ./file.jl:110
    [4] #32
      @ /usr/local/julia/share/julia/test/file.jl:401 [inlined]
    [5] mktempdir(fn::Main.Test32Main_file.var"#32#34", parent::String; prefix::String)
      @ Base.Filesystem ./file.jl:750
    [6] mktempdir(fn::Function, parent::String) (repeats 2 times)
      @ Base.Filesystem ./file.jl:748
    [7] macro expansion
      @ /usr/local/julia/share/julia/test/file.jl:400 [inlined]
    [8] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
    [9] top-level scope
      @ /usr/local/julia/share/julia/test/file.jl:400
   [10] include
      @ ./Base.jl:418 [inlined]
   [11] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:24 [inlined]
   [12] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
   [13] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:23 [inlined]
   [14] macro expansion
      @ ./timing.jl:380 [inlined]
   [15] runtests(name::String, path::String, isolate::Bool; seed::UInt128)
      @ Main /usr/local/julia/share/julia/test/testdefs.jl:21
   [16] #149
      @ /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:412 [inlined]
   [17] run_work_thunk(thunk::Distributed.var"#149#150"{typeof(runtests), Tuple{String, String}, Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}}}, print_error::Bool)
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
   [18] remotecall_fetch(::Function, ::Distributed.LocalProcess, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:437
   [19] remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:479
   [20] macro expansion
      @ /usr/local/julia/share/julia/test/runtests.jl:241 [inlined]
   [21] (::var"#37#47"{Vector{Task}, var"#print_testworker_errored#43"{ReentrantLock, Int64, Int64}, var"#print_testworker_stats#41"{ReentrantLock, Int64, Int64, Int64, Int64, Int64, Int64}, Vector{Any}, Dict{String, DateTime}})()
      @ Main ./task.jl:423
Test Failed at /usr/local/julia/share/julia/test/file.jl:1503
  Expression: rm(joinpath(d, "nonempty"); recursive = true, force = true)
    Expected: Base.IOError
  No exception thrown
Error During Test at /usr/local/julia/share/julia/test/file.jl:1488
  Got exception outside of a @test
  IOError: chmod("/tmp/jl_VOKPwm/nonempty", 0o777): no such file or directory (ENOENT)
  Stacktrace:
    [1] uv_error
      @ ./libuv.jl:97 [inlined]
    [2] chmod(path::String, mode::UInt16; recursive::Bool)
      @ Base.Filesystem ./file.jl:1114
    [3] chmod
      @ ./file.jl:1113 [inlined]
    [4] (::Main.Test32Main_file.var"#88#89")(d::String)
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:1504
    [5] mktempdir(fn::Main.Test32Main_file.var"#88#89", parent::String; prefix::String)
      @ Base.Filesystem ./file.jl:750
    [6] mktempdir(fn::Function, parent::String) (repeats 2 times)
      @ Base.Filesystem ./file.jl:748
    [7] macro expansion
      @ /usr/local/julia/share/julia/test/file.jl:1489 [inlined]
    [8] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
    [9] top-level scope
      @ /usr/local/julia/share/julia/test/file.jl:1489
   [10] include
      @ ./Base.jl:418 [inlined]
   [11] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:24 [inlined]
   [12] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
   [13] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:23 [inlined]
   [14] macro expansion
      @ ./timing.jl:380 [inlined]
   [15] runtests(name::String, path::String, isolate::Bool; seed::UInt128)
      @ Main /usr/local/julia/share/julia/test/testdefs.jl:21
   [16] #149
      @ /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:412 [inlined]
   [17] run_work_thunk(thunk::Distributed.var"#149#150"{typeof(runtests), Tuple{String, String}, Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}}}, print_error::Bool)
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
   [18] remotecall_fetch(::Function, ::Distributed.LocalProcess, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:437
   [19] remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:479
   [20] macro expansion
      @ /usr/local/julia/share/julia/test/runtests.jl:241 [inlined]
   [21] (::var"#37#47"{Vector{Task}, var"#print_testworker_errored#43"{ReentrantLock, Int64, Int64}, var"#print_testworker_stats#41"{ReentrantLock, Int64, Int64, Int64, Int64, Int64, Int64}, Vector{Any}, Dict{String, DateTime}})()
      @ Main ./task.jl:423


Test Summary: | Pass  Fail  Error  Total
  Overall     | 1038    10      2   1050
    file      | 1038    10      2   1050
    FAILURE

The global RNG seed was 0x1127b6e8024fad925df4df717276048.

Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:252
  Expression: isfile(t)
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:253
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:262
  Expression: length(TEMP_CLEANUP) == 1
   Evaluated: 0 == 1
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:276
  Expression: isdir(t)
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:277
  Expression: length(TEMP_CLEANUP) == 2
   Evaluated: 0 == 2
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:283
  Expression: length(TEMP_CLEANUP) == 3
   Evaluated: 1 == 3
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:287
  Expression: ispath(t)
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:287
  Expression: ispath(t)
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:293
  Expression: TEMP_CLEANUP_MAX[] == 4
   Evaluated: 3 == 4
Error in testset file:
Error During Test at /usr/local/julia/share/julia/test/file.jl:399
  Got exception outside of a @test
  TypeError: in typeassert, expected Base.IOError, got a value of type Bool
  Stacktrace:
    [1] test_stat_error(stat::typeof(stat), pth::String)
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:393
    [2] (::Main.Test32Main_file.var"#33#35"{String})()
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:417
    [3] cd(f::Main.Test32Main_file.var"#33#35"{String}, dir::String)
      @ Base.Filesystem ./file.jl:110
    [4] #32
      @ /usr/local/julia/share/julia/test/file.jl:401 [inlined]
    [5] mktempdir(fn::Main.Test32Main_file.var"#32#34", parent::String; prefix::String)
      @ Base.Filesystem ./file.jl:750
    [6] mktempdir(fn::Function, parent::String) (repeats 2 times)
      @ Base.Filesystem ./file.jl:748
    [7] macro expansion
      @ /usr/local/julia/share/julia/test/file.jl:400 [inlined]
    [8] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
    [9] top-level scope
      @ /usr/local/julia/share/julia/test/file.jl:400
   [10] include
      @ ./Base.jl:418 [inlined]
   [11] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:24 [inlined]
   [12] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
   [13] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:23 [inlined]
   [14] macro expansion
      @ ./timing.jl:380 [inlined]
   [15] runtests(name::String, path::String, isolate::Bool; seed::UInt128)
      @ Main /usr/local/julia/share/julia/test/testdefs.jl:21
   [16] #149
      @ /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:412 [inlined]
   [17] run_work_thunk(thunk::Distributed.var"#149#150"{typeof(runtests), Tuple{String, String}, Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}}}, print_error::Bool)
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
   [18] remotecall_fetch(::Function, ::Distributed.LocalProcess, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:437
   [19] remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:479
   [20] macro expansion
      @ /usr/local/julia/share/julia/test/runtests.jl:241 [inlined]
   [21] (::var"#37#47"{Vector{Task}, var"#print_testworker_errored#43"{ReentrantLock, Int64, Int64}, var"#print_testworker_stats#41"{ReentrantLock, Int64, Int64, Int64, Int64, Int64, Int64}, Vector{Any}, Dict{String, DateTime}})()
      @ Main ./task.jl:423
Error in testset file:
Test Failed at /usr/local/julia/share/julia/test/file.jl:1503
  Expression: rm(joinpath(d, "nonempty"); recursive = true, force = true)
    Expected: Base.IOError
  No exception thrown
Error in testset file:
Error During Test at /usr/local/julia/share/julia/test/file.jl:1488
  Got exception outside of a @test
  IOError: chmod("/tmp/jl_VOKPwm/nonempty", 0o777): no such file or directory (ENOENT)
  Stacktrace:
    [1] uv_error
      @ ./libuv.jl:97 [inlined]
    [2] chmod(path::String, mode::UInt16; recursive::Bool)
      @ Base.Filesystem ./file.jl:1114
    [3] chmod
      @ ./file.jl:1113 [inlined]
    [4] (::Main.Test32Main_file.var"#88#89")(d::String)
      @ Main.Test32Main_file /usr/local/julia/share/julia/test/file.jl:1504
    [5] mktempdir(fn::Main.Test32Main_file.var"#88#89", parent::String; prefix::String)
      @ Base.Filesystem ./file.jl:750
    [6] mktempdir(fn::Function, parent::String) (repeats 2 times)
      @ Base.Filesystem ./file.jl:748
    [7] macro expansion
      @ /usr/local/julia/share/julia/test/file.jl:1489 [inlined]
    [8] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
    [9] top-level scope
      @ /usr/local/julia/share/julia/test/file.jl:1489
   [10] include
      @ ./Base.jl:418 [inlined]
   [11] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:24 [inlined]
   [12] macro expansion
      @ /usr/local/julia/share/julia/stdlib/v1.7/Test/src/Test.jl:1283 [inlined]
   [13] macro expansion
      @ /usr/local/julia/share/julia/test/testdefs.jl:23 [inlined]
   [14] macro expansion
      @ ./timing.jl:380 [inlined]
   [15] runtests(name::String, path::String, isolate::Bool; seed::UInt128)
      @ Main /usr/local/julia/share/julia/test/testdefs.jl:21
   [16] #149
      @ /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:412 [inlined]
   [17] run_work_thunk(thunk::Distributed.var"#149#150"{typeof(runtests), Tuple{String, String}, Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}}}, print_error::Bool)
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/process_messages.jl:63
   [18] remotecall_fetch(::Function, ::Distributed.LocalProcess, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:437
   [19] remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, UInt128, Tuple{Symbol}, NamedTuple{(:seed,), Tuple{UInt128}}})
      @ Distributed /usr/local/julia/share/julia/stdlib/v1.7/Distributed/src/remotecall.jl:479
   [20] macro expansion
      @ /usr/local/julia/share/julia/test/runtests.jl:241 [inlined]
   [21] (::var"#37#47"{Vector{Task}, var"#print_testworker_errored#43"{ReentrantLock, Int64, Int64}, var"#print_testworker_stats#41"{ReentrantLock, Int64, Int64, Int64, Int64, Int64, Int64}, Vector{Any}, Dict{String, DateTime}})()
      @ Main ./task.jl:423
ERROR: LoadError: Test run finished with errors
in expression starting at /usr/local/julia/share/julia/test/runtests.jl:93
ERROR: A test has failed. Please submit a bug report (https://github.com/JuliaLang/julia/issues)
including error messages above and the output of versioninfo():
Julia Version 1.7.0
Commit 3bf9d17731 (2021-11-30 12:12 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PATH = /usr/local/julia
  JULIA_GPG = 3673DF529D9049477F76B37566E3C7DC03D6E495
  JULIA_VERSION = 1.7.0

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] runtests(tests::String; ncores::Int64, exit_on_error::Bool, revise::Bool, seed::Nothing)
   @ Base ./util.jl:632
 [3] runtests(tests::String)
   @ Base ./util.jl:617
 [4] top-level scope
   @ none:1

caused by: failed process: Process(setenv(`/usr/local/julia/bin/julia -Cnative -J/usr/local/julia/lib/julia/sys.so -g1 /usr/local/julia/bin/../share/julia/test/runtests.jl file`,["PATH=/usr/local/julia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "JULIA_VERSION=1.7.0", "JULIA_CPU_THREADS=4", "JULIA_PATH=/usr/local/julia", "TERM=xterm", "JULIA_GPG=3673DF529D9049477F76B37566E3C7DC03D6E495", "HOME=/root", "OPENBLAS_MAIN_FREE=1", "HOSTNAME=c99dfb853f88"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:531 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:446
 [3] run
   @ ./process.jl:444 [inlined]
 [4] runtests(tests::String; ncores::Int64, exit_on_error::Bool, revise::Bool, seed::Nothing)
   @ Base ./util.jl:626
 [5] runtests(tests::String)
   @ Base ./util.jl:617
 [6] top-level scope

My system:

Ubuntu 20.04.3 LTS (Focal Fossa)
Docker version 20.10.12, build e91ed57
julia:1.7 - is the official docker julia images
- https://github.com/docker-library/julia

$ docker images julia:1.7
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
julia        1.7       71e8ac9bcddb   10 days ago   524MB

other info:

replicated in other 2 ubuntu system.
docker run -it --rm --privileged julia:1.2 julia -e 'Base.runtests("file")' → is OK

ImreSamu · December 16, 2021, 12:15pm

Ouch …
Lesson learned … the testing is not working with “root” user!
in this case … Add a normal user for testing!

FROM julia:1.7
RUN  useradd --create-home juliauser
USER juliauser

$ docker run -it --rm juliauser17 julia -e 'Base.runtests("file")'
Test  (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
file       (1) |        started at 2021-12-16T12:08:56.250
file       (1) |    16.49 |   0.13 |  0.8 |     572.39 |   343.12

Test Summary: | Pass  Total
  Overall     | 1061   1061
    SUCCESS