Julia v1.3.0-rc4 is now available

ararslan · October 16, 2019, 2:00am

The fourth (and this time almost surely final) release candidate for Julia v1.3.0 is now available. You can get binaries at Download Julia for Linux (i686, x86-64, ARMv7, AArch64), Windows (32-, 64-bit), macOS, and FreeBSD (x86-64). Check out the NEWS file to see what will be new in 1.3.0 and see the (short) list of commits included since 1.3.0-rc3 here.

As a release candidate, this should not be considered production-ready; it’s intended to give users a chance to try out their code with 1.3.0 prior to a full release. Note that 1.3 on Travis, AppVeyor (with Appveyor.jl ), and Cirrus (with CirrusCI.jl ) now points to 1.3.0-rc4.

Let us know in the issue tracker if you run into any issues. Assuming no problems arise with this release candidate, we’ll move forward with a full 1.3.0 release soon.

Enjoy!

jstrube · October 17, 2019, 7:28am

I seem to be seeing a massive slowdown in code that calls C++ libraries (via CxxWrap) from rc-3 to rc-4. I’d need more tests to verify that this is indeed the new version, and not some other parts in the chain, but I’m seeing ~30% slowdown… My julia-only code did not see this slowdown…
What I’ve run has a lot of moving parts, though. @barche might have some better test cases handy.

fredrikekre · October 17, 2019, 7:50am

There are basically no changes between rc-3 and rc-4: https://github.com/JuliaLang/julia/compare/v1.3.0-rc3..v1.3.0-rc4 and in particular no changes that looks to worsen performance.

Juan · October 17, 2019, 11:21am

Any benchmark comparing v1.3 vs v1.2 vs v1.1 vs v1.0?

jstrube · October 18, 2019, 5:47am

Sure, why not. This isn’t reproducible at all, because the data isn’t public, but I’m running some simple code over a large data sample. But, I guess everybody likes benchmarks, so here we go:
julia-1.1rc1: 1080 events / s
julia-1.2rc2: 1368 events / s
julia-1.3alpha: 1349 events / s
julia-1.3rc3: 1158 events / s
julia-1.2.0: 1375 events / s
julia-1.3rc4: 789 events / s (just ran again: 852 events / s)

For comparison: With C++ I’m clocking 1707 events / s.

There is some variation due to I/O, but the drop from rc3 to rc4 is significant.

johnh · October 18, 2019, 9:10am

Would profiling the code help here?

Tamas_Papp · October 18, 2019, 9:15am

Can you bisect between the rc3 and the rc4 to pin down the exact commit which changed this?

dpsanders · October 18, 2019, 10:10am

Can you just use fake data to make a MWE?

ufechner7 · October 18, 2019, 1:26pm

I get strange segfaults with RC4 that I never had before, for example:

julia> include("local_tests/run_gen_queries2.jl")

signal (11): Segmentation fault
in expression starting at /home/ufechner/n4Server/local_tests/run_gen_queries2.jl:22
getindex at ./array.jl:742 [inlined]
macro expansion at /home/ufechner/.julia/packages/CSV/YgNdL/src/CSV.jl:486 [inlined]
macro expansion at ./simdloop.jl:77 [inlined]
macro expansion at /home/ufechner/.julia/packages/CSV/YgNdL/src/CSV.jl:485 [inlined]
#34 at ./threadingconstructs.jl:113
unknown function (ip: 0x7fca0ebe3c57)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2130 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2300
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1631 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:659
unknown function (ip: 0xffffffffffffffff)
Allocations: 53186854 (Pool: 53177422; Big: 9432); GC: 71
Segmentation fault (core dumped)
ufechner@TUD277255:~/n4Server$

Sometimes the code executes fine for a couple of times, but then I get this segault:

julia> include("local_tests/run_gen_queries2.jl")
ERROR: LoadError: ArgumentError: Reference array points beyond the end of the pool
Stacktrace:
 [1] PooledArrays.PooledArray(::PooledArrays.RefArray{Array{UInt32,1}}, ::Dict{String,UInt32}, ::Array{String,1}) at /home/ufechner/.julia/packages/PooledArrays/ufJSl/src/PooledArrays.jl:37
 [2] PooledArrays.PooledArray(::PooledArrays.RefArray{Array{UInt32,1}}, ::Dict{String,UInt32}) at /home/ufechner/.julia/packages/PooledArrays/ufJSl/src/PooledArrays.jl:36
 [3] copy(::CSV.Column{String,CSV.PooledString}) at /home/ufechner/.julia/packages/CSV/YgNdL/src/tables.jl:80
 [4] (::DataFrames.var"#DataFrame#91#94")(::Bool, ::Type{DataFrame}, ::Array{AbstractArray{T,1} where T,1}, ::DataFrames.Index) at /home/ufechner/.julia/packages/DataFrames/yH0f6/src/dataframe/dataframe.jl:130
 [5] Type at ./none:0 [inlined]
 [6] #fromcolumns#411(::Bool, ::typeof(DataFrames.fromcolumns), ::CSV.File) at /home/ufechner/.julia/packages/DataFrames/yH0f6/src/other/tables.jl:17
 [7] #fromcolumns at ./none:0 [inlined]
 [8] #DataFrame#412 at /home/ufechner/.julia/packages/DataFrames/yH0f6/src/other/tables.jl:32 [inlined]
 [9] DataFrame(::CSV.File) at /home/ufechner/.julia/packages/DataFrames/yH0f6/src/other/tables.jl:23
 [10] top-level scope at /home/ufechner/n4Server/local_tests/run_gen_queries2.jl:26
 [11] include at ./boot.jl:328 [inlined]
 [12] include_relative(::Module, ::String) at ./loading.jl:1105
 [13] include(::Module, ::String) at ./Base.jl:31
 [14] include(::String) at ./client.jl:432
 [15] top-level scope at REPL[1]:1
in expression starting at /home/ufechner/n4Server/local_tests/run_gen_queries2.jl:26

julia>

julia> versioninfo()
Julia Version 1.3.0-rc4.1
Commit 8c4656b97a (2019-10-15 14:08 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PKG3_PRECOMPILE = enable
  JULIA_NUM_THREADS = 4
  JULIA_EDITOR = gedit

(v1.3) pkg> status
    Status `~/.julia/environments/v1.3/Project.toml`
  [1520ce14] AbstractTrees v0.2.1
  [336ed68f] CSV v0.5.13
  [a93c6f00] DataFrames v0.19.4
  [1313f7d8] DataFramesMeta v0.5.0

(v1.3) pkg>

No problems with Julia 1.2.

jstrube · October 18, 2019, 2:42pm

I can try, but I won’t have much time to work on this over the next few days. The code doesn’t do much other than call the C++ library to read data, so it’s a bit puzzling…

I’ll try to get to it, but if somebody else would like to take a look, the package is LCIO#master, which needs CxxWrap#master. I can try to benchmark the test file of that package when I get back to my computer later today.

ufechner7 · October 18, 2019, 6:08pm

My current assumption: Is CSV using threads in 1.3 but not in Julia 1.2? This could explain the failure and it could be bug in Julia or in the CSV package.

barche · October 19, 2019, 7:45am

To check if this is linked to the calling of the functions itself, you can run the CxxWrap tests, and look for output like this:

Julia test:
  0.102698 seconds (128 allocations: 7.938 KiB)
  0.063683 seconds (4 allocations: 160 bytes)
  0.053773 seconds (4 allocations: 160 bytes)
C test:
  0.123567 seconds (135 allocations: 8.203 KiB)
  0.118030 seconds (4 allocations: 160 bytes)
  0.116455 seconds (4 allocations: 160 bytes)
C++ test:
  0.180980 seconds (7 allocations: 384 bytes)
  0.187293 seconds (4 allocations: 160 bytes)
  0.217453 seconds (4 allocations: 160 bytes)
C++ lambda test:
  2.567566 seconds (7 allocations: 384 bytes)
  2.514374 seconds (4 allocations: 160 bytes)
  2.470426 seconds (4 allocations: 160 bytes)

Here, Julia test is a pure Julia loop, C test is a loop using ccall directly, C++ test calls a C++ function pointer using ccall and C++ lambda passes through a C++ std::function (very slow here because I ran on a debug build).

jstrube · October 19, 2019, 4:03pm

I’m seeing a small difference in the C++ test, 0.188 s with rc4 vs. 0.177 s with rc3. Not enough to explain what I was seeing before.
Wrapping my package test cases in a @btime doesn’t show any difference between the two releases.
I’d have to benchmark my original script in pieces to find the culprit, but I won’t get to it for probably two weeks.

Tamas_Papp · October 20, 2019, 5:11am

Again, I would suggest bisecting. There are 11 commits between rc3 and rc4, so it would take about 4-5 runs to pin down the relevant commit.

Tero_Frondelius · October 20, 2019, 3:18pm

Travis log here: https://travis-ci.com/JuliaFEM/UMAT.jl/jobs/247565737

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x6b5f32b8 -- gc_try_setmark at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gc.c:1557 [inlined]
gc_mark_scan_objarray at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gc.c:1669 [inlined]
gc_mark_loop at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gc.c:1960

Only Windows 1.3 tests fails in Travis, all others passes.

xref: 1.3-RC4 Windows EXCEPTION_ACCESS_VIOLATION at 0x6b5f32b8 -- gc_try_setmark · Issue #33616 · JuliaLang/julia · GitHub

Tero_Frondelius · October 21, 2019, 6:06am

Do I need to open an issue or is the above sufficient bug report? Nightly build passes maybe the issue is already fixed at master.

Karajan · October 21, 2019, 6:58am

I think you should open an issue. That makes sure the right people see it and the fix gets backported. Otherwise you might have to wait for 1.4 for the fix.

Tero_Frondelius · October 21, 2019, 9:13am

Thanks. I did: https://github.com/JuliaLang/julia/issues/33616

yuyichao · October 21, 2019, 1:07pm

It’s very likely a user error, especially since there’s a warning for you.

Tero_Frondelius · October 21, 2019, 2:03pm

Do you mean this one:
WARNING: Method definition isotropic_elasticity_tensor(Any, Any) in module Materials at C:\Users\travis\.julia\packages\Materials\mSWeT\src\idealplastic.jl:5 overwritten at C:\Users\travis\.julia\packages\Materials\mSWeT\src\chaboche.jl:5.
I believe it’s unrelated, because it’s coming from different package. Of course, that should be fixed because it’s really annoying to wait Materials.jl building when running the tests.