Slowdown of Distributions.jl on 0.7?

General Usage Performance
1

I’m seeing a big slowdown in some code that uses Distributions.jl on 0.7 (built from master this morning), compared to 0.6.3. A MWE is

using Distributions

function main()
    a = zeros(100)
    y = rand(30)
    for i = 1:100
        d = Exponential(rand())
        a[i] = mean(logpdf.(d, y))
    end
end
main()

On 0.7 built from master today, the second run gives

julia> versioninfo()
Julia Version 0.7.0-beta.205
Commit e8dd4119b9 (2018-07-08 08:59 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, broadwell)

julia> @time include("TryIt.jl")
  **2.087699** seconds (84.38 k allocations: 4.667 MiB, 0.50% gc time)

While on 0.6.3, the second run gives

julia> versioninfo()
Julia Version 0.6.3
Commit d55cadc350 (2018-05-28 20:20 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

julia> @time include("TryIt.jl")
  **0.145669** seconds (48.17 k allocations: 2.572 MiB)

julia>

So, there is a tremendous slow-down. Is this the closure problem, perhaps? If so, how do I write the code properly?

I know that Distributions.jl was recently made to work on 0.7, and perhaps this is a known problem? If not, I can file an issue.

2

A post was split to a new topic: Arpack.jl install error with MKL

3

I think you are running into a silent deprecation warning. Use --depwarn=yes to get

ERROR: broadcast will default to iterating over its arguments in the future. Wrap arguments of
type `x::Exponential{Float64}` with `Ref(x)` to ensure they broadcast as "scalar" elements.

and then fixing as

        a[i] = mean(logpdf.(Ref(d), y))

yields

julia> @benchmark main()
BenchmarkTools.Trial: 
  memory estimate:  34.02 KiB
  allocs estimate:  102
  --------------
  minimum time:     33.536 μs (0.00% GC)
  median time:      34.602 μs (0.00% GC)
  mean time:        43.728 μs (13.85% GC)
  maximum time:     42.665 ms (99.88% GC)
  --------------
  samples:          10000
  evals/sample:     1

julia> VERSION
v"0.7.0-beta.204"

vs

julia> @benchmark main()
BenchmarkTools.Trial: 
  memory estimate:  34.02 KiB
  allocs estimate:  102
  --------------
  minimum time:     52.695 μs (0.00% GC)
  median time:      53.716 μs (0.00% GC)
  mean time:        60.395 μs (1.65% GC)
  maximum time:     837.051 μs (87.87% GC)
  --------------
  samples:          10000
  evals/sample:     1

julia> VERSION
v"0.6.3"

which is a reasonable improvement, demoing v0.7 awesomeness.

4

BTW, I wonder why these deprecation warnings are not showing by default.

5

Great! Thanks! I agree that 0.7 is awesome, and I’m really happy to see that this only slowdown I’ve encountered has a simple solution.

6

This is a good question. I wonder if it’s a big in the deprecation system. Was this deprecation perhaps shown earlier in the same session?

7

I believe that I did see the warning at the beginning of the process, but not while I was trying to figure out what to do.

8

I can confirm that it is shown, I may have missed it because of other warnings previously.

9

Ok, then this is all working as it should.