# Performance regression of Cuba.jl on Julia 0.6

**URL:** <https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424>\
**Category:** General Usage\
**Created:** [March 3, 2017, 1:20pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424 "2017-03-03T13:20:17Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 3, 2017, 1:20pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/1 "2017-03-03T13:20:17Z")

</div>

I noticed a performance regression of [`Cuba.jl`](https://github.com/giordano/Cuba.jl) on Julia 0.6, compared to Julia 0.5. It can be reproduced with these simple examples

Julia 0.5:

```julia
julia> using Cuba, BenchmarkTools

julia> @benchmark cuhre((x,f) -> f[1] = x[1])
BenchmarkTools.Trial: 
  memory estimate: 39.98 KiB
  allocs estimate: 982
  --------------
  minimum time: 25.191 μs (0.00% GC)
  median time: 26.890 μs (0.00% GC)
  mean time: 31.741 μs (11.10% GC)
  maximum time: 2.149 ms (97.00% GC)
  --------------
  samples: 10000
  evals/sample: 1
  time tolerance: 5.00%
  memory tolerance: 1.00%

```

Julia 0.6:

```julia
julia> using Cuba, BenchmarkTools

julia> @benchmark cuhre((x,f) -> f[1] = x[1])
BenchmarkTools.Trial: 
  memory estimate: 39.98 KiB
  allocs estimate: 982
  --------------
  minimum time: 32.348 μs (0.00% GC)
  median time: 35.141 μs (0.00% GC)
  mean time: 42.440 μs (10.04% GC)
  maximum time: 3.181 ms (94.20% GC)
  --------------
  samples: 10000
  evals/sample: 1
  time tolerance: 5.00%
  memory tolerance: 1.00%

```

On Julia 0.6 it’s almost 30% slower than in previous version. Any clue of what could be the culprit?

Before, Cuba.jl was really competitive with equivalent C/Fortran code (see [https://github.com/giordano/Cuba.jl#performance](https://github.com/giordano/Cuba.jl#performance)), but now it’s sensibly slower

**Edit** : contrary to what was written in the original version of the post, it’s not necessary to checkout master to reproduce the issue.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 12:08am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/2 "2017-03-04T00:08:39Z")

</div>

**Update** I bisected some 600 commits, with a possible [culprit](https://github.com/JuliaLang/julia/issues/17057) in mind.

On commit [f27c6f3ae50b45e0e6ff2305dd5031d07c8665a7](https://github.com/JuliaLang/julia/commit/f27c6f3ae50b45e0e6ff2305dd5031d07c8665a7) (one of the two parents of merge commit of [https://github.com/JuliaLang/julia/issues/17057](https://github.com/JuliaLang/julia/issues/17057)) performance of `Cuba.jl` was still fine, the same as in Julia 0.5. After that revision, it has been impossible to use the package for some time, because of the problem with world age. The first revision on which `Cuba.jl` became usable is [bfd9c7ab805f38298a04f6fd74e6c62fadb2494c](https://github.com/JuliaLang/julia/commit/bfd9c7ab805f38298a04f6fd74e6c62fadb2494c) (part of [https://github.com/JuliaLang/julia/pull/20167](https://github.com/JuliaLang/julia/pull/20167)).

So the problem seems to lay somewhere between

- [44d76771999d6a8ac23ee09d37f4a5f812010844](https://github.com/JuliaLang/julia/commit/44d76771999d6a8ac23ee09d37f4a5f812010844)
- and [bfd9c7ab805f38298a04f6fd74e6c62fadb2494c](https://github.com/JuliaLang/julia/commit/bfd9c7ab805f38298a04f6fd74e6c62fadb2494c)

bounds included.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 4, 2017, 12:28am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/3 "2017-03-04T00:28:47Z")

</div>

If it was slower after the world age change, maybe it’s due to the cfunction change. I tested the case I care about to make sure it doesn’t slow down but maybe you are hitting the slow path. Can you do a `@profile` with `C=true` in the printing?

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 4, 2017, 12:30am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/4 "2017-03-04T00:30:08Z")

</div>

I just took a quick look at the differences in the number of hits on each line with `@profile`. One line that stood out was [`.../Cuba/src/Cuba.jl:101; generic_integrand!(::Int32, ::Ptr{Float64}, ::Int32, ::Ptr{Float64}, ::Ptr{Void})`](https://github.com/giordano/Cuba.jl/blob/3766a5218a59d3ff653c773c4b88b8044ac4129d/src/Cuba.jl#L101). On 0.5, that line accounts for 7% of the profiling hits, whereas it accounts for nearly 20% on 0.6.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 12:35am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/5 "2017-03-04T00:35:03Z")

</div>

I faithfully implemented the callback as suggested here: [Passing Julia Callback Functions to C](http://julialang.org/blog/2013/05/callback) Maybe now something better should be done?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 12:37am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/6 "2017-03-04T00:37:00Z")

</div>

Uhm, removing `::Function` from that line makes the regression go away. Does it make sense?

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 4, 2017, 1:34am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/7 "2017-03-04T01:34:03Z")

</div>

That’s beyond my pay-grade, but I do know that the subtyping calculation got a little more expensive with the new type system. I think that landed within the commit range you’ve identified.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 4, 2017, 1:37am UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/8 "2017-03-04T01:37:02Z")

</div>

You can also use [https://github.com/stevengj/Cubature.jl/pull/23/files#diff-b34c7d94237ad3d15683f8eccabb72b4R116](https://github.com/stevengj/Cubature.jl/pull/23/files#diff-b34c7d94237ad3d15683f8eccabb72b4R116) to speed it up.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 2:16pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/9 "2017-03-04T14:16:33Z")

</div>

Ok, for the time being I simply removed type annotation of the function, and performance is now really exciting (see update [performance section](https://github.com/giordano/Cuba.jl#performance) of the `README.md`. Maybe the blog post on [julialang.org](http://julialang.org) should be updated (cc @stevengj)? The type annotation is also there.

> [@yuyichao](#):
>
> You can also use [Fix for 0.6 by yuyichao · Pull Request #23 · JuliaMath/Cubature.jl · GitHub](https://github.com/stevengj/Cubature.jl/pull/23/files#diff-b34c7d94237ad3d15683f8eccabb72b4R116) to speed it up.

I saw that PR (thinking that it could have been useful for `Cuba.jl` as well), but honestly I had a hard time understanding how it achieves what’s described in the commit message. Could you please give me a hint?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 4, 2017, 2:49pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/10 "2017-03-04T14:49:49Z")

</div>

Instead of using

```julia
function callback(ptr::Ptr{Void})
    unsafe_pointer_to_objref(ptr)()
end
ccall(cfunction(callback, Void, (Ptr{Void},)), Void, (Any,), f)

```

Use

```julia
function callback(f)
    f()
end
ccall(cfunction(callback, Void, (Ref{typeof(f)},)), Void, (Any,), f)

```

instead.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 3:30pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/11 "2017-03-04T15:30:49Z")

</div>

You mean this:  
[https://github.com/giordano/Cuba.jl/commit/1552d06ea55b626b5493f07353681034cc85a938](https://github.com/giordano/Cuba.jl/commit/1552d06ea55b626b5493f07353681034cc85a938)  
?

That’s amazing, now the `cuhre` benchmark is ~10% faster than Fortran!

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 4, 2017, 3:33pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/12 "2017-03-04T15:33:45Z")

</div>

That’s about right. You can also replace the `::Function` in the function signature by a type parameter to force specialization on that. It should improve performance on `cfunction` construction but that shouldn’t be called in the loop so it may not matter much.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 4, 2017, 3:49pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/13 "2017-03-04T15:49:28Z")

</div>

> [@yuyichao](#):
>
> That’s about right. You can also replace the `::Function` in the function signature by a type parameter to force specialization on that. It should improve performance on `cfunction` construction but that shouldn’t be called in the loop so it may not matter much.

I should replace

```julia
integrand_ptr(integrand::Function) = ...

```

with

```julia
integrand_ptr{T}(integrand::T) = ...

```

right? Only this? I confirm it doesn’t change much.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 4, 2017, 3:50pm UTC](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424/14 "2017-03-04T15:50:32Z")

</div>

Also on the function that calls `integrand_ptr`. Assuming this is constructed once and then the same pointer is used in the loop it shouldn’t matter too much.