# Improving ccall speed for many calls

**URL:** <https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594>\
**Category:** General Usage\
**Tags:** question\
**Created:** [March 10, 2017, 4:35pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594 "2017-03-10T16:35:43Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmgnve](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmgnve/32/1906_2.png) [@jmgnve](https://discourse.julialang.org/u/jmgnve)\
**Post date:** [March 10, 2017, 4:35pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/1 "2017-03-10T16:35:43Z")

</div>

Hi!

I am calling a shared library (a model written in Fortran) using ccall. I need to call the model many thousand times. The code is quite fast, but I am wondering if there is a way to speed it up since the current setup is rather slow for some applications. Is the shared library loaded somehow every time I use ccall? If so, does that take extra time and requires memory allocations that can be avoided somehow, some sort of preloading perhaps?

If it helps to answer my question, you find the code of the shared library [here](https://github.com/jmgnve/JFSM.jl/tree/master/deps). The Fortran subroutine I am calling is called FSM.f90. This [file](https://github.com/jmgnve/JFSM.jl/blob/master/src/run_fsm.jl) containes the Julia code where I use ccall. Obviously, you find my whole Julia package [here](https://github.com/jmgnve/JFSM.jl).

I hope to convince my collegues that Julia is the way to go, but then I need to improve the speed of this code a bit since it is currently not much faster than the current Matlab setup. Any help greatly appreciated 🙂  
Jan

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 10, 2017, 5:09pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/2 "2017-03-10T17:09:50Z")

</div>

> [@jmgnve](#):
>
> Is the shared library loaded somehow every time I use ccall

No.

If the code is mostly calling C code I wouldn’t expect to be faster than matlab code assuming both uses the same C library. It’ll be much easier to write (from scratch) though.

---

<div class="post-metadata">

**Author:** ![jmgnve](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmgnve/32/1906_2.png) [@jmgnve](https://discourse.julialang.org/u/jmgnve)\
**Post date:** [March 10, 2017, 5:41pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/3 "2017-03-10T17:41:51Z")

</div>

Thanks for the answer.

For the Matlab setup, I am passing data by writing and reading binary files since it is more difficult to call shared libraries in Matlab than in Julia. However, in Matlab I run the model for more time steps and grid points at once. Thus, in Matlab I am passing much larger chunks of data back and fourth for each call than in Julia, where I run the model for one time step and grid point in each call. The Fortran code is kind of big and contains quite a large number of constants etc that needs to be initialized. Does it take time to initialize the shared library each time it is called? Would you expect that the Julia version gets faster if I pass larger chunks of data at once, and reduce the number of calls instead?

---

<div class="post-metadata">

**Author:** ![tknopp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tknopp/32/3569_2.png) [@tknopp](https://discourse.julialang.org/u/tknopp)\
**Post date:** [March 10, 2017, 6:02pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/4 "2017-03-10T18:02:42Z")

</div>

no it does not take any initializing time if you call the shared library. `ccall` has negligible overhead and thus can even be called for scalar operations.

Why do you pass different chunk sizes from Matlab and Julia?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 10, 2017, 6:59pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/5 "2017-03-10T18:59:13Z")

</div>

> [@tknopp](#):
>
> Why do you pass different chunk sizes from Matlab and Julia?

I’m not sure it’ll really matter though, given that ccall’s overhead is so low.

I agree with:

> [@yuyichao](#):
>
> If the code is mostly calling C code I wouldn’t expect to be faster than matlab code assuming both uses the same C library. It’ll be much easier to write (from scratch) though.

I would suspect that with a large enough batch size, MATLAB is likely close to “optimal”. I took at look at your code and noticed that it’s pretty much all calling out to Fortran. I wouldn’t be surprised if the majority of the time is sitting in that Fortran code, and in that case both the MATLAB code and the Julia code should run for about the same time.

But yeah, writing a `ccall` is much easier than writing some MEX glue 🙂.

Julia’s speed vs MATLAB will really shine when you are writing the code in MATLAB. If you translate that Fortran code to Julia, I would suspect that you’d have a \<2x difference in runtime, and you’d get things like compatibility arbitrary precision numbers almost for free. I don’t think either of those would be true for MATLAB.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [March 11, 2017, 10:49am UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/6 "2017-03-11T10:49:02Z")

</div>

I test this overhead in CxxWrap, by dividing an array of 50M float64 numbers by 2 in a loop. Comparing pure julia and having a function ccalled for each number I see a factor 2 slowdown, which is good considering how little work this function does:

```julia
Timings:
Pure Julia test:
  0.061723 seconds (4 allocations: 160 bytes)
ccall test:
  0.092434 seconds (4 allocations: 160 bytes)

```

So in most cases the overhead should only be an issue if the called function is almost trivial, and then it usually is easy to rewrite in Julia.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 11, 2017, 7:07pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/7 "2017-03-11T19:07:20Z")

</div>

The benefit of inlining is AFAIU often not avoiding the overhead in the function calls but the extra optimization opportunities available to the compiler.

---

<div class="post-metadata">

**Author:** ![jmgnve](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmgnve/32/1906_2.png) [@jmgnve](https://discourse.julialang.org/u/jmgnve)\
**Post date:** [March 16, 2017, 5:02pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/8 "2017-03-16T17:02:51Z")

</div>

Thanks for all replies, and sorry for my rather late answer. Too busy.

I made some tests calling different Fortran subroutines that multiply a number or a vector by 2. You find details about those tests [here](https://github.com/jmgnve/ccall_test.jl/blob/master/README.md).

The overhead for using ccall and Fortran is approximately a factor of 2 compared to using a pure Julia function. So I am convinced, the overhead of ccall is small as you all pointed out above. One big plus point for Julia 🙂.

To tknopp: You asked “Why do you pass different chunk sizes from Matlab and Julia?” I did it because it takes less time to write a certain amount of data to one file, instead of writing the same amount of data to many files. In Julia, I do not pass data between Fortran and Julia using files, so there it does not matter.

Thanks again for all replies. Great community.  
Jan

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 16, 2017, 6:28pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/9 "2017-03-16T18:28:32Z")

</div>

Did you run the functions twice? The first time will include compilation time. I would expect multiplying a number by 2 to be almost instant after the first call.

---

<div class="post-metadata">

**Author:** ![jmgnve](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmgnve/32/1906_2.png) [@jmgnve](https://discourse.julialang.org/u/jmgnve)\
**Post date:** [March 17, 2017, 3:27pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/10 "2017-03-17T15:27:04Z")

</div>

Yes, I ran the functions twice, but I did not multiply only a single number by 2, but a vector with 1000000 entries. So that is why the results do not appear instantaneously I think. I guess you already noticed my link to the Julia package with my tests including the readme in my last post, but if not you find them here:

[https://github.com/jmgnve/ccall\_test.jl](https://github.com/jmgnve/ccall_test.jl)

---

<div class="post-metadata">

**Author:** ![m-j-w](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/m-j-w/32/4541_2.png) [@m-j-w](https://discourse.julialang.org/u/m-j-w)\
**Post date:** [March 17, 2017, 3:44pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/11 "2017-03-17T15:44:57Z")

</div>

You should add `-march=native` to the gfortran compile flags in your `deps/build.jl`. This allows gfortran to emit SIMD vectorized code that fits your CPU architecture. (`-O3` does not imply CPU architecture settings.) Otherwise it’ll default to the slower scalar code. Most likely then the speed difference goes away since Julia instructs LLVM by default to same ‘native’ setting when optimisation is enabled (as is by default).

To see the difference you may use `objdump -d array_routine.so` to show the disassembled code, which is essentially the same you would see from `@code_native` in Julia.

Also, `-O3` can lead to suboptimal code when the optimiser doesn’t have enough information on how functions are exactly used and, ultimately, makes too aggressive assumptions. This is in particular a problem when writing shared libraries that iterate over arrays. So sometimes `-O2` gives faster code. Be sure to check both settings.

**Addendum:**  
To clarify, the above is with regard to the comment ‘The array function is much slower, perhaps due to memory allocation in the fortran subroutine.’  
So, here’s the disassembly of the Fortran code using `-O2` and `-O2 -march=native`:

```julia
0000000000000610 <array_routine_>: 0000000000000610 <array_routine_>:
 610: 48 8b 0a mov (%rdx),%rcx 610: 48 8b 0a mov (%rdx),%rcx
 613: b8 01 00 00 00 mov $0x1,%eax 613: b8 01 00 00 00 mov $0x1,%eax
 618: 48 85 c9 test %rcx,%rcx 618: 48 85 c9 test %rcx,%rcx
 61b: 48 8d 51 01 lea 0x1(%rcx),%rdx 61b: 48 8d 51 01 lea 0x1(%rcx),%rdx
 61f: 7e 20 jle 641 <array_routine_+0x 61f: 7e 20 jle 641 <array_routine_+0x
 621: 0f 1f 80 00 00 00 00 nopl 0x0(%rax) 621: 0f 1f 80 00 00 00 00 nopl 0x0(%rax)
 628: f2 0f 10 44 c7 f8 movsd -0x8(%rdi,%rax,8),%xmm | 628: c5 fb 10 44 c7 f8 vmovsd -0x8(%rdi,%rax,8),%xmm
 62e: f2 0f 58 c0 addsd %xmm0,%xmm0 | 62e: c5 fb 58 c0 vaddsd %xmm0,%xmm0,%xmm0
 632: f2 0f 11 44 c6 f8 movsd %xmm0,-0x8(%rsi,%rax,8 | 632: c5 fb 11 44 c6 f8 vmovsd %xmm0,-0x8(%rsi,%rax,8
 638: 48 83 c0 01 add $0x1,%rax 638: 48 83 c0 01 add $0x1,%rax
 63c: 48 39 d0 cmp %rdx,%rax 63c: 48 39 d0 cmp %rdx,%rax
 63f: 75 e7 jne 628 <array_routine_+0x 63f: 75 e7 jne 628 <array_routine_+0x
 641: f3 c3 repz retq 641: f3 c3 repz retq

```

And it appears the `-O3 -march=native` is actually significantly faster on an AVX capable machine.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [March 17, 2017, 3:51pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/12 "2017-03-17T15:51:25Z")

</div>

These results (the factor of 2) are completely in line with the benchmark I do in CxxWrap.jl, which is pretty much identical except that I divide by 2 🙂 A factor 2 for such a simple function is very good, on par I think with calling a non-inlinable function on each array element in C.

---

<div class="post-metadata">

**Author:** ![jmgnve](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmgnve/32/1906_2.png) [@jmgnve](https://discourse.julialang.org/u/jmgnve)\
**Post date:** [March 17, 2017, 4:41pm UTC](https://discourse.julialang.org/t/improving-ccall-speed-for-many-calls/2594/13 "2017-03-17T16:41:13Z")

</div>

Thanks for all tips. I tried the different compiler settings on my machine. That doesn´t seem to affect the execution time very much. The ccall options are approximately a factor 2 slower than pure Julia. This slowdown is okay for my applications.