# Benchmark for latest julia?

**URL:** <https://discourse.julialang.org/t/benchmark-for-latest-julia/5991>\
**Category:** Community\
**Tags:** question\
**Created:** [September 20, 2017, 2:47am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991 "2017-09-20T02:47:24Z")\
**Posts on this page:** 20\
**Page:** 5

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [June 21, 2018, 8:03pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/81 "2018-06-21T20:03:17Z")

</div>

@ScottPJones, @John_Gibson, Do you have updated results for 0.7 Alpha?

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [June 21, 2018, 8:32pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/82 "2018-06-21T20:32:39Z")

</div>

Yes, but I don’t plan to publish benchmark results until 0.7 finalizes. Last time I ran them the geometric mean relative to C for julia-0.7-DEV was 1.08, a little slower than julia-0.6’s 1.05. But maybe it’s worth rerunning and posting results here now that 0.7-alpha is out.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 21, 2018, 8:38pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/83 "2018-06-21T20:38:13Z")

</div>

Yep. I’m just benchmarking (string stuff) on v0.7 to see what needs to be optimized.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [June 21, 2018, 8:39pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/84 "2018-06-21T20:39:44Z")

</div>

Can you compare to MATLAB and SciLua?

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [June 21, 2018, 8:50pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/85 "2018-06-21T20:50:21Z")

</div>

Scott, have you benchmarked portions of the C code for `parse_int`, or is that your assessment based on experience and reading the code? We could rename the benchmark “random\_numbers” . Or maybe time just the parsing of a pregenerated large array of stringified ints.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 21, 2018, 9:14pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/86 "2018-06-21T21:14:19Z")

</div>

There was a whole thread about it on GitHub, 2-3 years ago

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [June 22, 2018, 2:31am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/87 "2018-06-22T02:31:01Z")

</div>

LuaJIT is really fast and the compilation time is almost negligible. How does it fit to scientific computing?

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [June 22, 2018, 7:35am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/88 "2018-06-22T07:35:11Z")

</div>

> [@Yifan\_Liu](#):
>
> LuaJIT is really fast and the compilation time is almost negligible. How does it fit to scientific computing?

You can use [SciLua](http://scilua.org/) as an extension of LuaJit for Scientific Computing.  
It is shown on SciLua page it can compete with Julia on the micro bench created by Julia.

I’d like to see Julia beating SciLua (LuaJit) in tests.  
Julia is all about speed and I want to see more data proves it is faster than other competing languages.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 22, 2018, 2:17pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/89 "2018-06-22T14:17:47Z")

</div>

> [@RoyiAvital](#):
>
> Julia is all about speed and I want to see more data proves it is faster than other competing languages.

I disagree here (if your definition of faster is strictly about run-time performance of some algorithm written in Julia vs. language X)

I’d rather see more data showing how Julia is faster at the most important metric for many programmers:  
How long does it take me to get the answers I need, from initial specification of the problem, to valid results?

Julia is _many_ times faster than C, C++, Lua, Python, and _any other language I’ve used in 43+ years of programming_, using that metric, for what I’ve been writing the last 3 years.

[There _was_ some ramp-up time, because back in v0.3.x, Julia lacked a number of things that I had available in other languages, such as Caché ObjectScript, but I was able to quickly add those capabilities, all by myself, writing _only_ in Julia (and wrapping a couple open source libraries such as AeroSpike)]

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [June 22, 2018, 4:52pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/90 "2018-06-22T16:52:35Z")

</div>

> [@ScottPJones](#):
>
> > [@RoyiAvital](#):
> >
> > Julia is all about speed and I want to see more data proves it is faster than other competing languages.
> 
> I disagree here (if your definition of faster is strictly about run-time performance of some algorithm written in Julia vs. language X)
> 
> I’d rather see more data showing how Julia is faster at the most important metric for many programmers:  
> How long does it take me to get the answers I need, from initial specification of the problem, to valid results?
> 
> Julia is _many_ times faster than C, C++, Lua, Python, and _any other language I’ve used in 43+ years of programming_, using that metric, for what I’ve been writing the last 3 years.
> 
> [There _was_ some ramp-up time, because back in v0.3.x, Julia lacked a number of things that I had available in other languages, such as Caché ObjectScript, but I was able to quickly add those capabilities, all by myself, writing _only_ in Julia (and wrapping a couple open source libraries such as AeroSpike)]

While the metric you suggest is highly important it is very subjective.  
It might be a results of personal habit, style, etc…  
I for one find MATLAB to be heaven in that regard.  
Others find Python to be as elegant as a British Gentleman.

The first metric, pure speed, is both important and objective.  
Moreover, it may have a real chance bring users.  
Sayings, as yours, will seem to others as a subjective opinion and hence won’t change minds of those who have other preferences.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 22, 2018, 8:24pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/91 "2018-06-22T20:24:36Z")

</div>

There are other metrics that can be used as a proxy, such as LOC, along with total character count, [some languages like old-school Mumps have very few lines of code, simply because people stick a lot of code onto a single line, so low LOC is not by itself a good indicator], to make it less subjective.  
I wish there were a good way of getting metrics for _released_ bugs per lines of code for similar projects in different languages, as I feel that’s a very important issue (and one that Julia might not do as well as, compared to something like Rust).

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [June 22, 2018, 8:34pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/92 "2018-06-22T20:34:40Z")

</div>

The scilua seems to offer some basic numerical functionalities and the performance is great, also the syntax looks better than Python. It has been around for a while, why it has not got popular?

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [June 22, 2018, 9:05pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/93 "2018-06-22T21:05:37Z")

</div>

Indeed [SciLua](http://scilua.org/) looks impressive.  
I have no idea why it hasn’t gain popularity (Lack of proper IDE?).  
I wish it did as a light alternative to Python (LuaJit is really a miracle and its seems its developer, [Mike Pall](https://github.com/LuaJIT/LuaJIT/issues/45), is out of the chart in his skills for creating a JIT compiler).

Actually its developer, @stepelu, has joined this community.  
He is the better one to answer the question.

It seems SciLua (LuaJit) sets the bar regarding speed of JIT languages and I’d like Julia to pass that bar and get faster and more efficient (Actually only Julia is in the dial area of SciLua [LuaJit] so it is already impressive!).

**Remark**  
When I write **SciLua** I mean in the Scientific Progrmaming snese.  
Of course the heavy lifting is actualy done by [LuaJit](http://luajit.org/).

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [June 22, 2018, 9:33pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/94 "2018-06-22T21:33:10Z")

</div>

I was wondering if Julia can continue to improve its performance, since the other programming languages are catching up. For example, Numba is really fast and R 3.5.0 is also getting much faster than its old version. Some might argue that Numba or Cython are not using native Python, but actually it is quite easy to code in Numba or Cython for a basic Python user, and extra work needed is quite acceptable.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [June 22, 2018, 9:46pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/95 "2018-06-22T21:46:46Z")

</div>

There is still nothing like StaticArrays.jl in Numba. Because of inlining etc it is at least 10 times faster than Numba for small arrays. Furthermore automatic differentiation just works with Julia and can give another 10 times improvement in solving differential equations.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 22, 2018, 10:33pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/96 "2018-06-22T22:33:32Z")

</div>

> [@Yifan\_Liu](#):
>
> I was wondering if Julia can continue to improve its performance, since the other programming languages are catching up. For example, Numba is really fast and R 3.5.0 is also getting much faster than its old version. Some might argue that Numba or Cython are not using native Python, but actually it is quite easy to code in Numba or Cython for a basic Python user, and extra work needed is quite acceptable.

There’s pretty much a speed limit. But the issue is what kinds of codes can get there. Numba and Cython architectures don’t compose. I’ve already shown that if you stick a Numba function into odeint (a wrapper over a Fortran function) you still get something slow because of the Python in the middle and the function call cost:

[http://juliadiffeq.org/2018/04/30/Jupyter.html](http://juliadiffeq.org/2018/04/30/Jupyter.html)

Julia code all composes, even between other Julia code, because you don’t compile functions/packages/etc. separately but together when possible. Numba and Cython are like if you compiled everything as a shared library while Julia is like statically compiling everything together. This means that Numba and Cython can put out much better microbenchmarks than it can put out real-world solutions, like seen in that example above.

LuaJIT is a tracing JIT and compiles really late like Julia. Julia and LuaJIT are pretty much head to head. The Julia benchmarks show Julia slightly ahead of LuaJIT and this uses newer versions than the one posted on the SciLua page:

> **[Julia Micro-Benchmarks](https://julialang.org/benchmarks/)**
>
> The official website for the Julia Language. Julia is a language that is fast, dynamic, easy to use, and open source. Click here to learn more.

These are using SciLua as well, so same benchmarks and newer version of Julia. And in another test, you can see LuaJIT and Julia pretty much head-to-head in C FFI.

> **[GitHub - dyu/ffi-overhead: comparing the c ffi (foreign function interface)...](https://github.com/dyu/ffi-overhead)**
>
> comparing the c ffi (foreign function interface) overhead on various programming languages - GitHub - dyu/ffi-overhead: comparing the c ffi (foreign function interface) overhead on various programm...

Funnily enough, Julia and LuaJIT beat C itself. This is because of runtime magic to magic shared library calls compile more like statically linked libraries (further reinforcing how Julia keeps an advantage over bolted-on solutions which compile things in isolation)

[https://nullprogram.com/blog/2018/05/27/](https://nullprogram.com/blog/2018/05/27/)

LuaJIT is fine, Julia is fine. Maybe under different circumstances LuaJIT would’ve done well. But what Julia really has is the genericism. Julia has multiple dispatch and generic functions that let everything compose. All of these benchmarks are all testing the use of integers and floating point numbers because it’s not trivial to run those same benchmarks with dual numbers or numbers with uncertainty or numbers with units. In Julia, swapping that out is trivial though since Julia doesn’t treat operations on floating point numbers as special. I think this is a bigger feature than most people think, and I think I’ll only be able to convince people of that when I go show them all of the ways to use it so I’ve got work to do.

But yes, speed limit, in some cases other strategies can hit it, Julia consistently gets there and is easy to optimize. Oh and the creator of LuaJIT left. Julia is in a pretty good spot for many many reasons beyond the surface level.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 22, 2018, 11:26pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/97 "2018-06-22T23:26:42Z")

</div>

As a language implementor, I’ve seen how many languages end up running into a brick wall sooner or later because of fundamental issues in their design.  
You can throw a _ton_ of money and really smart people at speeding up something like JavaScript or Python, but the very design will still hold you back.  
Julia actually doesn’t do such a great job at optimizing at the moment (or so I’ve been told by some of the implementors) - it’s very fast _already_ because of its architecture.  
Many of the data structures used extensively can be improved a lot (such as Dict, IMO).  
Although most of the focus of v0.7/v1.0 is getting the language and the API “right”, the comparitively small amount of time spent on optimization has made v0.7 quite a bit faster in places already.

The core Julia team has said a number of times that v1.1 will be focused on optimization, and I can hardly wait to see what things they’ll manage to pull off by next year.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 22, 2018, 11:32pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/98 "2018-06-22T23:32:14Z")

</div>

> [@ChrisRackauckas](#):
>
> Oh and the creator of LuaJIT left.

Really? Looking at the git repo, Mike Pall made a commit as recently has June 5th. What happened?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 22, 2018, 11:41pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/99 "2018-06-22T23:41:48Z")

</div>

> [@ScottPJones](#):
>
> Really? Looking at the git repo, Mike Pall made a commit as recently has June 5th. What happened?

He stopped in like 2012. He will still commit now and then, but if you check there have been like 100 commits in the last 4 years or so. There’s a whole explanation somewhere, I think it was because he got a job.

Edit: Looks like it was more around 2016

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [June 23, 2018, 3:34am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/100 "2018-06-23T03:34:27Z")

</div>

Too bad he can’t be dragged into the Julia community!

[Previous page](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991.md?page=4)

[Next page](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991.md?page=6)