# Benchmark for latest julia?
**URL:** https://discourse.julialang.org/t/benchmark-for-latest-julia/5991
**Category:** Community
**Tags:** question
**Created:** [September 20, 2017, 2:47am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991 "2017-09-20T02:47:24Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)
#### Post date: [September 20, 2017, 2:47am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/1 "2017-09-20T02:47:24Z")
I noticed that there is a benchmark table for julia 0.4 on the official website. Why no one is updating it to julia 0.6?
---
### Author:  [@lobingera](https://discourse.julialang.org/u/lobingera)
#### Post date: [September 20, 2017, 5:37am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/2 "2017-09-20T05:37:14Z")
[https://github.com/JuliaLang/julialang.github.com/issues/420](https://github.com/JuliaLang/julialang.github.com/issues/420)
---
### Author:  [@cormullion](https://discourse.julialang.org/u/cormullion)
#### Post date: [September 20, 2017, 8:30am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/3 "2017-09-20T08:30:43Z")
Running the benchmarks looks like it involves having current licences for Mathematica and Matlab, which are quite costly. This prevents many people from being able to run them, or help out.
Someone working in a university with knowledge of most of the languages, and plenty of spare time, and access to suitably licensed machines could do the job, but where you would find someone with these qualifications? 😀
---
### Author:  [@lobingera](https://discourse.julialang.org/u/lobingera)
#### Post date: [September 20, 2017, 10:33am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/4 "2017-09-20T10:33:16Z")
I just checked, matlab is around 2k€ and mathematica around 3.5k€ (single person, local license, but there are now also cloud/pay per use schemes) which is not nice, but managable. But actually it (imho) should possible to redo the orginal campain at the place it was done (i’d assume MIT…)?
---
### Author:  [@cormullion](https://discourse.julialang.org/u/cormullion)
#### Post date: [September 20, 2017, 10:49am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/5 "2017-09-20T10:49:20Z")
Also I think academic users get cheaper purchasing opportunities. (Microsoft Office is now given away to first-year undergraduates!)
---
### Author:  [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [September 20, 2017, 11:46am UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/6 "2017-09-20T11:46:54Z")
It seems both matlab and mathematica have a free trial period, so one should be able to do it for free, it just requires a bit of time downloading and installing everything.
---
### Author:  [@lobingera](https://discourse.julialang.org/u/lobingera)
#### Post date: [September 20, 2017, 12:36pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/7 "2017-09-20T12:36:42Z")
I though about that, too. But, i guess there is a reason, why this is called ‘trial period’, so there are limitations what you can do with the results…
---
### Author:  [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)
#### Post date: [September 20, 2017, 3:22pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/8 "2017-09-20T15:22:00Z")
Thanks for pointing this out the need for updating the benchmarks. I’ll try to do this in the coming week. I have academic licenses, an interest in Julia benchmarking, and a benchmark that I’d like to add to the suite (if others deem appropriate). So it’d be a good exercise for me.
---
### Author:  [@tbeason](https://discourse.julialang.org/u/tbeason)
#### Post date: [September 20, 2017, 4:27pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/9 "2017-09-20T16:27:53Z")
@John_Gibson same here. I’m in a university, I have access to all of these on my office computer. I’m not super familiar with all of the languages used in the original benchmark, but I could definitely run 5/6 of them without trouble (especially if I’m just handed the code!). You take a crack at it when you get a chance. I’ll do the same when I can (probably not for a few weeks).
There is just the issue of getting my hands on an Intel(R) Xeon(R) CPU E7-8850 2.00GHz CPU with 1TB of 1067MHz DDR3 RAM machine … 😋
---
### Author:  [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)
#### Post date: [September 22, 2017, 7:07pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/10 "2017-09-22T19:07:52Z")
Here’s what I have so far. Machine is Intel Core i7-3960X CPU @ 3.30GHz 64 GB DDR3 RAM running openSUSE LEAP 42.2. Julia is julia-0.7.0-DEV compiled with MARCH = native (not 0.6.0 as stated previously). Had to hack the tests/perf/micro Makefile a bit, and I’m having trouble with the lua, java, octave, and scala benchmarks. Some of that is just getting those systems installed on the machine. But I expect I’ll figure it out.
---
### Author:  [@cortner](https://discourse.julialang.org/u/cortner)
#### Post date: [September 22, 2017, 7:23pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/11 "2017-09-22T19:23:35Z")
Could be nice to have various Julia versions on that plot?
---
### Author:  [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)
#### Post date: [September 22, 2017, 7:43pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/12 "2017-09-22T19:43:40Z")
Ah, it’s so satisfying seeing Python and R get smoked. 🚬
Oh god, I just had a scary thought: instead of looking at the above plot and concluding that they should use Julia, will some people look at the above plot and conclude that they should use… Javascript? The horror… the [horror](https://eamodeorubio.github.io/thejshorrorshow/#/) 💀
---
### Author:  [@NickNack](https://discourse.julialang.org/u/NickNack)
#### Post date: [September 22, 2017, 7:54pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/13 "2017-09-22T19:54:51Z")
> [@cortner](#):
>
> Could be nice to have various Julia versions on that plot?
I don’t think that would work graphically since the different Julia versions would presumably be very close to each other, and it would be hard to see the difference between them at that zoom level. Better to just use the latest Julia release in the cross language comparison and then make a separate figure for various Julia versions, all nice and zoomed in.
---
### Author:  [@mauro3](https://discourse.julialang.org/u/mauro3)
#### Post date: [September 22, 2017, 8:29pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/14 "2017-09-22T20:29:58Z")
The `rand_mat_mul` of Fortran seems wrong, otherwise a quick comparison with the table on [https://julialang.org/](https://julialang.org/) seem about right. Julia improved!
---
### Author:  [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)
#### Post date: [September 22, 2017, 8:33pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/15 "2017-09-22T20:33:00Z")
> [@mauro3](#):
>
> The `rand_mat_mul` of Fortran seems wrong, otherwise a quick comparison with the table on [https://julialang.org/](https://julialang.org/) seem about right. Julia improved!
I noticed that, too, and a few other discrepancies. I’ll look into it. I’ll also post the raw data table when I get back to the office Monday.
---
### Author:  [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [September 22, 2017, 9:31pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/16 "2017-09-22T21:31:57Z")
Also I think you have to start Matlab in single threaded mode (-singleCompThread), maybe there’s other gotcha of the sort. That javascript point is a bit suspicous…
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [September 22, 2017, 11:26pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/17 "2017-09-22T23:26:14Z")
That machine died so we can’t do that anymore. The core Julia devs aren’t at MIT anymore either so we don’t have access to educational licenses for those systems.
---
### Author:  [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)
#### Post date: [September 26, 2017, 4:50pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/18 "2017-09-26T16:50:18Z")
With further work I got the Octave and Java benchmarks to run.
Most of the Go benchmarks run correctly, but `rand_mat_mult` and `rand_mat_stat` seg fault during calls to the gonum-blas libraries. I haven’t been able to run the Lua benchmarks for lack of gsl-shell (no binaries available, can’t get it to compile).
I’m puzzled by two things: the improvement of `mandel` performance for Java and Javascript, and the worsening of `rand_mat_mul` for Octave, Python, and R. At this point think I should make a PR for my fixes to julia/test/perf/micro so that others can run the benchmarks and try to make sense of those changes, or maybe try to get Lua and Go working, too.
I’m inclined to wait to resolve these questions before making a PR to [julialang.org](http://julialang.org) to update the benchmark data and plot, unless people here think otherwise.
Also, I like showing C==1 on the plot, and I’d prefer ordering languages with generally faster towards the left and generally slower towards the right. How’s that with everyone?
---
### Author:  [@Chong\_Wang](https://discourse.julialang.org/u/Chong_Wang)
#### Post date: [September 26, 2017, 5:22pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/19 "2017-09-26T17:22:57Z")
Actually, where’s the test code? I am surprised by javascript performance and want to try it locally. I have all the licenses and a working machine, but I don’t know go and lua. Are go and lua popular among scientific communities? If not, we can skip their tests.
---
### Author:  [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)
#### Post date: [September 26, 2017, 5:42pm UTC](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991/20 "2017-09-26T17:42:42Z")
> [@Chong\_Wang](#):
>
> Actually, where’s the test code? I am surprised by javascript performance and want to try it locally. I have all the licenses and a working machine, but I don’t know go and lua. Are go and lua popular among scientific communities? If not, we can skip their tests.
The benchmark codes are in `julia/test/perf/micro` of the julia lang source tree, [viewable online here](https://github.com/JuliaLang/julia/tree/master/test/perf/micro). There are a few problems with the Makefile which prevent the benchmarks from compiling and running on julia-0.6.0 and julia-0.7-DEV, mainly changes in the location of a few libraries (e.g. libopenblas64\_.a) within the julia source tree.
I imagined Go and LuaJIT are included in the benchmarks not because these are good science languages, but because they’re prominent languages with just-in-time compilation.
[Next page](https://discourse.julialang.org/t/benchmark-for-latest-julia/5991.md?page=2)