Julia programs now shown on benchmarks game website

Well, I’m not sure if that would actually apply, since it’s a 1:1 port from the leading C++ solution and there isn’t even any official memory pool implementation in Julia (and mostly doesn’t need to be?).
My “memory pool” trick even ended up simpler than the C++ version that uses a dedicated memory pool library, so I’d find it pretty irritating if the solution wouldn’t get accepted :wink:

Isaac Gouy @igouy-guest commented 1 week ago

It is unfortunate that Go does not provide a pool/arena/… and obviously storozhukBM/arena is exactly what I ask people not to do.

Go does provide GC.

Sorry.

That’s a pretty weird attitude, considering that the “pool” basically comes with no extra effort and is even less verbose than in a language that doesn’t have a GC and hence would need much better pool support.

What annoys me is, that the argument is to “benchmark the GC”, but still includes languages without GC… So a language with equally powerful tools for memory management as a language without GC ends up worse off, just because it happens to have a GC.
So either everyone should be allowed to use all tricks, or languages without GC should be excluded.

1 Like

Chapel programming language has GC, but the performance shows good on the benchmarks game website

Chapel’s performance looks comparable to a fast Julia solution without memory pool.
But if I remember correctly, the memory pool solution in Julia is almost the same speed as the fast C++ solution - which is ~4x faster!

Would making each implementation a separate module and making use of precompilation be of any help, provided that Julia also get a compilation step in benchmarking?

I was also considering to publish results (perhaps only the current best vs. Julia) that considers the startup and JIT overhead if there is no licensing issue.

I started extracting parts of PackageCompiler into a single file to compile an executable from the julia script - which should end up in a very comparable solution to compiling a C program. If that works out, we should be able to remove the JIT overhead.

5 Likes

I hate to be the guy defending Java, but there have been millions of dollars poured into making it go fast, and it’s no slouch in the performance department anymore.

In particular, I think type-unstable Julia is going to be slower than poorly-written Java in most cases. Of course, I’m not denying that well-written Julia is usually faster–and “scary-code” Julia is as fast as anything out there.

Non-GCed languages get an unfair advantage and if the binary trees benchmark was fair, C++ would use smart pointers and be ranked slightly below Python.

However, it’s still a useful GC benchmark for the languages that don’t cheat. If we can beat Haskell or SBCL’s GC on the binarytrees benchmarks with the Julia GC, that’s already a major achievement.

Done the compilation to an executable: protype to compile julia scripts to exe by SimonDanisch · Pull Request #35 · JuliaPerf/BenchmarksGame.jl · GitHub

11 Likes

Sweet!

Is there any setting or flag on Julia to temporarily disable the GC and let the user do the work manually?

1 Like

Not to my limited knowledge, but if you’re really desperate, you can use Libc.malloc to get heap memory that isn’t owned by the garbage collector and just work with raw pointers, so the garbage collector has a lot less work to do. I’m NOT recommending this strategy, just pointing it out.

A better strategy would be just write code with all your arrays preallocated as constants and avoid allocation that way. Programming in this style also sucks, but it does tend to be pretty darn fast, and it’s slightly less crazy than using malloc directly.

Preallocated buffers are used (and considered idiomatic) in Julia (eg see LinearAlgebra), while manual memory management is arguably not how Julia is normally used, so it goes against the spirit of the benchmarks.

The point of Julia is not merely to write fast code, it is to write fast and elegant code.

4 Likes

Preallocating buffers is also idiomatic in C, but it doesn’t make it a fun way to program!

But I agree that is much preferable to manual memory management, and that should probably be avoided for the benchmarks game. As stated, it’s not an approach I recommed, but it is a way to avoid GC if one absolutely must. I do think it’s cool that Julia actually makes memory management available to the user, even if it’s not recommended. I think it sort of speaks to the Julia ethos.

As mentioned earlier in the thread, part of the point of some of the benchmarks is to evaluate the efficiency of the garbage collector, so I doubt an answer that circumvents it would be accepted anyway.

If you hang on to a reference for your variables, the garbage collector won’t have anything to do. You could have a global notgarbage Any[] vector that you push stuff onto you don’t want GC’d. I think they have a name for this sort of thing in Java. Unless you call a library function that churns memory in some way of course.

2 Likes

why don’t you like preallocating arrays? I always really enjoy putting ! after my functions, it feels like I’m really getting things done :slight_smile:

7 Likes

If you want to know what I really like, it’s getting good performance without having to think too much about performance! I’d have thought that would be obvious from the fact that I use Julia!

This conversation seems many stepped removed from the topic at hand.

3 Likes

I see that you, like me, like exclamation marks :joy:
Sorry Kristoffer.

2 Likes