How did we get here? (Why is Julia such a messy language?)

Internals & Design
1

Julia is a great language, but to be honest, I can’t say it’s ideal either. I love the language, but to not be able to criticize it is not a healthy love. It’s like Julia has been optimizing for things people say they care about and measure first, while issues that they didn’t know they cared about get sacrificed in the process.

Scope independence

One of my strongest objections is scope dependency. For example, closures actually allow you to write a pseudo-OOP. The Emergent Features of JuliaLang: Part I · Invenia Blog

Another issue is that if you do some mad stuff, for example, using eval inside macros and so on, the module is not a clean namespace. You cannot safely say that

module A
  function/macro f()
  ...
  end
end

A.f()/@A.f

is equivalent to declaring function/macro f and then using it in every case. Modules are supposed to be namespaces, but they aren’t clean namespaces.

GC issues
Julia has GC, so it is a memory-safe language, right? Nope. There are a bunch of "unsafe"s that can do something nasty. There is also the issue of compiled code not being GC-ed ever. And yes, the unsafe is indeed used to reinterpret an array in a performant way, which is itself used in order to store type stable “raw bytes”. This is useful, for example, in implementing efficient runtime virtual functions.

Development shenanigans
It is said that in Julia, it is relatively easy to contribute back to the language itself because the language itself is mostly in Julia right? Except… well, developments can and do involve wrangling with unsafe, dealing with undocumented features, etc. There are also many popular packages that use internal, undocumented features.

I thought the Julia compiler was just a thin wrapper that specialized all the functions, did some optimizations, and then passed the function into LLVM, while the lowest calls of the functions are themselves LLVM (for example, x+y eventually is an add instruction or something along the line). Except… shenanigans!

I thought the language was pristine, beautiful, and clean, but the truth is, it’s messy and there are dragons everywhere.

How did we get here?

2 Likes
2

This post seems like it’s going to generate a primarily emotional conversation rather than produce actionable insights.

44 Likes
3

I thought the language was pristine, beautiful, and clean, but the truth is, it’s messy and there are dragons everywhere.

It turns out almost nothing in this world is “pristine beautiful and clean” while also being useful for complicated tasks (at least if we take those things to mean "so simple that a random person can be expected to understand everything there is to to know about every level of it)

This very much just seems like a mirror of your experience with posting about video games in the transition from

to

Some things are just complicated. A good language can make it easier to think about and deal with complicated problems, but it won’t remove the complexity.

16 Likes
4

Uhh, I listed 2/3 concrete issues (with their own sub-issues) I have with the language. This is not me speaking up with nothing to back up my claim.

5

The point being that these types of threads never actually end up producing anything meaningful/actionable and just devolve. There have been many like it on Discourse in the past and the pattern continues.

I have similar thoughts to Mason though that sometimes complex problems are just complex, there is no magic language that removes all complexity.

9 Likes
6

As far as I can tell, Julia’s warts come from 3 major sources:

  1. User/library empowerment. Julia tries to make it so that just about anything can be implemented in libraries, and few things have to be added directly to the language. This means giving developers lots of possible tools and access, but also means it’s possible to use them unsafely.
  2. Performance. A lot of seemingly elegant solutions to various problems are hard to implement in a performant way, or in a way that works with multiple dispatch. The World Age problem is a major example.
  3. Legacy. This affects every programming language over time – you have to support old code, meaning you’re locked into old decisions.

I believe the issues you listed are all a result of 1 + 2. A lot of these issues would be easier to contain if we could get some guarantees on interfaces, but that presents its own challenges – there are several libraries attempting different implementations.

How do other languages handle these problems? Most languages are bigger and have more warts, but one popular language that’s tried to be stay small and performant is Go. Go sacrifices user empowerment, there are plenty of things that libraries just can’t implement (or couldn’t for the longest time), and this makes it much easier to prevent certain problems at the cost of more verbose and less flexible code.

Lisps are more flexible than Julia, but tend to sacrifice a lot of performance, and can lead to much crazier shenanigans.

7 Likes
7

Maybe just as preemptive reminder, I’ll post this:

TLDR: it’s fine to have discussions like this, and it’s fine to disagree with the OP and tell them why you disagree, but lets please try and avoid turning this into a big argument of one person against twenty separate people all at once.

22 Likes
8

In my experience, using Python, at least for the thing it’s good at, doesn’t incur issues with Numpy let alone the language itself. However, when I use Julia, I have a lot of issues with the language itself.

Go was designed for network/web applications. Most don’t require very complicated logic. It can do that because… well… its use case kinda allows it to. That being said, there have been some nasty issues in Go like people adopting crazy workarounds to implement generics like code generation or lisp interpreter.

9

As discussed Time limits for unfocused discourse threads?, we tend to discourage unfocused threads.

2–3 entirely different questions, but each complex topics in themselves (why does Julia have “unsafe” operations? why should packages be allowed to access undocumented internals? something about functions and macros and eval?), is already too much. Coupled with the fact that the overall topic (“Why is Julia such a messy language?”) is unfocused, this clearly falls under that policy so I’m going to put in a time limit.

If you want to open a specific topic with some question about Julia’s design, e.g. “Should Julia eliminate ‘unsafe’ operations?”, you are welcome to. We just don’t want meandering open-ended threads.

35 Likes
10

I like how people pointed out that typically these discussions are not fruitful.

In an attempt to be more helpful, what specific issues blocked you from progressing in your Julia implementations? Maybe we can be helpful if we know exactly what it is that’s blocking you, rather than abstract concepts that we all generally more-or-less agree with?

No tool is perfect, Julia has flaws, we can work together to make it better. You had a reason to complain, and I kind of like that you did instead of staying silent and stop using Julia. I guess we are all trying to help you complain constructively.

2 Likes
11

As said in the very first post, these are my issues. Yes, they’ve been issues that have caused me problems before. The scope dependence issue has caused me issues when developing a package because some weird hacks I did stopped working cross the module boundary. The GC issues are things I’ve met myself.

12

No like, do you have a MWE that we can run to reproduce any of your issues? Otherwise we can’t really help you.

4 Likes
13

As some have suggested Julia is complex it may seem messy for you, but decisions are taken considering a lot of use cases you may have not thought.

So to answer you on this issue:

How do you manage A.f()/@A.f when your module A is a bare module ? (without its own include and eval methods)

eval (or macroexpand) have a parameter to specify the module in which expression is evaluated, when you don’t specify the module, you use Base.MainInclude.eval method.
This is clearly documented at Core.eval, it prevents calling it in a baremodule inadvertently and allow you to define which eval method is called in your macro/function definition if required.

2 Likes
14

I’d like to point out that half of each of your first points are essentially caused by giving the user power. The user then needs to use that power wisely but this no language can do for them.

Also note that you can do “mad stuff” in any language. The question should be “do you need to do crazy things to achieve your goals?”. For Julia I would always answer this with “No”. As long as your goals are reasonable, you can write reasonable code to achieve it.

To underline this point with a funny blog post: Most people would think that Haskell is one of the purest languages in existence. But does that mean that you can’t do “mad stuff”? Well I’ll let you decide after reading
https://aphyr.com/posts/342-typing-the-technical-interview

2 Likes
16

You always have the complexity of a Turing complete programming language on top of modern OSs and hardware. The simplicity of a language is in the world of computation that it presents, not the world in which it has to live.

8 Likes
17

Every language has the problem that you can call out to the shell and sudo dd to directly plop bits into /dev/mem… doesn’t mean that language is bad. There’s always stupid crap you can do. In Julia I rarely need to do stupid crap. My biggest problem in Julia is that it wall-of-texts error messages with 300 layers of nested parameterized types defined deep inside Turing or SciML or whatever. Makes it hard to know where i went wrong. But I feel like this is getting worked on. Beyond that I have almost no complaints.

2 Likes
18

There are tradeoffs associated with every single decision that gets made. Julia got to where it is by making a long series of decisions over a long period of time, each of which involved a set of tradeoffs. Some people will like the balance of tradeoffs that Julia has achieved, others won’t. It all depends on what you value and what your pet peeves are.

5 Likes
19

This is exactly what I was thinking.

OP @Tarny_GG_Channie basically admitted:

so it should be no surprise that things stopped working, especially if they were using private features that are by design not supported from version-to-version.

This type of issue exists in every large language. No language can perfectly guard the user’s experience unless that language were so extremely limited in features and functionality.

A friend once taught me that many people get far in life by mainly focusing on their strengths rather than being bogged down or overfocusing on their weaknesses. That might be analogical here where OP needs to question why their doing hacky things in Julia, and be open to the community’s help on the “best” alternatives for accomplishing their end goal—use Julia’s strengths, rather than Julia’s weaknesses of internals/private functionalities that aren’t supported from version-to-version.

3 Likes
20

I did the hack to force the code to be run at compile time. I should’ve used CompTime.jl to do the work, but I was unaware of the package at the time, and I heard of nasty world age issues with generated functions, so I went full hacker mode and hacked out a solution. What did I do? It’s a package where, for example, let’s assume the counterfactual that a pokemon can have more than 2 types, and you want to calculate how effective an attack is, you store the elements of the defender as bits, mask the relevant bits, then do a perfect hashing lookup. The perfect hash takes a long time to find, which is why I needed it to be run at compile time.

21

Oh I see, that makes sense. And sounds like the type of workaround I also would’ve done. So does CompTime.jl solve your issues? Or are there other things we can help you with?