Question about Julia's goals

Community
21

I would like to see Julia being used for the development of a big CFD (or FEM) code, particularly as a framework for an open-source project composed of several packages which implement various physical and numerical models used for flow simulations together with a “driver” main application which implements the basic logic.

There is certainly a lot of interest for such a framework but the problem is that Julia has a different orientation as it is trying to provide a one-language solution for what is currently written in Python for scientific computing, a field that in my opinion Python covers very well and people will be reluctant to abandon. REPL, dynamic types, notebooks and running scripts are not very important to the CFD/FEM world unless you want to experiment with some particular model you are developing for testing purposes.

What is important is the ability to manage large codebases with ease (not good in Julia), building the application with a straightforward system which works on all platforms (works well in Julia), handling external packages automatically (also good in Julia), being able to create binary libraries and executables of a reasonable size (not good in Julia), a readable language which does not require a degree in Computer Science (good in Julia) and most importantly a highly performant code which can use the available hardware for parallel computing with features baked into the language and of course supported by various libraries which remain mainly invisible to the user.

Julia was not designed with these in mind. Of course the language designers can do whatever they like but from my point of view the real problem in today’s scientific / engineering computing is not a replacement for Python but a replacement for the C++/Fortran and the myriad different libraries and APIs which extend the languages for parallel processing. Yes, Chapel follows this philosophy but it has no ecosystem of libraries around it like Julia, the documentation is limited, books do not exist, applications are not open source or they do not exist at all. Julia is in a better position to be the leader in this field but only if the developers decide to move the language into this direction.

P.S. The reasons I talk about binary libraries and executables are these:
a. A binary executable will execute instantly without any potential pre-compilation delays.
b. Commercial codes cannot release source code, only binaries.

1 Like
22

What is important is the ability to manage large codebases with ease

Scaling a codebaseis is really not so bad in Julia once you realize all your internals functionality can be separate packages. Splitting things up has zero runtime overhead and imposes few structural limitations, and has major social implications: very often smart people you have never met will help maintain and improve your submodules for free.

Solving the expression problem changes how we can structure code, it just takes a while to get used to.

Of course some of this wont apply so much if your codebase is closed source.

7 Likes
23

??

2 Likes
24

BTW have you seen Oceananigans.jl?

1 Like
25

Reusable, fast generic code.

Given that you say that you are new to Julia, perhaps it would make sense to reserve judgement on this until you are more familiar with the language.

I am not sure how that is relevant for this discussion.

No manual communicates how addictive it is. Once someone programs in a language which integrates multiple dispatch as well as Julia for, say, 6 months, all languages without it will be extremely painful — you are, effectively, scarred for life. (The problem was, to a certain extent, present with Common Lisp but Lispers whined so much about all aspects of other languages that it was lost in the noise.)

16 Likes
26

Thank you for your suggestion. This looks like an interesting (for me at least) application to explore.

27

Julia is two things in my mind.

First and foremost it’s a science-oriented language, focused on the computation of scientific tasks.
That is its primary use case, and you will be hard-pressed, to find a lot of usage outside of that.

Second, it is a unique language in its feature set, as it combines a couple of pretty rare concepts into one language. Mostly, that is multiple dispatch, homoiconicity, and its flexible type system.

It is rare to find any of these features in a language, let alone combined particularly to this extent.

2 Likes
28

Not really.

2 Likes
29

Complaining about compilation delays for CFD codes sounds a bit fatuous: CFD simulations are known for long run times. What difference does a few seconds make?

3 Likes
30

While I agree with the idea you are expressing, there have been issues in the past where someone running Julia code on a cluster, and not thinking too much about the setup, triggers long (re)compilations of package dependencies on each node. I think there are ways around this, or perhaps it just doesn’t occur anymore, but I definitely recall issues around this in the past on Discourse.

Even recently, triggering an OrdinaryDiffEq (re)compilation was taking some people 20+ minutes. (Though I haven’t experienced that recently, so perhaps the underlying issue was fixed.) Triggering 20 minutes of recompilation on each node of cluster is pretty wasteful of one’s allocation. (But again, hopefully these issues don’t occur anymore and/or are easy to avoid.)

3 Likes
31

That does sound terrible. Is that still happening as of now (1.10)?

32

I haven’t experienced it recently, but I’m not sure if I’ve triggered a full recompilation of it either.

33

I honestly think this is even better.

Semantic homoiconicity is what matters to me, and Julia has even cuter syntax.

This is why one of the best pieces of advice, when people are trying to write macros in Julia, is to do the following:

  1. Write an example of the kind of code you want your macro to generate
  2. Call Meta.@dump on that code to see it as a data structure
  3. Write code to generate that data structure—this is your macro.

In Lisp, you don’t have to do step 2 because syntax for the code is already the same as the syntax for the data structure.

So we just have to automate the call of Meta.@dump (or do it manually) and we have the same benefits as in Lisp. Plus, additionally the subjectively better syntax.

To me, Julia has even excelled in homoiconicity.

2 Likes
34

Yeah, I used to think Julia was homoiconic but after thinking about that answer and some of the comments on it, I’ve realized I was wrong. The key to homoiconicity is that the syntax for code is the same as the syntax for the data structure that represents that code. That’s the case for Lisp but it’s not the case for Julia. The obvious next question is does it matter? Well, it’s not crucial but it does make it more obvious how to generate code. Julia Expr objects that are used to represent parsed Julia code use special syntax forms that you need to know about if you’re trying to generate the same data structure directly. We have code quoting that can do this for you, but it’s still a complication.

14 Likes
35

Yes, I try not to talk about things I don’t understand. How about for me? multiple dispatch is a wrapper code for additional behavior of the main solutions. Combined with Markdown and other things, program texts turn into rather extensive panels where you need to have a special way of thinking in order to keep everything in your mind, or have noticeable experience. So it seems to me that maybe I’m wrong, maybe I’ll even correct myself someday. That’s all, I don’t pretend to be a big debate and for now I’m writing in Julia as in Python. But you shouldn’t be responsible for others either. All of us want to write a lot, correctly, easily, and even without being gifted, or even even programmers. Or are you afraid of losing your bread if you lose the crown of an unattainable expert in the magic of ultra-deep scientific programming?

36

Certainly, but the only practical alternative that people have come up with is S-expressions for surface syntax (eg Lisp proper). Once a language like Julia has decided on something else, like the current Julia syntax, that is not an option, so the only choice we have is making working with our syntax easier, keeping in mind that it is never going to be as nicely integrated as Lisp.

That’s a trade-off all language designers have to face. Given how unpopular S-expressions are outside Lispers, I think Julia made the right call.

5 Likes
37

I read what you wrote twice and I still can’t make sense of it, nor understand how this could be relevant for your question. Sorry.

You may want to watch

3 Likes
38

maybe you can do this - homoiconicity is a matter for the developer, and Julia can only provide the opportunity

39

Thank you. I am facing the dilemma of career reorientation. Previously, a primitive approach to programming was enough, now I need to master something more serious. Because such questions are what people think about why they need Julia.

40

I believe Julia is actually proven for large codebases (only for special cases e.g. building web browsers is Rust better), e.g. SciML ecosystem proofs that, so what did you have in mind? People have complained for edge-cases for (generic) code with OffsetArrays.jl but that was overblown, and fixed already or fixable. What languages do you consider better for large code bases? I believe C++ and Python are worse, because of OOP, also if you opt out of using it. If you think OOP is actually better, then Julia has two OOP packages already. C++ didn’t have modules until recently, so it’s getting better that way, Julia was at least ahead if you had that in mind, and I believe still is (I believe there’s also a package for some perceived difficulty there). [C++ and Python do not have equivalent of OffsetArrays, and all languages that do would have same problems, for generic code. I at least think it’s unfair to complain about issues regarding optional capabilities other languages lack.]

If you had static typing in mind, then I believe it slows you down getting to large code-bases, it does help for knowing that you can just compile with changes and you will not get runtime errors. But tools for Julia help to, in same way as compilers for static languages. Maybe they are not optimal yet?

About “reasonable size” of executables, then I think it’s an overblown problem. You can compile to large (actually also tiny, with limited capability, i.e. same as for C code), and there are tricks to making them smaller. But ca. 200 MB (overhead) is not a problem in many cases, and will be laughable that it is, going forward.

Rust (and Mojo) has some advantages; for concurrent code, if you had that in mind. Julia is seemingly good enough, at least comparable to Go. You can reuse all libraries of Rust already (and C/C++), call to (or from) Rust, same as with Python.

You can already do that compile Julia code to binary executables and hide all the source code if important to you (optional flag). If all the code is precomiled there’s no startup delay, what I think you had in mind. No executable runs instantly. That would be rather useless. [You can tolerate not precompiling all your code, doing it after you start may not be bad.]

2 Likes