What don't you like about Julia for "serious work"?

[lmiq]

@paulmelis

. But with a Julia file that’s included as part of a package how should I approach the same granularity in testing?

Probably Revise is the best answer here. You will have your package loaded once, the function being worked on the editor, and the REPL setup for testing that function. You can than simply edit the code and test repeatedly the execution of the code.

That does not eliminate completely having to restart everything from scratch, but it becomes much less frequent.

If the test in case is well thought it can become a @test unit of the package.

@paulmelis

With Python a separate module (file) always declares its dependencies, making it easy to test one module/file separately.

I am not a frequent python developer, but I sort of like this idea. It has come out in some discussions here. Yet, I feel that while that is something that python forces one to do, it does not mean that you cannot do the same in Julia. One can write tests for every specific function which contain the dependencies for that test. A recent thread mentioned the idea that these tests could be placed alongside the function, an idea I like as well, but currently the suggestion is to name test files after the function names and put them in the test folder of the project.

[Skoffer]

One can easily do some sort of test suits, since Pkg.test can accept arguments which can be used to filter test files. For example like this https://github.com/PyDataBlog/ParallelKMeans.jl/blob/master/test/runtests.jl

Also there is new package ReTest.jl which provides incode testing (similar to rust tests) and regexp filtering. Together with Revise it makes code testing very simple. I’ve used it in last Advent of code at it was very smooth, for example https://github.com/Arkoniak/advent_of_code/blob/master/2020/12/day12.jl

[Plop]

Since we are asked for subjective matters (what we don’t like for “serious work”).

Julia, as a langage, is targeted to and optimised for computational scripting. It’s like a more general-purpose Matlab (with much much better typing paradigm).

So, if what you want to do is putting a quick-and-dirty 500-line script together to calculate stuff (as a single file or jupyter notebook, etc.), your experience will be excellent.

You can consolidate those scripts into libraries but that’s where the experience begins to plummet a bit: the langage offers very little guarantees to the programmer and does not promote proper software architecture.

Practically it means most of the largish code you’ll see out there is a mess to maintain or fix. You typically need to know far more about the overall code to make a change anywhere than in langages like modern C++, Go or Rust for instance… Same thing goes with Python, even though tooling has helped enforcing good practices for maintainable code there (but it’s “ok” or “good” at most, not excellent).

Now if you feel outraged, read again: I am not saying it is impossible to make a big maintenable Julia code base. It is just made harder. Julia is like a hammer: if what you need is to put a few nails here and there, it is the perfect tool for the job, but as the number of nails and precision requirements increases, your life will be made easier if you use a nail gun (though, again, a hammer is not horrible).

So I would say Julia is perfectly ready for “serious” research work.

I would not be reimplementing an OpenFoam, Comsol, Gromacs or Gaussian in there though (which is another form of “serious”, i.e. consolidated research work). Again, this is not impossible, but reaching such reliable performance in Julia is tricky to start with and again Julia offers very little in terms of architecture and guarantees. As a program goes bigger and bigger in size, being able to look at just a couple functions at most to solve a problem while being sure it won’t adversely affect unrelated pieces of the program is important. If you only count on unit testing for that, your guarantee is only as good as your tests, unfortunately.

[rikh]

@Plop

I am not saying it is impossible to make a big maintenable Julia code base. It is just made harder.

Why exactly do you think this is the case? What kinds of things do you miss in Julia what other languages do have?

[BLI]

@adamslc

Here’s what I came up with:

1. Time to first plot. Can be solved by working in a REPL, or using PackageCoompiler. But still a worse workflow than say python.

I have used Julia as the main tool since ca. 2018 (in addition to OpenModelica). I used MATLAB extensively from 1992 to 2011. Then I gradually switched to Python. Reason: MATLAB became too expensive for my university which had budget problems at the time. Today, I have excellent access to MATLAB, but I prefer to stick with one language, so Julia.

In April 2016, I bought a Dell XPS 13 laptop (8GB RAM, SSD; job), and in December 2016 I bought a Surface Pro 4 (16GB RAM, SSD; private). Some experience:

• It took more than 2 minutes to start MATLAB on the Dell XPS 13.
• It took some 25 s to start MATLAB on the Surface Pro 4

I don’t know if this difference was due to the amount of RAM, or the processor version.

In any way, I came to be annoyed by the start-up time of MATLAB. This increased quite a bit when MATLAB started to use Java for the GUI (long before 2016). Because of this, the time-to-first-plot in Julia doesn’t bother me very much, although I of course wish it was shorter.

In August 2011, I started to learn Python. There were quite a few things I liked (but then I also like to learn new things). And some things I disliked. At that time, I didn’t like the Spyder IDE, and much preferred the Wing IDE 101 (which also had superb customer support).

Going through the optimization packages available at the time, I tested a package built in pure Python. Using it was rather frustrating… in order to find the optimum for the Rosenbrock Banana function, I had to start within 10% of the optimal solution… a pathetic performance compared to the Schittkowski and other algorithms available for Python.

My personal overall thought on these three languages at the moment:

• Cost: Julia and Python are free, MATLAB is expensive
• Documentation: MATLAB is super good, Python has lots of decent books, Julia is lacking really good introductory books (but has some good topical books)
• Language: Julia is best/most modern, followed by Python, with MATLAB showing that it was designed in the 1980s or so
• Packages: MATLAB and Python probably covers most fields, while Julia has a more limited scope of packages (e.g., no simple GUI tool?) – but is growing fast being a young language (Julia is already much better than Python when it comes to Control Engineering packages, at least last time I checked)
• Quality of packages: MATLAB packages are generally good (but costly; still, the Optimization TB and its QP solver failed for problems ca. 2010 that were easily solved using the Lagrange formulation), many Julia packages are very good. I had mixed experience with numerical packages when I used Python.

In summary, I’m happy with Julia, and also like the quick development that keeps going on. The main improvement I’d like to see is some high quality introductory books for engineers (like myself; I prefer books over web pages…). But I shouldn’t blame that on all the developers who prefer to develop code. Instead, I and others who complain about lack of documentation should consider to contribute – as users of a free tool, we have a responsibility ourselves.

[Plop]

This is kind of the wrong question to ask for what I am talking about. If it was about what is missing, you would just answer and say “oh but library X does this” or “it is planned” or “but Julia has that other thing that (in my mind) solves the problem (even if it introduces other problems)” and those would indeed (probably) be excellent answers! Programming languages are extensible, Julia is not exception! So if something was truly missing, it would not really be a big deal.

Guarantees come with restrictions: it’s because you are not allowed to do X and Y (or it’s very difficult/it’s seen as an exception) that I can assume Z holds. I burning to give simple concrete examples here but in my experience people then only focus on those examples instead of getting the big picture. Look at go, look at Rust and for most design decisions, just ask yourself “why didn’t they go the Julia way” and you’ll probably start to see how the trade-offs differ and how Julia went for “ease of use for script” while the others went for “ease of developing big things”. Now that I think about it, you can compare with Java too… It’s the same thing (but computer science evolved since it was developed).

The thing is, restrictive scripting languages are cumbersome and a pain to use. Julia explicitly chose to avoid that but there are trade-offs. I hope this makes things clearer.

[lmiq]

@Plop

Gromacs or Gaussian

The first version of those packages are more than ten years older than Julia itself. I don’t think that implementing them in any language again to get the same performance and functionality will be easy. Probably not happening any time soon. Writing such a thing from scratch I would bet is easier in Julia than in C, C++ and Fortran, which I think are the main languages there.

It is a good question, however if one could get the same performance in principle with pure Julia. I think Gromacs has even some assembly code in the middle.

One thing that is remarkable in Julia is the Julia implementation itself. It has already more than a thousand contributors, and I don’t think that any other language offers the possibility of new users contributing to the core language like Julia does. Even I have contributed a small improvement, and I don’t use Julia for more than a year now. This is perhaps a counter example of your statement that one must know a lot of the package to contribute?

[Henrique_Becker]

@Plop

and does not promote proper software architecture.

What exactly do you mean by that?

@Plop

Now if you feel outraged, read again: I am not saying it is impossible to make a big maintenable Julia code base. It is just made harder.

I fell outraged, XD. I must be out of the touch with C++ (although, I have programmed at least five of the last seven years in the language). If cutting my little finger would guarantee me I would never need to maintain a large C++ project and instead I could do it in Julia, then I would start reaching for my sharpest knife.

@Plop

Again, this is not impossible, but reaching such reliable performance in Julia is tricky to start with and again Julia offers very little in terms of architecture and guarantees.

Ok, now I understand a little better what you mean. Your problem is building a large project that needs reliable critical performance. Eh, I don’t really fell the problem, my PhD needs the best possible performance (i.e., it is the kinda of area you only are sure to publish if you can beat the previous times) and I migrated from C++ to Julia. Now I am using solvers (CPLEX/Gurobi) so this does not make much difference, but when I started the migration using solvers was not in the plan.

What I can concede is that until today I find truly horrifying that Julia Arrays have a internal not-very-well-publicized hardcoded threshold length that, if an array grows larger than it, it stops doubling when reaching full capacity and instead have a smaller growth. In other words, the amortized O(1) insertion guarantee is thrown in the trash and nothing tells you about that.

[RogerP]

Well I think Julia is a proper language and I am using it for a very large robotic control problem. Special thanks to Rigid Body Dynamics , DifferentialEquations.jl, LibPQ.jl and more recently Rocket.jl and too many more to mention. All the blue boxes are written in Julia. I love Julia’s speed, multi-core support and I’m just daring to get into threading. OK, not hard realtime but the ability to control GC, and to benchmark everything, package compilation. What’s stopping anyone? (I just feel such a cheat relying on everyone else’s code!)

[ChrisRackauckas]

@lmiq

I miss the possibility of using the @code_warntype macro in other scope than the global one.

@show Main.@code_warntype ...

[Sukera]

@Henrique_Becker

What I can concede is that until today I find truly horrifying that Julia Arrays have a internal not-very-well-publicized hardcoded threshold length that, if an array grows larger than it, it stops doubling when reaching full capacity and instead have a smaller growth. In other words, the amortized O(1) insertion guarantee is thrown in the trash and nothing tells you about that.

Could you elaborate on that? Looking through the code in array.c, nothing jumps out to me as violating that guarantee

[Oscar_Smith]

This is actually intentional. The idea is that if you have an array which is using a sizable percentage of your ram, doubling it is a bad idea. The reason for this is that if you keep doubling, Julia would just crash, so it’s much better for julia to let you make as big an array as possible given your computer’s memory than to crash at a smaller size than that.

[Sukera]

Yeah, it certainly does make sense - nevertheless, I can’t seem to find anything having to do with the length of the array itself, but with the length of the requested new space (which is a very different thing complexity wise).

[Oscar_Smith]

These are the same thing. If an array is requesting a large percentage of the computer’s memory to expand, that means the array is already very large.

[Sukera]

I was referring to limit_overallocation in array.c:753, where the existing size is subtracted from the new size, resulting in “we need this much more space”.

It’s also not necessarily very large already, come to think of it If you have a = Int[] and do resize!(a, 2_000_000), you’ll get the memory you asked for (provided you have that much RAM).

[JeffreySarnoff]

What I can concede is that until today I find truly horrifying that Julia Arrays have a internal not-very-well-publicized hardcoded threshold length that, if an array grows larger than it, it stops doubling when reaching full capacity and instead have a smaller growth

The only hardcoded constant relating to array memory that I see is in array.c,

// at this size and bigger,
// allocate resized array data with malloc directly
// instead of managing them separately as gc objects
#define MALLOC_THRESH 1048576

// used here
        if (nbytes >= MALLOC_THRESH) {
            a->data = jl_gc_managed_malloc(nbytes);
            jl_gc_track_malloced_array(ptls, a);
            a->flags.how = 2;
            a->flags.isaligned = 1;
        }
        else {
            a->data = jl_gc_alloc_buf(ptls, nbytes);
            a->flags.how = 1;
            jl_gc_wb_buf(a, a->data, nbytes);
        }

Inquiring about that code, I am informed it plays no part in doubling logic:

However that logic deals with new buffers or reallocd buffers. In the if branch you get a big call to jl_gc_managed_malloc (allocate a big slice of nbytes ) versus jl_gc_alloc_buf (smaller slices) in the else.

[Henrique_Becker]

I do not remember the exact point in space-time that I found the info about Julia using this length threshold. Maybe this changed, and I would be happy to know. A fast search in Julia issues brought Grow arrays by a factor less than 2? #8269 and [WIP]: Fix array growth thresholding #32035. Someday I will dedicate to get again on par with the current situation but I think it will not be today.

[Sukera]

No, thank you for the issues! I did not know about the implications of the behaviour.

[PetrKryslUCSD]

I put all tests into their individual modules. Thus all tests have a tightly controlled env. Example: FinEtools.jl/test_basics.jl at master · PetrKryslUCSD/FinEtools.jl (github.com)

Whenever I write a substantial piece of sw, I put the functionality to test in a separate file/module which I run until all tests pass. Then the whole module is copied into one of the ‘test’ files in the ‛tests‛ folder. Then the package is tested to check the integration of the new functionality with the old.

[sijo]

the langage offers very little guarantees to the programmer and does not promote proper software architecture.

As a program goes bigger and bigger in size, being able to look at just a couple functions at most to solve a problem while being sure it won’t adversely affect unrelated pieces of the program is important. If you only count on unit testing for that, your guarantee is only as good as your tests, unfortunately.

Look at go, look at Rust and for most design decisions, just ask yourself “why didn’t they go the Julia way” and you’ll probably start to see how the trade-offs differ and how Julia went for “ease of use for script” while the others went for “ease of developing big things”.

Ha! I think you’re hitting the nail on the head here, especially this: “As a program goes bigger and bigger in size, being able to look at just a couple functions at most to solve a problem while being sure it won’t adversely affect unrelated pieces of the program is important.”

Julia is my favourite language, but I share the feeling that Julia code is more brittle than say, Go code for an equal amount of effort and discipline. I think it’s a perspective I acquired after working a few years as software engineer… Here are a few concrete points:

I used to think it nice when a clever line of code does exactly the right thing. Now for industrial-grade code I prefer several lines of boring code, maybe even with a little repetition (making junior developers go ). Boring is not fun but it’s easier to read and more robust. Julia however is so expressive… it’s a joy to write, but the code can be tricky to understand.

For example it has outstanding support for metaprogramming. Well, real-life experience: I wanted to tune the behavior of the Markdown standard library, to allow writing $ x = 2 $ instead of $x=2$. Maybe I found the relevant code here but I wasn’t sure. I looked at the macro definition and the rest of the file, and decided to stop procrastinating and go back to work. In Go there is no meta-programming. That code would look like this and this which is sooo pedestrian, but that’s a good thing! (from a software engineering perspective).

Another point: interfaces are a core concept in software engineering, but they are not formally supported in Julia:

• The most superficial (but real) problem is duck-typing. Does your type actually implement the interface it’s supposed to? In Go you get immediately a compile error if you forgot to implement a method, or made a mistake in a method parameter. In a large project, I can add a method to an interface and immediately find all places that must be fixed. In Julia I must run tests (much slower) and pray that people actually wrote tests for all uses of the interface.
• A bit more subtle: In Go, you define interfaces explicitly and this can serve as reference for people that need to implement them. In Julia there is nothing forcing you to document your interfaces. The “interface” is whatever list of methods your code happens to use at a particular time. Real-life example: I could not figure out how to implement the IO interface to solve a simple problem (counting bytes written to a stream).
• Also, the language doesn’t help you dispatch based on interfaces implemented by a type (Holy traits are a kludgy workaround and don’t help with existing code). Even if you’re lucky and the interface is one of the few documented, there’s no mechanism to dispatch on it. Real-life example: I wanted to implement a fallback last method for iterators. The interface is documented, but I can’t define last(::Iterator). I could not find a backward-compatible solution that dispatches without runtime penalty so I gave up.

The overall point here is that Julia has weak support for one of the most important tools to make robust software.

Last point: local definitions. In Go, you can’t include files: you can import packages and one package = one directory and each file has an explicit list of imported packages at the top, and importing names in the main namespace (like using in Julia) is strongly discouraged. In Julia, files can include files can include files… Open a file, you don’t know in which context it is included so you can’t be sure how it’s going to be interpreted. Maybe it’s even included several times in different contexts? Maybe the behavior depends on the order of includes?

Another way to put it: declarative (rather than imperative) code is generally considered more robust or at least easier to analyze and verify. In Go, you write imperative code in functions, but the overall organization (definition of packages and functions) is declarative. In Julia, this organization itself is imperative.

(That being said, maybe these Julia “downsides” are necessary for what makes it unique and wonderful, such as the interoperability of packages? I’d rather take Julia as it is, than have another Go or whatever).