Blog post about my experiences with Julia

It’s not really about the dynamic types, it’s about the REPL. It is really, really useful and fast to be able to simply run any little function your code is using to see what they’re doing. You don’t have to write a test for every single part of the program, you don’t have to wait and rebuild the entire program every time you make a slight change, like adding a quick @info to print whatever information. I’m a julia newbie and I’m finding this feature really useful for debugging. Once you get the mindset of how it works, it’s quite useful. For example, I have a script that declares and prepares functions to run a simulation, but that doesn’t run anything. I simply include it into the REPL, and then launch a run() function I’ve declared after having checked that the parameters are just right. I don’t have the fear of launching a simulation and discover I’ve missed a parameter that needed to be changed. I’m not saying that you can’t live without a REPL, I’m saying that debugging it’s not only about the type system (which is awesome in Julia, btw).

3 Likes

If you enable hot reloading of that file with Revise too, then you have incredibly fast iteration on your code—just edit in IDE, save, switch to REPL and rerun whatever part you’re working on. For me this is the most productive way of doing both development and debugging in any language.

5 Likes

How do you edit structs in this workflow?

That’s the only thing you need to restart for.

2 Likes

It is as Stefan said, but there are older threads with workarounds, like using NamedTuple, or using a Wrapper{T} type that you can change the fields freely but you will want to clean up when you decided on the final design.

1 Like

That is a fair point (in general). I’m providing some examples below. As I already explained my background and approach, all I’m asking in return is to please not engage in further discussions on how the particular problem can be solved (already been answered in the linked threads). I’m merely trying to illustrate why I felt the way I felt learning Julia and why I recognized much of the experience OP described as my own too. If you find the examples I provide here useful to understand the thinking of a newbie that is really great. But if not, let me go out on a limb here and say that I do understand that there is a particular solution and/or explanation for each particular problem, and I really do not with to go there right now. I’m describing a bigger picture here, not a particular issue. Thanks.

That being said, this thread is a good example of how a newbie can get stuck with Julia, even with a simplest problem and after reading the docs and trying to implement what was read. What’s not explained in that thread is how I went back and forth through the docs, and even tried to throw some obvious things against the wall as I would do in other languages to see if anything would stick. Ultimately, with Julia you just have to ask here. Another thread is a good illustration of how disconnected and cryptic the docs appear to be to a newbie (again - appear to a newbie, not as an an actual fact). Chances of making the progress without asking the question here is virtually zero. People are polite, on point, and very helpful, but man you just have to ask, no way out on your own.

We both mentioned python and MATLAB. Forgive me for not providing actual examples but describing a couple of things I did the other day. In python I dot-chained half a dozen functions in a truly disgusting fashion while semi-arbitrarily placing parenthesis and square brackets with arguments that were incremented in a loop. The key point here, and the main reason I do not provide the actual code, is that I did it highly intuitively, half-based on the docs I read, and half-based on my own non-programmer intuition. And guess what - the hodge-podge code worked. In MATLAB I similarly put together a horrible Frankenstein’s monster of a function that allowed me to dynamically change the variable name in a loop based on a string array, which in return allowed me to print 1000s of plots automatically. The key point again is that I did this by first reading a couple of help pages and a MathWorks thread strongly advising against dynamically changing the variable names. I took what I read and tried to put together something of my own and guess what - it worked. My experience in Julia is exactly the opposite, for whichever reason, even after reading the docs carefully I am unable to put together even the simplest and the most straightforward functions. That is just my own opinion, not a absolutist statement.

I’ll just stop here because this is one of those things where by now it is quite clear what I’m trying to describe or no amount of additional examples will be sufficient. I hope everyone can acknowledge that I am not a programmer, but merely a person who tries to utilize a tool to automate something.

2 Likes

That’s my biggest gripe with julia. My solution is to try type stuff I’m not sure about in the REPL using dummy names for structs. Once I’m good, I write it in the code base with the real name.

I’ve been using Julia for some years now and my biggest panic is when I have to interpret an error stack trace (for example those that arise from failing tests). The true culprit is never the first, nor the last line but always some in the middle that may also be the first or the last.

In Matlab, relax, it’s always the first.

9 Likes

My only explanations for this (anecdotal evidence;) are

a) either you were extremely lucky or
b) your tools try every possible implicit type coercion to make your code work (which Julia doesn’t always do AFAIU and which could also lead to interesting side effects)

Would be interested hearing other opinions.

1 Like

Yup, I think this is basically it, the “typeness” of the language. I think in Matlab, R and Python, they try that everything works if even remotely you would imagine that it should.

This is really nice because you normally just throw out some half-baked code, and it will probably work. Julia having a stricter type system and a lot less type coercion put you in a harder scenario, you need to know what you have in hands (types) and use it accordingly.

I still like a lot more Julia, but can see how other languages are more beginner-friendly because they don’t expose new users to a type system or a lot of other things.

(Also some docs needs improving, but this is going to get better with time.)

2 Likes

I agree error stack traces can be pretty intimidating. I’ve seen error messages while using Zygote + DiffEqFlux where the name of a single type in the error was over 2000 lines long :sweat_smile:

As discussed in another post though, it sounds like there are tools in the works to make errors more interpretable. And as mentioned here it sounds like pull requests are welcome for simplifying error messages.

Is part of the problem that Julia is stricter but without static typing? So you’re stuck in a potentially more frustrating middle ground?

@MK-1138 Are your frustrations with the language semantics, docs, or lack of dot chaining function discovery? (the latter isn’t in matlab) Or the interactions between all 3?

Edit: I think Dot chaining means it’s easy to find a function which works for your datatype, but I’m not sure why matlab would be easier. In julia hopefully something like Tim’s PR will help (can’t find it ATM)

2 Likes

A curiosity: with other languages, do you usually follow the docs to answer these kind of questions, or you search for them in google finding, with frequency, someone that asked a similar thing?

(I don’t know anything about Matlab docs, but for python I always go the google-way).

3 Likes

Yes, but I would not call it a problem “per se”, but more like a conclusion of the design of the language. The natural expressiveness of Julia multiple dispatch and type system allow getting into places that are hard, maybe impossible, to get on other languages.

2 Likes

@Akatz, not necessarily “frustration”, just a feeling of being lost way too often for my liking. As in, I’ll put Julia down and try something else that I’ll be able to figure out way quicker. Of the things you mentioned it is mostly the docs. Julia docs feel like I need to fully read and keep in mind the complete docs - for everything - before being able to form a working model in my head. It’s like each article holds only one piece of information devoid of context, examples, and application (I know that there are simple examples but stay with me here), and if you want to figure out those out the next piece has been explained in another article, then the next one, and so forth. Like, it’s too formal and dry as if it’s a legal explainer or academic paper, and not a help on how to use a tool. I do not have any issues with semantics or lack of dot chaining (if there is one), as that is something I need to learn and adapt to while learning a new language.

@Imiq, MATLAB does an excellent job in this regard, imho, they have a very, very practical approach regarding this. You can go to the MathWorks home page - not MATLAB help, but MathWorks home page - and type, say, “imread” in the search field, click on the first result and get the complete information, everything you need to complete your task. The page is formally correct, but not only formally correct, do you know what I mean? It’s not its main purpose to be academically formally correct utilizing the minimal language (albeit it is that too), the main purpose is to help you understand how to do your thing. Other languages arguably do not have it this nice, which is why I guess you asked the question - they do have a plethora of tutorials, discussion boards answers, etc. being more mature/popular languages than Julia. But that is the chicken and egg issue for Julia - in so many cases with other languages you do not even need to check the docs/web as I explained regarding my experience with python. Being far more intuitive (imho!) after reading the docs and far easier to pull something off in them simply makes me spend more time with them then with Julia. I imagine this makes more people use them, more people posting about them, and so on. So it’s not only about how old the language is, it’s also about how approachable it is. It all adds up and if it continues like this I do not see Julia ever be where python was at the identical age. It will just (“just”) remain a tool for a mathematician to solve a set of differential equations in 2.46 instead of 2.52 seconds and post the benchmark plot on Twitter.

6 Likes

:rofl: :joy:

Things where going well untill the end!

I think Julia achieved more than Python already for the age.

But it is not a python replacement for the people that is happy with python, it is a much more powerful tool, for who needs something that Python does not offer. And nothing to do with Twitter :rofl:

Just one small example from a daily use: yesterday we realized that we needed to compute energies in a molecular simulation in a specific way. A C++ implementation of something similar, not the same, available in a very popular package, results to be 20x slower than a code we put up in one day in Julia. I have no idea why that C++ implementation is slow, and we didn’t care until we needed to compute something that is not exactly the same thing. Even the “naive” algorithm, which we wrote for testing, is fast enough to be used. And this is one code in which nothing special at all is done to improve performance.

That was not about competing with that other code for performance, we were happy with it, except that now it didn’t do what we wanted anymore.

Being able to write code for productive use that easily is what Julia is about. That changed completely the way I can write useful tools for my group and distribute them.

11 Likes

Where in the DiffEq docs does it read like this? The reason why we’ve focused so much more on the documentation than the help docstrings is precisely to have the interwoven information. For example, focusing mostly on the tutorials

https://diffeq.sciml.ai/stable/tutorials/ode_example/

and things like the common keyword arguments that cover all differential equation types:

https://diffeq.sciml.ai/stable/basics/common_solver_opts/

I’d be curious to have some specific points where this inter-module documentation does not come through well so we can improve it.

The average would be more like doing parameter optimization in 1 hour instead of 15 hours with MATLAB (with MEX C code). Without C code, it’s usually closer to >100x. If you’re going to say lower numbers, please cite and justify it.

10 Likes

I mean, Julia is that right now (edit: as Chris said, normally with bigger numbers), hahaha. But jokes aside, for what I see happening, it is growing really fast and for good.

One signal of that is that we are having this conversation, getting feedback, and listening to it with honesty, is the way to grow on what you didn’t know you lack.

5 Likes

Volker, Thanks for your post. I agree that Julia still has problems in many areas… I am using C/C++ most of the time, Python, Java, am with Julia since its 0.4 version. I’m very happy how better Julia is now comparing to what it used to be. I agree with you that references / documentation / tutorials could be much more convenient, but actually many packages has very good documentation. I’m OK with Julia’s debugging, maybe because I used to embedded systems :face_with_peeking_eye: I do not like several other things in Julia (like loading modules, precompilation, local variables etc etc). For all those years I did many attempts to get used to Plots but failed all the time.

Saying that, I’m trying to use Julia language when I have a chance. It’s much more convenient and faster then Python, when math is around.

BTW, does Julia really recompiles all the modules when we do ‘using’? If yes, then maybe somebody can point me what/where to read about the reasons of this behaviour?

Best regards

@Imiq, @ChrisRackauckas, @suavesito,

Oh my, perhaps I too should’ve posted a smiley at the end of that sentence! I understand why it might read very differently than how I intended. :sweat_smile: I hope there is no hard feelings.

And I guess that sentence leans on heavily on what I saw as the very essence of the original post. If I know how to do something that takes 15h in MATLAB, then it will be done in 15h. But if I cannot figure out how to do the same thing in Julia it doesn’t matter that it would take an hour - it will never be done. It just won’t. Again, saying this as a daft layman, take it as you will. If the problem is me I can absolutely assure you I accept it!

4 Likes