Issues encountered teaching classes with Julia for the first time

I don’t. I taught a course out of the standard textbook by Copeland and Weston, which I used to teach to small classes fairly well-versed in mathematics. Then all of a sudden, it was announced I would be teaching this same course to a very large class of quite mixed mathematical background. After thinking about it, I realised that the basics of Financial Theory reduce to Constrained Optimization [minimize Variance subject to Budget and Expectation constraints, or Optimize Consumption Timing and Allocation to Production Opportunity subject to Budget constraints, etc]. The mathematically adept could work out and argue the solutions by hand, but the less mathematically strong found it easier to get the solutions using Constrained Optimization in JuMP, which I allowed. I don’t think I could have effectively taught this material to this audience without Julia. It might be fun to write a handbook based on this, but I don’t think I will have the time.

2 Likes

Oh indeed, that would be a difficult text for students “not well versed” in math. But I can see (looking at it briefly to remind myself) that this is precisely the sort of textbook that would benefit greatly from having the graphs and tables simulated and redone by students!

Too bad to hear you have dropped Neptune.jl.
I found it to be a great tool. Its non reactive property assists a lot when dealing with complex algorithms and large data.

3 Likes

Hi. Thanks for your kind words on Neptune. I didn’t say I would be dropping it. I may be using it less myself now, and I have no immediate plans to develop it further (unless I get a partner to help me), but I do intend to keep an eye on it to make sure it is compatible with new releases as I know quite a few people use it :slight_smile:

13 Likes

I’ll give another perspective. I am a very old student (I was on track for phd in compsci 25 years ago then stopped). This year I took a course in numerical optimisation (simplex, revised simplex, barrier method, conics, all the introductory stuff). The practicals (an electricity grid equilibrium) had to be done in Julia + JuMP + Gurobi. It was awful. Why ? Our professor insists on us to get the theory and be able to apply it. That is, we are given theoretical lectures (a lot) and then have to translate them on exercises and practicals. The theory explains each algorithm and proves that the algorithm works (lots of maths). So we have a number of demonstrations.

Now to JuMP+Gurobi. The huge problem with that pair is that it is very hard to know what the optimiser do. For example. We learned the simplex algorithm and derived the formula’s for sensitivity analysis. Although JuMP let you access to the sensitivity analysis report, it doesn’t give you access to the actual simplex tableau or reduced cost or dual variables. Therefore it is almost impossible to compare the theoretical formulas to the JuMP results. Therefore, although you can have the results, it’s very hard to explain them (especially when facing problems with thousands of variables). Since we couldn’t see “inside” JuMP or Gurobi, it was very “non pedagogical”. I basically used the optimiser as a black box, not knowing what it does.

Another problem was reoptimisation (that is, optimize a problem, change a constraint, reoptimize). JuMP or Gurobi doesn’t tell you it actually has re-optimized. You can guess it did, you can trust the documentation, but as a student, you’re very uncertain about what happens. So having explicit confirmations of all sorts is very important.

So I personaly would advise against JuMP+Gurobi for teaching. It’s too much of a black box. Having a simpler solver, that provides reliable information about its inner workings would be much better. (I’ve looked at GLPK, but it couldn’t solve our praticals problem).

Of course I have looked into Gurobi’s API and it is somewhat possible to figure out the necessary information but then you have to bridge that with your JuMP model which is super hard and not documented. I’ve tried and so ended up spending 25% of my time on that; as a a student that’s not acceptable: I should spend 100% of my time on learning numerical optimisation and not figuring out low level details.

I’d be happy to recommend another route to my teacher if someone suggests one. Ideally in Julia since it is a very nice language which maps very well on math concepts.

7 Likes

Just want to add another perspective here. I taught an Introduction to Computational Earth Science undergraduate course in Julia this year, after teaching it previously in MATLAB and more recently in Python. I set up a jupyterhub server on Google Cloud Platform (using the Zero-to-Jupyterhub instructions) and used a docker image with preinstalled Julia packages. I wouldn’t repeat this next year since the jupyterhub server was a bit slow at times and there were occasional GCP node connection issues; when that happened, some of my students installed Julia on their laptops and that went without any hitches, even with the IJulia + conda install steps.

At the start of the course there were some groans from a few students who didn’t want to learn a new language, but by the end of the coarse those same students said they really liked Julia’s capabilities and could see it being useful for future projects. About a quarter of the students had little to no programming experience, and it was easy to teach them Julia; maybe as easy as teaching MATLAB, except that Julia and its ecosystem offer much more functionality and so there’s more to learn.

I’m fairly new to Julia and hadn’t seen Pluto.jl when I set up my course last summer, but after checking out its capabilities and seeing the MIT course that uses it, I’m likely to move this course to Pluto next year. Although, the new standalone jupyterlab app sounds like another good option. This course is intro level and so using notebooks seems like the easiest way to get newbies writing code, but if I were doing this at an advanced level we would probably use vscode instead.

9 Likes

See comments above on Neptune.jl which is kind of a compromise between these two.

3 Likes

I am glad my post generated so much interest and attention.

Here I am 18 months after my first post, and I thought I would (firstly) report that my teaching with Julia in large (~200) classes has been successful, there is one serious issue that remains that has made the journey far from smooth for me, which has to do with Package Manager issues of longstanding.
In such large classes I have inevitably had a small subset (~10 per class) of students with issues, which have caused me hours of frustration. I mention them here because they are basically only two recurring ones (there had been a third which used to cause nightmares which has lately apparently been resolved).

The issue which appears to have been resolved is the interaction of commonly used packages with CSV.jl which had been resulting in it breaking in a way which was quite difficult to fix. Anyway (good news!) this one hasn’t cropped up in a long time.

The issue of longest standing is the Plots/GR issue, which dates back several years. There had been an easy solution of setting
ENV["GRDIR"]=""
which has worked up until now, but now no longer seems to resolve the issue (of Plots not finding GR).

The second recurring issue of the JuMP package breaking (see

) Unfortunately, this one has seemed pretty much unfixable beyond wiping everything
and starting over (I have chosen not to confuse this very elementary class with environments)

These appear to be the only recurring, unresolved, and longstanding issues, but they continue to
cost me many dozens of hours of frustration (mine and my students) each term.
I would be open to suggestions

Thanks

My experience with teaching with Julia is that taking some time to teach workflows is very important for a smooth ride - just after the first REPL experimentation. I would just include creating an environment into the list, to avoid (most?) of such issues.

2 Likes

Thank you. Ideally, yes. These are students who only previously may have had the rudiments of Excel, so even the concept of an environment is foreign (and a bit abstract), though. Even if we created new environments, there is no guarantee we wouldn’t encounter the same package tangles again.
Thanks for the suggestion, though - I may try this with students where I see no alternative.

1 Like

The chances of having problems with fresh environments is much smaller. I do agree that environments, and other Julia workflow tips, are somewhat convoluted for newcomers. Yet, I think that unless what one wants to teach is very rudimentary in terms of programming and ecosystem use, there is no good way out from learning the optimal workflows. And that includes using Revise, creating projects. It does take a full class only for that, but if one is planning a semester of Julia usage, it is worth the effort.

By the way, I think that we should have a very carefully written workflow basics tutorial in the Julia main page. A How to efficiently develop in Julia tutorial, something like that. The barrier for a smooth development in Julia that is associated with bad workflows drives a lot of people away.

30 Likes

Lots of classes are indeed very rudimentary in terms of programming — essentially they are just using Julia as a fancy calculator. I’m thinking of classes like linear algebra, statistics, physics or differential equations classes that want to show a bit of numerical ODEs, etcetera — classes that don’t teach “programming” at all (no data structures, little or no control flow besides loops for parameter sweeps, no modules or abstraction beyond simple functions…), and for which every “computational” assignment is just a set of very basic scripts in a Jupyter notebook.

7 Likes

Yes, that would be my case exactly. [You don’t want to be talking to MBA’s about environments :joy:]

2 Likes

Nothing to disagree there, but I’ve been in situations where even showing simple ODEs and letting the students play with parameters, modify the equations, etc, was much nicer after showing them Revise, setup a project, etc, even if only to avoid repeated TTFXs.

(a midterm alternative is Pluto, of course, you may consider it @compleat, I think it is more robust as a user experience than Jupyter, if one gets used to the reactivity and long startup time).

Hi. As a side note I forked a version of Pluto with reactivity turned off, (as well as run on startup likewise turned off so it starts immediately), called Neptune. It is under repair at the moment, though, because I haven’t had time to update it.

3 Likes

Using Pluto, one doesn’t need to learn about environments when doing basic calculations. I would introduce Pluto as the primary tool in non-programming classes that utilize Julia.

4 Likes

This is surprising since references/dependence on Grisu where removed in v0.9.19 of MathOptInterface way back in December 2020; so suggests use of a very out-dated version of JuMP?

1 Like

likely caused by the lack of use of environments…

I’m thinking something as simple as:

“when you want to start working on a new project in Julia, create a directory for your project, start julia in that directory, and then immediately type ] add X,Y,Z,P,D,Q where you replace those letters with the packages that are important for this particular project. This creates an “environment” for your new project where you are tracking the necessary packages and their versions separately from all other projects you are working on”

end of story. It’s really not that complicated. It can be considered just a “magical first thing you need to do to work in Julia”

4 Likes

Even this requires a bit more knowledge about command line options, or activate . and so on.

Note, that to do this effectively, you need to do at least one of:

  1. Start Julia with
julia --package
  1. In the REPL, do
]
activate .

Otherwise you risk adding everything to the global environment, which is what you’re trying to avoid.

1 Like