Convincing physicists that Julia is worth their time and effort

Community Teaching & Outreach
21

I could imagine an announcement like this:

Julia is a modern open source language which aims to make it easy to write reusable, high level code that is also very fast. It has a sophisticated parametric type system and multimethods, and compiles functions with specific concrete type signatures into highly efficient machine code using the LLVM backend. Calling code from C, Fortran is natively supported, while Matlab, R and Python interfaces are provided by libraries. It has a rich and rapidly developing ecosystem of packages, including differential equations, automatic differentiation, and MCMC.

which you may want to tailor further to your audience.

In the presentation I would emphasize that while the language is pretty mature, some libraries are work in progress and their interface may change.

3 Likes
22

Is the possibility to put these benchmarks in presentation without setting Numpy, Lua-JIT, etc. and running Jupyter nootebook on ones own machine and saving result in some png/pdf file? Julia Micro-Benchmarks

These format is designed to browsers display and look very poorly after downloading.

23

“they don’t want learn new stuff”

I have made a similar observation in the circuit design world as well. Though I have no solution to this issue, I think I have a bit of a grasp on what causes this mentality. Being aware of the mindset might help with how you address your audience:

  • People doing science/engineering are trying to solve real world, multi-facited problems. They are typically under time pressure to do this, so they have a hard time prioritizing where to put their efforts (Is it worth my time to learn this new language? Is it going to help me solve my problem faster - or is it just going to consume my time, and actually slow down how quickly I get to converge on to the solution?)
  • Unlike when you are a programmer, the software is NOT the solution, it is merely the path you chose to try to understand a phenomenon.
  • People have already suggested (QUITE OFTEN) that I to learn something that will supposedly make me more efficient. After much investment in my own time, most of those suggestions (feels like 75%??) just seem to have ended up being of little value. Why should I listen to you know?
  • It is very difficult to predict how long it will take to learn a new tool (like a programming language)… So you can’t really make an educated guess on how much time to invest on learning a new one. Your colleagues might feel more comfortable if they got to see real-world working examples that are already solving issues they are currently working on (especially if they are easy to understand/read).
  • Their current solution might not be blazing fast (either to modify or run), but they might not feel that the software is the real bottleneck to finding a solution a the moment. Sure, the software is a bit slow, but why would they waste time learning something new if what they are using does not appear to be consuming a significant portion of their overall time?
  • People might not actually be aware of how much brain power they are diverting from their issue in order to run the tools. They also might not be aware that they avoid looking at certain facets of a problem simply because the software implementation would be impractical (ie take too long) with the tools they currently use.

Hilight the familiarity of Julia

One thing you might want to do is try to focus on how Julia is not that different from what they know… But better. Though you can write somewhat similar looking code with Julia than you can with Fortran, my guess is that people want to use their Fortran code because it does complex algorithm that involved many man-hours to create. As others have suggested, it is probably better to show how you can easily call/use Fortran code from Julia, and focus on the similarity between Julia <–> Matlab for high-level code (what most scientists are probably dealing with):

  • Being in the field of science, most of your colleagues probably have experience with Matlab. You might want to show a typical Matlab solution for your field (maybe one that doesn’t involve specialized toolkits), and compare (side-by-side?) how it looks in Julia.
  • Hilight a few really cool things (even if they might seem mundane), like how you can create a fuction on the fly as simply as writing f(x)=3x. What’s more familiar/natural to scientists than defining a function the same way you would write any old mathematical function???
  • Oh yeah, did I mention you don’t even have to think about vectorizing every aspect of your solution… and there is a good chance that it will be fast anyhow??

You can talk about how your solutions end up being easier to maintain than with Matlab:

  • One of the things I dislike about Matlab is the idea of needing one file for each function you create. Though not conceptually a real issue, it seems that this simple overhead for writing a function sort of influences people to write large, monolithic functions that cannot be re-used easily… Nor can they easily be read/understood!!
  • Functions in Matlab used to be pass-by-value, which made decomposing a solution in multiple functions a slow proposition (in terms of run time). That has changed (somewhat recently) by allowing function arguments to be passed by reference, but from what I have seen, the syntax is somewhat unpleasant to look at, which kind of makes you shy away from using pass-by-reference - unless you really have to.

…Just remember to make sure you don’t over-sell it. That might your colleagues reject Julia after the first hicup:

  • There is a good chance you might not have all of the features of the Matlab toolkits with which you are accustomed. And if you do, you might have to do a bit of learning in order to figure out the quirks of using the Julia package (different API, etc).
  • I too like the idea of IJupyter notebooks. I wish I could recommend them. Sadly, I found that my kernel often dies, and the editor (browser) is slow for some reason that I simply cannot justify. I would possibly tell my colleagues about how interesting Jupyter is conceptually (especially how you can “inline” documentation with the code, using Markdown formatting)… But that the slowness might be aggravating to some.
  • (…)

Visualization tools

In my experience, the simulation is not actually the most important part of your solution. If you have no good way to see the data, then your simulation is basically useless:

  • Show how easy it is to do interactive plotting using Plots.jl. Can write plot(x,y) like in Matlab.
  • Show examples they can relate to (plots your colleagues are regularly looking at).
  • Suggest they choose/use a particular plotting “backend” directly when they write their final functions that plot results. The time to first plot of Plots.jl is relatively long still, and some people might find the wait to be annoying.

In case your colleagues use plots that are similar to those analog circuit designers use, you can look at the examples provided with my own plotting backend, InspectDR.jl:

They might also like the little applets I wrote that allow you to do simple interactive/exploratory work with plots:

Calling C/Fortran

I have a concrete example I used to wrap simple (yet very important) pre-existing C code that reads in SPICE simulator data:

This example is very nice, because it is quite simple. My guess is that you will not feel overwhelmed by the amout of Julia wrapper code.

Note that it was later ported to a pure-Julia implementation… because it is just simpler to maintain since you don’t need to compile the C library separately:

17 Likes
24

I think you’re right on all points. It’s also worth remembering that your colleagues that don’t think it’s worth their time to learn Julia might be right. Your point about software not being the solution is an important one, and we may do a disservice to Julia if we push it on people that don’t need it.

6 Likes
25

I am a little bit sick and deliver presentation on Friday 23 November (here is 00:16 am, 22 November), so I probably don’t correct all my mistakes on time (my English is not to good). Regardless of that I will be grateful for all corrections and pointed errors. Two Jupyter Nootebooks are also parto of presentation.

Talk is probably to long for 60 minutes, I try to make it more compact today, there all also big things missing. Notebooks have lots of staff that will not be shown, there are designed to use at different talk.

When this end I replay to all your comments. Thank you for help.

1 Like
26

This thread reminds me to comment. What scientists and engineers need is to work alongside the new breed - Research Software Engineers. https://researchsoftware.org/
There are RSE organisations in DE, UK, NL, Nordics.
I think this fits well with the concepts above - the scientists should concentrate on their models and algorithms, with the RSE being on hand to advise on the languages and platforms - the RSEs having some experience in science/engineering of course.
Warming to my theme, think of the surgeon performing operations on a series of patients. His efficiency is boosted by having around him a team of experts dedicated to each function - assistant, nurses, technicians who sterilise and prepare the instruments. You dont get more efficiency by hiring more surgeons and having them work elbow to elbow.
In this analogy I see the RSE as the anaesthetist - a doctor (scientist) in their own right, but dedicated to keeping the patient (the code and platform) in top condition. Indeed in keeping the patient alive!

5 Likes
27

While I can see the appeal, am sceptical of this model in basic research conducted by small teams. Implementation usually goes in tandem with methodological development, and the requirements are often difficult to specify. Delegating programming to someone who is not deeply immersed in the project itself usually makes turnaround times much longer.

So, while it would be great to have someone do my programming, I think the reality is that some scientists need to program (this does not imply that everyone on a team of collaborator needs to). This is similar to typesetting: in principle using LaTeX + plots has little to do with my core skills as a researcher, but delegating it is not practical.

I think the appeal of Julia is precisely that it makes programming simpler.

6 Likes
28

Maybe! So I then see the RSE as a magus who sits in a gilded cave at the top of a mountain. OR more likely an office at the top of the building. They will hand you a wand and a robe and tell you “GO FORTH and do good work with this wand” and “Cover yourself with this robe while using corrosive chemicals” (*)

The next time you visit the cave you may be given a new, powerful wand. The Wand of Julia.
You will be told to “Undertake a quest to slay PYTHONS”

(*) Programming language pun intended.

29

I also find that coding something makes me understand it better. In fact, I would say that I don’t fully understand something until I have been able to put it into code. The act of coding is a crucial part of solving problems and developing ideas.

This is similar to the mental process you go through writing a paper or a report (but here I don’t mean typesetting). If you cannot express your ideas in a way that is readable to someone else, you’re not really done with it.

3 Likes
30

“Programming is understanding”, :wink:

2 Likes
31

As an addition: If you figure out a cool new algorithm, then you want to beat the established optimized C/C++ libraries. If you happen to code your inner loops in python, then this is freaking hard. If you code in julia, then the cross-over is much lower, and writing fast julia is easier than fast C/C++.

1 Like
32

This was not easy month, so I apologized for not responding for so long. Now I write some remarks about seminar and respond in few days to posts here.

Talk was in form of combination of slides and programming using Jupyter Notebook, mostly in the form of running written before code, with some short, true live coding. Given time constrains, 50 or so minutes, I think that was good choice, showing in what way Julia is dynamic language and how compilation of code and reusing it really looks like. Even if compiling take some time I think that is the best way to show nature of working in Julia (at least how experinced working with it). I only reloaded some packages like Plots to not used to much time for that.

I get one major criticism: that I started from reason to not learn Julia. Maybe I also listed to much such reasons, since I really don’t want oversell anything and I generally not very optimistic person, so naturally talk about problems. I don’t know how much this critic is valid, since colleagues of mine who makes this comment is quite odd fellow, hard to say if others have similar feelings.

For I think was big mistake, I don’t make one slide for methods of optimizing Julia. I was thinking that things like type stability was to specific topic for such talk, but in retrospect even this is in someway esoteric topic, make information how optimization looks like was big mistake. Even if type stability may sound scarier that it is, it better make a point, that you must learn something new to make your code fast.

There were also two comments from younger part of audience, that I don’t consider as strict criticism. One that Julia optimization is very odd, while C is very natural. Personally, I don’t see anything natural in C optimization, we just learn it from the cradle, so it looks like natural thing. Also Julia optimization don’t look odd to me, just different than in languages like C.

Second was, that C can be hard to learn, but after learning it we can do anything since, we know what compiler is doing with our code and this is reason why we should prefer C. Recently I lost my personal faith in that I understand what compiler is doing and I start thinking since every language separate (assembler and things like that excluded) me from hardware, it should separate me as much as it can reasonably do it. I didn’t have any other answer to this person.

Response. From older people one was interesting in Julia, one decided that is better to learn again Maple (he was never computer stuff guy, so this was predictable), about younger part I write above.

I want to thanks everyone from Discourse, that help to make this presentation or say how can be improved.

Seminar with used Jupyters Notebooks is here. But this is note version that I presented, since this is too much for 50 minutes and many parts need more work, so I removed them from presentation. Also references are really long, because they sometimes as for sending presentation to study it, so I want them to be in some way guide to people. Also nootebooks are too long for 50 minutes, but I make them hopping that I will reused them in the future.

4 Likes
33

I know that in my presentation they are mistakes that I don’t notice and some part can be dubious, so I happy to here any of your corrections and comments.

34

IMO. Julia did a great job in interoperability. I tested creating function pointers in Julia for c/c++ library. Works great for large problems. ccall and Cxx are silver bullets

35

I apologize for silence, I must now end few projects and smaller things, still don’t have enough time to respond to many very insightful comments.

Maybe on Sunday I finally find a time.