Convincing physicists that Julia is worth their time and effort


#21

I could imagine an announcement like this:

Julia is a modern open source language which aims to make it easy to write reusable, high level code that is also very fast. It has a sophisticated parametric type system and multimethods, and compiles functions with specific concrete type signatures into highly efficient machine code using the LLVM backend. Calling code from C, Fortran is natively supported, while Matlab, R and Python interfaces are provided by libraries. It has a rich and rapidly developing ecosystem of packages, including differential equations, automatic differentiation, and MCMC.

which you may want to tailor further to your audience.

In the presentation I would emphasize that while the language is pretty mature, some libraries are work in progress and their interface may change.


#22

Is the possibility to put these benchmarks in presentation without setting Numpy, Lua-JIT, etc. and running Jupyter nootebook on ones own machine and saving result in some png/pdf file? https://julialang.org/benchmarks/

These format is designed to browsers display and look very poorly after downloading.


#23

“they don’t want learn new stuff”

I have made a similar observation in the circuit design world as well. Though I have no solution to this issue, I think I have a bit of a grasp on what causes this mentality. Being aware of the mindset might help with how you address your audience:

  • People doing science/engineering are trying to solve real world, multi-facited problems. They are typically under time pressure to do this, so they have a hard time prioritizing where to put their efforts (Is it worth my time to learn this new language? Is it going to help me solve my problem faster - or is it just going to consume my time, and actually slow down how quickly I get to converge on to the solution?)
  • Unlike when you are a programmer, the software is NOT the solution, it is merely the path you chose to try to understand a phenomenon.
  • People have already suggested (QUITE OFTEN) that I to learn something that will supposedly make me more efficient. After much investment in my own time, most of those suggestions (feels like 75%??) just seem to have ended up being of little value. Why should I listen to you know?
  • It is very difficult to predict how long it will take to learn a new tool (like a programming language)… So you can’t really make an educated guess on how much time to invest on learning a new one. Your colleagues might feel more comfortable if they got to see real-world working examples that are already solving issues they are currently working on (especially if they are easy to understand/read).
  • Their current solution might not be blazing fast (either to modify or run), but they might not feel that the software is the real bottleneck to finding a solution a the moment. Sure, the software is a bit slow, but why would they waste time learning something new if what they are using does not appear to be consuming a significant portion of their overall time?
  • People might not actually be aware of how much brain power they are diverting from their issue in order to run the tools. They also might not be aware that they avoid looking at certain facets of a problem simply because the software implementation would be impractical (ie take too long) with the tools they currently use.

Hilight the familiarity of Julia

One thing you might want to do is try to focus on how Julia is not that different from what they know… But better. Though you can write somewhat similar looking code with Julia than you can with Fortran, my guess is that people want to use their Fortran code because it does complex algorithm that involved many man-hours to create. As others have suggested, it is probably better to show how you can easily call/use Fortran code from Julia, and focus on the similarity between Julia <–> Matlab for high-level code (what most scientists are probably dealing with):

  • Being in the field of science, most of your colleagues probably have experience with Matlab. You might want to show a typical Matlab solution for your field (maybe one that doesn’t involve specialized toolkits), and compare (side-by-side?) how it looks in Julia.
  • Hilight a few really cool things (even if they might seem mundane), like how you can create a fuction on the fly as simply as writing f(x)=3x. What’s more familiar/natural to scientists than defining a function the same way you would write any old mathematical function???
  • Oh yeah, did I mention you don’t even have to think about vectorizing every aspect of your solution… and there is a good chance that it will be fast anyhow??

You can talk about how your solutions end up being easier to maintain than with Matlab:

  • One of the things I dislike about Matlab is the idea of needing one file for each function you create. Though not conceptually a real issue, it seems that this simple overhead for writing a function sort of influences people to write large, monolithic functions that cannot be re-used easily… Nor can they easily be read/understood!!
  • Functions in Matlab used to be pass-by-value, which made decomposing a solution in multiple functions a slow proposition (in terms of run time). That has changed (somewhat recently) by allowing function arguments to be passed by reference, but from what I have seen, the syntax is somewhat unpleasant to look at, which kind of makes you shy away from using pass-by-reference - unless you really have to.

…Just remember to make sure you don’t over-sell it. That might your colleagues reject Julia after the first hicup:

  • There is a good chance you might not have all of the features of the Matlab toolkits with which you are accustomed. And if you do, you might have to do a bit of learning in order to figure out the quirks of using the Julia package (different API, etc).
  • I too like the idea of IJupyter notebooks. I wish I could recommend them. Sadly, I found that my kernel often dies, and the editor (browser) is slow for some reason that I simply cannot justify. I would possibly tell my colleagues about how interesting Jupyter is conceptually (especially how you can “inline” documentation with the code, using Markdown formatting)… But that the slowness might be aggravating to some.
  • (…)

Visualization tools

In my experience, the simulation is not actually the most important part of your solution. If you have no good way to see the data, then your simulation is basically useless:

  • Show how easy it is to do interactive plotting using Plots.jl. Can write plot(x,y) like in Matlab.
  • Show examples they can relate to (plots your colleagues are regularly looking at).
  • Suggest they choose/use a particular plotting “backend” directly when they write their final functions that plot results. The time to first plot of Plots.jl is relatively long still, and some people might find the wait to be annoying.

In case your colleagues use plots that are similar to those analog circuit designers use, you can look at the examples provided with my own plotting backend, InspectDR.jl:

They might also like the little applets I wrote that allow you to do simple interactive/exploratory work with plots:

Calling C/Fortran

I have a concrete example I used to wrap simple (yet very important) pre-existing C code that reads in SPICE simulator data:

This example is very nice, because it is quite simple. My guess is that you will not feel overwhelmed by the amout of Julia wrapper code.

Note that it was later ported to a pure-Julia implementation… because it is just simpler to maintain since you don’t need to compile the C library separately:


#24

I think you’re right on all points. It’s also worth remembering that your colleagues that don’t think it’s worth their time to learn Julia might be right. Your point about software not being the solution is an important one, and we may do a disservice to Julia if we push it on people that don’t need it.


#25

I am a little bit sick and deliver presentation on Friday 23 November (here is 00:16 am, 22 November), so I probably don’t correct all my mistakes on time (my English is not to good). Regardless of that I will be grateful for all corrections and pointed errors. Two Jupyter Nootebooks are also parto of presentation.

Talk is probably to long for 60 minutes, I try to make it more compact today, there all also big things missing. Notebooks have lots of staff that will not be shown, there are designed to use at different talk.

When this end I replay to all your comments. Thank you for help.


#26

This thread reminds me to comment. What scientists and engineers need is to work alongside the new breed - Research Software Engineers. https://researchsoftware.org/
There are RSE organisations in DE, UK, NL, Nordics.
I think this fits well with the concepts above - the scientists should concentrate on their models and algorithms, with the RSE being on hand to advise on the languages and platforms - the RSEs having some experience in science/engineering of course.
Warming to my theme, think of the surgeon performing operations on a series of patients. His efficiency is boosted by having around him a team of experts dedicated to each function - assistant, nurses, technicians who sterilise and prepare the instruments. You dont get more efficiency by hiring more surgeons and having them work elbow to elbow.
In this analogy I see the RSE as the anaesthetist - a doctor (scientist) in their own right, but dedicated to keeping the patient (the code and platform) in top condition. Indeed in keeping the patient alive!


#27

While I can see the appeal, am sceptical of this model in basic research conducted by small teams. Implementation usually goes in tandem with methodological development, and the requirements are often difficult to specify. Delegating programming to someone who is not deeply immersed in the project itself usually makes turnaround times much longer.

So, while it would be great to have someone do my programming, I think the reality is that some scientists need to program (this does not imply that everyone on a team of collaborator needs to). This is similar to typesetting: in principle using LaTeX + plots has little to do with my core skills as a researcher, but delegating it is not practical.

I think the appeal of Julia is precisely that it makes programming simpler.


#28

Maybe! So I then see the RSE as a magus who sits in a gilded cave at the top of a mountain. OR more likely an office at the top of the building. They will hand you a wand and a robe and tell you “GO FORTH and do good work with this wand” and “Cover yourself with this robe while using corrosive chemicals” (*)

The next time you visit the cave you may be given a new, powerful wand. The Wand of Julia.
You will be told to “Undertake a quest to slay PYTHONS”

(*) Programming language pun intended.


#29

I also find that coding something makes me understand it better. In fact, I would say that I don’t fully understand something until I have been able to put it into code. The act of coding is a crucial part of solving problems and developing ideas.

This is similar to the mental process you go through writing a paper or a report (but here I don’t mean typesetting). If you cannot express your ideas in a way that is readable to someone else, you’re not really done with it.


#30

“Programming is understanding”, :wink:


#31

As an addition: If you figure out a cool new algorithm, then you want to beat the established optimized C/C++ libraries. If you happen to code your inner loops in python, then this is freaking hard. If you code in julia, then the cross-over is much lower, and writing fast julia is easier than fast C/C++.