Well, for me, the bigger issue for me is not a language, but the culture that comes with it. If the language is Java or Go, but I’m allowed to write Python that generates its code or a Lisp interpreter in it, then I think I’m quite okay with that. On the other hand, if a project chooses Go for its simplicity, well, I’d just say that the project is not the right fit for me. If they can’t handle overloading, then they likely can’t handle Lisp interpreter either.
3 Likes
I remember seeing, but have yet to watch this presentation about using Julia as a backend to Python. The initial ease of transition to Julia via using it as a backend can also assist people in getting comfortable with Julia, especially when they see its speed. I’m also curious as to their motivation for using Julia instead of C as a backend.
I wonder if it would be a good idea to make videos about the Hackathon to show what can be achieved in Julia in such a short time. It might help convince people that programming in Julia is more productive than Python+Rust or Python+C++.
3 Likes
Once you wrap your head around multiple dispatch, single dispatch feels very limiting and very much like an artificial constraint.
14 Likes
Well, not “artificial” per se, more like design limitations. Multiple dispatch is really difficult to design and implement. You need to have all sorts of rule governing specificity.
3 Likes
I’m was a long-time user of R in a non-production environment, got curious about Julia and Bayesian at the same time I discovered what a fantastic self-learning tool AI can be if I remember to prompt with what I’m trying to do rather than asking how to perform some perhaps poorly chosen routine. I can also get walkthroughs and it even generates the doc strings that I’m usually too lazy to get around to.
After implementing a Bayesian presidential polling model, I’m starting to feel fluent, I never get bolloxed up on simple things, enjoy sensible defaults.
Me and Julia Down on the Keyboard
4 Likes
It’s particularly hard to do efficiently. Clojure’s multimethods are even more general than Julia’s multiple dispatch, but they’re slow, and I can’t think of any other even moderately popular language that does runtime multiple dispatch efficiencly.
1 Like
Thank you for sharing your experience and I hope that you can take some solace in knowing that there are others out there who experiencing similar frustrations in our workplaces - I’m one of them!
Your post reminded me of an anecdote. I hope it might offer a reason for optimism, despite workplace frustrations. It did for me 
A couple of years ago I spoke with a group of scientists at a local Julia event whose work included running computationally intensive calculations on C++ infrastructure.
For some time they had been aware of ways to improve their C++ tools and the benefits that such improvements would bring to their work - essentially reduced need for computational resources.
Nevertheless improvements were rarely made, because the cost-to-benefit tradeoff was rarely favourable such that highly trained, subject specific scientists could justify investing their limited resources into the improvement of their C++ infrastructure.
Then Julia arrived within the research group and it was trialled starting with implementations of some smaller parts of the existing infrastructure, to see if it would offer improvements. The Julia version of their infrastructure saved them computational resources, like they had expected.
Within a relatively short period of time, the group had begun transitioning the majority of their infrastructure into Julia.
Yet the expected performance enhancements were not the primary reason they chose to migrate their infrastructure from C++ to Julia - after all they had been working for a long time without these improvements and migrating came with costs.
They told me that Julia’s highly accessible code analysis tools empowered non-computer science experts to regularly contribute improvements for the tools which they relied on, which had not been practical for the C++ infrastructure.
The research group valued this and motivated them to transition to Julia. The greatly reduced costs for making improvements to their infrastructure were much more significant than the significant performance gains for that group at that time.
So in a way, the argument put to the decision makers was still ‘Julia is faster than our current infrastructure’, but we should be mindful to finish that sentence: ‘Julia is faster than our current infrastructure, because it is more accessible for our colleagues’.
Not every organisation will value this, but some might, particularly any similar to this research group with non-computer science experts requiring computationally intensive techniques for their work. I guess there must be many such organisations these out there!
This kind of accessibility for code analysis and improvement inherent to Julia (particularly compared to incumbent languages) is to its credit. The easier it is to spot problems and implement improvements, the more likely that people in an organisation might do so, and the more valuable that an organisations tools should become for what they need.
I believe this should lead to a competitive advantage for organisations which adopt languages like Julia in the long run.
Even if this is just a one-off anecdote, I found their story to be motivating in the face of headwinds in my workplace (and industry) to being able to make use of Julia where it has a lot of potential. Perhaps others may also find it a bit motivating as well
Cheers!
19 Likes
Sadly, my experience is a bit more bumpy about Julia-Python interop so far.
I am also at the position that trying to convince my workplace about considering some parts of our Python madness to be replaced by Julia. Transparent inter-operability is absolutely the key to make this acceptable to everybody here.
I have demostrated its superiority in a number of cases, but sadly, I had to realise that neither PyCall not PythonCall support multi-threading in Julia in a full extent, one of the key selling point of the change. So I am forced to call out to a Julia process via RPC.
Then came another disappointment: I was not able to find a robust off-the-shelf RPC library for doing this. Some RPC protocols are implemented, but still, lots of boilerplate has to be written to achieve that underlying Julia code is fully “hidden” from the viewpoint of the Python caller. I am trying to put together something myself, but obviously, that holds up real work.
I believe my situation is quite typical. Julia can only spread by evangelists injecting it whenever the situation allows it, and the community should concentrate on making this the easiest possible.
2 Likes
Well, for my simulations Julia offers a 100x performance improvement compared to Python without multi-threading.
And if you need multi-threading you can always use RPC as you explained. I have good experience with ZeroMQ, but I agree that generating the code for serializing and de-serializing data is a pain, even if you use protocol buffers. There is room for improvement.
2 Likes
You may like to try this unregistered, experimental package: GitHub - cjdoris/PyRun.jl: Reliable Python-Julia interop
It’s a Python-Julia interoperability library but runs python in a separate process. This avoids loads of the issues with PythonCall/PyCall - e.g. multithreading just works.
I suspect you want the reverse though (calling Julia from Python)?
3 Likes
Thank you! Unfortunately, my case requires the opposite direction indeed.
(BTW I believe calling Julia from Python is a more typical setup, as agents working for our cause likely need to smuggle J into an existing P pipeline, rather the reverse.)
I am also working on something similar in the other direction: JSONRPC over a domain socket + transferring dataframes as references to temporary parquet files. I could not find an easy way to transfer them in-memory. It is OK for my use case, but it is obviously not going to be as robust, fast or universal as a joint effort from seasoned coders could do.
I also need some more functionality, for example, to configure the location of the executable, choose virtualenv / environment, and define shell environment variables and parameters for the launched subprocess. It should be also able to create / maintain the julia environment on its own, that is, install J packages from scratch, and also update them on request from P.
Returning to the topic: we have understand that we need to spread the word in teams that are often reluctant to change. Ideally, if a team member, using our partially J code, does not want to know anything about J, our tools should make it possible that he does not need to.
1 Like
You can use pyjuliapkg for lots of that, it’s what JuliaCall uses.
1 Like
Honestly, I am on the other side of the fence. I get all excited and want to switch to Julia and every time I find the time to try something its so much slower that all my precious time feels completely wasted and I end up being the boss that decides against it 
.
Some explanation: I have been looking at Julia and lurked here for I don’t know how many years (started sometime between 0.4 and 0.5 when Atom was still a thing). I use primarily Matlab (about 17 years of experience with the last 13 almost daily).
My problem is, I am really struggling to find time to learn new stuff. Every project has to be done in a rush so I take what I have and make it work quickly through years of experience. Also (and very important) I am not a programmer. I really had to learn basic concepts like difference between x=1 and x=1.0 to even start.
So I spend my weekends or holidays if I want to learn something new, I get totally excited about the promised speed for julia, then I try to port or rewrite my matlab code and it ends up like 50% slower after battling one stupid issue after another, just because I suck at programming and don’t grasp basic julia concepts. Or there is a completely non-julia related issue that goes behind my back - I went crazy on one weekend when julia in atom just didn’t work after it had flawlessly only to find out after pulling my hair out and two reinstalls that my antivurs has decided that it doesn’t like the program anymore. Also I feel I have to come to the discourse for every little thing.
So I do want to switch to julia because I also like a lot of the things you mentioned but in the end ease-of-use and performance are my two biggest things and I just can’t write code from scratch and get it fast. For it to come I really need to know what I am doing and I don’t. I really don’t get the language. And in Matlab, honestly, I either don’t have to think about it or basic performance concepts are so engraved in me after close to two decades that I do it automatically.
I am currently writing a new program (again decided to try to port a Matlab code) and would soon post some code here to get some help (again) because I have no idea what I am doing. And there are many people here helping me and others but I can’t go every time to a person for help.
6 Likes
From what I’ve seen here the performance advantages of Matlab code ported naively fall into basically three things: 1) the use of MKL and multi threading setup in linear algebra; 2) slices vs views, and 3) not putting code in functions.
Fixing those generally gets the Julia code in par with the original Matlab code.
15 Likes
I actually do relate to this. My experience with programming out of University was some MATLAB, one course in C (which I almost failed) and one course in Julia. I forgot about Julia and started working at my current company, which used MATLAB mainly, and eventually they also got me helping out in C. My boss and colleagues were patient with me in learning both languages, and eventually I started majorly contributing in both languages.
As I learned both languages over the years, I would have so many experiences of the following forms:
-
Something so unintuitively or contradictingly designed. There would usually be some rationalising story behind it e.g. one design was made to try and improve on the other, so they were made years apart, not simultaneously. Alternatively another unintuitive or contradicting design I’ll say comes from MATLAB. I live in Australia, and the analogy I like to share is that MATLAB was built like Sydney - just add features as they come without forethought of the design. Melbourne was built like Julia: planning ahead the major features to work coherently (to say nothing of Melbourne’s mishaps but that’s beside my point).
-
The two-language problem. Need I say more? My company has a big program in C++ with a lot of mathematical algorithms. In order to test their fidelity to literature what do I need to do? Re-implement them in MATLAB. Discover and prove what needs to be improved. Make the equivalent changes in the big program in C. Kill me now.
Along the way I also remembered my Julia experience at University and became exposed to the proliferation of programming languages and the reasons for their creations. Julia stuck out to me not because I used it in University, but because it was made by scientific programmers for scientific programmers, who have a deep understanding of what’s going on at the low level. That alone along with demonstrations of its use and performance was enough for me to find Julia a language to invest in and goal-wise call my primary programming language.
So I got to work, started learning Julia. And there have been so many moments of confusion and hair-pulling like your experience @Boris that I’ve taken many breaks from Julia, but with the plan to tackle my problems in the future with another attempt and a fresh mind. Sometimes my problems relate to mental health; a lot of the time due to my lack of knowledge with programming languages. But either way learning Julia has been both frustrating but also satisfying when things work out, or when I benchmark against MATLAB and find Julia wonderfully better, or against C++ and I find myself justified in its competitiveness.
The thing is, I’ve also had those hair-pulling moments in C++ and MATLAB as well. Take boilerplate for example: I’ve recently got the hang of Julia macros and have started replacing my repetitive Julia code with macros and it’s so satisfying. And the speed is still present. Can’t do this in C. Can’t do this in MATLAB. Additionally, I can take a break from Julia throughout my years of learning it. Not so at work - I have to go to work the next day to continue using MATLAB and C. Even with how much I know about MATLAB, I am cursed with my knowledge and experience of Julia in knowing that things could be so much better - the original motive of this thread.
I relate to your experience @Boris, and empathise with you. I find Julia a worthwhile investment of time in the present for an improvement of experience and efficiency in the future. I accept the learning curve I am going through while still pulling my hair out, albeit less frequently as time goes on, and am improving in when to ask people for help, when to take a break, and when to take a step back and try to understand something from another angle. Hang in there, I believe this journey to be worth it.
8 Likes
Thank you @jlawrie, that anecdote is THE DREAM.
And your bold-face paragraph is the point. All the C++ programmers can keep C++ as much as they want to. But scientific programmers with little-to-no experience in low level languages will find it hard to understand and contribute to. And the low level programmers will experience their own learning curve of understanding the esoteric subject they’re implementing for. Mumble mumble leading to the two language problem. There’s a graph somewhere of this divide between low level programmers and subject matter experts that I can’t find, that illustrates this phenomenon.
Julia solves my two-language problem at work, demonstrably so for me. My boss and colleagues just don’t understand it - hopefully yet. They’re praising and encouraging of me, but not adopting of what I’ve demonstrated, possibly illustrating a praise out of kindness but not of empathy. But I’ll keep trying, thank you for your encouragement and justification to you and everyone else in this thread.
As my workplace uses MATLAB and C++, I like to say “Julia is legible like MATLAB and performant like C++.” This hasn’t caught on as much as I’d like. I feel that’s partially due to how abstract my statement is.
I could probably improve it with “…so it’s easy for me - non-programmer - to contribute to, unlike C++, yet it’s also easy to get performance like C++.”
Does anyone have any better ways to express this concept? “It solves the two-language problem” is a nice hook because those who’ll bite then ask “what’s the two language problem?”
I also really like:
2 Likes
That is absolutely true and I’ve had that in my codes. Actually the last time I tried to learn I took a baby example and with the help of the forum and one individual in particular who went above and beyond the code did beat the Matlab time by a factor of 10 and all those three were in play.
By the way, I don’t want it to be on par, I need it to be faster. I am putting a lot of time to learn the language and the majority of the academic work in my field is in Matlab so if I switch I always have to translate code from others and pay high fixed costs. So I need it to compensate for that.
And @kapple I totally get what you are saying. That is why I also keep coming back again and again. And I can especially relate to the point of grasping and using something existent in Julia and then going back and missing it a lot. For me, maybe a stupid thing, but having ALL functions in one file (module) is just awesome and I very quickly began to hate the folders full of support functions that I have in my Matlab projects . And I haven’t even grasped the really nice and powerful concepts in Julia yet.
2 Likes
MATLAB classes and packages are such a huge pain. Julia’s flexibility with files and modules are such a relieving escape from the inflexibility of MATLAB. And don’t get me started on the MATLAB testing framework. In Julia, tests are for the most part normal code.