Great post @frylock. I can imagine that’s how Python was adopted by many companies, as various specialized libraries were developed to allow it to participate in different niches.
I will bet that Python, R, MATLAB, and many other languages have Julia on their radar, and are asking “What can we do to be more like Julia?”
I believe that when they look deeply, they will find a lot of challenges, especially how they have evolved over time. Someone more experienced with Julia can correct me, but I have read that most (maybe all) Julia libraries are written in Julia. This is not the case with other languages, so even if Python, R, etc… offer more JIT compiler support, there will still more inherent challenges. They can create wrappers for Julia… pyjulia… why not just use Julia?
Now, I am not saying that Julia is the “chosen one”, the final programming language… but perhaps for our current architectures it is close to it?
I think blog posts help A LOT. I have subscribed to Towards Data Science because they do a great job of writing articles about Julia and providing excellent use cases. Industry blogs and newsletters would be a great way to have Julia be noticed, the articles can differentiate Julia from the current meta solution.
Also, the training material that I have come across is indeed well written. The authors of the material do a great job at targeting potential readers. For example, Flux has some really nice ML examples with more technical explanations… which is exactly what I would expect, while Pluto and Genie examples are less technical, but focused on showing how it is easy to create dashboards or web apps using Julia.
My only complaint (and again, I did it to myself ) would be configuration of Julia environment. I already had VS Code installed, chose to install Julia manually, and with zero experience… I did not realize that VS Code resets the PATH variable constantly… and you can actually do a whole lot in a REPL that is not properly configured, like add the packages for dependencies… and then find out you don’t have them installed when you try and run your project. It was worse for me because Genie Builder creates its own project structure in VS Code… but again, it was my fault, because I rushed right into write code, write code, write code!!! and caused my own problems
Fortunately, juliaup and reading the “Get Started” docs was the answer to this…
I see a lot of other people have difficulty getting the environment up and running… so take my advice, and use juliaup and read the docs. Everything (so far) is working out really well now that it was sorted out.
Yes, I think the barrier to creating a proper workflow without guidance is one of the major problems Julia has for early adopters. It gives bad first impressions.
Maybe it would help if juliaup could be ported into a VS Code extension. Maybe still have it available for download for those that use other IDE. I think it might be more straightforward.
The thing is… it’s not Julia language’s fault, but it does turn off new users.
Dropbox was a pretty significant catalyst in achieving this goal. Back when Python was less popular/mature in 2007, Dropbox’s success demonstrated Python’s scalability to the corporate world - it was shedding its reputation as a toy language for one-off projects. Rust started off in Mozilla and very early on, their executives were open-minded enough to see the value in rewriting their backend. (Of course, the real situation is more complicated than this - Dropbox wasn’t the only player in town, etc.)
The point is, every popular language has had to overcome a chicken and egg problem somehow, and the commonality is being spearheaded in industry and demonstrating a value proposition to the bigger players. If the question in this thread is being asked, then Julia has largely not done this yet. As to why this hasn’t occurred, I’m not sure - there’s probably multiple reasons. My speculation:
Problems with scalability - the discussion around FromFile.jl a while back suggested that the “idiomatic” way to organize file dependencies in large codebases is unnecessarily painful, and perhaps this is a problem that happens to be run into much less often in the scientific world
More generally, maybe people have underestimated the differences in priorities between a general-purpose programmer and a scientific programmer, and Julia’s design decisions crystallize those differences in a way other languages don’t. It’s not inherently designed for a specialized domain like Mathematica is, but maybe there are forces independent of the existing ecosystem that nonetheless push the language there.
One of the largest parts of scientific computing used in industry, AI/ML, isn’t yet mature in Julia, owing to a combination of shortage of manpower and more ambitious autodiff capabilities with the libraries that are in development. The responses I’ve seen with Zygote so far have been very mixed. Julia certainly has the potential to be very capable here, but again, you have to have engineers being paid full-time for this work.
The bar is higher for a startup now because the ecosystem is so much more developed. (Higher interest rates don’t help either.) The product produced would have to demonstrate some significant improvement in capability. One of the things I’ve read is that frameworks like Pytorch/Jax softly limit the space of problems being investigated in industry to those which are readily tackled by these tools, and therefore there should be regions in this space being overlooked where Julia has the potential to shine. But to be attractive to industry, it has to also do most everything else that Pytorch/Jax can do.
The alternative is for a group like Meta AI to charitably pour some resources here into an unfamiliar language for them, but I think this is even more unlikely given their interests in Pytorch.
One question to ask is “What is Julia really FOR?” And I think the answer is “it’s for power users of scientific and numerical computing”. It’s biggest competitors in some sense are Matlab, Mathematica, R, and Python. You wouldn’t use Julia to write a new C compiler or to make a realtime 3D game or a command line tool for collecting ping statistics on an OpenWrt router or stuff like that.
Some of that may change. If there’s eventually a way to compile small binaries maybe ping statistics on a router will work out. If the GC latency drops dramatically maybe you could make some games.
But for the most part, Julia is for numerical and scientific users. IMHO a good way to make it more popular would be some kind of reproducible but interactive Engineering tools. There are about a bazillion Excel spreadsheets used in design offices each day to calculate what kind of pump is needed or how much head loss there will be in a pipe or what the bending load that causes lateral buckling in a W flange beam is etc.
How about interactive analysis tools for Bioinformatics data for bio labs. Most biologists don’t program at all, though some do. But there are many more who get RNA seq data and would currently shove it through something expensive and site licensed like Partek flow or whatever.
A data explorer and analysis tool like JMP except with more Bayes and more connections to things like Turing would be really good.
Basically there are a lot of commercially viable ideas for taking Julia and making good interactive science and data analysis tools for science end users. The real limitation IMHO is that all of these things have relatively heavy competition and doing this would require a few tens of millions of dollars and a decade of development. It’s not clear how you get that investment.
Maybe you read it on the blog of Chris Rackauckas (can’t find the exact reference though)? In any case, I believe that Julia was one of the reasons that PyTorch, Tensorflow etc started implementing their own ODE solvers – in Julia you could just take the existing solvers and use them with ForwardDiff or define a Zygote adjoint. In PyTorch you have no choice, but to reimplement the solver.
As mentioned in another thread already, I believe that the composability of Julia actually hinders its adoption. I.e., the above example of combining autodiff with ODEs is an instance of that issue: PyTorch, Tensorflow etc all bring their own incompatible tensor types – effectively locking the user into the ecosystem of its creator. In Julia this would be much harder to achieve as different data types can be extended and exchanged much more easily. You could even mix and match parts from different vendors – how dare you!
Genie isn’t just about the web. The GenieFramework also includes no-code and deployment solutions. If you’re working in a corporate setting, you’ll likely need these tools at your disposal. I believe that very few people in the Julia community want to build serious full-stack web applications, but many people want to build visualization dashboards, deploy these dashboards in the cloud, and add authentication on top of them. They want to render important data in tables and share it across the organization. If you want to build a web service or a full-stack website, I would recommend Python, Node.js, Golang, or Java. This recommendation could, however, change if Genie attracts more open-source contributors.
"IMHO a good way to make it more popular would be some kind of reproducible but interactive Engineering tools. There are about a bazillion Excel spreadsheets used in design offices each day "
True. Most companies in my experience depend heavily on Excel for many tasks.
For the engineering tools you refer to that is a perfect place for Pluto notebooks IMHO.
You can imagine a Pluto notebook where you put in the values and plots are interactively produced.
OK - Excel can do plots. But how about full 3D renders such as Makie can do? Or plugging in Julia’s rich automatic differentiation framework etc. etc.
My biggest problem, having tried using Pluto to replace Excel, is that you can’t just have someone double click in File Explorer to open the notebook. They have to install Julia, install Pluto into an environment, open the REPL, using Pluto, Pluto.run(), then use the not-very-good built-in file browser box in Pluto to find and open the Notebook. Everyone gave up on me before they even got the notebook open.
Something similar perhaps but the other problem with Pluto notebooks is they are permanently modifiable. If you want a kind of interactive pocket calculator, you want a kind of AC button that resets it to it’s out of the box state. I need to play with Genie but for the stuff I’m thinking of that might be a better option.
@AbhimanyuAryan I think you are right on! By the way, thank you for all of the excellent plot examples using Genie/Stipple!! They have been excellent resources!
Many companies claim to develop low-code to no-code solutions for building business processes. Everything from Microsoft 365, Google Workspace, down to more specialized packages like Automation Anywhere and Laserfiche.
They are quite expensive, and overkill for most businesses that are not bon-a-fide enterprises. Google Workspace and Microsoft 365 are not very effective because their Forms processes rely on spreadsheets and actions must be taken by Google Apps Scripts or Power Automate. Laserfiche is great, and it’s competitors are probably pretty good too.
I think it would be very interesting if Genie, and specifically Genie Cloud, could one day become the very important graphical building blocks for business process automation. Julia and your favorite database may power the back end and provide functionality like SQL queries, OCR, automatic form population, automatic email, etc… to actually build end-to-end digital business processes.
) would be configuration of Julia environment. I already had VS Code installed, chose to install Julia manually, and with zero experience… I did not realize that VS Code resets the PATH variable constantly… and you can actually do a whole lot in a REPL that is not properly configured, like add the packages for dependencies… and then find out you don’t have them installed when you try and run your project. It was worse for me because Genie Builder creates its own project structure in VS Code… but again, it was my fault, because I rushed right into write code, write code, write code!!! and caused my own problems 
They have been excellent resources!