What steps should the Julia community take to bring Julia to the next level of popularity?

As a long time user of R, and occasional user of Python, Fortran, etc., my perspective is that one uses the tools that work for you. Julia has passed a threshold in utility where not only are cutting edge developers doing their thing (which is what you want in terms of creativity, in the long term) but also base level utility for the every-day stuff has stabilized to the point of being able to do what needs to be done in compact and elegant ways. So aesthetics had some influence in my transition. However, that was not enough. I was held back initially because many tasks were already implemented in one language or another. Why recreate the wheel? The incentive to transition the mundane stuff was low.

The critical factor was access to cutting edge approaches that are rapidly and easily deployed in Julia. This is where incentive comes in for me. Brute speed and the new methods that this makes possible.

So, all in all, I would say, keep doing what you are doing to the devs. I am happy. (Hat’s off)

However, to make the initial transition easier for others once the use case becomes clear, perhaps make a concerted effort to remove outdated tutorials and documentation. Amalgamate best practices with working examples in one central area. And did I mention, documentation, with many examples as use cases are so varied, especially where in XX language it is done this way, here it is in Julia … and so much faster, clearer, etc,

Thank to all the tireless devs.
Jae

13 Likes

Agreed. The more relevant question is: assuming that broader adoption is the goal, what action can each person take individually to move towards that goal? And the answer probably is to contribute in areas that are likely to be useful to most. That may or may not intersect with one’s area of expertise for instance.

My area of (relative) expertise is quantitative finance, where there are few users of open source solutions (not a dig at them, especially as I contributed some C++ code to the project over 10 years ago, but QuantLib doesn’t have a high level of adoption in banks or hedge funds).
Where I think Julia could attract more users? As stated before, in the area of small glue scripts (where Perl, now Python, are pretty good at).

So my area of expertise doesn’t really intersect with the area where I think Julia could gain broad acceptance. If I was involved in Julia internals though, I would work on static compilation and tooling. If I was a package developer, on system stuff. In the meantime, I can only use Julia where it makes sense for me, hoping that the above happens (as I would stand to benefit from it, from the point of view of a more mature language and ecosystem).

Back to the original question, I would suggest we take a look at areas suffering from 2(+) languages problems. Game development could be one aspect. There could be many more. Other than that, make Julia a top-choice programming language for newer projects.

4 Likes

IMHO Julia is already the top choice for programming technical/scientific computing (TSC) packages (by far).

Yet it is not dominant in this specific area.

I am more interested in Julia keeping on growing in the TSC community (TSCC) than reaching other domains (games…).

Hence the reasons that are slowing down the adoption in the TSCC tend to be more relevant to my eyes.

  • In my personal experience, the inability to produce .so/.dll from Julia functions is one of the major problem because it prohibits a progressive introduction of Julia in large scientific projects. I know that this issue is already known by all the people able to improve this point.

  • Another reason is the difference of maturity of GPU programming for non NVidia vendors. Being able to easily program for all GPU vendors would be a strong and unique feature. Don’t get me wrong : I am a super fan of CUDA.jl and brothers. I just point out an opportunity for Julia to propose something really unique.

  • A more opinionated way for package creation and development (e.g. via a button click in vs-code) would also help beginners coming from Matlab and FORTRAN.

A lot of new TSC projects continues to be launched in FORTRAN or Python/C++ and this makes me sad because of my strong opinion that it is a vast loss of time. For Matlab, it is less clear because for now, I think that Julia still requires more CS skills than Matlab and that some scientists will never invest enough in CS to enjoy the power of Julia.

11 Likes

“FORTRAN or Python/C++ and this makes me sad because of my strong opinion that it is a vast loss of time.”

Why? Keep in mind, workstyle, experience with established tools, quality mindset and test coverage has (still) way more impact on productivity than the implementation language. If the F77 math kernel written by experienced programmers is well documented and tested, moving it to e.g. a julia implementation is a simple job. And ccalling a lib from julia is a managable problem (as it seems, my recent experience is missing on that).

The (ongoing) problem in TSC is reuse of working code. Because a lot of developments do not try to establish stable infrastructure (platform-independent-GUI?), but solving a local problem.

1 Like

I guess your experience is very different from mine. I have seen ton of super large Python/C++ or FORTRAN projects that could have been implemented with at least 10 times less lines of code and a much higher maintainability in Julia. Code dedicated to language interfaces just disappear in Julia, package management, scientist who make tremendous effort to adopt OO style where it is totally irrelevant for their needs… Too many reasons. I was paid to write C++ code called from R package and we have shown to the clients that their library could be written in no time and a few line of codes in Julia (they were shocked).

For Legacy code, it is OK to call FORTRAN code from Julia but I can’t see no good reason to start a new project with this language.

Last time I checked this was experimental. If I recall correctly, this issue was the main topic of the last JuliaCon Keynote by J. Howard. A very good news indeed if this can be easily done.

9 Likes

Not to derail the main convo, but I think KernelAbstractions.jl might come close to that goal?

8 Likes

It is true. Yet a large fraction of GPU operations do not require to write kernels and are properly handled via high order functions (mapreduce, fold, broadcasts…) using CUDA.jl. This abstract style allows to write the same code running on both CPU and GPUs which I find super cool and productive.

Unfortunately, Metal.jl or oneAPI.jl offer the same functionality without the performance (e.g. slow broadcast copy in 2D · Issue #41 · JuliaGPU/Metal.jl · GitHub)

I wish there was a fourth edition of Numerical Recipes in Julia.

7 Likes

Wonderful idea ! I guess it could be possible to start a collaborative github project translating the numerical recipes implementations. I did use N.R. a lot many years ago. I suspect that some mathematical parts should also be modernized.

P.S. Initial attempts :wink:

7 Likes

Web frameworks may be another similar area where both speed and code rapid development matter. When starting a new project, the suggested technology is often compared, and there are already popular benchmarking sites for this purpose that also carry a marketing effect.

For instance, the TechEmpower Web Framework Benchmarks

If anyone has the time, they could update/improve/extend the Julia benchmarks.
Here is the current Julia version on the benchmarking github folder:

And updated 2y…3y ago ; and Http.jl still using julia-1.5.3-linux-x86_64.tar.gz
It’s worth knowing that rivals will use every trick in the book,
such as " -flto -march=native -mtune=native -O3 " . In other words, it wouldn’t hurt to recompile almost everything that could affect the benchmark and using the latest v1.10.0-alpha1 (July 6, 2023) .

3 Likes

Could you give me more information on that?

I doubt the license of Numerical Recipes allows doing that: Boycott Numerical Recipes. And their algorithms weren’t even the best ones at the time the original books were written.

13 Likes

I don’t know much about license but I don’t see why one could not translate published algorithms.

The idea would not be to provide best algorithms but to demonstrate the power and the elegance of Julia implementations compared to FORTRAN and C++ ones. I don’t know if nowadays N.R. is still a famous book for computational scientists but it could introduce Julia to N.R readers. In addition to each of the translation, one could introduce state of the art implementations and Julia packages.

As a first example let one compare the implementation of integration methods (Romberg et al) when you can pass function operands as first class citizen :wink:

2 Likes

Close enough:

9 Likes

That’s a great reason for not doing it then.

You said you wanted to translate their code, not implement the algorithms. The former is copyrighted. Numerical Recipes is really silly, they provide code that by the terms of their license can’t be used for much. Yet people ignore that and have used their code for ages, but that’s not a great argument.

7 Likes

OK I did not know about this. It is silly indeed.

Not such a good idea after all.

Julia’s tour mentioned by @gdalle is nice enough !

1 Like

Actually, as far as I can tell, that repository doesn’t have a license either, but it does have a copyright notice. So you have to assume that code is fully copyrighted and cannot be used for anything.

1 Like

On that note,

One of these doesn’t have a license, and the other is GPL3, which isn’t compatible with the MIT license that most Julia projects use. Both of them probably violate the original Numerical Recipes copyright

What would not violate any copyright (caveat: IANAL) is to take the methods from numerical recipes and translate them to Julia (but only Julia, unlike the previous two attempts that also includes original NR code). That’s because there is no Julia edition of NR, and I really hope there never will be. If you do something like that, though, please make sure it’s public domain.

2 Likes

Quite honestly any basic “numerical recipes” like that are mostly useless unless they’re placed in the public domain. That’s what made netlib so nice: most of those libraries are written by US government employees, automatically making them public domain

4 Likes