What steps should the Julia community take to bring Julia to the next level of popularity?

I feel the whole SciML not new user friendly yet argument to heart ig, I have tried couple of times to find what it might be useful for me and contribute. But then there is a lot of jargon to go through first before you get to the good stuff I guess. I never did a GSOC in SciML where the person involved just has to go through with the hard part, Chris is amazing, so much work being put in there by contributors with good intentions. I work 6 days a week ;-;(bit busy i mean) and whenever I try to learn something to do with SciML in evenings, I just get frustrated and have to let it be. It feels like it’s something I’ll pick up in masters where I might have more time.

I mean we need more documentation and not academics documentation, some documentation that makes me go “this might be something I can pick up”. It’s lot different than the instructor-student or mentor-mentee when you are learning from documentation and asking questions in #helpdesk or #sciml-bridged. Also it was starting point problem for me that Chris had already pointed out in this discussion.

[Edit 1] : It might be just that a proper use case to learn it for me hasn’t arisen yet

2 Likes

R and Python at one time were fairly competitive in the same domains. The article “How Python Became the Popular Choice” by Jun Wu highlights the big packages and applications of Python as it rose to meet big and often overlapping demands, like finance, data science, machine learning, web development.

3 Likes

Strictly speaking, the question is meaningless, because the subject (“Julia”) is a computer language, and as such it is not a conscious actor: it “wants” nothing.

I recognize that this is a figure of speech, meant to represent a group of people (“the community”, “the core devs”, “package authors” etc). But most of the groups you can plug into that sentence are a very diverse bunch, with nothing close to a single unified goal.

Discussions like this topic are pretty pointless, since no consensus will be reached, and in any case very few people will write FOSS code for anything unless they need it in the first place, and then they do it anyway.

Good luck with that. But, with respect, I hope you realize that whatever you agree on, you have more or less zero effect on what people will work on in the package ecosystem. Which is as it should be — it would be weird if a central committee allocated effort to FOSS projects.

It is as if my neighbors got together and voted on what I should plant in my garden. I would shrug and plant what I want.

For the curious

It will be Lonicera japonica, with flowers that smell very nice around dusk.

To end on a positive note: everyone here has the potential to steer the Julia package ecosystem with contributions. Report issues, write docs, make PRs, or fill in niches of missing functionality with new packages.

22 Likes

As a long time user of R, and occasional user of Python, Fortran, etc., my perspective is that one uses the tools that work for you. Julia has passed a threshold in utility where not only are cutting edge developers doing their thing (which is what you want in terms of creativity, in the long term) but also base level utility for the every-day stuff has stabilized to the point of being able to do what needs to be done in compact and elegant ways. So aesthetics had some influence in my transition. However, that was not enough. I was held back initially because many tasks were already implemented in one language or another. Why recreate the wheel? The incentive to transition the mundane stuff was low.

The critical factor was access to cutting edge approaches that are rapidly and easily deployed in Julia. This is where incentive comes in for me. Brute speed and the new methods that this makes possible.

So, all in all, I would say, keep doing what you are doing to the devs. I am happy. (Hat’s off)

However, to make the initial transition easier for others once the use case becomes clear, perhaps make a concerted effort to remove outdated tutorials and documentation. Amalgamate best practices with working examples in one central area. And did I mention, documentation, with many examples as use cases are so varied, especially where in XX language it is done this way, here it is in Julia … and so much faster, clearer, etc,

Thank to all the tireless devs.
Jae

13 Likes

Agreed. The more relevant question is: assuming that broader adoption is the goal, what action can each person take individually to move towards that goal? And the answer probably is to contribute in areas that are likely to be useful to most. That may or may not intersect with one’s area of expertise for instance.

My area of (relative) expertise is quantitative finance, where there are few users of open source solutions (not a dig at them, especially as I contributed some C++ code to the project over 10 years ago, but QuantLib doesn’t have a high level of adoption in banks or hedge funds).
Where I think Julia could attract more users? As stated before, in the area of small glue scripts (where Perl, now Python, are pretty good at).

So my area of expertise doesn’t really intersect with the area where I think Julia could gain broad acceptance. If I was involved in Julia internals though, I would work on static compilation and tooling. If I was a package developer, on system stuff. In the meantime, I can only use Julia where it makes sense for me, hoping that the above happens (as I would stand to benefit from it, from the point of view of a more mature language and ecosystem).

Back to the original question, I would suggest we take a look at areas suffering from 2(+) languages problems. Game development could be one aspect. There could be many more. Other than that, make Julia a top-choice programming language for newer projects.

4 Likes

IMHO Julia is already the top choice for programming technical/scientific computing (TSC) packages (by far).

Yet it is not dominant in this specific area.

I am more interested in Julia keeping on growing in the TSC community (TSCC) than reaching other domains (games…).

Hence the reasons that are slowing down the adoption in the TSCC tend to be more relevant to my eyes.

  • In my personal experience, the inability to produce .so/.dll from Julia functions is one of the major problem because it prohibits a progressive introduction of Julia in large scientific projects. I know that this issue is already known by all the people able to improve this point.

  • Another reason is the difference of maturity of GPU programming for non NVidia vendors. Being able to easily program for all GPU vendors would be a strong and unique feature. Don’t get me wrong : I am a super fan of CUDA.jl and brothers. I just point out an opportunity for Julia to propose something really unique.

  • A more opinionated way for package creation and development (e.g. via a button click in vs-code) would also help beginners coming from Matlab and FORTRAN.

A lot of new TSC projects continues to be launched in FORTRAN or Python/C++ and this makes me sad because of my strong opinion that it is a vast loss of time. For Matlab, it is less clear because for now, I think that Julia still requires more CS skills than Matlab and that some scientists will never invest enough in CS to enjoy the power of Julia.

11 Likes

“FORTRAN or Python/C++ and this makes me sad because of my strong opinion that it is a vast loss of time.”

Why? Keep in mind, workstyle, experience with established tools, quality mindset and test coverage has (still) way more impact on productivity than the implementation language. If the F77 math kernel written by experienced programmers is well documented and tested, moving it to e.g. a julia implementation is a simple job. And ccalling a lib from julia is a managable problem (as it seems, my recent experience is missing on that).

The (ongoing) problem in TSC is reuse of working code. Because a lot of developments do not try to establish stable infrastructure (platform-independent-GUI?), but solving a local problem.

1 Like

I guess your experience is very different from mine. I have seen ton of super large Python/C++ or FORTRAN projects that could have been implemented with at least 10 times less lines of code and a much higher maintainability in Julia. Code dedicated to language interfaces just disappear in Julia, package management, scientist who make tremendous effort to adopt OO style where it is totally irrelevant for their needs… Too many reasons. I was paid to write C++ code called from R package and we have shown to the clients that their library could be written in no time and a few line of codes in Julia (they were shocked).

For Legacy code, it is OK to call FORTRAN code from Julia but I can’t see no good reason to start a new project with this language.

Last time I checked this was experimental. If I recall correctly, this issue was the main topic of the last JuliaCon Keynote by J. Howard. A very good news indeed if this can be easily done.

9 Likes

Not to derail the main convo, but I think KernelAbstractions.jl might come close to that goal?

8 Likes

It is true. Yet a large fraction of GPU operations do not require to write kernels and are properly handled via high order functions (mapreduce, fold, broadcasts…) using CUDA.jl. This abstract style allows to write the same code running on both CPU and GPUs which I find super cool and productive.

Unfortunately, Metal.jl or oneAPI.jl offer the same functionality without the performance (e.g. slow broadcast copy in 2D · Issue #41 · JuliaGPU/Metal.jl · GitHub)

I wish there was a fourth edition of Numerical Recipes in Julia.

7 Likes

Wonderful idea ! I guess it could be possible to start a collaborative github project translating the numerical recipes implementations. I did use N.R. a lot many years ago. I suspect that some mathematical parts should also be modernized.

P.S. Initial attempts :wink:

7 Likes

Web frameworks may be another similar area where both speed and code rapid development matter. When starting a new project, the suggested technology is often compared, and there are already popular benchmarking sites for this purpose that also carry a marketing effect.

For instance, the TechEmpower Web Framework Benchmarks

If anyone has the time, they could update/improve/extend the Julia benchmarks.
Here is the current Julia version on the benchmarking github folder:

And updated 2y…3y ago ; and Http.jl still using julia-1.5.3-linux-x86_64.tar.gz
It’s worth knowing that rivals will use every trick in the book,
such as " -flto -march=native -mtune=native -O3 " . In other words, it wouldn’t hurt to recompile almost everything that could affect the benchmark and using the latest v1.10.0-alpha1 (July 6, 2023) .

3 Likes

Could you give me more information on that?

I doubt the license of Numerical Recipes allows doing that: Boycott Numerical Recipes. And their algorithms weren’t even the best ones at the time the original books were written.

13 Likes

I don’t know much about license but I don’t see why one could not translate published algorithms.

The idea would not be to provide best algorithms but to demonstrate the power and the elegance of Julia implementations compared to FORTRAN and C++ ones. I don’t know if nowadays N.R. is still a famous book for computational scientists but it could introduce Julia to N.R readers. In addition to each of the translation, one could introduce state of the art implementations and Julia packages.

As a first example let one compare the implementation of integration methods (Romberg et al) when you can pass function operands as first class citizen :wink:

2 Likes

Close enough:

9 Likes

That’s a great reason for not doing it then.

You said you wanted to translate their code, not implement the algorithms. The former is copyrighted. Numerical Recipes is really silly, they provide code that by the terms of their license can’t be used for much. Yet people ignore that and have used their code for ages, but that’s not a great argument.

7 Likes

OK I did not know about this. It is silly indeed.

Not such a good idea after all.

Julia’s tour mentioned by @gdalle is nice enough !

1 Like