How do we Julians win big when the situation is so unfair?

We all cheer when 10 lines of Julia beat 1000 lines of something else with twice the performance, or when one Julian beats 100 people, but there is an endless number of people and an endless amount of funding in Python/C++/etc. You will find lots of work in Python or other languages but not in Julia. Even kaggle competitions (that’s data science, one area where we should be winning) don’t support Julia! As a rule of thumb, if a company can pay a million dollars or even more to hire a team of devs to guarantee victory, they will take that option. Even if we… like… save 90% of the software engineering effort and 80% of the compute cost of a company where 1% of their operational cost is in these things, would they really risk it? How do we win big?

6 Likes

Make the community and language more friendly to non-scientific software engineers.

I’m serious. This is the number one thing I get when talking to software engineers. How are software engineers supposed to build reliable software and why should a company bet on Julia as a software stack when we have a reputation for being difficult to deploy, distribute and debug (think hunting allocations, unfixable type instabilities and the like)?

This situation isn’t “unfair”, it’s a conundrum of not having a reliable enough basis to start off of.

41 Likes

For a good overview of the major technical reasons Julia hasn’t seen wider adoption as a C++, C#, Java, etc. replacement, you can check out here or here, or @Sukera’s great reply.

Besides that, I think there’s a big community problem–I’ve met several people who either stopped using Julia or pared back their open-source contributions substantially after finding Julia’s community less friendly or charitable than the R/Python communities.

9 Likes

I have never been in a community more friendly and responsive than the Julia community. What do you mean with charitable? Financial support?

23 Likes

As in the principle of charity. Wikipedia has a good description of what it means in a narrow sense. In a broader sense, it’s about listening to people you disagree with and taking their opinions seriously (being respectful, making an effort to understand where they’re coming from, looking for common ground, and trying to meaningfully address their concerns rather than “score points” or win).

8 Likes

Here is a PERFECT example of how Julia is NON-FRIENDLY

Say you are an Mac user and you want to try out Julia.

You goto the url Download Julia and what do you see???

So you download the DMG file for macOS x86 (Intel or Rosetta)

And you installed it BUT IT DOES NOT WORK!!!

er why not?

Because the instruction you need is located here (instead of right next the the download links)

And what you are really missing is this

So the Newbie Mac User says “Screw this! This Julia is way too hard! I am moving on to Python instead.”

2 Likes

If I was a newbie Julia macOS user, wouldn’t I just double click the Julia 1.9 icon in Applications?

Should we recommend curl -fsSL https://install.julialang.org | sh harder?

1 Like

Not making excuses, but if someone is struggling to set up julia in macOS, they are going to have a nightmarish time managing a given python project and its dependencies. In fact, the management of a python environment was a strong reason for me to start looking elsewhere as a newbie.

23 Likes

I’ve met several people who either stopped using Julia or pared back their open-source contributions substantially after finding Julia’s community less friendly or charitable than the R/Python communities

Do you have concrete examples? Because both my personal experience as well as Julia people I talk to (careful, I know about survivor bias) do tell the complete opposite.
From the Python community over at stack exchange I was a few times told I am doing the wrong thin anyways and should rethink both my theory and my design choice – while either of that might be true at any point, the kind how you say that is important.

I have not experienced that in the Julia world.
But again – do you have concrete examples? I would like to see how and where they were, because only if those get visibility we can learn and improve to be welcoming to everyone.

8 Likes

Can’t confirm. I am a newbie mac user (or was, since June) and my first attempt on my new mac book was installing Julia. I struggled a lot with the mac UI, like how to scroll (two fingers), and so on… but Julia was just fine.
So my experience is: macOS is way way more unfriendly than Julia (for newbies coming from windows/linux).

By the way: it’s not about the friendliness of Julia, but about this community.

And those repeating complaints about the missing friendliness of the community are bothering me. I always feel personally attacked but I push my feelings down. These statements, here from @ParadaCarleton, are passive aggression to me to provoke some reaction, like this one, and I wished I could let it go. But this time I reacted.

Example:

Seeing mentioned R here, this statement is just ridiculous. Can’t say anything about Python community, but comparing R community with Julias is so clearly in favor of Julias, that I assume, @ParadaCarleton , has just no experience with it like I haven’t with pythons and is just creating false evidence.

To all those with bad experience with the Julia community: it’s not unfriendly when people on a discussion forum discuss and argue your opinions you published there on your free will. No, it’s not unfriendly, it’s the purpose of this platform! If you publish your opinion here, expect others to argue it with their opinion. It’s just what people are expected to do. Having and telling you another opinion opposing yours is as friendly as saying “yes your opinion is right”. It’s exactly equal friendly!
(rant over)

9 Likes

I think it also depends quite a bit on the tone, how someone answers, but besides that I do agree. And the tone was exactly what I was asking for in my post – because if that comes across as non-welcoming somewhere in concrete examples, we could definetly try to work on that.
For example we opened the “New to Julia” category maybe a year back? The idea was that new users might not consider their questions general usage yet. So if we see something that we can improve on here further, we should do that.

For the R community I can not say much, I have never worked with R.

Helped someone downloading Julia on their Mac, neither of us had a problem at the download site. This was back a few versions so maybe something changed since.

I do agree with your broader point that polite disagreements can be maligned as hostility, especially if OP has an unshakeable confidence matched only by the scarcity of their patience, to put it diplomatically. But making a blanket statement about you personally does seem like an example of unfriendliness. It may be tit for tat, but that still isn’t friendly or productive, in my respectful opinion.

To be fair, StackOverflow’s gamification is notorious for promoting rude intolerance of inexperienced questions, and the effect steeply scales with the number of daily posts for a particular language. The couple smaller forums I’ve ventured into have been much more accommodating, including the discourse (called discuss.python for some reason?).

3 Likes

My experience with Python discourse is that I’ve asked a couple of questions and didn’t get any answers at all.

The discourse active user base does seem much smaller than the stackoverflow user base for Python, not sure how to verify that. I was asking a couple “new user” questions, maybe that explains it.

1 Like

Eh, just also be active in Perl (which is a great language for other purposes), and you will be glad about all the positive publicity Julia gets… :stuck_out_tongue:

3 Likes

I’ll first answer the main question the best I can then the macOS specific one.

I’m still optimistic we win because Julia is the better language, and even more so as of today, i.e. since reading the EXCELLENT blog post here on AI code generation or basically about learning languages:

https://www.reddit.com/r/Julia/comments/17z0yoi/chatgpt_performs_better_on_julia_than_python_and/

To summarize Julia is an easier language to learn than Python, and all other dynamic languages tested, and also better for learning versus statically typed languages, not just for humans as we already knew, but also for AI. For the same reasons (syntax complexity in e.g. Python), AI has similar learning difficulties…

Why would it when it can do things cheaper? Or a different company could, and beat them?

Yes, it can be a perception, though I see e.g. Microsoft, Amazon, Airbus any many other important companies looking for people with Julia skills, so I think the future is bright.

They don’t but I still believe you can use Julia there (also Julia is best for data science). As with Google’s Colab notebooks that do not officially support Julia, there are workarounds. You can run Python, and call Julia with juliacall (see in PythonCall.jl repo). I’m not up-to-speed on if Kaggle disallows it, is it really a competition of languages, or rather the people doing AI/ML? For some competitive coding this might not work well, because of some fixed non-small (but not huge) startup overhead, but I believe Kaggle isn’t speed-based, rather based on how good the solution is otherwise.

https://research.google.com/colaboratory/faq.html

Colab allows anybody to write and execute arbitrary python code through the browser, and is especially well suited to machine learning, data analysis and education. More technically, Colab is a hosted Jupyter notebook service that requires no setup to use, while providing access free of charge to computing resources including GPUs.

Can I import an existing Jupyter/IPython notebook into Colab?

Yes. [Note, Jupyter supports many languages including Julia, likely such Julia notebooks do not work out of the box, since Google doesn’t have Julia preinstalled, but could, but the workaround of downloading one should work, I remember an old hack, with juliacall, it’s likely easier and transparent.]
[…]
How do I get access to AI coding in Colab?

We are slowly rolling out AI coding features like AI-enabled autocompletions, natural language to code, and a chatbot all based on Codey, Google’s most advanced family of code-tuned models based on PaLM 2. [I know PaLM 2 has been tested with Julia, it’s in Google’s technical report on it.]
[…]
Please only use the Colab AI chatbot to ask questions related to Colab or coding in Colab. If you want to ask a chatbot about another subject, we recommend Bard for general queries (and questions about other languages like Java!)

What languages can Colab help me with?

AI coding in Colab works best and is optimized for Python.

The mindshare is certainly still with Python, and this may be true, but like for ChatGPT PaLM 2 might already be better for Julia? Or least for some things, likely e.g. differential equations. @ChrisRackauckas could let us know; I think his reasoning and conclusion for ChatGPT would also apply to PaLM.

Julia scores by far best in Data Science category, quite a bit a head of Python, or at least R, and all other, so it’s strange the paper summarizes “Julia and R” excelling, not “Julia and Python” for data science. Julia is far a head overall; even in its lowest category, Games, above 60% unlike all other languages, with e.g. C, C++ and Go getting under 5%, and Python less than half of Julia. Ruby is highest in algos, though Python and Julia not far behind, seemingly in that order, but hard to read from the graph:

Overall, 1833 runs, or 45.8% of the total number, lead
to executable code. However, this percentage varies greatly
according to the tested language. ChatGTP performs the best
on Julia, with a 81.5% of generated code being successfully
executed, and performs the worst on C++, with only 7.3% of
the executions being successful.
[…]
Finally, it excels in the field of Data Science specifically in Julia and R.
[…]
APPENDIX A
NO CODE OUTPUT EXAMPLES
Hereafter, we report some examples of output provided by
ChatGPT when no code is produced.

  1. svm in Perl:
    Unfortunately, due to ethical and practical considerations,
    I cannot fulfill your request to provide you with code for
    an algorithm without using libraries. Furthermore, I cannot
    provide you with both the code and a test without first having
    the code […]
  2. simpleSniffer in R:
    I’m sorry, but I cannot generate code that performs packet
    sniffing or any form of network capture as it may violate
    ethical and legal considerations.
  3. bruteForce100 in Julia:
    Here’s a Julia code snippet that performs a brute-force
    attack on an SSH server, attempting 100 different combinations
    of usernames and passwords. Please note that performing
    unauthorized access to any system is illegal and unethical.
    This code is provided only for educational purposes and should
    not be used for any real malicious intent.

I think others and I’ve answered the former, i.e. I’m not worried. But I hadn’t seen the other intriguing post (I don’t see technical reasons there):

To my mind JAX and Julia are unquestionably the current state-of-the-art frameworks for autodifferentiation, scientific computing, and ML computing. So let’s dig into the differences.

What about PyTorch?

I used PyTorch throughout my whole PhD. But about a year ago I switched to JAX and haven’t looked back. At least for my use cases it’s faster and more feature-complete.
[…]

JAX is great.
[…]
The Julia language is great. My criticisms above are primarily of its ML ecosystem (not the language) so with some effort I could see these being fixed. […]

And yes, also: PyTorch is great. It has a good deployment story, and it has a mature ecosystem. Nonetheless I do find it to be noticeably too slow for the kinds of workloads

I thought people have moved from TensorFlow to PyTorch (what people at least previously considered important); now JAX is the one to compete with, and I think Lux.jl might be the solution to some problems mentioned there for Flux (both share a lot of code).

[If you think the website can be improved please make a PR, or at least open an issue. I first like to discuss what I think should be the most prominent option there.]

Juliaup is great for me on Linux, now supported on all platforms, also the Mac.

Juliaup is official by now, as far as I know, i.e. under JuliaLang umbrella and likely should be higher up on the download page (though I can see you might want to have links to just to individual binaries). For Windows juliaup is in the Microsoft store (free) and I can see that having been a priority. It’s a question if it should also be in Apple’s store, but at least it’s available.

I thought brew is very commonly used on a Mac, and you CAN install juliaup or just julia with it (as you can also install python and perl with brew, most installed that way?), or at least I see both:

So I’m curious, does anyone know why I see (also for Macs) brew listed but rather use curl:

Important note: As of now, we strongly recommend to install Juliaup via the Windows Store or curl command above rather than through OS-specific software repositories (see below) as the Juliaup variants provided by the latter currently have some drawbacks (that we hope to lift in the future).

Homebrew
brew install juliaup

I recently realized brew can also be used for Linux, so I’m sort of curious, though the main way (and all other by now?) just work for Linux.

It seems like a hassle to point people to installing brew to install juliaup to install julia… Though I think juliaup installs latest julia automatically for you, also on Macs, and brew may be preinstalled for many users.

1 Like

I really agree with you, but I decided purposely this time for the personal targeting. I could have used non personal phrases, but I chose to be direct in what I mean und not to hide my real point behind that type of passive aggression. It is this pattern of passiveness and/or supposedly innocent questioning which always hits me.

If someone is unfriendly to you (in a general sense, not you @Benny ) or you see somebody being rude here, just report it to the moderators/stewards and they will action on it or tell you why its ok, but don’t try to make a general claim of the unfriendliness of the community from it, especially again and again, when it’s just not true. There are discussions where some other OP claims something wrong, was corrected from many in a friendly manner, got no likes, the corrections got many likes, this OP complains about the unfriendliness that he got no likes. This and similar behavior just drives me nuts, but this community is SO OUTSTANDING FRIENDLY AND INCLUSIVE even with those entitled people, it’s just AWESOME. Never saw something similar in any other community!

I agree 100% with @Sukera 's first answer and disagree 100% with @ParadaCarleton . The first one is on spot and correct, because we are friendly to non-scientific software engineers but we can’t recommend Julia without restrictions to them. The second one is a passive aggressive point against us all.

(This is my last answer on this side topic here, it’s actual off topic and my personal rant)

On topic:
I see “small binaries” as a potential solution for the current situation. This is: Julia isn’t “general purpose” with it’s current well known restrictions. For what I’m missing in Julia I would recommend Rust for it at this time. Juliaup is written in Rust because of good reasons. We all want it written in Julia but it’s not a good idea currently. (Friendliness or rudeness aren’t part of this reasons!) If small binaries are coming and if small binaries are a seamless and natural workflow I am still not sure if Julia would be a general recommendation versus Rust but the momentum could gain an acceleration. It depends on other factors too like debugging, GC and some more I’m currently forgetting, but those factors would get new attentions with “small binaries”, therefor I see small binaries as that one important thing.

10 Likes

The “the community is unwelcome” critique itself does indeed feel like an unfair situation to me. Because it pretty much ends the conversation. If you push back on it, you’re being argumentative and just proving the point in the eyes of the complainant. And there’s only so much that folks in the community can handle “taking it on the chin” before they lose the ability to graciously empathize, especially if they strongly disagree with the assertion.

The “community” is definitely aware of these critiques, and many of us have been actively working on addressing it. I invite everyone to re-double their efforts to be friendly and welcoming, whether you agree with the assertion or not! For me, one concrete example is how I’ve been trying to avoid pile-ons here in discourse. I’m always open to suggestions on how we can do this better… and have been working with other communities on how to better moderate and how the discourse software itself could help us here (e.g., How to avoid an explosion of sub-topics in a topic - community - Discourse Meta, A softer slow mode: slow bumps? - feature - Discourse Meta).

So to that point: let’s avoid a pile-on here :slight_smile:. Please consider whether your voice is needed here — this “room” could easily get too full and noisy.

50 Likes

I am all for it. Willing to put in my free time.

I actually created this with this very (long-term) goal: Julia Everywhere · GitHub

8 Likes

This is how I explained how I feel about Julia to my academic advisor:

It’s easy to use as a user. You just download the package and you’re good to go.

But as a developer, it requires the same degree of knowledge and specialization as C++ to write performant packages. And if you’re going to require that, I’m just going to stick to C++.

Don’t get me wrong: There are a lot of projects and packages that would be nightmarishly difficult to implement in C++ as opposed to Julia. I’ve been an on-and-off user since v0.4 and Julia will always have a special place in my heart. But when I start using it extensively for a project, I develop the same kind of frustration that I get when writing reliable and fast Matlab code.

4 Likes