We as a community should be more understanding of Julia's flaws

Community
41

Thanks for the updated info. My status quo comment wasn’t meant to offend. I did build a HelloWorld type of standalone binary a few years ago using PackageCompiler, and it ran fine as I recall, but required a somewhat-large library (DLL or .so) file in addition to the binary executable. I’ll check out the latest version of this package to see how much smoother things are now . . .

42

No, it isn’t:

julia> function f!(result_array)
           for k in eachindex(result_array)
               value = sin(k)
               result_array[k] = value
           end
       end

f! (generic function with 1 method)

julia> result_array = zeros(10^6);

julia> @btime f!($result_array);
  11.649 ms (0 allocations: 0 bytes)

The situations in which “it should do something” but it is not doing it are, for instance, like this:

julia> function f!(result_array)
           for k in eachindex(result_array)
               value = [sin(k), cos(k)]
               result_array[k] .= value
           end
       end
f! (generic function with 1 method)

julia> result_array = [ zeros(2) for _ in 1:10^6 ];

julia> @btime f!($result_array);
  41.204 ms (1000000 allocations: 76.29 MiB)

The compiler should (and it will soon :crossed_fingers:) recognize that the intermediate vector [sin(k), cos(k)] does not escape the scope of the (iteration of) the loop, and thus should be stack allocated, or completely eliminated ( to read here result_array[k] .= (sin(k), cos(k)), which is non-allocating then).

7 Likes
43

No offense taken. I am just here to give the clear facts and guide the discussion. I think that’s one of the big issues in these kinds of discussions. A lot of the time the “facts” are just plain old. Not necessarily incorrect back in 2016, but things change. Usually correcting someone throws someone into a “defensive stance” and then you see people on Hacker News who suddenly fall down a weird rabbit hole of absurdity :person_shrugging: . It’s really not worth anyone’s time. Sharing information will always help in the long run though, so we need to just keep sharing the updates.

I think the answer is, cite your sources, share links, and write more blog posts showing how things are improving. Just keep sharing all of the good that’s coming out. Some people will say that’s not acknowledging the flaws, but whatever that’s a silly stance: how can you be writing about flaws you fixed if you didn’t acknowledge there was a flaw to begin with? :joy: Just keep swimming.

27 Likes
44

Every minor version release’s “Highlights” on the Julia blog is actually a list of “here’s something many users griped for years and we did something about it” and discourse is at least 70% a complaint forum.

45

Github issues are the list of flaws. They just don’t get trending on Hacker News.

3 Likes
46

I don’t see how this could be a big problem. It is trivial to give someone a package to execute whatever program they desire, and the package manager makes it a breeze to set up the appropriate environment.

4 Likes
47

Let me give some of the major reasons people complain about Julia and possible ways to solve them.

  • Perceived Problem 1: There are little to no questions on Julia’ s stackoverflow tag, so if you have an issue, you’ll have a hard time to solve it.

  • Perceived Problem 2: Python has more libraries than Julia. In Julia, there are little libraries in the ecosystem, and they’re not actively managed.

    • Solution: Put your expectations in check, programming languages and softwares don’t work that way. Real world applications take time and they move in trends. Using python as an example, below are the most popular libraries you would mostly use if you use python a lot:

      • pypy a just-in-time compiler for python to help speed up the language published an initial release in 2007, 16 after python was created.
      • cython an extension of python for those who want to speed [python] up published an initial release in 2007, 16 after python was created.
      • Pandas the most popular python library for dataframes published an initial release in 2008, 17 years after python was created.
      • Numpy the most popular python library for numerical computing published an initial release in 2006, 15 years after python was created.
      • Matplotlib the most popular python library for plotting published an initial release in 2003, 12 years after python was created.
      • Tensorflow one of the most popular python library for ML and AI published an initial release in 2015, 24 years after python was created.
      • Pytorch one of the most popular python library for ML and AI published an initial release in 2016, 25 years after python was created.
      • Scikit-learn one of the most popular python library for ML published an initial release in 2007, 16 years after python was created.
      • Selenium one of the most popular Python library for browser automation published an initial release in 2002, 11 years after python was created.
      • Django one of the most popular python web framework published an initial release in 2005, 14 years after python was created.
      • Flask of the most populat python web framework published an initial release in 2010, 19 years after python was created.
        None of the libraries were developed 10 years after python. Any genuine person would ask this: Then what were people doing with Python for the first 10+ years… I guess they were using Perl, BASIC and bashing on Python as a lot of people are currently doing to Julia. So if you claim to love python and hate julia because of libraries availability, you’re just a user who loves python because its easy to use with add-ons (libraries), not necessarily the language design itself; and moreover you would have said same thing about python had you come into it exactly 10 years after the language was developed (just like you say about Julia now).

      Now contrast the above with how Julia is evolving:

      • DataFrames.jl one of the most popluar julia library for tabular data published an initial release in 2013, 1 year after Julia was created.
      • Plots.jl one of the most popular julia library for plotting published an initial release in 2015, 3 years after julia was created.
      • UnicodePlots.jl one of the most popular julia Unicode-based scientific plotting for working in the terminal published an initial release in 2015, 3 years after julia was created.
      • Makie.jl one of the most popular julia high level plotting on the GPU published an initial release in 2018, 6 years after Julia was created.
      • Flux.jl one of the most popular julia ML library published an initial release in 2017, 5 years after julia was created.
      • Knet.jl one of the most popular julia deep learning framework published an initial release in 2016, 4 years after julia was created.
      • Franklin.jl one of the most popular julia static website generator published an initial release in 2019, 7 years after julia was created.
      • Genie.jl one of the most popular julia web framework published an initial release in 2018, 6 years after julia was created.
        There are over 8000 registered packages in the Julia ecosystem, so if you care to search well enough, you might find what you need. 10 years into a language and they’re already over 8000 registered packages. This should tell you how fast the language is evolving.

    The above comparisons are not given to say Julia is better than Python. The message it’s trying to convey is that softwares and especialy open source softwares takes time. Since no one is getting paid for what they do, why do you think it will get out of the door just with a winkle? It takes time, so have that in mind. Anyways, if you want a library to be developed instantly for you, then talk to Julia Computing (but be ready to pay for it as its not free).

    All the growth of those python libraries came as a trend of the growth in ML and AI, and python was the only simple and versatile language dominant among users to glue them to at that time - so you see the trend now. The libraries weren’t developed because of Python (most of the development were done in C++ and C, not in python as we all know), they were only glued to Python so users can easily call them. There are more libraries you know of in Python than the ones listed above, chances are they were initially released 20+ years after Python already existed. So yes Julia don’t have all the libraries you might need now, but its just 10 years old and chances are it has enough of the libraries to do most of the stuff you would need now (except in very niche cases where you either have to develop one yourself or wait for others interested in what you do to develop it as time goes on).

  • Perceived Problem 3: There are a lot of tutorials out there to learn other languages, Julia’s tutorials on the internet is very small.

    • Solution: Julia’s documentation isn’t the best you’ll find on earth, but it does a good job of explicitly telling you what and what you need to know and master to effectively use the language. If you have a problem the docs doesn’t address, then instantly ask questions on any of the channels highlighted in the solution to problem 1 (you’re guaranteed of an answer in 30mins) and also pull a request for it on github and you’re sure to get a review in a day (at most). Contrasting it with a language like python again, the tutorials boom on python only started gaining momentum 20 years after python already existed, as a result of the growth of ML and AI; and it didn’t happened overnight to get to where it is, it took time.

  • Perceived Problem 4: Julia is 1-based indexing and the other languages are all 0-based indexing, so its hard to get used to this new behaviour and it will slows my productivity.

    • Solution: Julia is a isn’t the only language using 1-based indexing. FORTRAN, APL, MATLAB, R, Wolfram, Lua, COBOL, etc., all use 1-based indexing and they have millions of developers using these languages and great softwares out there (so you see its not much of a big deal as most people would try to make it). Within a month of development in Julia, you will probably get used to it. If you’re still keen on using 0-based indexing, then JuliaArrays provides OffsetArrays.jl for you, so you can use 0-based indexing and even negative indexing as you wish to in Julia.

  • Perceived Problem 5: In other langauges I can quickly build standalone binaries, this is hard or not even possible in Julia.

    • Solution: Honestly, there is work going underground for this and its best left to say “NO Julia can’t build standalone binaries for now”. Of course, there are multiple workarounds like PackageCompiler.jl if you currently wish to build one still, but it’s not yet effective like other languages have.

  • Perceived Problem 6: In python for example I can easily call C and C++ libraries, this is hard or not even possible in Julia.

    • Solution: You can do same and possibly even more in Julia. With Julia you can call most programming languages and use their code and libraries effectively.

      A lot more others showing you how to use other languages from Julia can be found here: JuliaInterop

To conclude, I will add this: If a language/environment is “good enough” for someone, no amount of persuasion will convince them to switch. In my experience, people who gravitate to Julia do so because it really solves the problems they had with other languages (in terms of speed, elegance, abstraction, “two-language problem”), or expect that it will, given time, so it is worth investing in.

26 Likes
48

If you by “little to no” mean “thousands”, then you are definitely right.

2 Likes
49

This is somewhat derailing the thread so apologies for doing so, but on the StackOverflow point: I know many in the community actively dislike SO, and it’s certainly not a great place to have many of the in-depth “how can I speed up inference in this Turing PDE model 1000x” discussion happening on here, but there’s still a decent community of very good Julia coders who are monitoring the Julia tag on SO and provide excellent answers (or at least comments to the badly posed questions which can’t be answered) to most questions almost instantaneously.

So if you come across someone complaining about the lack of SO Julia content, please encourage them to post a question there, and they’ll likely be surprised how quickly they’ll have their problem solved.

Of course actively posting a question on SO is more effort than just googling something and copy-pasting the solution from an already existing question, but there’s little we can do about that other than encouraging people to ask more questions.

9 Likes
50

(Perhaps OT, yes, but I’ve never understood what’s so bad about SO, especially the Julia tag.)

1 Like
51

Do you have a guide on how to do that? Because I have been trying unsuccessfully for months. Environments and package version management are not trivial to me. They are a big problem.

3 Likes
52

By “them”, I am not sure whether you were referring to the problems or those people who where complaining…

I’d say that your argument in Problem 2 doesn’t quite stand. Intensive development of Python’s computational capability (thanks to the advent of machine learning, of course) largely started around 2010, slightly before the inception of Julia. In a way, one may argue that Julia started with a cleaner slate than Python (e.g. the Python 2 to 3 transition mess), so the difference is not that big.

IMHO, the biggest complaints the HN crowd have about Julia are:

  1. The language is a bit over-sold. When people tried the language and it failed to deliver all the promises, they feel they were “conned”.
  2. Some part of the Julia community can be quite dismissive when responding to valid criticisms on Julia. When people complain about Julia’s problems on HN (probably because they were disappointed), you often see replies arguing along the line of “You are doing it wrong; Julia is different than Python/Foo/Bar”, or when it comes to performance comparisons, “Your implementation is not type stable / is allocating too much; you should be apply these optimization to speed it up”. On itself this is not really an issue, but a lot of people perceived Julia as a language as easy to use as Python while as performant as C/C++ — they then found out that they have to choose between ease of use and performance, which reinforces their opinion that the language over-promised.
  3. There is also the problem of developer experience (e.g. tooling, static analysis, workflow best practices).
8 Likes
53

You’re using Julia and I believe you want it grow big.

So when you see such problems like the great ones you’ve highlighted. Think of a possible solution to circumvent it. If you don’t have a solution to a problem, then I feel its a “problem” to you as well, you understand?

For example, the language is oversold. Ok, so how do you help so we stop making it oversold? Put up the discussion on discourse and let others contribute, that’s how I see it from an angle of someone who wants Julia to grow.

So, whenever you see a problem, even if you don’t have a solution but love to see the problem being solved. Turn on your computer and post a thread in discourse, its free.

The problems I posted above are the ones I think I’ve seen countless times. Now you have to add the ones you’ve highlighted as well and also solutions (if you have them), or ask for help? Let your actions be someone who wants to help the language grow, other than stating problems A, B, C, with no help or no sign of asking for help.

54

Well, recognizing the problems is certainly the first step towards solving them.

I am not quite sure I follow you here…

I guess my point was that some of the solutions you proposed sounded more like defences — materials one can use to argue with critics of the language. Unfortunately, I don’t think these critics will change their mind after seeing the defences presented; they will more likely to just walk away, telling others on HN that the Julia community is toxic. Perhaps a better way of phrasing them would be for the Julia community to promote better user education?

As for my own observations: It is my hope that the language and ecosystem can continue to grow and I appreciate the effort Julia’s developers put into growing the language. However, I am but a mere user of the language, and I am only here to provide my feedbacks; I don’t think I am in a position to tell people how to solve those problems.

3 Likes
55

You’re not a mere user. It took you more than a second to install Julia on your local machine, so you’re part of the statistics when downloads is counted.

Please in your own small way provide solutions to any problems you encounter, so it can be solved. And when you don’t have such a solution, ask the question anyways, so others can learn from it.

1 Like
56

How about this: GitHub - PetrKryslUCSD/FreeVibrationExample: Simple Example of Free Vibration Computation?
If you’d like, download the repo and follow the instructions. Some minutes later
one can run the simulation. (Precompilation takes the most part of the start up time.)

1 Like
57

I use DrWatson to build a reproducible environment for data analysis. It doesn’t do anything you couldn’t do manually but does make the process a bit more straightforward. It is particularly helpful for ensuring that scripts run in the correct environment.

1 Like
58

Thank you for the post. Point well taken and I think appropriate. New perspectives joining the is very useful. Helping those getting started guarantees the inflow of potentially creative and useful ideas. Sometimes a simple question can open a dialogue that leads to a new perspective, with group input.

59

I’d hardly call writing a .bat and a .sh script for every project to make it reproducible “trivial” or “a breeze”, nor is it a Julia solution.

I think Dr. Watson is overkill for what I need, but I will give it another look.

But I am off topic now and will leave it here.

60

Ok, perhaps it wasn’t as obvious as I thought.

  1. This workflow (GitHub - PetrKryslUCSD/FreeVibrationExample: Simple Example of Free Vibration Computation) is only needed if the person receiving the instructions does not have an installed Julia at all.
  2. To adjust this workflow for a different application project it doesn’t take more than changing a single variable: FreeVibrationExample/install.sh at 8be88c2e9687c3629fe71890d90407bfaa8699a8 · PetrKryslUCSD/FreeVibrationExample · GitHub
    No other messing with batch files is required. In a pinch, the user package name could be solicited
    from the user at runtime.
  3. If the person receiving the program already has Julia installed, all that is needed is to download
    GitHub - PetrKryslUCSD/RunVibrationExample.jl: Run an example of a free-vibration solution and run run.jl in that folder. That is a pure Julia workflow. In doesn’t take more than say include("run.jl").
1 Like