What steps should the Julia community take to bring Julia to the next level of popularity?

[PetrKryslUCSD]

Is it a big deal to run julia -q (which is what you do anyway when using julia as a script runner)?

[DNF]

@Per

that’s how almost every other scripting language works.

Is it really? For Matlab, definitely not. Python, maybe partially, though for scientific workflows I strongly suspect most use the REPL, since you want the interactivity. And for R, I thought everyone worked inside RStudio.

In Bash, Perl, Javascript etc. I guess things are different.

[xiaoxi]

@AMJ

Do we have a bullet point of ideas/concerns shared in this thread?

I would like to see that:

• Julia community members submit Github repositories, blog posts, and JuliaCon videos to Slashdot, Hacker News, or any similar website
• The Julia Lang website better reflects the adoption of Julia in academia and industry with case studies, blog posts, and/or logos of companies and institutions that use Julia.
• Developers of Julia and major Github organizations write blog posts about upcoming major features from time to time
• Julia Stewards explore the possibility of turning Julia into a foundation under the Linux foundation

[stevengj]

@xiaoxi

Julia Stewards explore the possibility of turning Julia into a foundation under the Linux foundation

Julia is already a sponsored project under the nonprofit NumFocus foundation — similar to NumPy, SciPy, Jupyter, and many other prominent open-source science projects. This seems like a much more appropriate umbrella foundation than the Linux Foundation.

[Palli]

@Per

I think a lot of novice users run their Julia code by typing julia my_script.jl from the shell, since that’s how almost every other scripting language works.

That may be sub-optimal (now), but we also want it to work fast. Is there anything wrong with my suggestion:

@Palli

What I propose is that every .jl file you run compiles to .ji binary file, and if you run a .jl file then the cached code in .ji file is used if the timestamp is newer.

[xiaoxi]

@stevengj

Julia is already a sponsored project under the nonprofit NumFocus foundation — similar to NumPy, SciPy, Jupyter, and many other prominent open-source science projects. This seems like a much more appropriate umbrella foundation than the Linux Foundation.

Julia becoming a foundation would reinforce the idea that Julia is the right language for scientific computing. Although Julia could become an independent foundation as Python is, being under another foundation such as the Free Software Foundation (as R Foundation) or the Linux Foundation (as R Consortium) seems more convenient.

Bringing an open source project to the Linux Foundation

[GregVernon]

For what it’s worth, I’m sorta a novice, though I’ve been using Julia since ~2015. My workflow is to open a Julia REPL and copy/paste code from a “scratchpad” (unsaved) file, seeing what works (or doesn’t) and then doing exit() to clear functions etc., edit my scratchpad, rerun REPL, rinse-repeat.

[jishnub]

@DNF

Bash, Perl, Javascript etc. I guess things are different

I think these are the ones that are generally considered to be “scripting languages”, rather than Matlab. Maybe R can be placed in the same category.

[lmiq]

These days a student wrote me asking how to run a Julia script, which was quite simple. Just a package being used a couple of lines calling some functions. I told him that he had to first install the package in a Julia section, and then run the script. Even, it felt suboptimal, as ideally I would like to tell him to create an environment for his script, and not add the package to the global one.

That given, I think it would be interesting to have a keyword argument, -s or anything else, such that:

julia -s script.jl

would:

• Create an environment (not temporary, and uniquely identified with that script).
• Install all using packages without prompting.
• Run the script.
• Maybe: cash some compiled code given the functions used?

On the second call, the environment and the packages would be already created and installed. How much automation is a question (should a second run remove the packages that are not anymore used, for example?).

That would help distributing scripts. The user would just have to know that on the first call to the script some installation kind of thing would happen.

[adienes]

@lmiq

Install all using packages without prompting.

[lmiq]

Is that good or bad?

[adienes]

auto-installing packages without confirmation on the name (as the REPL currently gives) could come with increased security risks. probably not a lot, but these things add up

[jbytecode]

MathOptInterface.jl would require an infinite! amount of installation time and I can except some users to break the freeze process by pressing ctrl+D

[lmiq]

@adienes

auto-installing packages without confirmation on the name (as the REPL currently gives) could come with increased security risks. probably not a lot, but these things add up

@jbytecode

MathOptInterface.jl would require an infinite! amount of installation time and I can except some users to break the freeze process by pressing ctrl+D

Well, I don’t think most users (particularly new ones) think twice before typing “enter” when prompted, but anyway, not prompting is not the an important feature there. It could be -s --prompt-install=no, for that.

@jbytecode

MathOptInterface.jl would require an infinite! amount of installation time

ps: how can that be?

[jbytecode]

Joking, but it takes a significant compilation time

[lmiq]

That true for many packages, but there is no escape from it currently, so I don’t see that being a problem, given it will happen on the first call of the script only (particularly because of the environment created).

I’ve been using some julia script as scripts by manually creating an environment, adding the packages, and activating automatically the environment on top of the script, and it works relatively well, even with some heavy (DataFrames, Plots) packages. The above behavior would be similar to 1) create an environment, 2) add all packages, 3) add import Pkg; Pkg.activate("script_environment") to the top of the script. Just automatically.

[Palli]

@lmiq

I told him that he had to first install the package in a Julia section, and then run the script. […]
That given, I think it would be interesting to have a keyword argument, -s or anything else, such that:

julia -s script.jl

would:

• Create an environment (not temporary, and uniquely identified with that script).
• Install all using packages without prompting.
• Run the script.
• Maybe: cache some compiled code given the functions used?

We already have that capability.

You tend to think of Pluto.jl just for notebooks and interactivity (and possibly graphs), but it’s a pure Julia file with all you need, an implied Manifest.toml file, in just one file. I haven’t checked if I can run it from the shell (or as a REPL, as with ipython), but I would at least want that. That said I’ve made a 30% smaller sysimage, with 20% faster startup that would be good for scripting. Another problem with your “Install all using packages without prompting” is that it’s not reproducible, it doesn’t know the exact package used, so must assume latest version ok, and install that. In many cases would be ok, but better to specify. I guess that’s what Pluto does if handed a .jl file without the (otherwise hidden) markup. Another problem is the implied --startup-file=yes so in effect anything could happen at runtime, Pluto disables that, I believe. [I believe also it adds threads, a non-default in Julia, and possibly not wanted…]

🎁 Package management

Starting with version 0.15, Pluto has a built-in package manager, which means:

1. Packages are automatically installed when you use import or using.
2. Your package environment is stored inside the notebook file. When someone else opens your notebook with Pluto, the exact same package environment will be used, and packages will work on their computer.
These two features are designed to make it easy to write and share reproducible notebooks.

Your last point about caching compiled code is part of my tiny executable idea, i.e. would make .ji files with compiled code automatically, which would be good for default Julia, and then help Pluto also automatically:

Tiny executables with all capabilities. was: PackageCompiler.jl a bad name(?) And why can't modules and scripts not be compiled?

People want compiled binary executable for a few reasons, some good, some bad: To distribute just one file, already possible, but it’s not small. It can also be tiny with my idea. To have everything needed in that file, and it being an .exe. There’s strictly no need that everything is in that file, it just needs to feel that way, give that illusion. To hide (all) the source code. Already possible. Would still be possible. Or you could choose to have source and/or binaries for some files, there…

[lmiq]

@Palli

You tend to think of Pluto.jl just for notebooks and interactivity (and possibly graphs), but it’s a pure Julia file with all you need, an implied Manifest.toml file, in just one file. I haven’t checked if I can run it from the shell (or as a REPL, as with ipython), but I would at least want that.

No, I don’t think Pluto files can cover that use case. They cannot (they can, but they are not thought to) be edited directly. They carry all the meta-data required for reproducibility, which is good for its use case, but not for simple scripting running.

@Palli

Another problem with your “Install all using packages without prompting” is that it’s not reproducible

Someone wanting reproducibility is already in an advanced stage. There could be an option to provide a manifest file separately for that. But, most of the time, users just want to run code with the latest package versions available. This is the use case that a new user faces.

(the --startup-file=yes could be disabled in the scripting run mode above).

@Palli

Your last point about caching compiled code is part of my tiny executable idea, i.e. would make .ji files, that would be good for default Julia, and then help Pluto also automatically:

Yes, but that is the more complicated part. All the “scripting mode” above just would work without this part, as it is just a shortcut to creating an environment, installing everything, and finally adding the activation of the environment created to the top of the script.

[ParadaCarleton]

@ChrisRackauckas

if the package is setup with PrecompileTools

Which they often aren’t.

[ParadaCarleton]

Besides everything mentioned previously, I’d like to share some observations and concerns about the culture of the Julia community, particularly in relation to user feedback and suggestions. While I appreciate the many strengths and features of Julia, I have noticed that the community could improve its communication and attitude towards user requests, criticism, or suggestions.

In contrast to my interactions with Python package developers, who are generally helpful and friendly, I have seen instances where users expressing their opinions or difficulties with Julia or its packages are met with elitism, snark, or rudeness. This can create a negative and discouraging environment for users who want to learn and contribute to Julia.

I believe that the Julia community should strive to adopt a culture of openness and inclusiveness, similar to that of the Python community. This would help foster a positive and constructive atmosphere where users feel comfortable and welcome to share their thoughts and ideas. It would also help attract new users and retain existing ones, as well as improve the quality and usability of Julia and its ecosystem.

I hope this feedback is taken in the spirit of constructive criticism and improvement. I respect the hard work and dedication of the core developers and contributors, and I appreciate the amazing language they have created. I just hope that the community can also improve its culture and communication, as I believe this is essential for the long-term success and popularity of Julia.