Just wondering where the stability of Julia is currently. I have given up on Julia many times because every time I go back there are so many things that have changed. I do research and many times implement something and then don’t come back for months possibly (I use R more but need speed at times for genotypes and things of that nature).
I know there is an LTS, but this is the core language and not packages correct? Are the packages that we rely on such as DataFrames.jl stable yet?
Just looking for thoughts on this and if biologists can move over yet. I know so many people that have given up on Julia due to stability.
Julia uses “semantic versioning”. Packages at version 1.X are stable until they decide to break compatibility and move to 2.X. Packages at 0.X may make breaking changes whenever.
Julia LTS is not needed for most people since the regular 1.X releases don’t make breaking changes.
DataFrames.jl is on a 1.X release and doesn’t make breaking changes.
Visual Studio Code is the most-supported editor and is free of charge.
Julia v1.0 was released in August 2018, a bit more than four years ago, and the language has been stable (though actively developed) since.
Maybe you can clarify whether you are talking about the language itself or the ecosystem. The former is clearly stable, and the latter varies from package to package.
Julia + EcoSystem makes it really easy to control stability of every dependency. Just stick to your current Julia version and pin packages you depend on. I came from R to Julia and I am so much happier now regarding stability.
Also worth noting: If you use Project.toml and Manifest.toml files your environment is completely reproducible. Even if all the packages you use decide to release completely breaking new versions (which does not happen), you will still be able to reproduce your exact environment and package version with a single command. This is quite a bit better than the Python status-quo.
That’s not exactly semantic versioning. In semver and in Julia ecosystem, breaking changes happen when the first non-zero version digit changes - rules are same for 1.x and for 0.x.
So, 0.1.2 to 0.1.20, or 1.2.3 to 1.20.30 are non-breaking, while 0.2.3 to 0.3.4, or 2.3.4 to 3.4.5 are breaking.
To expand on @Krastanov’s excellent answer, you can actively control compatibility: as largely described here,
create a folder for the project when you start it
activate the folder’s environment and add the packages you need
highly recommended: add or expand a [compat] section similar to this one. Every package you add should have [compat]. This becomes a more permanent solution than relying on the Manifest, as you can do bugfix-upgrades while disallowing breaking upgrades.
From this point, you don’t care whether the packages you rely on make breaking changes, because you won’t get them. You’re in control of which versions you want to use.
One last tip, if you git init this project and check your Project.toml file into it, then you can experiment with updates and easily get back to a working version if something went wrong.
I have been around for the changes. It is true that in the days before Julia v1 some noticable aspects of the language continued to evolve. In my experience, using Julia has become much smoother, and that, in turn, has allowed important packages to become considerably more approachable.
People are having more success with productively applying Julia. Half of that is more packages are built to work seamlessly or nearly so with other people’s contributions. The other half is that the community is so forthcoming with their help. I’m sure to have omitted the other two or three halves…
Here are packages that are heavily used to support other packages’ functionality
(apologies to those who belong in this list and are omitted). With each one is a count of package manager downloads for that repository in the past 30 days. These downloads are a mix of requirements when using a package of interest (e.g. MacroTools.jl) and direct requests to install the package (e.g. Plots.jl).
With more than 15,000 downloads of these 20 packages each day, they get responsive attention and community support. Our library has grown; it is broader and has more well-respected packages than it did a few years ago. As you know, almost all of the content is provided by members of the Julia Community.
Additional participation by the biological among us is always welcome.
Both I guess. I understood they went through v1, but I still was unsure if that means things still aren’t changing. i think for devs small changes are not a big deal, but they break our code we have to then go and relearn some weird part of Julia and figure it out. The packages I was very concerned with as well. I’ve learned a lot of packages (like the plotting package no longer developed I think) and it’s very annoying to have to learn new packages every 6 months.
Excellent answer Tim, this helps a lot. Since you seem the most knowledgeable, I have another question. You guys answered a good part of it, I can stabilize old code for a long time.
The other thing I was wanting to know if you feel like there is enough stability in packages going forward. For instance, ggplot2 hasn’t changed for me since I started learning in 2013. Yet seems like packages and developers come and go all the time in Julia still. This is still important as for us to use, we have to know it’s mostly going to be the same. Let’s say a new package comes out and we need julia 2.x.x. I have to use that to use the latest package in our field but the old plotting package I use (i have used UnicodePlots.jl a lot) is gone and no longer developed and no one cared on in julia 2.0. Do you feel like this sort of thing is stabilizing and devs are hanging or still losing big packages to a dev leaving? This was more of my question I was thinking of, vs what was answered, but I saved these links for when I do work for my company (routine stuff we need to maintain).
I think the existing answers have addressed stability, which I think is easy to achieve in Julia given the Project.toml and Manifest.toml files.
But @austin-putz, I think you are also looking for maturity, is that right? As in, I gather you don’t have much interest in writing your own packages, or adding features to existing packages, or hunting through code because the documentation isn’t comprehensive.
Some parts of the ecosystem are very mature, but others are not. It will really depend on what sort of functionality you need, so it would be worth checking in about specific packages you want to use, and/or asking about what packages folks use to achieve what you want to do.
Plotting for instance has a pretty comprehensive set of features: I find I can achieve just about anything I want with Makie and AlgebraOfGraphics. But there are still sharp edges, and gaps in the documentation. I occasionally have to pull up my sleeves and fix a bug, add a new feature, or improve some documentation.
Like wise, I find that DataFrames.jl is quite mature at this point, and many of the companion statistical packages (e.g. GLM, MixedModels, StatsBase, StatsFunctions) serve me quite well, and some are best-in-class (e.g. MixedModels.jl is far faster than equivalents, like lmer4, in R). But the feature set of statistical analyses available in R is clearly much broader, so so it really depends on your field and what analyses you plan on doing.
Yes, that’s a better way of saying it. I want to know if I come back to it in a few months I don’t have to learn new packages when I develop a new pipeline. I used in this example in another reply but I can be very confident that tidyverse will be there. I’ve come back to Julia and packages don’t exist in the new version I download (which I need for a new package). So I have to learn a new package and that package often has little to no documentation. This cycle is unsustainable for someone in industry as we don’t have time to deal with learning new packages all the time. Most responses only addressed fixing the environment which is important but not really what I was asking. Thanks for your answer. It helps. Seems like we still have to worry a bit on what may or may not be maintained into the future, but I hope it’s stabilizing, at least for the main packages we would use.
It is improving day-by-day. I think it depends a lot on your exact field and the nature of the organization, whether or not it will work for you in an industry setting. For what it’s worth there are a a number commercial organizations that use Julia already. I have the impression that many of them are using Julia to do things that would be much less productive or essentially impossible to do in another language, making the cost of immaturity worthwhile.
This is much less likely to happen now that Julia 1.0 is out: essentially any package supporting 1.0 should work across all the 1.x releases.
Thanks. The main reason I would use it is for speed. The speed isn’t comparable and memory management for large files we have (such as genotypes on animals). Most things R works great, but terrible on memory, even if speed is okay. It’s mostly a development time thing, even if something is inefficient in R, it’s far faster when I don’t need to spend weeks trying to fumble around with a julia package that isn’t documented (which I’ve struggled with in Julia but if you bring it to their attention you get attacked so much). So I just want to know if you all think stable in terms of packages being developed long term so if I need a new version or start developing again in 6 months, packages will still be there and maintained.
There are a lot of packages with a single or few primary authors. Flux.jl’s author left but it got taken over by other developers. It’s hard to say what would happen for other packages, though it probably depends on each package’s maintenance burden and how many people want it maintained.
Which R packages do you use that you’d be looking for equivalents for?
IME this works better if you just open an issue on github for each specific documentation problem, so they can be handled one at a time and avoids triggering the defensive emotional reaction you’ve witnessed.