Yes I agree. I just don’t know why devs on here get so irate about it. Okay user wants more documentation… I don’t know why the Julia devs freak out about this. I’ve had many conversations with biologists who agree. I have not come back to Julia for 3-4 years due to this issue. I just hope every time I come back it gets better. I will check again and see how it’s going, at least for main packages. It seems the DataFrames guy really did a good job and sent me his book. I have bought and will read it. But who knows on other packages. Hopefully all main ones got better, I will document and report back when I have time.
you shouldn’t confound package for a language, Tidyverse is just a package, not the language, you CAN separate the two. so saying “Tidyverse is xxx” has nothing to do with how Julia compares to R and I’m pointing out the fact that Tidyverse dominating R’s user mindset may not be a good thing.
you seem to have only been a user of packages, maybe you should care more about “if XXX application has a mature Julia package”, not “if Julia itself is stable”
No one is freaking out about more docs, we all know that’s needed.
People are put off by your demanding tone: you are failing to see developers as just people doing things for free with limited time, who owe you nothing at all. But you call them lazy.
What we have been saying is make clear github issues with your problems. That is a way to help and let people know whats missing.
Anyway, I hope you find some better docs this time, and can communicate your needs to the specific packages you have problems with.
(and the Makie.jl evidence is the amount of devs working on it - thats the advantage of a dev stacked community, and why it will stick around)
20 Likes
I think the questions why developers in general behave this or that way belong rather to the realm of psychology, and there must be better places to discuss such things.
The fact is, many developers do register their packages, some better and some not as good documented. The fact is also, many users find it useful, even if a package in question is not perfect. Furthermore many packages are getting better with the help of the users feedback, which were not possible without registering the package in the first place.
2 Likes
No, the point is such a statement isn’t helpful. “Make better docs”, but then
okay so not DataFrames? Is it a complaint in one of the definitions of the ODE solvers? https://diffeq.sciml.ai/stable/solvers/ode_solve/ No you’re a data science person so not that library. That rules out 2 libraries, >7000 libraries left to ask you about?
“Better docs” just isn’t helpful when there are thousands of libraries you could be talking about. Some are well-documented, some have no documentation. What would help the ecosystem is to figure out where there seems to be more user interest than developer interest (and thus, where users seem to be seeing bad documentation that no developer is working on). Saying something specific like “I tried using GLM.jl but could not find out how to do X, it seems to be missing tutorials”, that’s useful. But “better docs” and calling people “lazy” means that you’re literally downing on people’s documentation that you yourself haven’t even read, unless you do literally have an opinion on all >7000 documentations out there which we’d be happy to read the commentary.
13 Likes
I am highly skeptical this thread is going to reach a good point by continuing further. I encourage everyone to pause and re-evaluate what the aim of further discussion would be.
32 Likes
That’s a somewhat fair question (the first one).
I’ve often used (small) packages by just reading the code (code can and should be self-documenting to some degree, and is with Julia which is very readable, more so than most languages), also to learn from it (yes, that would also apply for unregistered packages).
Julia has now over 9000 8400 packages (if counting with the 30+ waiting in the queue such as DWave.jl), half of CRAN numbers (and R is 29 years old, or 46 years old if you count S, which I believe R cloned or is compatible with so decades of head-start). http://www.modulecounts.com/
JavaScript (npm) has most: 2148968 (1044/day) followed by Java “Maven Central (Java)” at 501955 (280/day) then Python “PyPI” 405980 (243/day), then 10th largest package ecosystem:
CRAN (R) has “18700 -1/day” (I’m not sure how that happened).
[To e.g. DUB (dlang) 2244 1/day (much older language, aiming to replace C++, so ecosystem is different, e.g. more systems/low-level code, not competing with Julia, no REPL, though in many ways similar).]
I’m sure the ecosystem of R, like Julia and all the other language have a long tail of varying quality of code and documentation. My view it’s better to have the code without docs than no code. Of course better with docs, even if it’s minimal. Just registering would/should signal the code is useful. Otherwise I don’t know with some public repo.
If you don’t like the docs (or code) you can help with it by making a PR (not just open an issue). That would be constructive.
1 Like
For a basic user it doesn’t make sense to separate them. I work in R, with the Tidyverse ecosystem… Your argument is very technical and doesn’t matter to me how it arrives with in R or Julia. For devs it certainly does. Non for me. I don’t care how the pudding is made.
It’s not demanding. I don’t care. I’ll just use a different language. I’m only frustrated that when I come to Julia and there is holes places, why do people get irate if you simple ask them to add some documentation? I’m writing a pipeline right now as we speak, I’m documenting it as I go in a README file in my home directory for the project so I know when I come back in 6 months what I did. It’s not even going public and I do that. I just find it odd that devs don’t always do that. There is no demanding tone, just disappointment for a project that could be great, basic users can’t figure out how to use it and leave (many within my circle have told me about Julia, not just me). You all can be as sensitive as you want with an issues. It’s like when people say anything bad about your favorite sports team and people get irate for no reason. Julia dev community (i.e. this thread) is the same. It’s mostly dominated by advanced programmers who get overly defensive when anyone suggests this language is not the very best of all languages. It’s like asking the Yankees to root for the sox. I get you guys love it, but why not just admit there are problems instead of attacking users if they do point something out?
For sure there is varying quality, some R packages are terrible but still the baseline is often 10x better than the baseline of Julia code due to CRAN I presume forcing it on devs… I was only wondering from my original post how good it was and how stable these packages would be over time (i.e. would they still be maintained in 6 months or a year or when julia moves to 2.0 in the future). I will do my own homework on the main packages and see how well they are documented and see what I find. Thanks. I see Julia is growing and that’s why I’m on here interested in going back to it.
Could it be that your impressions are (partly at least) framed by how intensely you’ve used R/tidyverse vs Julia? The impression you describe about “instability” I have experienced too with R/tidyverse, Julia, Python/matplotlib (when ggplot2 went from 1 to 2, when Julia went from 0 to 1, or Python went from 2 to 3!). The features I need are in constant motion and if I don’t touch my code for a couple of years, many things have to be updated. Once you start using Julia regularly and leave your R code for a couple of years, your impressions could very well be reversed. Just something to ponder. I think a more focused and potentially productive question is about whether the group of Julia users in your research area has reached a critical mass.
2 Likes
Sure, it’s gotten way off track. That’s how conversations go and how threads go… The main question was if I come back to learn Julia is it stable enough that when julia goes to 2.0, I’m still going to have the same code for the most part from DataFrames and base julia. I’ve upgraded from R 3.0 to 4.0 and nothing really broke. Hadley has some packages that yes he implements new functions but always keeps old functions (usually like reshape2 and other functions in dplyr but yells at you to tell you they are deprecated). So to me, there is nothing new when I go to a major release. The question was about maturity as one user pointed out not “stability” as I called it because I’m just a dipshit user who knows very little programming. I’m not a dev, excuse my ignorance. I only want to know that I don’t need to relearn Julia in 2 years if I come back, many of us in my circle have given up on Julia due to this in the past, it’s been 3-4 years for me, I just asked this question to learn as much as I can, but turned out to just be devs attacking me and being sensitive about their favorite language. I’m language agnostic, I’ve used many and don’t care if they work and get the job done, if you love Python over R, great, it’s a great language too.
The answer to your questions, then are
- Documentation has dramatically improved. We have given you many examples of tutorials and documentation in mainstream Julia packages.
- Proper package management is how to maintain stability, not promises the API won’t change. If you use
Manifest.tomlto record your dependencies, the situation for Julia is vastly superior to R.
3 Likes
My original comment stemmed from an issue 3 years ago or so on CSV.read() where I looked for 2 days on solutions to reading in animal IDs as a string vs a numeric value. This was very frustrating and had to go down to emailing the dev, which I get is soooooo wrong and get so much attitude about on this thread… sorry… Anyway, yes it was frustrating. Yes I understand every package is different. My generalization was to ask “in general” if it was getting a little better (2 out of 10) or way better (9 / 10 score) on documenation for KEY package such as DataFrames or CSV, Plots, etc. Not to attack any one of the 7000 packages. Just IN GENERAL can you give me some idea how the last 3-4 years I’ve been away from Julia, how has this documentation progressed please. Not trying to attack Julia here, just wanting serious information. The book he linked me to was very helpful and instead of attitude such as that, I could get serious replies like his that shows me exactly what my question was asking for. Just answer how it’s gotten better so I can know if I want to come back to learn Julia or it’s still too premature and I should wait for 2.0 or whatever as it developes and “matures” as one user put it.
I agree. The only thing I struggle with is if I point out missing documentation it’s an on slot of attacks on here… Okay, relax guys… I was still talking about when I was learning Julia 3-4 years ago. If it’s gotten better just show me is all… I just wanted to know on 1-10 scale how much better it’s gotten and if we expect the code to mostly be the same into Julia 2.0.
I think it’s safe to say that the situation now is vastly different from 4 years back. Julia has been stable ever since the 1.0 release (which was around 4 years back, IIRC). At that point I had to unlearn sub2ind and learn CartesianIndex, but I’ve not had to relearn anything ever since, other than how to optimize code. The API has been remarkably unchanged across much of the ecosystem that I’ve used, and semantic versioning makes it easy for package developers to not accidentally break code.
I am sympathetic to your point about documentation, as I’ve suffered from sparse documentation myself. However, this may be improved, and it’s a different point from stability anyway. It’s better to create a separate thread that lists poorly documented packages, so that volunteers might know where they might be able to contribute.
As for your point about Julia 2.0, the developers have promised that (a) it’s not coming anytime soon, (b) they’ll try to break as little as possible if it ever arrives
3 Likes
Well, you were told several times, that Julia going to 2.0 is not going to happen in the foreseeable future. That is the official statement of the creators of the language, and this is the thinking of every participant of the discussion except you. So there is no point in asking here “what if” as nobody is considering such a thing. I personally think you will be simply too old to learn Julia by the time when Julia goes to 2.0 should that ever happen 
1 Like
Alright, let’s talk about “in general”. The DifferentialEquations.jl documentation is fairly complete:
https://diffeq.sciml.ai/stable/
One note of stability is that if you take a look at the tutorial from 2018:
Then you’ll notice that the syntax is the same today for the ODE, SDE, DDE, etc. solvers. That’s because the last major breaking change in DifferentialEquations.jl to the top-level syntax was in 2018.
With respect to differentiation, the long-awaited complete documentation of SciMLSensitivity was completed this year, splitting that out from DiffEqFlux:
https://sensitivity.sciml.ai/stable/
https://diffeqflux.sciml.ai/dev/
And as mentioned in the recent blog post, error messages and docstrings got overhauled this year:
Each of the SciML libraries now has a documentation, so for example here’s the SciMLBase API set:
https://scimlbase.sciml.ai/stable/
Being a package downloaded by 25% of unique IPs to the package server, that’s quite representative of a “core package”, and you can see how that documentation is leaps and bounds over where it was a few years ago, without making major breaking changes.
Then of course there was the splitting of Symbolics from ModelingToolkit.
https://symbolicutils.juliasymbolics.org/
https://symbolics.juliasymbolics.org/dev/
While I will admit MTK still needs a bit more docs, giving them both well-defined homes has really increased the amount that is documented. Most of it is all covered in docstrings, and most of these libraries were new as of a few years ago and have stabilized into a pretty core syntax that is new kept stable.
That’s a quick tour through some core packages, and you can see over the last 3-4 years that if go back through the tutorial videos for these, the same code works, there’s just more features, more performance, and more documentation. That is sans the experimental parts like DiffEqFlux which finally stabilized earlier this year.
8 Likes
I don’t think this question is really answerable in general - there’s lots of changes in the package ecosystem in the last 3-4 years, mostly I’d say documentation has gotten much better but as others have said it’s definitely uneven across the ecosystem so your experience will depend a lot on the actual packages you are using.
For DataFrames/DataFramesMeta I’d say docs are excellent, and for CSV they are good. For plotting it will depend on the package you’re using, I’d say Makie has made an excellent effort as has Plots.jl, if you’re a ggplot2 person you might prefer Gadfly or the newer (and probably not as well documented) AlgebraOfGraphics so ymmv.
It’s also a quesiton of what you are looking for I guess. Given that you’ve brought up the issue of column types in CSV.read() a few times, I’ve gone back to the docs for version 0.4 (released in 2018) which are here, which include:
CSV.File— Type.CSV.File(source::Union{String, IO}; kwargs...) => CSV.FileRead a csv input (a filename given as a String, or any other IO source), returning a
CSV.Fileobject.
(…)>
Column Type Options:
types: a Vector or Dict of types to be used for column types; a Dict can map column indexInt, or nameSymbolorStringto type for a column, i.e. Dict(1=>Float64) will set the first column as a Float64, Dict(:column1=>Float64) will set the column named column1 to Float64 and, Dict(“column1”=>Float64) will set the column1 to Float64typemap::Dict{Type, Type}: a mapping of a type that should be replaced in every instance with another type, i.e.Dict(Float64=>String)would change every detectedFloat64column to be parsed asStrings
which to me seems pretty comprehensive but maybe you don’t consider this sufficient compared to R’s vignettes or other things that exist in the R world, in which case you might still be disappointed by the docs for most Julia packages today.
5 Likes
I didn’t read this whole thread super carefully and so maybe I’ve missed somebody else making this suggestion, but I’d just like to propose another option for @austin-putz:
Just actually try the language again! All you need to do is make a directory and create a Project.toml file (for example, with ]activate . at a REPL), add the packages you want to use, and give your test case a shot. Conversations like this can of course also be productive, but I have to imagine the bandwidth you have spent on this thread is already comparable to what it would take to just try doing something with it again. And if you don’t have the time right now or a small project that would make sense to give it a shot, maybe this isn’t the right time to try for reasons that have nothing to do with the language. And if you do try it and something comes up, I think this thread has also given some helpful advice for how to open issues and see if doing so helps you to get back to a level of productivity you’re happy with.
What could be a better way to assess whether or not the language meets your needs than that?
14 Likes