coincidentally enough, I disagree with your preference about the right choice of default as well as with your characterization of the opposition to that choice for each and every item on that list…besides maybe 4.
Yes, but the original question in this thread regarded steps to take for 2.0, and I think that structured documentation is a step the community could take which would allow for automated checks. And, that might imply a breaking change and packages get let behind at 1.x. R has had these checks with cran for decades, and I think its a major strength.
It looks like good work being done in other areas of improving package quality Aqua.jl
For instance, I’d love to be able to simply do
Pkg.add_organization( "JuliaMath" )
Organizations could already now publish a wrapper package that reexports all (or at least the best supported) packages in the org. You probably want to exclude experimental ones.
I’m not surprised! All of these defaults are examples I chose because they appeal to the kinds of people who are extremely used to writing Julia, have been using it for a long time (possibly pre-1.0) and post on the Discourse and Slack. I picked them because they’re examples where the opinions of a developer and a user would strongly diverge. I’ve noticed that Python developers have a consistent philosophy of prioritizing user preferences, while Julia developers have a philosophy that prioritizes their own needs and those of heavy, experienced Julia users.
Other examples of poor defaults that are easy to miss when you’re learning Julia:
- FMA (not default because it could confuse devs if someone ran a computation on a pre-2008 computer)
- The inability to use positional arguments with keywords (because then, developers wouldn’t be able to change the names of positional arguments without breaking user code)
- Defaulting to single-threading instead of threads=n_cores (held up by complaints this would be a bad default for large-scale computing)
- Extremely long stacktraces filled with nested type declarations (implementation held up by complaints that it could result in users not posting the full stacktrace in Github issues)
- Performance footguns like type instabilities and eager functional primitives (maybe 80% of the mistakes I find in PRs by newer users)
- No compile-time errors when static analysis catches likely mistakes, like method ambiguities.
100% agreed on all of those, though that’s not a surprise since I might be the one who has opened each of those issues in Base , or at least brings them up in gripes when they aren’t fixed.
The philosophy should be: whatever is best for the inexperienced user for the 90% use case
HPC users have to figure something extra out to get a bit more performance? Whatever. That’s the kind of user that comes from C++/Fortran that knows how to read intense API docs. Why should an undergrad get 1 threaded runs and not know why Julia is slow in order to preference the PhD in computer science doing the 1000 core HPC runs not having to add -t 1
to their build script? Why give a giant stacktrace just in case a dev might want the full thing in the future? Just ask for more when you need it (and honestly, what’s chopped off is essentially never useful… at least the {...}
new stuff).
Not particularly. People liked PyTorch because eager mode was more intuitive and produced readable error messages, and also because it was more readable than TensorFlow. These are the main reasons any ML developer will give if you ask them why they switched.
We look so much at Jax because it’s faster, when in reality PyTorch has almost all of the mindshare because it’s easier to use.
Other examples of poor defaults that are easy to miss when you’re learning Julia:
- FMA (not default because it could confuse devs if someone ran a computation on a pre-2008 computer)
That’s not the reason. In a language for people that care about numerics, you aren’t allowed to arbitrarily re-associate floating-point operations, because then you are computing something different from what the programmer specified. Even clang & gcc only do this with -ffast-math
.
See e.g. X * y + z does not automatically use FMA instruction - #2 by GunnarFarneback
The inability to use positional arguments with keywords (because then, developers wouldn’t be able to change the names of positional arguments without breaking user code)
That’s not the main reason. The basic issue with using keywords for positional arguments is that it makes multiple dispatch ambiguous, unless you disallow re-ordering (and even then it is a problem if different methods use different argument names). See e.g. Allow use of named-argument syntax for positional arguments?
Anyway, this is not an example of a “poor default”, but a language choice that you disagree with. Fair enough, everyone has their own tastes, but people who have different taste than you are not being hostile by saying so.
Performance footguns like type instabilities
Making type instabilities impossible would require an entirely different sort of language, not a dynamically typed language. (There are lots of great static languages, and liking them is fine! But that’s not what Julia is.)
…Anyway, this kind of unfocused thread, into which every sort of complaint is dumped, is not particularly productive. I’m doubtful I should even be responding, because it just goes around in circles with the subject continually shifting to a new grievance.
I really dislike seeing this thread continue like this.
This is not because of the original post, the content, or the replies. Most of the content is potentially useful and productive. What concerns me is that much of this will be difficult to reference or find in the future. It is exactly because of the potential that I worry about what a waste all the effort in this thread may be. I hope I am wrong.
Please consider starting a new thread. Your time, thoughts, and opinions are worth it.
I do think I’ll create separate threads from my earlier post, but let me make a final-final comment. When we (Coreform) needed to choose a language to serve as our user-subroutine interface for our (in-development) isogeometric analysis solver (in the field of FEM), we chose Julia (although our codebase is otherwise C++).
The “granddaddy codes” typically use Fortran/C for their subroutines, which are used to define custom material models, element formulations, boundary conditions, etc. So users (typically younger users who’ve learned Matlab or Python) have the challenge of learning Fortran/C and manually compiling their code prior to execution – not always trivial to make sure you’re using the right compiler, and you need to compile for each operating system you want to run on (common to “develop” with GUI tools on Windows, simulate full-scale problem on Linux). And of course then there’s the question of how do you debug your code - from stepping through a debugger, to plotting the function, to benchmarking / profiling your code. Even six years ago, when we were just beginning to spec out our code, it was clear that Julia was the best language for our user-subroutines: the best blend of ease-of-use and performance.
Just a simple example, from our “booth” video that we share at conferences we attend (go to 1:40 if the timestamp in the link doesn’t work):
I hope you’ll excuse my self-promotion, but I thought it would be an uplifting note to mention its use.
EDIT: And what finally convinced me of Julia being the perfect fit was the response from this community on a question I asked back in 2018. Even though I’d been dabbling in Julia for three years at that point, that was the first time I finally saw the full potential of Julia. So what’s one step the Julia community should take, you ask? Answer:
Keep on answering user’s Discourse questions, knocking them out of the park with the power and simplicity of Julia. Any user can do this, I know that I personally am quite hesitant to contribute to source, but I can certainly try to answer other user’s questions.
What concerns me is that much of this will be difficult to reference or find in the future. It is exactly because of the potential that I worry about what a waste all the effort in this thread may be.
Perhaps, not every thread needs to be “efficient, single issue, useful” thread. This is not a commercial concern anyways.
May be there is a reason this is some kind of an attractor thread (despite snark threats from @mbauman )
For me, anything that increases interaction and lets people speak their mind is good.
I personally stopped trying to read this thread about 500 posts ago.
I suggest collecting all the ideas, and eventually spawning threads based on that.
Similar to how @ChrisRackauckas did it with the Revise issue.
Actually this has already been proposed in this thread, If I remember correctly. As is so often, “somebody” should implement it. I understand a person who proposed some Base code change may feel unqualified for a PR. But everyone is qualified to make such a list
I find it odd that LinearAlgebra has something as “obscure” to a layperson as a generalized-eigenvalue solver function, but not a conjugate gradient or GMRES solver.
I’m focusing on a small thing and am hesitant to take part in a broader way in this sort of open ended discussion. But in my opinion, this makes perfect sense. Julia provides a fairly complete set of dense matrix algorithms in LinearAlgebra
, providing solutions to most problems that can be solved using LAPACK. Dense linear algebra is an area in which algorithms can be treated as black boxes that will work well without much guidance from the user, at least on problems that are of a manageable size. There are more choices when dealing with iterative methods and they are in a totally different class in terms of what is required of the user to use them effectively. I think that separating that complexity from the much simpler to use and more problem independent dense matrix algorithms is a good call.
Also, I’m not sure who counts as a layperson, but I would argue that generalized eigenvalue problems aren’t really all that rare if you are in the right field. Having access to the dense matrix algorithms for that is certainly helpful to people in some fields in engineering and physics when they have problems that are not too large.
Yes, let’s bring this thread to a close. Where it’s been most useful, I think, is where we’ve been able to split out into focused topics. Thanks everyone for your participation!
Folks are always welcome to share what they think would make their life better, but dedicated topics are always going to be the most productive. If you want to continue the discussion, the best way forward would be to create new topics focusing on a single issue at a time — or better yet search to see if your issue has been discussed before and use an existing topic!