That’s an interesting file to pick for OrdinaryDiffEq.jl. While the line lengths are a little long and it should probably be squashed a bit, that file is really concise and easy to understand. If you wanted to pick on me, pick on this file:
What don’t you like about the OrdinaryDiffEq.jl code you linked? Or for that matter, the ApproxFun.jl code? I would consider those easy to read and pretty standard Julia files. I generally don’t care about the style too much as long as the algorithm is easy to read, so I’d take suggestions/PRs.
To be more correct, people are focused on writing packages for the things they care about and will use. And frequently the functionality is incomplete and non-scalable. I agree with the sentiment, but until package developers are on a payroll, they frequently will not build much more than they need.
I assume you see some obviously horrible code here, but I think they were strange choices to make your point…
EDIT: as much of the OP and the title of the thread has been edited (to the better ), much of what I say here may not make sense any more.
I have learned a few lessons from my time in the open source community, and I think some of them are relevant here:
Vague critiques are seldom welcomed. The reason is they convey a negative sentiment without giving a clear course of action to alleviate it.
Identifying specific problems, and identifying specific solutions to those problems, are generally welcomed in an open spirit. In this case, it seems that the problem you are identifying (in your bullet point 1 and 2) is the lack of succinct advice in the julia style guide on file length.
Actions count – words don’t. In this case, an approach you might take is to write a recommendation on file length and submit it as a PR on the style guide.
With regards to your bullet point 3, it is a little unclear to me what exactly you would propose as a solution. And I disagree with your critique. I think the focus on writing good packages is very clear in a fairly large group of important packages, especially when compared to other highly decentralized open source projects such as R.
Payroll is a valid excuse for everything in this non-utopian world we live in.
Actually this is the opposite complaint. You don’t want to improve existing code, so you make your own out of stubbornness or laziness.
In general, I think there should be non-profit money that is funneled to develop some of the most important packages and maintain robust/clean code, and there should be other sources for the community to register packages that are limited/prototype/untested/etc.
I think this is a really interesting problem for mathematical codes. I don’t think that any amount of comments will make this code:
understandable to someone who doesn’t know how PI-stepsize adaptivity works: it’s a transformation of a very in-depth algorithm, and has not been implemented very many places because not many people know it / know how to do it. But just because this code will not be understood by the vast majority of potential developers, does that mean it should be replaced with something similar but easier? No, this is production-quality differential equation solver code, and the benchmarks show that PI-stabilization helps a lot in terms of runtime and stability properties, so it’s staying.
That’s the nature of academic software. Sometimes you have to write a whole paper just to explain and test a small portion of the code. So by that very fact, the number of people who will understand the full code is minuscule (I would bet the answer for “who knows all of the algorithms in JuliaDiffEq” is… just me). And even for those who understand it, it will probably require developer documentation, some academic papers, and some pencil/paper to workout what’s going on, because it’s inherently a math problem.
But then the fact that it’s a mature software ecosystem means that, any algorithm that any undergraduate has ever learned has some optimized implementation somewhere. So the barrier to entry is already going to be much higher than “okay I took an undergraduate class and I’d like to implement ____” since anything at that level is a quick weekend project (and I probably got all of them done already, or Sundials + ODEInterface picks up the slack).
It’s still an issue. I am trying to address it by continuing to keep up with devdocs, being very accessible in my Gitter channel, and writing blog posts along the way to explain the overarching ideas. There’s other methods which address this problem in parallel. One way is the modular architecture which means that someone can contribute to the ecosystem by creating a brand new package, so there’s no reason to force someone to adding to a specific package (though in this case, for it to be useful, it probably needs all kinds of optimizations, callback interfaces, advanced stepsize algorithms, all of which you get for free with the *DiffEq packages). For this reason I am also leaving around ODE.jl as a non-production quality (but decent) ODE package which people can hack away on with a much lower barrier to entry (I would definitely use that for teaching).
But line spacing / breaking up files is the least of my worries. My main worry is writing more of the papers to explain what some of the new algorithms are, and continuing to test/benchmark every detail. That stuff will let me advance to candidacy. Moving code around to look a little (subjectively) prettier won’t.
I think it’s very similar to using Parameters.jl’s @pack and @unpack. I think it would be an interesting addition to that set of macros. But is
But yeah, that code definitely has a case of "it’s not broken, it has a ton of tests to make it stays that way, and I have other stuff to work on, so I’m not going to touch it until I both have the time to implement the beautification, but also properly test it out to make sure that this very core piece of code doesn’t break. However, I instead have to spend my time wowing academics and implementing novel algorithms, otherwise I’ll be out of a job. It’s a sad reality, and yes it does always go back to the fact that open source software is woefully underfunded (if at all…).
if a programmer feels the need to separate logical units into smaller files to increase readability, you don’t need suggested limit. if a programmer does not feel such need, numerical limits won’t help. also, there are good arguments for and against chopping up the code base. if you have good tools to navigate large files, what is your argument against it? at the end, we are dealing with types and functions. let us have some freedom where to put them. (i always hated languages that impose a file structure on you.)
It appears to me that a big reason for disliking long files is the difficulty of navigating through them. This is an issue of imperfect IDEs and not of files lengths. From a user experience perspective, an ideal IDE or editor lets you navigate between functions easily, no matter whether they are in the same file or in another file. Functions are the natural logical and navigational units. Files are irrelevant.
UX-wise, and maybe for the style guide, one should therefore think about function length rather than file length. But, as several have pointed out, there are constraints on how short one can make a function that implements a complicated algorithm.
Even if you managed to get this highly contentious restriction into the style guide, note that the whole guide is a suggestion, not anything binding. So people could still write code as they please. Which, when you think about it, is great.
There is no Style Guide Police. In the open source world, the only person whose coding style you can constrain is yourself. You can try submitting PRs to the abovementioned repos, but aesthetic reorganizations or mere reshuffling of code into separate files might not be a priority. Which is also great, when you realize that developer time is a scarce resource.