It’s arguably a package of fundamental importance and I’d love to see it ship with Julia. Maybe the @btime macro should even be available without any using? 
Thoughts?
2 Likes
I personally think this would make sense – almost at no occasion you would want to include compilation time in a benchmark. I would even argue @btime should be the default behavior of @time macro?
1 Like
I think everyone will agree that BenchmarkTools is of fundamental importance, but that doesn’t necessarily mean it’ll be easy to convince people it should be in stdlib. The idea of @btime always being available is definitely appealing, especially considering how many people complain about something being slow after using @time, but I can’t say I see all that many advantages of putting BenchmarkTools in stdlib.
Hopefully this won’t be seen as maliciously derailing this thread, but I’m of the opinion that it is significantly less important for BenchmarkTools to be in stdlib than MacroTools. If you need to do benchmarking, you use BenchmarkTools period, nobody would seriously question that. On the other hand, the fact that MacroTools is not in stdlib has resulted in the fact that there is a huge amount of unnecessarily bewildering and complex Julia macro code out there in the world. (Again, sorry if this is impolite, feel free to split my comment into another thread if it is too derailing.)
By the way, all my discussion here is predicated on the premise that at some point the stdlib will be under separate version control from base Julia.
5 Likes
The general tendency is to make standard library packages less and less special. They were moved out of Base for that reason, and now you pretty much need to treat them as any other package.
I think that the ideal solution is to have a setup where you don’t need to care if a package “ships” with Julia, since correctly setting up your Project.toml and perhaps providing a Manifest.toml will just take care of providing it.
Not having libraries in the standard library collection has a lot of advantages: bugfixes and features can be provided independently of the Julia release cycle, making the latter also more efficient.
Of course, for an offline setup there are a few issues to solve, but nothing a competent sysadmin can’t handle.
4 Likes
Moving out of Base and moving out of stdlib are two very different things!
And a lot of disadvantages. Bugfixies are less guaranteed and in some environments installing additional packages imply additional bureaucracy effort.
I’m not really sure what’s being implied by your latter comment. If it is that you are subject to a corporate bureacracy that makes it difficult to install additional packages, I sympathize, but at the same time I absolutely do not think that the development of Julia should make concessions to accommodate dysfunctional corporate bureaucracies.
As for bugfixes being guaranteed, that’s putting a lot of responsibility who are volunteering their free time to work on Julia. In smaller packages it’s easier to go in and make your own PR to fix bugs.
3 Likes
Sure, that’s kind of the point. Once functionality is outside Base, it makes very little difference if it is in a “standard library” package or not.
I am not sure why you think this. (In any case, bugfixes are not “guaranteed” in any sense whatsoever.)
Thinking the standard libraries as “batteries included” or having some extra level of QA and then trying to stuff everything into that is a mistaken approach, as it increases the complexity of the project, slows down releases, and forces people to cope with additional API changes at the time of a new Julia release.
Instead, Julia provides the necessary tools (the new Pkg with fine-grained dependency management, CI frameworks, etc) so that users can work and keep up with any package they want seamlessly, regardless of whether it is in the standard library or not.
3 Likes
I completely agree. Companies restricted by such rules could perhaps contribute to maintaining a curated and quality controlled set of “standard libraries”, or pay Julia Computing for doing it (it is very close to what they are doing with JuliaPro anyway).
It is not always dysfunctional. See if somebody depends on Cxx.jl they are in troubles now. It could be fine if you just write articles to science magazine or teach with small scripts - you could (and probably want!) to switch often. If you will have multi-thousand line production code you are in different situation.
a) we probably have to start thinking about Julia in bigger picture.
b) fixing other work bugs cost money and is difficult/unpleasant to plan if you are working in bigger teams with time schedule
1 Like
I’m not saying that these things don’t require good judgment, I’m saying that slapping labels will not be substitute for such judgment. Cxx.jl is a case where there are clearly very few qualified maintainers. That is one of those cases where you probably have to consider that it might not be so easy to go in there and make the changes you need. The BenchmarkTools.jl case is somewhat special: Julia obviously needs to have this functionality. If it is not available (from this package or an equivalent) the language is dead, so you run the same risk using BenchmarkTools.jl that you run using Julia in the first place. Putting it in stdlib would not change that.
Yes, this is one of the concessions that must be considered when using open source software. The fact that, in most cases, open source has so comprehensively defeated proprietary software shows that it is a better approach nevertheless. Again, this is something that requires some judgment. Deciding to use a specific Julia package is different than deciding to use the linux kernel. In any case, slapping an “official” label on something doesn’t necessarily mean more people will come along and maintain it.
I don’t mean to imply that I am absolutely opposed to the idea of BenchmarkTools being in the stdlib. I made the argument for MacroTools because I felt that the visibility was important, and that the benefits would far outweight the cost of the relatively tiny amount of code in that package that would need to be maintained as part of the stdlib. I’m just saying that so far, were I one of the maintainers of the BenchmarkTools or julia repos (which I’m not) I wouldn’t yet see any convincing reasons why it should be moved.
1 Like
You may be missing the following: if you have a working setup, you just commit your Manifest.toml and don’t need to change anything until you are ready to deal with it.
It is why so many projects and packages stayed at python 2.x for a long time.
You don’t “need” to change anything if you don’t need to go ahead if you are prepared to resign at progress (which is accelerating into singularity 
),
Yes you could keep working with Julia 0.6 and using Cxx.jl. But do you really thing it is a sustainable and competitive method how to create software? 
It is quite unlikely that this will happen. The argument that things should be moved to stdlib so they are guaranteed to be maintained is likely not that compelling to the people that now have to maintain it ;).
BenchmarkTools, Documenter etc work very well as separate packages.
7 Likes
I don’t understand why you think I suggested this.
I just want to make the following point:
- Pkg has reasonable defaults based on SemVer, and optionally even more fine-grained control for reproducing the exact environment, if that is desired.
- Upgrading can and should happen, but it can be done in a controlled way, without disrupting production work in any way.
- This whole issue is pretty orthogonal to whether a package is in the standard library or not.
If Cxx.jl would have been part of the standard library, there would not have been a julia version 1.0 where it is broken. How is that not relevant?
1 Like
Cxx.jl would never have been considered for inclusion in the Julia repo but assuming there had been development time moved from other things to maintaining Cxx, it likely that would have meant no new optimizer and no new iteration protocol. Thinking about the opportunity cost is very important.
6 Likes
Perhaps someone could clarify what the purpose of stdlib is, and what the criteria for inclusion are?
So far, I’m getting the impression that nothing should be in stdlib.
2 Likes
Ref What is a Julia standard library? · Issue #27197 · JuliaLang/julia · GitHub.
The stdlib are currently things that were moved out from Base with the purpose of at some later point be able to be versioned separately from the language itself. Therefore in the new package management system you have to do using Stdlib and declare dependencies on the standard libraries so even if one of them got moved out of the Julia repo the dependency on it would still be recorded and existing code would still work.
2 Likes
Does that mean that in principle, there is no need for a stdlib? It’s more of an “intermediate storage” for things moving out of Base?
1 Like
Sure, one has to make decisions. I am not saying that anything is wrong with julia 1.0 or the choices of what to include and what not. I just pointed out that being in std implies that the package is functional in the next release. While there won’t be magically more maintainers, people are forced to keep it working. For some people this is therefore a good argument for putting stuff in std or at least keeping it there.