The problem is when a bunch of other packages depend on this unmaintained package. It’s unrealistic to expect all of them to suddenly change their Project.toml to point to the new UUID / package name.
8 Likes
Being able to fork is pretty much orthogonal to picking up maintenance of orphaned packages. Someone with the skills to maintain the package is a necessary condition either way. If the package can be maintained without creating a needless fork the community is saved from the churn of changing the name of the package everywhere for no good reason. Not to talk about the split of the discussions in issues and pull requests across different repositories, which makes investigating motivations of past decisions much harder than necessary, again, for no good reason.
5 Likes
E.g.,
2 Likes
I am not sure why you think this. If the new package exposes a compatible API, then rewriting the Project.toml is a trivial change that can be readily automated with mass PRs, or a script (and even without those it is trivial).
Moreover, these changes are going on all the time in the Julia package ecosystem. A recent example is SimpleUnPack.jl, which was first released in March 2023, and it restricted but API compatible to UnPack.jl but more compiler-friendly. 14 packages have switched to it already, the change is trivial.
I fully agree with this, but I was talking about the case when a package is unmaintained, or worse, when this is not possible to ascertain or some kind of low-level maintenance is going on but not at the level that would ensure a coherent vision for the package.
A typical example is eg KernelDensity.jl: it is one of the oldest packages for KDE. The original author is no longer active, so it is “community maintained” by the JuliaStats contributors. This gets trivial PRs merged, but a lot of PRs languish unmerged and look abandonned. In the meantime, people start their own packages specializing in some aspect of KDE, there are now 9 on JuliaHub and a couple more unregistered. Many of them are a significant improvement on KernelDensity.jl (mostly in terms of speed for multivariate KDE, using modern methods).
I don’t know what the best solution is in similar cases. Ideally, if no one wants to maintain a package, I think this should be clarified in its README and we should marke the package as abandonned. But failing that (sometimes authors of practically unmaintained packages do not make this clear), forking should be an option.
1 Like
Yes:
-
Essentials for any Julia programmer - anyone hacking away “trying to make something work”: Debugger, Revise, TestEnv - and the VS Code extension (and other editor integrations).
-
Essentials for professional Julia programmers - e.g., a programmer in a team - and programmers focused on correctness and deployment: JuliaFormatter, JET, PackageCompiler, Aqua.
And perhaps LocalRegistry should be highlighted more - to mitigate registrations in General purely due to needing to develop package B dependent on package A.
2 Likes
Now imagine if those 9 authors had added their special things to KernelDensity.jl, instead, and started to help maintaining it. After some time you’d have ~9 people being familiar with a code base, so the odds of long-term survival and maintenance are probably much larger than 10 disparate efforts that are each a little bit maintained (if that).
Seriously, the amount of “bus factor<1” tragedy (instead of banding together for more resilience) going on in the wider OSS ecosystem (not just julia) makes me so sad sometimes. 
(I realise that’s not on topic for this thread, sorry)
IIUC using MultiDocumenter would add links and searchability for essential libraries.
If I am reading your proposals right, @ChrisRackauckas, I think you are also suggesting elaborating on the documentation in JuliaLang/julia, e.g. to highlight the brief intro to Pkg more prominently - and to make it not “wrong”, e.g., so the note could be at the bottom of that page (“for more info see …”).
And to bring things in sync. between the Julia website and the Julia Documentation.
It could be beneficial to elaborate on “the wall of text” on the landing page of the Julia Documentation and on the Getting Started section along the lines of adopting the divio documentation system as suggested by @Chris_Foster (https://documentation.divio.com) - to complement “the programming language manual” (under Manual) with tutorial guides for beginners of various sorts, e.g., the student/researcher in some science field, and the professional software engineer/developer.
Related issue: Better documentation of stdlibs · Issue #28712 · JuliaLang/julia · GitHub
2 Likes
Totally agree, LocalRegistry is surprisingly underadvertised. It’s not mentioned at all in Julia docs, and a very brief note is given in Pkg docs.
Personal-use packages that are properly versioned and can be registered/updated with no constrains and no delay, would be nice to promote this usecase officially!
9 Likes
Revise is listed under the essential tools rubric. As a newcomer, I would read that to mean “power user tool you should really check out”. (people overuse “essential”). No, Revise really is essential. I understand that the VSCode plugin has some native features that give you more or less the same. I’m sure some people who use neither of these still manage to be productive. But I guess that for the vast majority of us, Revise (or VSCode) is essential (for real). I have heard several stories of people and groups who don’t know about Revise abandoning Julia in anger about the constant recompilation.
Here’s an idea (I already posted on slack). We ought to tell people that you need Revise to develop a project in Julia (or VSCode, with a link). add Revise is part of the instructions for installing Julia. I’ve started to adopt that myself in package documentation and tutorials and talks. There’s no harm done if they are one of the few who decide they don’t need it. It may be a bit tricky to avoid cluttering a first-steps tutorial, where Revise would just be in the way, and at the same time make sure it’s part of a first package tutorial.
As Chris said, the prominence of Revise is currently very small. So any number of steps could be taken to improve the visibility.
I think it’s great that this topic thinks about the wider context and how we treat similar tools. But improvements could be attacked piecemeal. If it sounds like it might be supported, I’d make a PR about Revise to some document.
9 Likes
If there’s something I can do to make this easier, do let me know!
This sounds like something that might need to be plumbed into the runtime rather than JuliaSyntax, though? Is the point that each Method needs to have metadata to know which top level expression it originated from?
Anyway, perhaps better discussed on github. Is there an issue tracking this?
1 Like
The divio system is amazingly clarifying and I use it in all my documentation now. It would be great to rearrange the main Julia documentation in such a way. It’s just a huge amount of effort so we’d need to find a way to chip away at it incrementally.
6 Likes
If the load time of package gets down to PackageCompiler levels (which right now is perhaps 2 orders of magnitude difference on large packages) I would gladly remove that part of PackageCompiler. However, note that that is only a small part of PackgeCompiler. The deployment step of creating apps and libraries that are callable without an existing installation of Julia is most of the code in PackageCompiler and the need for that will probably always be there.
13 Likes
This is basically Julep: Expr provenance when lowering · Issue #31162 · JuliaLang/julia · GitHub. JuliaSyntax will solve this nicely for parsing (mapping expressions back to the sequence of characters in the original source file) but not (yet) for lowering. We will likely also need to handle the lowering part before this could really work.
@c42f, I’m super-excited about your work in this direction because I think a lot of things will get better if we can track provenance throughout the compilation process (from parsing through type-inference):
- the debugger(s) will be able to step through code as it is written and not require complex filters (e.g., “is this a bunch of
NamedTuplecode added by lowering to handle kwargs?”) in order to provide a decent user experience - Revise will be able to ditch about 7/8 of its dependency stack and internal code while also becoming more robust
- TypedSyntax won’t be a hack anymore and we could confidently add type-annotations to VSCode (now it’s a bit at-your-own risk and IMO doesn’t deserve top-tier billing)
7 Likes
To add to that list - Coverage can be more granular than line-based. For example, something like
x = foo ? bar : baz
really is required to be covered on at least a per-branch basis, not a line basis.
5 Likes
I had missed that replacing Main was now possible from the v1.9 news. Would teaching Revise to automatically replace modules upon struct redefinition be overly difficult to implement? If not, I might take a look.
1 Like
I don’t know, I haven’t looked into it.
I wanted to point out that I find this proposal very apt. As a frequent user of Julia for the last 6 years (from the perspective of an engineer who needs to write data analysis and other code a dozen times a year but doesn’t program every day) I learned from this thread that Pkg has its own separate documentation that goes into much more depth and detail. I had no idea and perusing it briefly it would’ve been quite helpful earlier in my Julia journey and as a resource to point colleagues too when they have needed to use a tool or code I have written.
15 Likes
Aha, yes this is a lot of work then but I do hope to rewrite lowering in Julia at some point. We’ll see, it could well be a multi-year project 
2 Likes
Another nice feature of the MATLAB doc is that they have a separate page for each function.
Currently, julia’s documentation lists too many functions on a single page and lacks a general description and classification of the functions.
Compared to MATLAB’s doc, they have a neat multi-level classification:
Mathematics > Elementary Math > Trigonometry
sine
sin: Sine of argument in radianssinh: Sine of argument in degrees- …
Some examples:
sin
- sin - Mathematics · The Julia Language
- numpy.sin — NumPy v1.26 Manual
- sin - MATLAB
- Sin: Get the sine of an expression—Wolfram Documentation
Here are some attempts: sin · Base.Math/Trigonometry/Sine
- Generated a multi-level menu using Documenter.jl
- Have a separate documentation page for each function
Currently you need to manually add an md document for each function, perhaps Documenter.jl can provide some support for this in the future. - Added at least one example for different types of input
- Added some images.
By usingUnicodePlots
Splitting function document pages may require more work than rearranging the document pages because we need new content to fill those new pages.
Of course, we can also split the document pages incrementally. Start by writing a separate documentation page for each trigonometric function, then replace the documentation strings in base/math/ with a table or links, and gradually replace the long list of documentation strings.
I would like to know what people think about separate function doc pages.
Is this a good direction to move forward?
8 Likes
Two upsides of long pages
- ctrl-f works
- scrolling shows related functions nearby (docstrings’ “see also” section are generally incomplete)
I think these are important.
5 Likes