High Quality Packages to Learn From

As a newcomer to Julia I found reading through docs and the underlying code of high quality packages to be one of the best ways to learn idiomatic ways to do things, style and best of all a bunch of useful functions in base and how they are applied.

So far I have gone through DataFrames, I would love other recommendations of packages that would show a reader and exploring of the language good techniques.

7 Likes

Images.jl or basically anything @timholy has written :wink:

7 Likes

Some of the JuliaNLSolvers packages are very nice “introductory” packages which do things in a solid way with good outcomes. The easiest package to dig into is probably DataStructures.jl. Its code is dead simple and you probably already know a lot of the algorithms so it’s a nice introduction to Julia.

On the larger end, Optim.jl and NLsolve.jl are great packages and their source code is some of the easiest to read, and you’ll see a few things in there like caching to reduce allocations and function barriers. These packages also rely on a base package NLSolversBase.jl so it’s a nice introduction to the standard org setup. For example, if you didn’t know the algorithm for Trust Region Newton methods, this is pretty much pseudocode for it:

https://github.com/JuliaNLSolvers/NLsolve.jl/blob/master/src/solvers/trust_region.jl

Take notice on how they’re using .= and mul! commands and you got yourself some good idiomatic Julia.

If you want something complicated, I would say check out OrdinaryDiffEq.jl. The reason why it’s complicated is because it’s probably the best example right now of how generic a code can get: you can put types with units and symbolic values from SymEngine.jl etc. in there and it’ll all work. The reason why I say it’s complicated is because there is no reference to this anywhere at all in any of the code: that’s why it’s all generic without having dependencies on the numeric packages it uses. Each piece of code is solving some kind of deeper riddle, for example take a look at how the absolute tolerance is automatically applied:

https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/solve.jl#L118-L126

If you had an array of arrays you need to make sure you have a nice way to get the unitless bottom element type (of the recursion), which is done like:

https://github.com/JuliaDiffEq/RecursiveArrayTools.jl/blob/master/src/utils.jl#L96-L105

so it exploits the generic difference between one and oneunit without having to rely on which unit package a user is using, and it is careful to do this in a way that the compiler can figure this all out at compile time. Then it also is sure to take the real part of the element type in case it’s a complex or quaternion number. This file itself is something I usually have GSoC students ignore due to these kinds of complexities, but if you want a pretty thorough documentation of how you make a generic algorithm truly work in full in a very general sense, that’s the nitty gritty details.

Another quite complicated good Julia package is IterativeSolvers.jl

It sets its algorithms up on the iterator interface and it utilizes all sorts of fast linear algebra tricks in there. If what you do is scientific computing then the source code is well worth a look (and ask a few questions in the #linear-algebra Slack channel!)

8 Likes

https://github.com/oxinabox/MultiResolutionIterators.jl

I think, illustrates some worth-wild techniques,
like Holy traits (Scalarness),
and programming with higher order functions.
And a fair bit of dispatch.

And is is not terribly written.

2 Likes

Thank you all, these look great. Over the next few days I will dig into each of them and see what I can learn.
I especially like the idea of learning algorithms I am not familiar with by reading Julia code, great idea.

I used Distances.jl for exactly this purpose when I was starting out. It is a great example of how to use the type system in really simple packages, but it also uses lots of the little tricks for making code run as fast as possible (eg using @inbounds and @simd where possible, etc).

2 Likes

Cheers, thats an incredible package I didnt come across previously, thank you for pointing it out, already learning a lot!

I especially like LightGraphs.jl as an example of a well designed and utilized interface (complete with a nice concrete implementation).

1 Like