I’ve done an application in C++ that is taking some million lines of code.
And I’ve paid the price.
By starting reading the book ‘Large Scale Software Design - John Lakos’ I can see I made different mistakes, which I don’t want to repeat now that I’m migrating to Julia.
Tips and guidelines on large scale software design on Julia?
One bit of advice I once gave someone who apparently found it useful was:
Don’t just write programs, write the API that you’d want and then use it.
This is hardly particularly original advice (Lispers have been using this approach since before I was born), but if you think of your code in terms of more general APIs, then test those general APIs thoroughly (not just in the way you intend to use them, but more broadly) then you get a bunch of nice high-level abstractions that you can compose your application out of. The application will be easier to understand and change. The internal APIs are also the natural point for testing the correctness of your abstractions, so there’s a bit of a nod to test-driven development philosophy in here too. Of course, designing good APIs is one of the hardest things there is in software engineering, but internal ones don’t have to be perfect. Do respect your own abstraction boundaries though, or you’ll end up with a mess anyway!
I doubt that you will find books specific to Julia about this topic yet. By today’s standards, the only large scale code I know of is Base; even larger packages and umbrella organizations have much less code in comparison (possibly because Julia is very terse).
I find it instructive to read the source and follow the discussions and PRs (mostly in read-only mode).
Celeste.jl is around 16K Julia LOC (counting all **/*.jl files), which is rather large. In comparison, Base is 266K LOC by the same measure.
I doubt if I can legitimately claim that Julia programs require less code in general but I do often wind up amazed at how succinct Julia packages are. I suspect the powerful extensability resulting from meta-programming and multiple dispatch has some part in that. Sometimes I advertise Julia to friends and colleagues by introducing them to the expression problem, to which the usual response is something along the lines of “well, so what if I have to go back and add a function to my object? you have to write the new function somewhere regardless”. But the point is, this is usually impractical, because most code lives in self-contained packages. Multiple dispatch makes it much easier to write a package without worrying about every possible use case because it is so much easier to extend after the fact, I suspect this is one of things that cuts down on the size of Julia code.
Metaprogramming can also greatly reduce the amount of boilerplate code that you typically have to write in other languages.
I remember very early on in my investigation of Julia (to recommend it or some other solution to the company I was consulting for), being totally amazed by what Steven Johnson had accomplished in a minimal number of lines of code to wrap the Intel decimal floating point library (DecFP.jl).
Multiple dispatch, the wonderfully expressive type system, metaprogramming, multiple return, and type inference all come together beautifully, IMO, in Julia, to really cut down the WET factor, keeping my code nice and DRY 
I think if you are writing a large code-base you want to take the recommendation to keep function names as short as possible in lowercaseallsmusheddtogether ( exaggeration ) with a grain of salt.
Quick peek at Celeste and I see names like strip_keyword_argument_prefix. I think this is a better name than stripkp or something like that.
Do nothing until you know that you want to do something a certain way.
One of the endemic difficulties that impede a very large software project is an overreliance on the design one feels without clarity of purpose. As @StefanKarpinski has said “…write the API that you’d want and then [program what is needed for you to be able to] use it.” When the manner of use and modalities of interaction with an unfolding software are simpatico one with the others, then the pace of progress is increased and bad roads avoided.
That will get you going more feasibly. Equally important is not becoming too insular with your development process. Large efforts benefit from wise eyes. And it is always good to have a few others who know where you are at and why you are there at the time.
Best,
Jeffrey Sarnoff
