I’m new to Julia and I’m learning Julia working on small private project for the HF antenna design but if I’m thinking about the “large” scale software then I don’t see currently any way to do this in Julia. For example one of our libraries, my daily work, is about 1.16 x 10^6 of C and 250 x 10^3 of C++ lines of code. Even if I assume that Julia source code is much shorter that C/C++ code then potential new source code will be large. How is your experience with large scale software packages in Julia?
it is not a simple question of source code size limits but more the question of source code organization, build and test at the daily work. Our software package (library) is spited into 20 subpackages and the makefiles manage the build on the differed abstraction levels. Because our development is test driven I have a set of 6000 test case which should be check before a single push into the repository. Integration and regression tests are on the top of the 6000 “basic” test cases.
I have a look into jeff-regier/Celeste.jl but this “only” 16k line of code package with few test cases – so far I can recognized this.
I’ve recently switched to managing my own registry. This makes it surprisingly easy to manage multiple interdependent specialised packages and controlling compatibility between them. Most Julia packages use test driven development, there is nothing different here. If you want to be cautious - first pass all tests locally, then push, then the tests run again on Travis in multiple environments. And only when you are satisfied with both layers of testing you bump the version number. If you start a new “project” based on your package eco-system you specify which compatible versions of your packages are allowed. I’m not a software developer by any measure, but I’ve found the tools available in the Julia eco-system extremely helpful to me in these regards. It may be worth reading
TDD is pretty much the norm for Julia packages too, usually set up on Travis for open source projects (but for a large, non-OSS project, you may want to get a paid plan, or set up your own CI environment to get faster turnaround).
Splitting code into multiple packages is also very common in Julia. A nice example is the DiffEq ecosystem:
It is still not clear to me why you imagine Julia could pose problems for your setup. Everything you are asking for is available and commonly used.
If you are seriously considering Julia, I would recommend programming in it for a few months to get a feel for the language and the tooling. Then you should be in a position to make a good decision.
It this what I’m currently doing. I started a small “test” project for the hf antenna design and this will take awhile to finish it. I’m focusing myself in this project on using Julia and functional programing. At the moment my test program (package) is at about 1000 lines of code spited into 3 modules and 10 files. As you see is is small at the moment.
Because my daily job is a large scale C/C++ software I’ve been thinking if I could write this kind of software in Julia. From my point of view isn’t practicable to compile from scratch the full source code by every testrun and one of the solutions could be the incremental compilation. At the moment I’m looking for the way to precompile a part of my source code and this is the main reason of my post in this group.
Break it into modules. Let each module be a fully tested unit with clear APIs so that one module doesn’t need to change when changing or developing other modules. It might take a few iterations to figure out a good splitting, so start with sub-modules then when compile or test times get too large, split them. Also check out Revise.jl which will only recompile the parts changed when developing so you can run your tests in a single Julia session without the need to recompile the whole package again and again. Perhaps other people have better advice.