See for example in the TOML stdlib:
5 Likes
An alternative way to mark private functions is to prepend an underscore (e.g. _my_private_function(x)). This convention is widely used in Python and partly also in Julia.
3 Likes
The issue is not double inclusion, but the inclusion order. This can cause a lot mental burden and should be handled by the compiler since the compiler knows which module depends on which, e.g when you have
include("B.jl")
include("A.jl")
where module B depends on A, manual include will result in undefined error, but if we implement issue 4600, then we can just write something like (the syntax is not decided yet)
using .B from "B.jl"
using .A from "A.jl"
or
using .A from "A.jl"
using .B from "B.jl"
the order does not matter anymore, since the compiler will evaluate module A when loading B. The include order problem is something the programmer should not and do not need to care about - the compiler has sufficient information to infer that, now using manual include just increases the programmers’ burden.
I don’t think relative module loading is something that is not encouraged, and in fact I believe file loading should be handled by compiler, the programmers should not take care about the orders and dependencies of files themselves - if it’s something the machine can easily do why are we humans doing it?
6 Likes
module Printer include("print.jl") end
I’m curious: why are the module and end lines in the (what I’ll call the) ‘parent’ file rather than the first and last lines of print.jl?
Also just want to say that I’m learning a lot reading this thread!
1 Like
No good reason really. I think it is mostly because the original file came from somewhere else and I wanted to minimize the diff against that. But it doesn’t really make sense, you are right.
2 Likes
I never had a problem with what PatModules is trying to solve (I’ve used the utils.jl submodule pattern mentioned above to good effect), but I think there are some larger themes here that were mentioned in passing that deserve attention:
But in a math book I will often refer to other parts. When an important Lemma is used a book will tell me “using Lemma 4.5.1 (Abraham’s Lemma)” where 4.5.1 will mean Chapter 4, section 5 first Lemma, enabling me to quickly glance back. It would be a disaster if a book would just use names and assume I have memorized everything that has come before. Further, glancing back I can check what the assumptions and result of Lemma 4.5.1 are without having to read or understand the proof. I can build on it without even looking at the internals. Which is what @Skoffer mentions here:
To me the single largest issue with reading (and designing) Julia code is that I don’t know (can’t specify) what the assumptions the author made are. This includes understanding what context the code I am looking at will actually run in. Existing tools do not solve this (short of reaching for the debugger and actually running the code). Due to highly generic code, and due to assumptions being encoded in documentation only (often slightly wrong/out of date, as documentation is hard to test) the IDE tools will often lack the context to jump to the right method when I ask it to jump to the definition of a function used.
This is not really an issue if I am working deeply on a package. Then I will read the code start to finish and have a good working memory of the overall layout and where to get the info I need. Even then I believe Julia could do a lot better at supporting people in structuring code well, but in this context it’s doable. But if I am building on top of a 400k line universe of code that is highly modular and intricate, then that is not a feasible strategy.
The specific combination of features of Julia has brought amazing things that are completely inconceivable in other languages I know. But they do pose real challenges, too.
A final analogy: At one point “be a better and more disciplined developer” was considered an appropriate rebuke to people objection to C-style memory management. I feel like this is where we are at with code organization/structure in Julia right now.
A happy new year to you all!
7 Likes
Highly generic code has its own issues that I don’t think can be captured in the code. The way people try to capture it in the code would be like Haskell’s type system, but there are plenty of examples which would immediately fail if trying to use derived types (units is always one tricky example which breaks simple things like f(u::T,p,t)::T). So anything in the code would only break examples, other than the code itself (i.e. looking at it and saying “the scalars should support sqrt, …”). I’m not sure there’s a better solution than mass combinatoric testing, other than imposing an interface that locks out otherwise compatible types.
3 Likes
Why would an interface lock out compatible types? You need a set of methods anyway, so just have a traits that tags a type or value as implementing those methods
See the example. Common descriptions of interfaces usually don’t just lay out what methods are required but also have restrictions on the types in the methods. But that can be tricky. You may think that f(u::T,p,t)::T is required for u, and it needs sqrt(u::T)::T, etc. but then units are the counter example: sqrt(u) when you have m^2 returns m, and in an ODE f(u,p,t) should output something with units u/t because it’s a rate. So the details in some cases on what can work in terms of types is very difficult or impossible to explain in generality, meaning that if you don’t want to lock useful cases out, then you want to just leave it at functions.
But you don’t want to mention all functions (Base.broadcasted(…)), so you instead try to document it at a very course “it should support sqrt, *, + on the element types”, which of course has some extra undocumented assumptions on type calculations and properties under broadcast and … It can go pretty deep too, like you can say “should be an AbstractArray”, but in reality things that broadcast don’t need to be an AbstractArray, and implementing broadcast is separate from implementing “indexing” (example: CuArrays). So what it comes down to is that with every reasonable description of the interface I’ve thought of, there are useful edge cases which violate it, meaning that documentation of interfaces are more of a heuristic than a hard boundary.
That’s continuing to improve in SciML as we build out:
And solidify higher order properties like “whether A has fast scalar indexing”, but it’s still in progress.
11 Likes
Wow yes, this is much more involved than I had assumed. Thanks for laying it out
Thanks for the detailed explanaition @ChrisRackauckas on what you’ve thought about on this front. C++ Concepts are the solution the C++ standard committee arrived at. They are carefully designed so that you can retrofit them to a library without breaking any existing uses of the library.
I have to disagree with your example, though. If my code assumes that a square root with this type signature exists sqrt(u::T)::T then type based unit systems will break my code, and attempting to use units with my code will provide a sane error message. If my code doesn’t assume it I have provided an incorrect interface (so open an issue and we can fix it). Overly conservative type constraints can be an issue in some languages, but seeing how far Julia errs on the other side I doubt it would be to bad…
The whole situation with AbstractArray, which is an abstract type but should be an interface (e.g. has_index) exactly demonstrates this specific problem of Julias design, and any new ideas need to work around this (and especially the problem that it means that if you use AbstractArray then nothing else can be encoded at the AbstractType level…).
3 Likes