My organization is writing a small ecosystem of packages for physics simulations. The goal is to make them as modular as possible, of course.
One issue we’ve run into is the following: Package A contains some useful stuff, which both Package B and Package C will depend on. However, B will not depend on C. E.g.
# In file A.jl
module A
export AType
struct AType{T}
a::T
end
end
# In file B.jl
module B
include("A.jl")
using .A
struct BType{T}
atype::AType{T}
end
BType(a) = BType(AType(a))
end
# In file C.jl
module C
include("A.jl")
using .A
function foo(atype::AType)
return 10*atype
end
end
Now I’d like both B and C to talk to each other. Specifically, I’d like to do
include("B.jl")
include("C.jl")
using .B, .C
b = B.BType(7)
C.foo(b.atype) # error! Function expecting C.AType not B.AType
The way you’ve written it, this would not happen because both modules B and C imported A.Atype from the one package A. I assume you have 2 packages named A somehow, and there’s no way to make them interchangeable in a nominal type system, even if they have the same structure.
Different packages with the same name can exist in the same dependency graph as long as they are not both direct dependencies of any one package, so import A is unambiguous. This is discouraged, especially for duplicate code.
No there are not two packages. Package B has its own instance of A, and Package C has its own instance of A. To my understanding, that is how packages handle dependencies without clashing. The reason I put the explicit comment there about this is because I want to be clear that B and C are two separate packages depending on some other package A, and not two modules sharing some other module A in the higher namespace.
Thanks for your tips! I was mostly checking if there is some Julianic way of doing this, but it seems situation-dependent. My goal above was to give a simple MWE. In actuality our package C will contain a struct which has AType as a field. But of course, I cannot construct said type using a B.A.AType, only a C.A.AType. Therefore it seems the way to go for use is option 3 if my understanding is correct
You can choose what versions of A each package is compatible with. Whenever another package or environment depends on both B and C, the package manager will try to find a single version of A that both B and C are compatible with. If it can’t find one it gives an error.
You set compatible package versions in Project.toml. The actual versions of packages that are going to be used when running your program are defined by Manifest.toml.
When you create a package P that depends on B and C, package A will be a (transitive) dependency of P, too. Once you instantiate P, Manifest.toml will be created and versions of all dependencies will be resolved. If B fixes A to be at version x (in Project.toml of package B), and C fixes A to be at a different version y (in Project.toml of package c), then the dependencies cannot be resolved and you get an error. However, packages usually define a range of compatible versions of their dependencies, in which case a version for A would be found that works for both B and C. That version is going to be used when running P.
I modified the example slightly so how about you just run it and see? Here’s the error you’ll get
julia> C.foo(b.atype)
ERROR: MethodError: no method matching foo(::Main.B.A.AType{Int64})
The function `foo` exists, but no method is defined for this combination of argument types.
Closest candidates are:
foo(::Main.C.A.AType)
@ Main.C ~/Test/C.jl:5
Stacktrace:
[1] top-level scope
@ REPL[5]:1
That’s a different situation though. Now, you are including a file that defines a module A in modules B and C, which does create two different modules B.A and C.A. Then the types B.A.AType and C.A.AType are truly different. That’s not what’s happening when you create a proper package A and add it as a dependency to other proper packages B and C.
This printing demonstrates you evaluated module B ... end and module C ... end into Main, which indeed may be done by include-ing a file. This is not the same thing as installing packages into an environment and importing them in a session. It also demonstrates that you evaluated a module A ... end expression twice to make 2 separate modules, which again is not what happens when importing one package, or any module for that matter, several times. To emulate loading the package on the first import then referencing the same module for subsequent imports in the session, evaluate a module once then import it again elsewhere however many times you want.
In the broader picture, some languages like Rust and Javascript do let you use >1 version of the same package in the same project, with the disadvantages of multiplied memory usage and of the need to segregate or bridge the distinct versions. This is much more of a problem in dynamic languages where users can interactively dig into different packages and throw their types at each other, so it’s notable that Javascript has structural instead of nominal typing. If Julia were structurally typed, then C.foo(b.atype) would work if both versions share the structure struct AType{T} a::T end. On the other hand, AType would also be indistinguishable from struct Blah{T} a::T end, which isn’t conducive to type-based dispatch.
OK I have tested this by dev-ing out B, and C in the examples here as packages created with PkgTemplates.jl, and using a random package (DualNumbers.jl) for A (letting AType be Dual) and there is no error now. I see my misunderstanding. This is good news as it makes our lives simpler as we develop the ecosystem. Thanks everyone!