Hi I realized during the development of a Julia package that one of the big drawbacks of importing/using local modules is that the signatures of all functions tied to a Modules namespace are changed to be nested in the including Modules namespace when include is called. Which makes sense, because that is exactly what include does; evaluating the included file in the enclosing Module. This creates issues however when you want to construct objects in one module and then evaluate them in another based on a variable known only at runtime because the method signatures tied to the type definition in the evaluating module don’t match those of the module where the object was constructed. I’ll try to construct a simple example below:
config.toml
runtimevars=["Evaluate in ModuleA.", "Evaluate in ModuleB."]
MyModel.jl
module MyModel
struct Model
data::String
end
function getdata(model::Model) = model.data
end
MyEntryPoint.jl
module MyEntryPoint
include("MyModel.jl")
import TOML
using .MyModal
configs = TOML.parsefile("config.toml")
runtimevars = configs["runtimevar"]
for var in runtimevars
model = Model(var)
if var == "Evaluate in ModuleA."
include("ModuleA.jl")
elseif var == "Evaluate in ModuleB."
include("ModuleB.jl")
end
end
end
ModuleA.jl
include("MyModel.jl")
using .MyModal
processdataA(model::Model) = println(getdata(model))
#=
SOME CONDITIONAL TO ENSURE THAT THIS ONLY OCCURS
IF IT IS INCLUDED IN THE ENTRY POINT.
=#
using ..MyEntryPoint: model
println(getdata(model))
println("We are evaluating in ModuleA.")
ModuleB.jl
include("MyModel.jl")
using .MyModal
processdataB(model::Model) = println(getdata(model))
#=
SOME CONDITIONAL TO ENSURE THAT THIS ONLY OCCURS
IF IT IS INCLUDED IN THE ENTRY POINT.
=#
using ..MyEntryPoint: model
processdataB(model)
println("We are evaluating in ModuleB.")
I know there is/must be a way to simply import/use the method definitions tied to the Model type as they would be identified in the MyEntryPoint.jl in both Modules A and B. But I don’t like this solution because I lose the modularity of my Modules by enforcing that Modules A & B only make sense so long as they are included in MyEntryPoint. At the moment my workaround is FromFile.jl because methods imported using this macro all have the same signature, which keeps my Modules “modular”. I was prompted to look for a Julia native solution however because the Julia LSP doesn’t work with FromFile imports, and sufficiently large projects get difficult to keep all in my head. Thanks for taking time to figure out what I mean here, hope I was able to be clear
It’s not clear to me what your actual complaint/question is, but one problem is clear: you include multiple copies of the same files (that happen to contain modules).
What you almost surely want to do instead, especially as a beginner, is to only use include at the top-most level.
Similar thread:
NB: corrected the category from “internals” to “general usage”.
My question/issue is very much related to your response to write the snippets and include them at the top level: How do I accomplish writing code, like the dummy example i mocked up, that maintains a sensible module structure and doesn’t utilize include “snippets”?
I am new to this forum and newish to Julia, but not to programming; I don’t want to write scripts, I want to write modular code. I see what you’re saying though…
If i didn’t wrap the code, in the various files I am including, in modules and moved their dependency up to the “top level” everything should run as expected. My issue with this solution is that avoiding wrapping the code in a module would mean of course that the included code is no longer contained in a module. This would mean that the code would only have any real meaning in the context of the “top level” where it is included. I generally am against requiring requiring anybody that reads the code to have to understand files A and B to understand how file C really works. Furthermore, the whole module “toolkit” would no longer be available for these files (importing, exporting, extending, namespace control). So I do understand that essentially writing a number of snippets would be possible, but I structurally I don’t think it would be the right choice.
I initially was just using Revise to avoid having to restart the REPL every time I changed source code. I was unaware of the includet method. Looking at the REPL help of the function I’m a little unsure if it solves the problem I’m trying to address. Do you mean to say that using includet would dynamically recognize that the signature of a module I am referencing has changed if I switch from evaluating in the namespace of MyEntryPoint.jl to that of ModuleA or ModuleB (given that said Module is a dependency of all three)?
That’s not what I tried to suggest. Have you seen the thread I linked, and the linked message specifically?
I’m suggesting you have a top-level file (which could also be a module, if you so desire), and that top-level file would be the only place where you use include. All the other files would then be modules, but none of them would include anything.
This design would prevent you from doing mistakes like having multiple identical module copies in various places in your module tree.
As an example, see my package FindMinimaxPolynomial: this is the top-level file, which only includes other modules:
The other files are always modules, but don’t use include.
I believe each of those statements follows an include in the example, so it has to preceded by a dot according to the docs: Modules · The Julia Language
Or at least that’s what I always understood, and I never had problems doing it this way.
Yeah thanks for the reference. I don’t think that would necessarily solve the problem though, because MyEntryPoint.jl, ModuleA.jl and ModuleB.jl all require the MyModel.jl dependency. I suppose you’re suggesting always pull the MyModel.jl dependency from the top level scope, but again I feel I lose flexibility here because Modules A and B would then only make sense in the context of the top level scope. If I were to write a further module in the future extending lets say the functionality of ModuleA.jl then I would run into problems because ModuleA’s dependencies don’t exist in ModuleA.
To reiterate, though, it’s almost certain that you do not want multiple copies of the same module in your code. Compare with #include in C/C++: it does the same thing as the include in Julia; but C/C++ headers usually have include guards or #pragma once, which prevents multiple copies of the same source from being included in the code.
Sorry If i’m not being clear. I’m referring to the situation that I tried to demonstrate at the beginning. That being the situation where three modules all have a common dependency, and I want the methods of all modules to be interoperable so that methods and objects constructed with the methods of one module can be handled by another module. Simultaneously I don’t want to achieve this by making the presence of any one of these modules necessary to use/import any of others.
That being said I don’t come from a C/C++ background (but python and java) so maybe this type of issue and how it’s handled in those languages has just been hidden away from me to some extent.
Ok yeah this was what I was wondering, thanks for the response. Is this not an issue? Why does Julia require an entirely new package to achieve this, I don’t remember similar structural requirements in other languages I’ve used?
