While developing a package, it is common to include a lot scripts into the main module (not Main), let’s say package FooPackage.jl
in the main module
module Foo
include("xxxx.jl")
export xxxxx
using xxxxx
# some code
end
There are two solutions I can come up with
Manually include and manage them in order
first arrange files manually, this is actually how some small julia package arrange their scripts
module Foo
# be careful with the order
include("core1.jl")
include("core2.jl")
include("core3.jl")
end
and in each file
# core1.jl
####
# code for foo1
####
export foo1
similar with other cores. However, this could cause conflicts if one script depends on other scripts
# core2
include("core3.jl")
use module in each script
this is similar with Python, the only difference is that you have to write an include each time.
# core
module Foo
include("core1.jl")
include("core2.jl")
include("core3.jl")
using Core1, Core2, Core3
export xxxx, xxx, xxx
# some codes
end
and in each core
# core1.jl
module Core1
export xxx, xxx
# some codes
end
the second solution can solve the problem, and it doesn’t matter for the order of includes. But isn’t those includes here is unnecessary(I mean if we encourage users to define a module in each file)? Is there any macros that and import local files like Python? Or what is preferred in Julia? (at the moment at least)
module Foo
include("file1.jl")
include("file2.jl")
end
as equivalent to
module Foo
# ... contents of file1.jl
# ... contents of file2.jl
end
It just helps you organize your code into files, either for because of conceptual reasons or just editing convenience. The files included should not be “scripts”, ie runtime code, just method, type, and occasionally constant definitions. For initialization, see __init__.
OTOH, using submodules allows you to compartmentalize your code further into separate namespaces. It is an orthogonal feature: you can have code in a single file, yet define submodules.
FWIW, I keep code up to 2-3000 LOC in a single file, using form feed (^L) to separate parts, but that’s a personal preference. In practice, I find submodules to be a rarely used (outside Base) middle ground for code that should have a separate namespace, yet not its own package.
I see, but this can be dangerous when files’ dependency is complex, the order of each file have effects on other files.
Let’s say, there is a team work on one package, if someone include a new file, then it will be totally mess…
Besides, module name in Julia will binding itself (cannot use the name inside).
just found a possible solution, I define a very simple macro
module Depends
export @depends
DEPENDENCIES = String[]
macro depends(names...)
for dep in names[1].args
path = join([dep, ".jl"])
if !(path in DEPENDENCIES)
include(path)
push!(DEPENDENCIES, path)
end
end
end
end # module
and then all includes are safe and all the names are only imported and there is no extra name pollution.
This is a possibly bad solution to a problem that I am not sure exists. First, you should use macros for source transformations, not for changing global state (you can’t rely on when it is called). Second, all uses of the macro (in possibly different packages) will share Depends.DEPENDS.
Supposedly one can figure out a linear ordering and include them that way.
I am not sure how. Refactoring code and moving it between files is a pretty standard operation, not inherently more prone to error than anything else.
One possible situation could be that while developing a package contains several files.
module PackageName
include("file1.jl")
include("file2.jl")
end
First, if I want to check the behaviour of some methods in file2.jl, I have to include all its dependencies and delete those includes after this check. Second, if include files directly in the main script, it cannot show the dependencies of a file, and each time when there is problem involved with dependencies, one have to check it manually. With similar macros/built-in behaviours If someone look into each file, its dependencies is clear and elegant.
Currently it does have this problem, that it cannot be used in multiple packages, therefore, one can simply copy it to one’s package for temporary. Or I will try to figure out a better way in the future.
I just want a temporary solution since this problem will be solved or to say this feature will be added according to issue #4600 after 1.0. (repeatedly using/import will not cause error and is safe)
Comparing to other languages: Python have the same behaviour for import, C/C++ has preprocessors like
thus I do not believe manually refactoring code and moving them is a good solution. Code dependencies should be resolved by the compiler itself. And each file itself should contain the description of its dependencies.
BTW, I am quite confused about why I cannot rely on when a macro is called? Is there any reference or demo? (shows that Julia’s macro command could be called in random order…)
It seems that you are trying to carry over habits from some other language to Julia. While the Julia language is extremely flexible and extensible, learning its idiomatic use is often a good thing to try first.
compartmentalize logically independent units of code into packages, with their own tests etc.
Also, my impression is that while #4600 is an issue (and will be resolved accordingly), no one is in a hurry because it affects a corner case for the commonly used workflows. If your workflow depends on this crucially, the best fix would be changing that.
It is not a big problem indeed, and as you mentioned I can use unit tests for test. But I cannot use it for examples, benchmarks, etc. It would be more convenient to have something that can load local modules safely.
And personally, I prefer to use just editor for development and I don’t like to use interactive development (but I tried this one, this package is awesome). Thanks for your suggestions I have been using coding with Julia since v0.3. And I have translated the whole Julia docs to Chinese with other collaborators (JuliaCN). And I believe I know most of its idiomatic.
To have a summary of my concern is just that I would think it would looks more elegant to have several lines in each file that indicates its dependency and could be more convenient that the programmers do not need to think about the include order, which is not a big problem indeed, but it will make your code looks better.
Basically I don’t think it’s about Julia’s idiomatic and other language. Resolve local code dependencies is something you won’t need crucially but will make your work easier.
And I was just making an comparison, it’s not about building other language’s into Julia because of Julia’s flexible feature, we can always make use of other language’s advantages. And I am not the only one concerning this (ImportMacro) You can check the Julia’s announcement. Julia itself brings a lot advantages from other languages.
Personally, I just do not think arrange files manually is elegant (if this will be a Julia’s feature, I will choose not to use it) and do not like to write 2-3000 lines in a single file. We can say that C/C++'s #include has similar effects with Julia’s include, but why using such preprocessors in C/C++ is a de facto standard. nobody wants to arrange them in order. And C/C++ are not the only language to do so.
And I bet after #4600 is solved, people will use using and import more than include. As the developers say, it is just because it is quite urgent to have 1.0 and this issue won’t break (if it will break, it will be fixed before 1.0).
Thanks, this is very helpful. But using a package to do this simple job is not what I was thinking (IMHO it is not elegant), but this is a temporary solution indeed before #4600 is solved. I think pasting that simple macro to one’s package will cause less side effect (like slow loading time) at the moment. And it can be replaced when local module import is done by Julia team.
