I followed this thread with interest, but one thing that seems to stay a bit out of focus is the differences in philosophy/design between Julia and (say) Python when it comes to packaging and reuse. This influences the way modules work in Julia and perhaps (partly) causes the perceived mismatches with other languages.
In Julia the package is the main unit of reuse and distribution, as it contains all necessary details on dependencies for a set of source files. Plus packages have explicit support for easy deployment via e.g. git repositories and management using Pkg.
Even though Julia packages are usually structured in modules, using a Julia module by itself is not really a supported goal (you need to go through a package instead) and it shows in the way they are implemented. For example, a module might be split over multiple source files and there is no general way to handle such a module by itself except by using the package the module is contained in.
In contrast in Python any .py file comprises a module. And as such it (implicitly) list its dependencies through the relevant import statements contained in it. So a Python module can usually be reused without too much trouble and is much more of a standalone unit than a Julia module. For example, this helps in writing test code for individual modules as you simply import the module to test in a driver script and don’t need to worry about the package the module is part of. The same goes for a Python package, which is not much more than a directory of files, leading to a set of nicely namespaced modules.
But support for module/package deployment is not as tightly integrated in the Python ecosystem, even though there are things like PyPI, setuptools and pip to handle packaging and deployment. All in all, handling dependencies is quite a bit more involved and requires more manual labour in Python than in Julia.
At least, this is my 10,000 feet view, based on limited experience with Julia (but much more with Python).
This is not true. The general non-package way to use a module defined in one or more files is to do include("file-with-mymodule.jl"); import .MyModule, where "file-with-mymodule.jl" is the file that contains module MyModule ... (and which possibly includes other files).
But it seems you have no guarantee in general that the included module file will pull in all necessary dependencies? As dependencies are quite often listed at the top-level package .jl file and not in the module file, with individual parts making up the package include()d below all the dependencies. I.e. structured like
module M
import ...
using ...
include("file1.jl")
include("file2.jl")
...
end
Including file1.jl as you suggest will most likely not work.
Yes it will, as long as file1.jl defines a module (i.e. it is of the form module ... end). Any import and using statements outside of the module statement (in the enclosing scope) don’t affect it.
Okay, but this nicely highlights the semantic differences of a “module” in Julia and one in Python. Even if file1.jl defines a module and lists all dependencies it can include() dozens of other source files which individually will not list their dependencies, and cannot easily be used by themselves. So indeed a Julia “module” will be usable outside of its enclosing package the level of use in general is coarser than in (say) Python where every source .pyfile is guaranteed to be importable by itself (and also each make up a module). I guess the main point I was trying to make was that in general in Julia a single .jl file does not equal a Julia module and is not expected to be easily usable by itself (yes, you can include() it, but that might require dependencies to be manually specified as well).
Yes, Julia is not Python—believe me, we know . But saying “files ≠ modules” in Julia is different from saying that “using a Julia module by itself” (not as a package) is not supported.
I think people here are generally open to ways that make it easier to import modules from files without having an explicit include, e.g. by import "Foo.jl" or some other syntax. But I think it’s unlikely that Julia will ever require each file to define a new namespace, ala Python.
This really captures where I find these threads lose me – the threads oscillate between two requests, one of which seems great and one of which seems not great:
Great: It should be easier to treat a file like a module that has a sealed namespace so that importing it makes all introduced names explicit.
Not Great: It should be impossible to break up a single module into multiple files.
I actually wasn’t aware you could still use a module separately like you showed. I mostly saw them as the building blocks for packages. In fact, most package repos I’ve looked at so far seem to be a single top-level module including many separate files.
I actually wasn’t advocating Julia moving towards the Python module scheme. Just trying to make an observation on the differences.
I am not sure if it is by design or accidental, but I find it really great that using and import currently resolve names via the stacked project environments, not the file system.
I understand that it is occasionally tedious for some use cases, but the benefits are also significant for writing modular code.
The issue is that if you have a moderately complicated dependency structure on internal modules, you can’t simply do include("module.jl") over and over again, because that introduces a newModule every time.
OP is (essentially) asking for a dependency resolver among different file-modules that are used internally in a package. This is currently clunky to do, requiring messing with LOAD_PATH or creating a very complicated directory structure.
This is a perfectly reasonable request and deserves developer consideration.
Please see earlier in this discussion (it’s long enough already). Basically, it’s a common design pattern to have sub-modules with a networked structure. People like submodules because it restricts the name-space they have to think about when writing code.
Registering and making a package takes a lot more time than just defining a module especially when the project is not mature enough. If there’s 10 components of a mono package that can be seperate to small modules it will take 30 days to register does this sounds reasonable to you? Not to say there are private commercial users who wants to write a mono package within one private repo
Liking submodules does not explain why would someone do include("a.jl") repeatedly. It is simply a wrong design decision/implementation, nothing else. No messing with LOAD_PATH is required either.
. But saying “files ≠ modules” in Julia is different from saying that “using a Julia module by itself” (not as a package) is not supported.