[ANN] ThinModules.jl - lightweight module dependency resolver

Package Announcements
1

Hello!

This is a small package ThinModules.jl, which tries to avoid the manual maintenance of module orders. Please look at the example:

using ThinModules 

@thinmod module MyModule
# all submodules inside can have free orders. 
# ThinModules would do the re-order.
    module B
        module B1
            import ...C.C2: x
            @show x
        end
        module B2
            y = 2
        end               
    end
    module C
        module C1
            import ...B.B2: y
            @show y
        end
        module C2
           x = 1
        end  
    end
end

Julia modules have to be written done in the right order when some depend on others. And include is just copy-and-paste, so when you have multiple files in your source tree, you also need to remember what’s the correct order of the includes, which could have been avoided easily and has already been avoided in almost every modern language.

There are already some attempts, e.g. FromFile.jl by @Roger-luo and @patrick-kidger, which provides a Node.js-similar module management, where every file is a standalone implicit module and module dependency is just file dependency. That is very natural to me, and I love that a lot, but that seems not the favorite of many in the community. I guess probably because that is too different from the so far standard Julia codes.

So I made ThinModules.jl, which focuses on module-dependency resolving and tries to keep things looking as similar to standard Julia as possible, without an assumption of file-module mapping. When you use include without module, nothing happens. When you use module no matter they are in the same file or included from different files, they can be put in arbitrary order, and it works as long as there are no cyclic dependencies. Nothing else.

Look forward to your feedback!

P.S. you would probably find it look like Rust’s module system, if you always use module mymod; include("./mymod.jl"); end or module mymod; include("./mymod/mod.jl"); end and avoid bare include, which is very like what Rust’s mod mymod; does.

8 Likes
2

Is it possible to do the same with include? I mean, in a situation, when we have

# file funcs.jl

function(x::A) = nothing

# file structs.jl

struct A end

to make a macro, which can pull structure definition to the top of the code, so

@multiinclude begin
  include("defs.jl")
  include("structs.jl")
end

generated a proper order of structures and functions?

3

Actually what you describe is not “do the same with include”, it is about the granularity of re-ordering and you want the single declarations within a block to be the granularity, while in ThinModules.jl the granularity of re-ordering is the module. include itself does not matter.

This could be possible, but it should be harder, with such flexibility than re-oreder modules. Taking module as the granularity, there is hard to be something wrong as long as the module top-level and __init__ don’t have codes with side-effects. But if we can re-order every declaration, I don’t know - there are too many possibilities. On the other hand, analyzing the dependency relationship between the declarations could also be harder than modules, where you mainly need to look at using and import.

I am not sure how hard is the above difficulties now, but it would indeed reduce more manual maintenance of codes order if it works.

1 Like
4

I would note that when reading Julia code it is a recurring issue to find the location of referred types and functions. Auto-reordering by a preprocessor may make this even harder.

5

You are right. If no re-ordering, your searching space for a symbol in the same module is halved - you only need to look at the part before the reference. But, to really solve this problem, I think we have to rely on intelligence. And we may need a more suitable code structure, e.g. smaller modules. If every module does not cross files, such re-ordering should be fine.

This is also one reason why I made ThinModules.jl, I do prefer smaller modules, with explicit mutual dependences, and from import X: y, I can easily know where each symbol is from even when I don’t have LSP installed at hand. Though there would be some redundancy codes to write.

1 Like
6

Yes, that kind of direct importing everything non-trivial is on my list of habits to build. That may also help the tools to find the correct locations. Language server fails to even start on some of my projects at the time, but I do not complain, static analysis of Julia is hard (if not impossible).

I really like the minimalism of ThinModules.jl, and I think its goal of looking standard-like may make it attractive for others too. Great work!

1 Like