Implicitly loaded modules in the future?

Internals & Design
, ,
265

I could get behind @Skoffer’s idea regarding the keyword depends. I would favor the word depends over import, since import is already used to mean that you are importing objects from a module.

@patrick-kidger My apologies if the depends idea is essentially what you had in PatModules.jl… perhaps I didn’t look closely enough at the details of that package.

The advantages of depends, from my point of view:

  • I no longer have to manually order my include statements.
  • It does not enforce a file-module correspondence.
    • Thus, I can still use the flat namespace that I prefer.

However, the example that @Skoffer provided is not the best example, because you can define your methods of foo in whatever order you want. There’s no sense in which the A.jl and B.jl files depend on the utils.jl file. (function foo end is not more primal than any other method.) If we ignore modules that run imperative code, the only thing that matters is that types are defined before they are referred to.

Let me elaborate a bit more on why I don’t like turning every file into a module. Suppose I have a generic function foo, with various methods, that gets used throughout a package DemoPackage.jl. With Julia as it is now, foo “belongs” to the DemoPackage module. But if every file now has to be a module, I have to arbitrarily pick a submodule to “own” foo, and then I have to import foo from my arbitrarily chosen submodule every time that I want to extend it.

Below is a runnable example of what this would look like, using the current module system with nested modules. I think it would look basically the same if the nested modules in the example were separate files with the file-module correspondence enforced.

module DemoPackage
       
	module A
		struct S end
		foo(::S) = 1
	end
	
	module B
		import ..A: foo
		struct T end
		foo(::T) = 2
	end
	
	module C
		import ..A: foo
		struct U end
		foo(::U) = 3
	end
	
	import .A: S
	import .B: T
	import .C: foo, U
	export foo, S, T, U
	
end

julia> using .DemoPackage

julia> methods(foo)
# 3 methods for generic function "foo":
[1] foo(::S) in Main.DemoPackage.A at REPL[1]:8
[2] foo(::T) in Main.DemoPackage.B at REPL[1]:14
[3] foo(::U) in Main.DemoPackage.C at REPL[1]:20

Note how I arbitrarily picked module A to “own” foo. But in reality, none of the modules really owns foo—it is a generic function who’s full definition spans multiple modules. There are other arbitrary choices I had to make here:

  • In module C, I could have done import ..B: foo instead of import ..A: foo, because, now B “owns” foo just as much as A does!
  • In the third to last line, I could have done either import ..A: foo, import ..B: foo, or import ..C: foo. It’s an arbitrary choice, because they all refer to the same generic function.

To make matters worse, I’ve actually introduced an artificial code dependency that wouldn’t have existed otherwise. Look what happens if I transpose the definition of module B above the definition of module A:

DemoPackage module with order of A and B flipped 
module DemoPackage
       
    module B
		import ..A: foo
		struct T end
		foo(::T) = 2
	end	

    module A
		struct S end
		foo(::S) = 1
	end
	
	module C
		import ..A: foo
		struct U end
		foo(::U) = 3
	end
	
	import .A: S
	import .B: T
	import .C: foo, U
	export foo, S, T, U
	
end

If I run the new DemoPackage with the order of A and B flipped, I get ERROR: UndefVarError: A not defined. You might say, “That’s exactly how it’s supposed to work. The code dependency has been enforced.” But the point is, there should not be a code dependency here, because I can define methods in any order I want! I could have done the following, where I can flip the order of defining foo(::S), foo(::T), and foo(::U) to my heart’s content:

module DemoPackage

    struct U end
    struct T end
    struct S end

    foo(::U) = 3
    foo(::T) = 2
    foo(::S) = 1

    export foo, S, T, U

end

To summarize:

  • Using the same generic function across multiple submodules requires arbitrary import choices and introduces artificial and unnecessary code dependencies.
  • You can’t really encapsulate a generic function inside a module, because the generic function is defined by its global method table.
6 Likes