# Import types "above" a module with ..: for consistency and brevity?
**URL:** https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754
**Category:** General Usage
**Tags:** module
**Created:** [March 30, 2020, 6:17pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754 "2020-03-30T18:17:45Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)
#### Post date: [March 30, 2020, 6:17pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/1 "2020-03-30T18:17:45Z")
You can import stuff from “above” with `..Thing`, such as:
```julia
module Mod
# Define some common types.
struct T1; field::Int64; end
struct T2; field::Float64; end
# Define a sub module. Maybe this would even live in a different file. It assumes that
# T1 and T2 are available in its parent, but it doesn't need to know who its parent is.
module Sub
import ..T1, ..T2
f(x::T1) = T2(2 * x.field)
end
end
```
That works fine. But once you start importing a bunch of stuff, it would be really nice if you could change this:
```julia
import ..T1, ..T2
```
to
```julia
import ..: T1, T2
```
which would be consistent with importing from a module with a given name, like so:
```julia
import ...Mod1: T1, T2
```
The problem with this final line is that it requires `Sub` to know who its parent is (`Mod1`). If `Sub` is intended to be used by a lot of different utilities, then it naturally can’t know who its parent is.
So I think `..:` is nice, clean, and consistent. Does this seem like a good idea, or am I missing something?
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [April 1, 2020, 8:30am UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/2 "2020-04-01T08:30:47Z")
I get the consistency argument, but I’m not sure I understand this one:
> [@tuckermcclure](#):
>
> The problem with this final line is that it requires `Sub` to know who its parent is ( `Mod1` ). If `Sub` is intended to be used by a lot of different utilities, then it naturally can’t know who its parent is.
I might have misunderstood something, but I think that in any case, `Sub` has to know a lot of things about the module from which it imports things: at least it has to be sure that it defines types `T1` and `T2`, and these types have to follow an specific interface. And how would `Sub` know all that, but not the name of `Mod1` itself?
In other words, I think that putting a line such as `import B: foo` in module `A` defines some form of dependency of module `A` to module `B` (and if they were packages, this would be enforced by the packaging system). But how can `A` depend on something it does not know?
In short, I agree with you about the syntax for the sake of consistency, but I would be interested in a (maybe less contrived) use case to motivate it in practice.
---
### Author:  [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)
#### Post date: [April 1, 2020, 7:13pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/3 "2020-04-01T19:13:48Z")
Hi @ffevotte. Good question. Here’s a simple example that captures the essence of a real problem we’re having at my company: Suppose we have a MyTypes module that exports a bunch of common types. Those types are on the interface for functions in the Subsystem module. Now we want to build _two_ separate systems that use Subsystem. Here’s an example. Suppose the modules are included from separate files (they’re just written this way so there’s a single working example file).
Here’s a system that needs to use the functionality of Subsystem directly.
```julia
module SystemA
# Included from MyTypes.jl.
module MyTypes
export Type1, Type2
struct Type1; field::Int64; end
struct Type2; field::Float64; end
end
using .MyTypes
# Included from Subsystem.jl.
module Subsystem
export f
import ..MyTypes: Type1, Type2 # This line breaks modularity!
# import ..SystemA: Type1, Type2
f(a::Type1, b::Type2) = Type2(a.field + b.field)
end
using .Subsystem
# Now for whatever this module was supposed to do.
x = f(Type1(1), Type2(2.0))
end
```
Here’s a different system, one that needs to use the functionality of Subsystem indirectly. [Edit: This is intended to be totally separate from SystemA, like a different program.]
```julia
module SystemB
# Included from MyTypes.jl.
module MyTypes
export Type1, Type2
struct Type1; field::Int64; end
struct Type2; field::Float64; end
end
using .MyTypes
# Included from SomeOtherSubsystem.jl.
module SomeOtherSubsystem
export g
import ..MyTypes: Type1, Type2
# import ..SystemA: Type1, Type2
# Included from Subsystem.jl.
module Subsystem
export f
import ..MyTypes: Type1, Type2 # This line breaks modularity!
# import ..SystemA: Type1, Type2
f(a::Type1, b::Type2) = Type2(a.field + b.field)
end
using .Subsystem
g(a, b) = f(Type1(b.field), Type2(a.field))
end
using .SomeOtherSubsystem
# Now for whatever this module was supposed to do.
x = g(Type1(1), Type2(2.0))
end
SystemA.x
SystemB.x
```
This doesn’t work. In order for Subsystem to work in the second case, its `import` statement would need an extra period, because it’s deeper here. Hence no single Subsystem.jl file works in both cases.
If the `import ..: Type1, Type2` syntax worked, then this type of modularity would work.
This is all part of a bigger picture that looks like this: My company needs all of this code in a single repo. We want modularity. Solutions:
- If we break things into modules like this, modules actually can’t be used in multiple contexts, so this doesn’t work.
- If we break things into local packages and `]dev` them, then there are new problems with modularity [["Unsatisfiable requirements" when local packages use local packages (fixed in v1.4)](https://discourse.julialang.org/t/unsatisfiable-requirements-when-local-packages-use-local-packages/36807)].
- If we create a local registry for our own package, that’s also problematic [ibid].
- This only working solution we’ve found is to avoid making modules and packages and just make a bunch of .jl files with functions and types in them, and then include all those files in one big module (possibly Main). This obviously has its own problems, like name conflicts and the fact that nothing can be precompiled, etc., so the startup time is now really slow.
That’s all just context to answer your question, François. This thread should still be about whether or not `..:` is useful and should be prioritized.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [April 1, 2020, 8:01pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/4 "2020-04-01T20:01:42Z")
> [@tuckermcclure](#):
>
> That’s all just context to answer your question, François. This thread should still be about whether or not `..:` is useful and should be prioritized.
Well, I’m sorry to derail this further but, on the other hand, I feel like this proposal is more likely to be prioritized if people understand why it is so useful to you.
One problem I see with the way you do things, is that the same code is `include`d in several different contexts, creating types (`Type1` and `Type2`) and functions (`f`) that should be the same, but actually are not. To be clear: in what you describe above, `SystemA.MyTypes.Type1` and `SystemB.MyTypes.Type1` are two different types that happen to have the same structure. I would say that this could cause unexpected behavior later on, especially if these types are supposed to be part of a common interface.
Here is how I would structure things:
```julia
# Only one MyTypes module
module MyTypes
export Type1
struct Type1; field::Int; end
end
# Only one SubSystem module
module SubSystem
using ..MyTypes
export foo
foo(x::Type1) = x.field
end
# Module SystemA uses the top-level MyType module
module SystemA
using ..MyTypes
using ..SubSystem
test() = foo(Type1(1))
end
module SystemB
# This makes MyTypes and SubSystem available in the current namespace
# ...and therefore also in submodules, using the .. syntax
import ..MyTypes
import ..SubSystem
# Submodules of SystemB can also use the same top-level MyType module
# without having to know about their depth in the module hierarchy
module SomeOtherSubSystem
using ..MyTypes
using ..SubSystem
export bar
bar(x::Type1) = 2*foo(x)
end
using ..MyTypes
using .SomeOtherSubSystem
test() = bar(Type1(1))
end
SystemA.test() # --> 1
SystemB.test() # --> 2
```
Now, I would even go one step further and imagine that everything is in its own package. In a real application, the code base should reside in a package, that defines a top-level module of the same name. When the package environment is activated, this top-level module can be referred to in absolute syntax (without leading dots) from anywhere. Therefore, submodules can also be accessed to using an absolute path starting from the top-level module.
The following example is designed to run in the REPL, where everything is defined in the `Main` module. If you want to transpose it to a project, simply replace every occurrence of `Main` with the name of the top-level module:
```julia
# Only one MyTypes module
module MyTypes
export Type1
struct Type1; field::Int; end
end
module SubSystem
# Refer to MyTypes using an absolute path starting with the top-level module
# (replace Main with the name of your top-level module, i.e. the name of the package)
using Main.MyTypes
export foo
foo(x::Type1) = x.field
end
module SystemA
using Main.MyTypes
using Main.SubSystem
test() = foo(Type1(1))
end
module SystemB
module SomeOtherSubSystem
# No need to know how deep we are, since we now use absolute module paths
using Main.MyTypes
using Main.SubSystem
export bar
bar(x::Type1) = 2*foo(x)
end
using Main.MyTypes
using .SomeOtherSubSystem
test() = bar(Type1(1))
end
SystemA.test() # --> 1
SystemB.test() # --> 2
```
Again, this has almost nothing to do with your proposed syntax and I’m sorry for the thread derailment, but would such a structure make sense in your project?
---
### Author:  [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)
#### Post date: [April 2, 2020, 8:46pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/5 "2020-04-02T20:46:47Z")
Wow, @ffevotte, thank you for that amazingly detailed reply!
Let me clarify two things:
1. `SystemA` and `SystemB` are intended to be totally separate programs. I understand that the two instantiations of `MyType`s are different there. That’s intended. The two systems just happen to be next to each other in the example above. Maybe I should have used two separate code blocks. I’ve edited my post to try to make that a little clearer.
2. Your proposal is essentially to put all possibly-common modules at the top level, and then import their functionality for each level going down. What I don’t like about this is the lack of modularity. Somehow, the top level needs to know what to import, so it needs to know that something inside of `SystemB` uses `Subsystem`.
3. Also, instead of having types/function “cascade” down each level, you suggested importing them directly from the named top level, which I like. However, the only way things can imported from the “top level” is if they know the name of the top level. If SystemA and SystemB are in two separate packages, then this still doesn’t work.
4. I would like to use packages. What’s great about this of course is that any module can say, e.g., `using MyTypes`, and everyone gets the same types. There’s no necessary “routing” of types down a level, and down a lever, etc., to the intended user. _Nor_ does the top-level module need to know about everything that everyone might possibly use and load it at the top level. It’s sort of automatically loaded at some kind of top level. BUT! I need everything in the same repo. It’s the only way my code base makes sense (there are a bunch of reasons for this that have nothing to do with Julia). That leaves me trying to use unregistered local packages via `]dev path/to/LocalPackage`. The problem with this approach is that `dev`ing local packages doesn’t really work as a sustainable workflow (see my “Unsatisfiable requirements” link above).
Maybe the best way forward is some kind of first-class support for local, unregistered packages. I’ll make a new post about that, and we’ll see where it goes.
In the meantime, I’m going forward with _both_ local modules (including them, and cascading their types/functions down and down and down) and local, unregistered packages (via ‘dev path/to/Package’) to see which is the least painful. Though the least painful thing so far is to simply do away with the idea of little modules and just make a bunch of functions and types in separate files, and then include them in either the module for SystemA or SystemB. This is more like mixins, and it’s hard too because the order of the includes has to be just so.
---
### Author:  [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)
#### Post date: [April 3, 2020, 1:00am UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/6 "2020-04-03T01:00:56Z")
Here’s a link to the new post about first-class support for local, unregistered packages.
> [@Could we make first-class support for packages that are "just files" and not repositories?](https://discourse.julialang.org/t/could-we-make-first-class-support-for-local-unregistered-packages/36931):
>
> I keep circling around the same issue from a variety of different vantage points, and I think that having first-class support for local, unregistered packages (just local files, not repositories) would solve a lot of problems that I see in structuring Julia programs. We have a repo that has our aircraft simulation in it. The current package manager does a great job of managing our dependence on external packages, like HDF5 and YAML. I love it. We also have local modules for reusable functional…
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [April 3, 2020, 5:45am UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/7 "2020-04-03T05:45:01Z")
> [@tuckermcclure](#):
>
> I would like to use packages. […] BUT! I need everything in the same repo.
>
>
> Currently, a single Git repository can only contain one registered package, whic…h must live in the top folder of the Git repository.
>
> I would like to request the ability to store more than one registered package in the same Git repository.
>
> For example, I might have two packages \`Foo\` and \`Bar\` that I store in a single GitHub repository \`DilumAluthge/FooBar.git\`. In this case, the contents of \`https://github.com/DilumAluthge/FooBar.git\` might look something like this:
> \`\`\`
> FooBar
> ├── Bar
> │ ├── Project.toml
> │ ├── src
> │ │ └── Bar.jl
> │ └── test
> │ └── runtests.jl
> └── Foo
> ├── Project.toml
> ├── src
> │ └── Foo.jl
> └── test
> └── runtests.jl
>
> 6 directories, 6 files
> \`\`\`
>
> For example, the contents of \`https://github.com/DilumAluthge/FooBar.git\` might instead look something like this:
> \`\`\`
> FooBar
> ├── myfolder
> │ └── Foo
> │ ├── Project.toml
> │ ├── src
> │ │ └── Foo.jl
> │ └── test
> │ └── runtests.jl
> └── otherfolder
> └── subfolder
> └── Bar
> ├── Project.toml
> ├── src
> │ └── Bar.jl
> └── test
> └── runtests.jl
>
> 9 directories, 6 files
> \`\`\`
>
> \---
>
> \## Possibly related issues/pull requests:
>
> \- #691: "Allow packages to live in a directory of repo"
> - This issue discusses relaxing the requirement that a package is located in the top folder of the Git repository. However, the issue doesn't discuss whether or not you would be allowed to have multiple registered packages in the same Git repository.
>
> \- #1233: Proposal for "sub-projects"
> - This issue discusses sub-projects. However, a sub-project is an "incremental addition of packages to an already existing main-project". This is different from what I am proposing here, in which there is no "main project" from which the other projects derive.
---
### Author:  [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)
#### Post date: [April 3, 2020, 7:02pm UTC](https://discourse.julialang.org/t/import-types-above-a-module-with-for-consistency-and-brevity/36754/8 "2020-04-03T19:02:01Z")
That’s cool. Thanks @Tamas_Papp. Still a work in progress, I see, but it’s good to know about.
I’ll have to think about how this would possibly work in the context of private repos. E.g., our sim’s going to run in Docker without Git credentials, so it won’t be able to hit our private repo, but as long as we already have it checked out somewhere, then… maybe something like this would work?