# Importing packages by directory

**URL:** <https://discourse.julialang.org/t/importing-packages-by-directory/43140>\
**Category:** General Usage\
**Created:** [July 15, 2020, 5:59pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140 "2020-07-15T17:59:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 15, 2020, 5:59pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/1 "2020-07-15T17:59:57Z")

</div>

### Question

Is there a way to perform a “proper” import using a local path as an argument?:

```julia-auto
import CMDimData
import_by_path joinpath("/path/to/CMDimData/subpkgs/EasyPlotMPL")

```

Otherwise, does someone have a better suggestion given the problem I have below?

### Background/Premise

I have a package providing glue code for a bunch of backends:  
(not important, but: [GitHub - ma-laforge/CMDimData.jl: Parametric analysis/visualization +continuous-f(x) interpolation](https://github.com/ma-laforge/CMDimData.jl))

If a calling module wants to load the glue code for PyPlot (Matplotlib), it simply has to “import” the package in `subpkgs/EasyPlotMPL`.

This way, I can limit the direct dependencies of my module, and avoid having julia add/install packages for all the supported plotting backends.

### Hack Solution 1: `LOAD_PATH`

When I `import CMDimData`, I could register the `subpkgs` directory with Julia’s `LOAD_PATH` variable:

```julia-auto
push(LOAD_PATH, joinpath(@ __DIR__ , "../subpkgs"))

```

That could work: it would add a new search path for packages to Julia’s session. The calling module could then simply do:

```julia-auto
import CMDimData
import EasyPlotMPL #Now visible in the search path

```

The problem is I am changing Julia’s state at a global level, unbeknownst to the user, so I don’t really like this idea.

### Hack Solution 2: `include` macro

What I am doing now is hacking my way in with `include`:

```julia-auto
macro includepkg(pkgname::Symbol)
	path = realpath(joinpath(rootpath,"subpkgs/$pkgname/src/$pkgname.jl"))
	m = quote
		if !@isdefined $pkgname
			include($path)
			$pkgname. __init__ ()
		end
	end
	return esc(m) #esc: Evaluate in calling module
end

```

Note that I have to manually ` __init__ ()` the module, because only “proper” packages get initialized automatically.

Now, the caller must instead run:

```julia-auto
import CMDimData
CMDimData.@includepkg EasyPlotMPL

```

What I don’t like here is that code gets copied into the caller’s module. That means that each caller module that wants to use my glue code must compile a new instance of it.

On a true package `import`, packages get pre-compiled somewhere (where??), and each subsequent call to `import` returns a reference to that same “compiled” instance.

So re-compiling the glue code is probably bad because technically, the types declared in these modules would be distinct from each other. Consider this simple example:

```julia-auto
module MA
   import CMDimData
   CMDimData.@includepkg EasyPlotMPL
end
module MB
   import CMDimData
   CMDimData.@includepkg EasyPlotMPL
end

```

Now technically, `MA.EasyPlotMPL.SomeType != MB.EasyPlotMPL.SomeType`. This might eventually cause unexpected behaviour.

### Any suggestions?

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [July 15, 2020, 6:12pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/2 "2020-07-15T18:12:10Z")

</div>

It seems like [https://github.com/JuliaPackaging/Requires.jl](https://github.com/JuliaPackaging/Requires.jl) would solve the problem. The glue-code example in the readme seems to be exactly what you’re looking for.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [July 15, 2020, 7:57pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/3 "2020-07-15T19:57:04Z")

</div>

I have a package that makes available a command-line script for solving an optimization problem and `Requires.jl` really helped. Using it I can isolate the code that uses CPLEX/Gurobi/GLPK/… (some of which are commercial solvers that the users are not able to get access) and only load it if the user loads the respective package. So no specific solver package needs to go in the dependencies, and the user can use any solver package I support effortlessly just loading it first.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 2:10am UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/4 "2020-07-16T02:10:20Z")

</div>

# Requires.jl

Well, I looked into Requires.jl, but I essentially get the same issues (though what triggers the package loading changes slightly).

I tried adding the `JSON/Colors` example found in `Requires.jl.README.md` to my CMDimData.jl package:

```julia-auto
module CMDimData
   using Requires #Must also be in CMDimData.Project.toml [deps]
   function __init__ ()
      @require HDF5="f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f" @eval using Cairo
   end
[...]
end

```

(I’m using packages that are easily found so you to try it for yourself - they don’t make sense in my context, though).

The problem is that, Julia (1.3.1) doesn’t really like you to `import` “new” packages that aren’t registered in your Project.toml file (even if you load it dynamically with `@require`):

```julia-auto
julia> using CMDimData

julia> using HDF5
┌ Warning: Package CMDimData does not have Cairo in its dependencies:
│ - If you have CMDimData checked out for development and have
│ added Cairo as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with CMDimData
└ Loading Cairo into CMDimData from project dependency, future warnings for CMDimData are suppressed.

```

And that seems reasonable, I suppose. I could see this might cause issues with Julia having to recompile code if a newly imported module tried to overwrite a function or something.

# Circumventing this issue

This is why I decided to find a means to only load my glue code _ **in the calling module** _. By doing this, you just need to make sure the caller’s environment registers the dependent module (`Cairo.jl`, in this example) in its own Project.toml file.

That’s easy enough to do. In any case, if she forgets, the user will get a very useful error message informing her of this:

```julia-auto
ERROR: ArgumentError: Package Cairo not found in current path:
- Run `import Pkg; Pkg.add("Cairo")` to install the Cairo package.

```

### But the issue still remains

Essentially, I can only `include` the glue code in the caller module. That way, my glue code can properly `import Cairo`, because the caller will have added it to her environment’s `Project.toml` file.

☹ And that means my code has to compiled in each module that tries to use the glue code.

🤔 Well, unless I can find a way to properly `import` my glue code module, instead of simply `include` ing it.

### Unless I am missing something??

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [July 16, 2020, 3:24am UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/5 "2020-07-16T03:24:50Z")

</div>

> [@MA\_Laforge](#):
>
> The problem is that, Julia (1.3.1) doesn’t really like you to `import` “new” packages that aren’t registered in your Project.toml file (even if you load it dynamically with `@require` ):

Just swap `using Cairo` with `using .Cairo` (note the dot) to get rid of that warning. Its mentioned at the end of the first section of the Requires readme.

Edit: ah sorry just realized in your example the package you want to `using` is not the same one being `@required`. I’m not sure this will work in that case. But you can (and probably should) just `@require` all the packages you actually need, you can nest them, then it should be fine.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 16, 2020, 6:24pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/6 "2020-07-16T18:24:30Z")

</div>

> [@marius311](#):
>
> But you can (and probably should) just `@require` all the packages you actually need, you can nest them, then it should be fine.

I definitely agree wit the part where you said “all the packages”.

That’s basically my problem: I need for Julia to recognize `EasyPlotMPL` _ **as a package** _ irrespective of whether `@require` is used or not.

### Using my minimal-dependency package

The idea is that I want people to be able to add my core package:

```julia-auto
]add https://github.com/ma-laforge/CMDimData.jl

```

which doesn’t have plotting backends as dependencies, thus avoiding a forced download/install/compile of _ **all** _ of their respective packages. I want users to install the only the backends they choose.

### Too many .git repositories

So now, users should import the glue package corresponding to their desired plotting backend - but I don’t want each one of those glue packages to be in their own separate repository:

```julia-auto
]add https://github.com/ma-laforge/EasyPlotMPL.jl

```

⚠ Repository overload: too many mini-glue packages!

### v1.5 support for `subdir`

If I stick with Julia 1.5+, I could start using the multi-package repo capbability:

```julia-auto
]add https://github.com/ma-laforge/CMDimData.jl:subpkgs/EasyPlotMPL

```

(See [add subdir functionality by KristofferC · Pull Request #1422 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/pull/1422), and other PRs with “subdir”).

However, even that is a bit heavy for my purposes. I don’t really want the user to have to `]add` my glue packages if they don’t have to.

### Adding to `LOAD_PATH` instead

As I mentioned earlier, CMDimData could just modify Julia’s global `LOAD_PATH` when it runs. That basically registers my package’s `subpkgs` directory as a sort of additional package “library”:

```julia-auto
push!(LOAD_PATH, joinpath(@ __DIR__ , "../subpkgs"))

```

⚠ But I don’t like this, as it modifies Julia’s global state in a unsettling way.

### Import by path

So the simplest solution to me would be to import a package by specifying the package path.

I suppose I could do the following in an `@require` statement:

```julia-auto
push!(LOAD_PATH, joinpath(@ __DIR__ , "../subpkgs"))
import EasyPlotMPL
pop!(LOAD_PATH)

```

⚠ But despite that global state is restored, it still seems a bit hacky for a well designed language like Julia.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [July 16, 2020, 7:02pm UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/7 "2020-07-16T19:02:55Z")

</div>

I am not sure if I understand your problem. But I have never used `Requires.jl` to import some module only if some other module is available. What I do is to check if some module is available, and then, if it is, define methods that make use of the module inside them. If you need a module you have two options: (1) have it in your dependencies; (2) instead of loading some module only if another module is loaded, do something (define functions, types, etc…) only if _both_ are loaded. I think these are the most clean approaches to this problem.

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [July 17, 2020, 1:35am UTC](https://discourse.julialang.org/t/importing-packages-by-directory/43140/8 "2020-07-17T01:35:31Z")

</div>

> [@Henrique\_Becker](#):
>
> if it is, define methods that make use of the module inside them

What you are effectively suggesting here is that I “include” my glue code somehow when I detect my dependent module is present.

That is essentially the solution I am using right now:

> [@MA\_Laforge](#):
>
> ### Hack Solution 2: `include` macro

But like I said, that means my glue code needs to be compiled multiple times for each caller module that uses it, so I find it somewhat suboptimal.