Loading from parent scope in package

Originally, none of these modules were packages… just modules. That’s where I originally discovered this problem. I’m right in the middle of making them into packages and due to the problem I had with pure modules, I was trying to design my package structure to avoid the problem. So you’re saying that I should be able to do using MatTypes in various modules/packages and it will be the same types everywhere?

I’m confused what the parent module of MatML is.

Since MatTypes is an independent package, just do the following in every module you want to use these types. There is no need to reference the parent module.

using MatTypes: Crystal, DataSet

A slightly different style is to the following.

using MatTypes as MT

foo(x::MT.Crystal) = x
1 Like


1 Like

I’m almost home on this issue. I’ve created four packages that all interact with each other and I think I’ve correctly specified all of the dependencies. The packages are:

But when I try and add one of these packages:

Using Pkg
Pkg.add(url = "https://github.com/lancejnelson/CrystalUtils.jl.git")

It claims to have trouble installing VASP. It only works if I install them in a certain order which doesn’t seem right because each of them have the right dependency information.

Note that you can specify multiple packages to Pkg.add at once. That should eliminate your order issues.

julia> using Pkg

julia> Pkg.activate(; temp=true)
  Activating new project at `/tmp/jl_2AyYu8`

julia> Pkg.add(
           ]) do pkgname
               PackageSpec(url = joinpath(                                              
                   pkgname * ".jl"

You may want to consider publishing a meta-package with a Manifest.toml that includes all your packages. The Manifest.toml includes the URLs of all of your packages. If someone installs the meta-package, the Manifest.toml will help the resolver discover the URLs of all of them.

julia> Base.active_project() |>
           x->replace(x,"Project" => "Manifest") |>
           x->read(x, String) |>

deps = ["LinearAlgebra", "MatTypes", "StaticArrays", "StatsBase", "VASP"]
git-tree-sha1 = "786a8c27a509440461bc1d1b3ba1a0a0c3291a70"
repo-rev = "main"
repo-url = "https://github.com/lancejnelson/CrystalUtils.jl"
uuid = "03766455-12b7-4a6c-9499-86e52762f3ab"
version = "0.1.0"

deps = ["CrystalUtils", "Distributions", "LinearAlgebra", "MatTypes", "StaticArrays"]
git-tree-sha1 = "efb4fd2f705998dec1f6113270d681cd5bc64f56"
repo-rev = "main"
repo-url = "https://github.com/lancejnelson/MatML.jl"
uuid = "272dadef-06aa-45c4-9a1d-b36e12ce8ea5"
version = "0.1.0"

deps = ["LinearAlgebra", "StaticArrays"]
git-tree-sha1 = "da38da5a0eb2d01a3fb17b0cf1566da52194d8e6"
repo-rev = "main"
repo-url = "https://github.com/lancejnelson/MatTypes.jl"
uuid = "b127de0a-484a-4dc8-ac0a-1765876f1d62"
version = "0.1.0"

deps = ["DelimitedFiles", "LinearAlgebra", "MatTypes", "Printf", "StaticArrays"]
git-tree-sha1 = "668dbb354e64f606fc3d361a848c21ce8287ac1f"
repo-rev = "main"
repo-url = "https://github.com/lancejnelson/VASP.jl"
uuid = "3d64381f-12ea-4605-8d7a-6644de55dbc1"
version = "0.1.0"

Something else I wanted to mention is that it is possible to have multiple packages in the same git repository.


I hope you’ll forgive me for saying this is a bit of a letdown. I thought the main advantage to packages is that all of the dependencies are handled automatically. Come to find out… if the packages are unregistered, you have to manage those dependencies manually. :frowning:

This was really helpful actually. Thank you. I was “including” the module files in the main file and then using them (with a dot) and the result was different types for different modules.

Try this:

Clone this repository:

Then activate the environment contained within.

using LibGit2

using Pkg

using VASP

Surprisingly, the below also seems to work.

using LibGit2

using Pkg
using DemoForLanceJNelson.jl

If you want to run this as a new user, set the environment variable JULIA_DEPOT_PATH.

$ git clone https://github.com/mkitti/DemoForLanceJNelson.jl
$ JULIA_DEPOT_PATH=`mktemp -d` julia -e '
using Pkg

using VASP
1 Like

Thank you. I understand your workaround. Now, I’m left pondering which workflow will be less complicated, packages or just modules, considering that students will be collaborating with me on this project. Thanks for all your help.

This last solution will download all of the packages into JULIA_DEPOT_PATH so that they can be modified and then pushed back to github?

This statement doesn’t seem to work for me:

JULIA_DEPOT_PATH=`mktemp -d` julia -e '

What I’m trying to do here is simulate what a new student would experience if they just downloaded Julia.

Yes, working with an ecosystem of unregistered packages is indeed less straightforward than with registered packages.

My personal advice would be to start with only one package. It might not even be necessary to introduce any further structure in it (by which I mean that splitting the sources in several aptly-named files or even subfolders might be enough to organize things, without having to introduce submodules).

It could look like:

+ src/
  + MainPackage.jl
  + analyze.jl
  + types.jl
  + vasp.jl
+ Project.toml

With the following contents:

# src/MainPackage.jl

module MainPackage
  # 3rd party dependencies (declared in `Project.toml`)
  using LinearAlgebra
  using StaticArrays

  # the API you want to export for final users
  export build_crystal, sample

  # all package sources organized in different files (possibly directories)
  # as you see fit
# src/types.jl

struct Crystal
  # ...

struct DataSet
  # ...
# src/vasp.jl

function build_crystal(file::String)
  # ...
  return Crystal(...)

Now if you really want your code to be structured in isolated namespaces, my personal opinion would still be to structure it (at least at first) in only one package, with a main module and several submodules.


Thank you. Great advice and thoughtful insight. Side note: My student doesn’t speak Julia yet so he decided he would replicate my code in Python. He was stunned at how slow Python was in comparison to my Julia version.


Since you’re doing this for a classroom, you may consider setting up a local registry, this is more work for you up-front but makes things very simple for your students.


Part of the original prompt was this would be a “pretty big library”, which is why we suggested the multiple package route.

I’m unclear if this project will be so that it requires a separate registry, but that is an option. One example for an academic group is @tim.holy 's lab’s registry: