Suppose I am developing a package MyPkg. How can I get the path to MyPkg.jl? I used Pkg.dir() * "MyPkg/", but that doesn’t work in Julia 1.0 anymore
1 Like
import MyPkg; pathof(MyPkg)
3 Likes
But am I supposed to import MyPkg inside MyPkg??
Oh no, sorry I didn’t realize you meant within the package. For that you can do
pathof(@__MODULE__)
Or, more likely you’ll want to use @__DIR__ which gives the directory of the current file.
3 Likes
It seems unreasonable to me that Pkg.dir is deprecated, because it makes a lot of sense to have a quick way of navigating to a package directory without importing it first. I know there is another way to do it, but it isn’t a simple and straight forward syntax.
3 Likes
This would also require importing, but eliminates one step:
Well, that’s only if you’re already within the module! I also find the loss of Pkg.dir a little bit frustrating (even though in the context that the OP was referring to, @__DIR__ or even pathof(@__MODULE__) is preferable). I think one of the reasons this may have been done is to allow for the possibility that the actual location of the package is ambiguous until it is imported. It would however be nice if there were still some options for getting the path of a package that’s not currently imported.
Could someone clamoring for the return of Pkg.dir() please tell me how it should work in the new system? I didn’t just delete it because I’m a jerk, there is no reasonable answer for it to return anymore.
5 Likes
See my quote above. As often happens with crticisms, this one has often come from us simply not knowing the details.
Still, import evaluates something, maybe some options for showing a list of possibilities of what might be imported were you to call import? (Again, I’m saying this without knowing the details of how it actually works.) (Also, in case you will accuse me of sounding crazy, keep in mind that many of us spend a lot of our time on packages checked out with dev).
To add some details.
When running import Foo there are two steps happening.
Step one:
Determine what “Foo” means in this context. This is determined by finding the unique UUID for “Foo”. If you are in “top level” (for example in the REPL) this is determined by looking in Project.toml for an entry Foo = "aefe..." where the right hand side is the UUID. If we are in a package (Bar) and execute the import the UUID is determined by looking at Manifest.toml, finding the [[Bar]] entry with the correct UUID and look up the UUID for Foo there.
Step two:
Ok, so now we know what “Foo” means (i.e we have determined the concrete Foo package out of all possible packages that are named Foo). Now we need to know what concrete path of Foo to load (there can be multiple versions of Foo installed at the same time). This is done by looking in Manifest.toml entry for the uuid we just found (the one for Foo) and if there is a path entry there, that will be the path to Foo, if there is a git-tree-sha entry (as is the case for versioned packages) a “slug” will be computed according to a hashing strategy (based on UUID and git-tree-sha). Every package directory in DEPOT_PATH is now searched for a path with that slug and if it is found that will be loaded.
Concrete example:
I have installed Example and want to know what will be the loaded path when loading it in the REPL. The Project.toml file contains the entry
Example = "7876af07-990d-54b4-ab0e-23690620f79a"
so the UUID will be "7876af07-990d-54b4-ab0e-23690620f79a".
The Manifest.toml contains
[[Example]]
deps = ["Test"]
git-tree-sha1 = "8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8"
uuid = "7876af07-990d-54b4-ab0e-23690620f79a"
version = "0.5.1"
So there is a git-tree_sha1 entry so we compute the slug as
julia> sha = Base.SHA1("8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8")
SHA1("8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8")
julia> uuid = Base.UUID("7876af07-990d-54b4-ab0e-23690620f79a")
UUID("7876af07-990d-54b4-ab0e-23690620f79a")
julia> Base.version_slug(uuid, sha)
"kH44X"
We will now search every package folder inside DEPOT_PATH for "Example/kH44X":
julia> isdir(joinpath(DEPOT_PATH[1], "packages", "Example", "kH44X"))
true
So that will be the path that we load. A short form of doing this whole process for top level scope is
julia> Base.find_package("Example")
"/Users/kristoffer/.julia/packages/Example/kH44X/src/Example.jl"
If I now do dev Example
(v1.1) pkg> dev Example
Updating git-repo `https://github.com/JuliaLang/Example.jl.git`
Resolving package versions...
Updating `~/.julia/environments/v1.1/Project.toml`
[7876af07] ↑ Example v0.5.1 ⇒ v0.5.1+ [`~/.julia/dev/Example`]
Updating `~/.julia/environments/v1.1/Manifest.toml`
[7876af07] ↑ Example v0.5.1 ⇒ v0.5.1+ [`~/.julia/dev/Example`]
then the Manifest looks like
[[Example]]
deps = ["Test"]
path = "/Users/kristoffer/.julia/dev/Example"
uuid = "7876af07-990d-54b4-ab0e-23690620f79a"
version = "0.5.1+"
so we will just directly use /Users/kristoffer/.julia/dev/Example as the path:
julia> Base.find_package("Example")
"/Users/kristoffer/.julia/dev/Example/src/Example.jl"
Note that a dependency can depend on a completely different package to Example but that happens to also be called “Example” with a different UUID. So the answer to Pkg.dir("Example") is dependent on who you ask.
5 Likes
This requires me to be calling from within the package, it doesn’t work outside the package.
How about some way of determining the package path that does not require the import statement, just a simple Pkg.dir command that uses the Manifest to determine if the name of the package is unique, and if the name is unique, return the only path that is possible. Otherwise, if there are multiple choices, give a multiple choice menu in the REPL using TerminalMenus, for example… that is one solution.
As I understand it, calling import also loads the package, precompiles, etc? I don’t want that, only the dir.
For what purpose?
Very reasonable question.
Very reasonable answer: say, you want to edit a package before loading it. then cd to it and edit the change desired, and then load the package afterwards. The package I am trying to edit causes the Revise module to crash, so using Revise is not an option for me at the moment.
The current import way does not allow me to reload the package after editing.
Packages that are just added are considered immutable (in will in the future be read-only so you shouldn’t hand-edit those). Packages that are “developed” (those that have a path entry), have their path printed when using st in the Pkg REPL.
It seems Base.find_package should work well for your interactive use.
3 Likes
Yea, that would do what I want. Thanks. I can alias it as Pkgdir = Base.find_package in startup.jl.
1 Like
When I was missing this feature, I was thinking of something like splitting off the directory of the result of Base.find_package, or equivalently Base.pathof without loading the module.
But now that I have used 1.0 a bit more, I don’t miss Pkg.dir that much. I think that removing it was the right decision, since it served as a basis for various other use cases, all of which have better solutions. Namely,
- testing, coverage, and docs generation in
.travis.ymland other CI servises (solution: no need to have the directory directly, and one can always use projects), - locating files relative to source (solution: use
@__DIR__as a basis for this, ideally via a function), - just want to browse the files, eg for reading the source (solution:
import, thenpathof, orpkg> devfor editing as suggested above).
Having read @kristoffer.carlsson’s explanation, I actually have gone back to thinking that having something like Pkg.dir was indeed a reasonable request. I think what we were looking for was just something that would give you whatever would be the path of the package that would be imported by import. (It looks like this functionality is indeed provided by Base.find_package.)
Now, I should temper this somewhat, and say that probably the reason we think this is reasonable is because many of us here are probably very used to living on the bleeding edge of packages, and when we think of a package we are always thinking about the latest release or the checked-out master. I have a feeling that if Base.find_package got out in the wild where people would be using it for whom this would not be the case it might start to cause massive confusion.
I’m not so confident that this is always true (granted, I’ve only just started seriously messing around with CI CD). You may still need it for some packages such as coverage. Certainly you migt need it if there’s any not-explicitly-Julia-related stuff in your package directory, or perhaps for deployment of scripts.
Otherwise I agree, for the most part you shouldn’t need it. Indeed, for the original purpose of this thread, @__DIR__ is a much more appropriate solution and using Base.find_package for such a purpose would only invite problems.
You don’t need any Pkg.dir for this because for CI you are already in the repo. There is no reason to ask Pkg what path the package is in because you are already in it!
1 Like