Suppose I am developing a package MyPkg. How can I get the path to MyPkg.jl? I used Pkg.dir() * "MyPkg/"
, but that doesn’t work in Julia 1.0 anymore
import MyPkg; pathof(MyPkg)
But am I supposed to import MyPkg
inside MyPkg
??
Oh no, sorry I didn’t realize you meant within the package. For that you can do
pathof(@__MODULE__)
Or, more likely you’ll want to use @__DIR__
which gives the directory of the current file.
It seems unreasonable to me that Pkg.dir
is deprecated, because it makes a lot of sense to have a quick way of navigating to a package directory without importing it first. I know there is another way to do it, but it isn’t a simple and straight forward syntax.
See this answer. No need to import the package first.
This would also require importing, but eliminates one step:
Well, that’s only if you’re already within the module! I also find the loss of Pkg.dir
a little bit frustrating (even though in the context that the OP was referring to, @__DIR__
or even pathof(@__MODULE__)
is preferable). I think one of the reasons this may have been done is to allow for the possibility that the actual location of the package is ambiguous until it is imported. It would however be nice if there were still some options for getting the path of a package that’s not currently imported.
Could someone clamoring for the return of Pkg.dir()
please tell me how it should work in the new system? I didn’t just delete it because I’m a jerk, there is no reasonable answer for it to return anymore.
See my quote above. As often happens with crticisms, this one has often come from us simply not knowing the details.
Still, import
evaluates something, maybe some options for showing a list of possibilities of what might be imported were you to call import
? (Again, I’m saying this without knowing the details of how it actually works.) (Also, in case you will accuse me of sounding crazy, keep in mind that many of us spend a lot of our time on packages checked out with dev
).
To add some details.
When running import Foo
there are two steps happening.
Step one:
Determine what “Foo” means in this context. This is determined by finding the unique UUID for “Foo”. If you are in “top level” (for example in the REPL) this is determined by looking in Project.toml for an entry Foo = "aefe..."
where the right hand side is the UUID. If we are in a package (Bar
) and execute the import
the UUID is determined by looking at Manifest.toml, finding the [[Bar]]
entry with the correct UUID and look up the UUID for Foo
there.
Step two:
Ok, so now we know what “Foo” means (i.e we have determined the concrete Foo package out of all possible packages that are named Foo). Now we need to know what concrete path of Foo to load (there can be multiple versions of Foo installed at the same time). This is done by looking in Manifest.toml entry for the uuid we just found (the one for Foo) and if there is a path
entry there, that will be the path to Foo
, if there is a git-tree-sha
entry (as is the case for versioned packages) a “slug” will be computed according to a hashing strategy (based on UUID and git-tree-sha
). Every package
directory in DEPOT_PATH
is now searched for a path with that slug and if it is found that will be loaded.
Concrete example:
I have installed Example and want to know what will be the loaded path when loading it in the REPL. The Project.toml file contains the entry
Example = "7876af07-990d-54b4-ab0e-23690620f79a"
so the UUID will be "7876af07-990d-54b4-ab0e-23690620f79a"
.
The Manifest.toml contains
[[Example]]
deps = ["Test"]
git-tree-sha1 = "8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8"
uuid = "7876af07-990d-54b4-ab0e-23690620f79a"
version = "0.5.1"
So there is a git-tree_sha1
entry so we compute the slug as
julia> sha = Base.SHA1("8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8")
SHA1("8eb7b4d4ca487caade9ba3e85932e28ce6d6e1f8")
julia> uuid = Base.UUID("7876af07-990d-54b4-ab0e-23690620f79a")
UUID("7876af07-990d-54b4-ab0e-23690620f79a")
julia> Base.version_slug(uuid, sha)
"kH44X"
We will now search every package
folder inside DEPOT_PATH
for "Example/kH44X"
:
julia> isdir(joinpath(DEPOT_PATH[1], "packages", "Example", "kH44X"))
true
So that will be the path that we load. A short form of doing this whole process for top level scope is
julia> Base.find_package("Example")
"/Users/kristoffer/.julia/packages/Example/kH44X/src/Example.jl"
If I now do dev Example
(v1.1) pkg> dev Example
Updating git-repo `https://github.com/JuliaLang/Example.jl.git`
Resolving package versions...
Updating `~/.julia/environments/v1.1/Project.toml`
[7876af07] ↑ Example v0.5.1 ⇒ v0.5.1+ [`~/.julia/dev/Example`]
Updating `~/.julia/environments/v1.1/Manifest.toml`
[7876af07] ↑ Example v0.5.1 ⇒ v0.5.1+ [`~/.julia/dev/Example`]
then the Manifest looks like
[[Example]]
deps = ["Test"]
path = "/Users/kristoffer/.julia/dev/Example"
uuid = "7876af07-990d-54b4-ab0e-23690620f79a"
version = "0.5.1+"
so we will just directly use /Users/kristoffer/.julia/dev/Example
as the path:
julia> Base.find_package("Example")
"/Users/kristoffer/.julia/dev/Example/src/Example.jl"
Note that a dependency can depend on a completely different package to Example but that happens to also be called “Example” with a different UUID. So the answer to Pkg.dir("Example")
is dependent on who you ask.
This requires me to be calling from within the package, it doesn’t work outside the package.
How about some way of determining the package path that does not require the import
statement, just a simple Pkg.dir
command that uses the Manifest
to determine if the name of the package is unique, and if the name is unique, return the only path that is possible. Otherwise, if there are multiple choices, give a multiple choice menu in the REPL using TerminalMenus
, for example… that is one solution.
As I understand it, calling import
also loads the package, precompiles, etc? I don’t want that, only the dir
.
For what purpose?
Very reasonable question.
Very reasonable answer: say, you want to edit a package before loading it. then cd
to it and edit the change desired, and then load the package afterwards. The package I am trying to edit causes the Revise
module to crash, so using Revise
is not an option for me at the moment.
The current import
way does not allow me to reload the package after editing.
Packages that are just add
ed are considered immutable (in will in the future be read-only so you shouldn’t hand-edit those). Packages that are “developed” (those that have a path
entry), have their path printed when using st
in the Pkg REPL.
It seems Base.find_package
should work well for your interactive use.
Yea, that would do what I want. Thanks. I can alias it as Pkgdir = Base.find_package
in startup.jl
.
When I was missing this feature, I was thinking of something like splitting off the directory of the result of Base.find_package
, or equivalently Base.pathof
without loading the module.
But now that I have used 1.0 a bit more, I don’t miss Pkg.dir
that much. I think that removing it was the right decision, since it served as a basis for various other use cases, all of which have better solutions. Namely,
- testing, coverage, and docs generation in
.travis.yml
and other CI servises (solution: no need to have the directory directly, and one can always use projects), - locating files relative to source (solution: use
@__DIR__
as a basis for this, ideally via a function), - just want to browse the files, eg for reading the source (solution:
import
, thenpathof
, orpkg> dev
for editing as suggested above).
Having read @kristoffer.carlsson’s explanation, I actually have gone back to thinking that having something like Pkg.dir
was indeed a reasonable request. I think what we were looking for was just something that would give you whatever would be the path of the package that would be imported by import
. (It looks like this functionality is indeed provided by Base.find_package
.)
Now, I should temper this somewhat, and say that probably the reason we think this is reasonable is because many of us here are probably very used to living on the bleeding edge of packages, and when we think of a package we are always thinking about the latest release or the checked-out master. I have a feeling that if Base.find_package
got out in the wild where people would be using it for whom this would not be the case it might start to cause massive confusion.
I’m not so confident that this is always true (granted, I’ve only just started seriously messing around with CI CD). You may still need it for some packages such as coverage. Certainly you migt need it if there’s any not-explicitly-Julia-related stuff in your package directory, or perhaps for deployment of scripts.
Otherwise I agree, for the most part you shouldn’t need it. Indeed, for the original purpose of this thread, @__DIR__
is a much more appropriate solution and using Base.find_package
for such a purpose would only invite problems.
See https://github.com/fredrikekre/Literate.jl/tree/master/test/coverage and similar.
You don’t need any Pkg.dir
for this because for CI you are already in the repo. There is no reason to ask Pkg what path the package is in because you are already in it!