How to install non-registered dependencies in Julia 0.7/1.0

General Usage
21

It’s a bit confusing, but what I think he meant was that he added those dependencies to the environment/Project .tomls of MPIReco.jl.

22

In the LinearSolver Project.toml, I have

name = "LinearSolver"
uuid = "12d7aa1e-a7d4-11e8-33ea-6753c3e87491"

but Pkg says

 Info: Assigning UUID 1906c763-83da-50c9-aa61-18e2016498cc to LinearSolver

and if I correct the environment files after installing LinearSolver and MPIFiles, then MPIReco installs without trouble here.

23

That seems to fix this locally.

1 Like
24

@Ralph_Smith:

if I remove MPIFiles and LinearSolver, I get

(v0.7) pkg> dev https://github.com/MagneticParticleImaging/MPIReco.jl.git
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/MagneticParticleImaging/MPIReco.jl.git`
[ Info: Assigning UUID cffeea2d-44ef-50d9-9a8a-3f5d8306ed6d to MPIReco
[ Info: Path `/Users/knopp/.julia/dev/MPIReco` exists and looks like the correct package, using existing path instead of cloning
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package LinearSolver [1906c763]:
 LinearSolver [1906c763] log:
 ├─LinearSolver [1906c763] has no known versions!
 └─restricted to versions * by MPIReco [cffeea2d] — no versions left
   └─MPIReco [cffeea2d] log:
     ├─possible versions are: 0.0.0 or uninstalled
     └─MPIReco [cffeea2d] is fixed to version 0.0.0

Do I need to manually install MPIFiles and MPIReco?

25

That’s what I did, in order to set the git branches and checkout viable commits.

26

Yes but I want MPIReco to run tests on travis. It thus needs to install those packages and I thought it will do so since the information is located in the Manifest.

27

I created GitHub - fredrikekre/PackageB.jl which depends on (a specific branch from) GitHub - fredrikekre/PackageA.jl and passes test: Travis CI - Test and Deploy Your Code with Confidence

Adding locally you can do:

(v1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`

(v1) pkg> add https://github.com/fredrikekre/PackageA.jl.git#somebranch
 Resolving package versions...
  Updating `~/.julia/environments/v1.1/Project.toml`
  [6efbc5e2] + PackageA v0.1.0 #somebranch (https://github.com/fredrikekre/PackageA.jl.git)
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [6efbc5e2] + PackageA v0.1.0 #somebranch (https://github.com/fredrikekre/PackageA.jl.git)

(v1) pkg> add https://github.com/fredrikekre/PackageB.jl.git
 Resolving package versions...
  Updating `~/.julia/environments/v1.1/Project.toml`
  [74f46274] + PackageB v0.1.0 #master (https://github.com/fredrikekre/PackageB.jl.git)
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [74f46274] + PackageB v0.1.0 #master (https://github.com/fredrikekre/PackageB.jl.git)

julia> using PackageB

julia> PackageB.greet()
Hello from PackageA.jl!

Hope this is useful.

3 Likes
28

Thanks. This means in principle this works.

29

But locally you had to manually install PackageA right? Just installing B does not work here:

(v0.7) pkg> add https://github.com/fredrikekre/PackageB.jl.git
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `https://github.com/fredrikekre/PackageB.jl.git`
  Updating git-repo `https://github.com/fredrikekre/PackageB.jl.git`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package PackageA [6efbc5e2]:
 PackageA [6efbc5e2] log:
 ├─PackageA [6efbc5e2] has no known versions!
 └─restricted to versions * by PackageB [74f46274] — no versions left
   └─PackageB [74f46274] log:
     ├─possible versions are: 0.1.0 or uninstalled
     └─PackageB [74f46274] is fixed to version 0.1.0

So how did you do this on travis?

30

Yea, but this is not new, you had to do this in Pkg2 too. (Edit: Store some location info for deps in Project.toml · Issue #492 · JuliaLang/Pkg.jl · GitHub is probably the issue to track regarding this.)

On Travis we start julia with PackageB/Project.toml as the active project, and since I checked in Manifest.toml (which includes the url and branch of PackageA) then Pkg knows what PackageA means, and can instantiate it.
This is equivalent to:

$ git clone https://github.com/fredrikekre/PackageB.jl.git PackageB
$ cd PackageB
$ julia --project

julia> using Pkg; Pkg.build(); Pkg.test()

The key thing here is that we are running inside ProjectB:

(PackageB) pkg> st
Project PackageB v0.1.0
    Status `Project.toml`
  [6efbc5e2] PackageA v0.1.0 #somebranch (https://github.com/fredrikekre/PackageA.jl.git)
31

Oh well, this is so complicated. this means that the Manifest files are actually not used, just in this very specific case.

I know. In the first post you can see my workaround for this. And I am now seeking for a workaround for Julia 0.7.

1 Like
32

What do you mean? The workaround is to install the unregistered packages in order, which is what we did in Pkg2 also, so this has not changed really. Whats wrong with doing

(v1) pkg> add https://github.com/fredrikekre/PackageA.jl.git#somebranch
(v1) pkg> add https://github.com/fredrikekre/PackageB.jl.git
33

I had a workaround that in Julia 0.6 that allowed to just add the leaf package (see my original post). And that particular workaround does not work anymore.

34

Right, but !isdir(Pkg.dir("LinearSolver")) was not necessarily correct in Pkg anyway, if the user had packages in other places than the standard location.
Just put in the README the commands to install the package. Copy-pasting

add https://github.com/fredrikekre/PackageA.jl.git#somebranch
add https://github.com/fredrikekre/PackageB.jl.git

is not really more work than copy-pasting

add https://github.com/fredrikekre/PackageB.jl.git
35

Its not just these two packages but in the end more than 10 packages. My workaround was nice since it kept the dependency local. I am looking forward to https://github.com/JuliaLang/Pkg.jl/issues/492 which then will hopefully solve this issue.

36

I want to thank you @fredrikekre for your very good support and explanation. Travis now works: Travis CI - Test and Deploy Your Code with Confidence.

My impression is that the overall experience with unregistered packages is not good yet. This is not a complaint but totally expected for a brand new package manager. I hope that https://github.com/JuliaLang/Pkg.jl/issues/492 will solve this issue which will then result in a better experience.

That travis runs the Manifest environment of the project is now a way to circumvent my problem but it does not feel correct. It means that things pass on travis but the do not work for the user. @keno: You argued in another thread that this is not desired.

37

For registered packages that is.

38

And for unregistered packages hopefully it is also desired that a passing travis implies a passing local experience, no? I have strong hopes that unregistered packages will become first class with Pkg3 (which they will be once https://github.com/JuliaLang/Pkg.jl/issues/492 is solved).

1 Like