Pkg3 for package developers

[denizyuret]

What are the steps to prepare an existing package for 0.7/1.0? Do we keep REQUIRE, add a new Project.toml, what is my UUID etc? To change dependencies do we use pkg> add or do we modify REQUIRE? How do the two interact? Where do we specify the julia version? Is there a guide for developers?

[Keno]

There’s nothing you need to do. REQUIRE will keep working for now and we’ll keep using METADATA. Eventually, there’ll be an automated PR to add Project.toml s to every package and registration will be switched to JuliaRegistries/General, but that’s not happening yet.

[denizyuret]

Here are some issues I am having.

1. AutoGrad/REQUIRE currently contains “julia 0.7” and “SpecialFunctions”. There is no Project.toml in the repo. When I try to develop with “pkg> dev AutoGrad” and try to run a test file that starts with “using SpecialFunctions” I get a “Package SpecialFunctions not found in current path” error. Is this because SpecialFunctions is in the manifest but not in the project? Does this mean SpecialFunctions is visible from inside the package AutoGrad but not from Main?

2. “pkg> up” does not seem to update the packages that I checked out with “pkg> dev”. I need to go into dev/PkgName and manually do a git pull. Is this by design?

3. In spite of what it says in the documentation “pkg> dev AutoGrad#master” does not work. I get “ERROR: a git revision cannot be given to develop”

4. I am not sure how packages in stdlib are supposed to be handled. When I try using them in a package I get “Warning: Package Knet does not have Libdl in its dependencies”. If I put them in REQUIRE, that does not seem to solve the problem. Do I put these in Project.toml?

[fredrikekre]

@denizyuret

Does this mean SpecialFunctions is visible from inside the package AutoGrad but not from Main?

Exactly, so if you want to e.g. include("testfile.jl") and have that work in Main, then you also need to add the modules that you want available, e.g. SpecialFunctions in this case.

@denizyuret

Is this by design?

Yes, if you have deved a package, and thus track it by a local path, then you are on your own there. The package manager intentionally does not touch any of that code (since it is “owned” by you in your local path, and not by Pkg).

@denizyuret

In spite of what it says in the documentation “pkg> dev AutoGrad#master” does not work. I get "ERROR: a git revision cannot be given to develop "

Yea, this was removed in the last minute, but the documentation was not updated. It has been now though (`develop` branch checkout fails · Issue #636 · JuliaLang/Pkg.jl · GitHub, update docs for develop by KristofferC · Pull Request #639 · JuliaLang/Pkg.jl · GitHub)

@denizyuret

I am not sure how packages in stdlib are supposed to be handled. When I try using them in a package I get “Warning: Package Knet does not have Libdl in its dependencies”. If I put them in REQUIRE, that does not seem to solve the problem. Do I put these in Project.toml?

If you have a Project.toml you need to add them there, if you only have a REQUIRE it is enough to add using Libdl in the code, and then you might need to pkg> resolve to make Pkg aware of the change.

[denizyuret]

Do we have a replacement for the old Pkg.installed? Can I found out whether a package is available and/or what version is available?

[rened]

It exists for me on 1.0:

using Pkg

Pkg.installed()

Dict{String,Union{Nothing, VersionNumber}} with 2 entries:
“AutoGrad” => v"1.0.0"
“JSON” => v"0.19.0"

haskey(Pkg.installed(), “AutoGrad”)
true

[ajaym]

Is there a document on how to add packages to GitHub - JuliaRegistries/General: The official registry of general Julia packages ?

[Evizero]

@fredrikekre

Yes, if you have dev ed a package, and thus track it by a local path, then you are on your own there. The package manager intentionally does not touch any of that code (since it is “owned” by you in your local path, and not by Pkg).

In general I think this is nice. But I have to say, when you know you are on master with no uncommited changes (and it is set up to track origin/master), it is a bit frustrating that I have to go in there and manually go git pull everywhere just to make sure everything is fast forwarded.
Add to this that status doesn’t show me the checked-out branches anymore. To me as a package developer/maintainer this aspect feels a little bit like a usability regression.

[kristoffer.carlsson]

@Evizero

In general I think this is nice. But I have to say, when you know you are on master with no uncommited changes (and it is set up to track origin/master ), it is a bit frustrating that I have to go in there and manually go git pull everywhere just to make sure everything is fast forwarded.

Why are you using dev and not add in this case?

[Evizero]

because I am working on those packages - as are others. Concretely I felt this while helping out on updating the JuliaImages packages over the last couple days. Every time I had some spare time available to do a little update session, I bassically had to cd in all the (still untagged) images packages, look which branch that I am on, and pull.

Its not like its a huge burden. It is just a deal less convenient than it was. And it feels a bit more opaque because I lack to ability to quickly look at which branches are checked out and if they are outdated.

I realize that maybe I just need to change my workflow, but so far this “environment-per-package” idea hasn’t come in that handy, because we are still transitioning from 0.6 to 0.7. (so I am using --shared environments with multiple packages marked as dev). I can imagine it being a lot more common for me afterwards to work on packages in isolation.