Recommended way to migrate an existing package to Pkg3


#1

I know that Pkg3 is still in active development and detailed documentation is coming soon, but right now I have a couple of issues that block me from updating my packages and thus want to overcome them, either using official tools or recommended workarounds.

  1. It seems like in Pkg3 you can’t be using another package without adding them to the project. Adding to the project requires Project.toml, but there seems to be no official way to generate it for existing package. My current workaround: generate a fake project, copy its Project.toml and updated required fields. Is it the recommended to fix the issue? Should I make PRs to METADATA using Project.toml generated this way?

  2. As far as I understand, dependencies for a package should be both - described in REQUIRE and added to the project using the same versions. Is there easier / less error-prone way to handle deps?

  3. adding a project using git URL fails if any of its dependencies cannot be satisfied, so the code isn’t even downloaded to the disk and can’t be modified. My current workaround: git clone into .julia/dev, but I’m not sure I don’t break any internal machinery using this hack.

Please let me if there are better ways to fix these issues.


#2
  1. There should be a default Project.toml for your version Julia version. Project specific ones override the default one.
  2. When everything is working the dependencies are computed from the Project.toml not from REQUIRE. You don’t need the REQUIRE file anymore.
  3. Projects are similar to .Rproj and other implementations. You need the whole shebang. You can download them to any directory and work with it there.

#3

Thanks for your reply. Unfortunately, your suggestions are relevant only for newly generated or already migrated projects, not the ones that you migrate from Julia 0.6 where there’s just no Project.toml.

I’ve found a relevant thread here, especially this and this comments. In particular, they state that one should not create their own Project.toml and still add dependencies to REQUIRE for now. But here’s a problem: my package Espresso.jl depends on (now external) LinearAlgebra module. I’ve added it to REQUIRE, but when testing the package fails with:

ERROR: The following package names could not be resolved:

  • LinearAlgebra (37e2e46d-f89d-539d-b4ee-838fcccc9c8e in manifest but not in project)
    Please specify by known name=uuid.

If I manually create Project.toml and run add LinearAlgebra, tests run fine. So either I do something that isn’t recommended at the moment (adding Project.toml), or my package gets broken.


#4

All registered packages already have a Project.toml which was generated so that these can be handled by the new stdlib. You can check those from the registry. For example, for DataFrames. You can just copy that file and modify it to include additional metadata (e.g., license, maintainer, project status, etc.). For stdlib you can check the UUID from their project.toml, (e.g., LinearAlgebra).

A few comments (right now due to a bug those that depend on stdlib/Dates need to wait a bit). There were a few issues that broke Travis for the last couple days (related to upgrading a few libraries in nightly). If you refer to packages for Julia 0.6 that have not been registered yet, you can either first register it and migrate or way a bit longer. There isn’t a “proper” way to register it directly for the moment.


#5

Aha! So that was the missing detail - Project.toml files are generated in the registry and not present in the project’s own repository yet. I’ll test it out and come back. Thanks for the tip!

Edit. Taking my Project.toml from the registry worked just fine!


#6

You can also just stick with using REQUIRES by using Compat since it appears that all the stdlib modules are also exported from there (that’s one way you can get v0.7 and v0.6 compatibility at the same time).