Response to Pkg.jl and Julia Environments for Beginners by Jules

General Usage
, ,
1

jkrumbiegel.com - Pkg.jl and Julia Environments for Beginners

Thank you so much @jules for the great writeup. That clarified a lot for me.

Consider adding this as a tutorial to the Pkg documentation. The existing documentation is pretty barebones and missing some crucial details e.g. I don’t think shared environments and the @ symbol are mentioned at all.

I have a few follow up questions that I hope you don’t mind if I ask openly here. My main use case is developing an unregistered packaged that will be shared with colleagues through a shared file system.

  1. Is changing the version = line in the package Project.toml during some git commit the only thing I need to do to “release” a new version?
  2. What is the difference between dev path/to/package and add path/to/package from some project environment?
    From this answer, I think dev would ignore versions and always use the latest #master code stored at path/to/package directly. I think (hope) add will copy the code to .julia/packages/ and store the package version used in the project Manifest.toml.
  3. How can I resolve Julia version conflicts with instantiate?
    When instantiating a Manifest.toml I got a warning that I am using a different Julia version than the manifest. Is there anything that can automatically fix that or am I going to have to do something fancy with juliaup? Users will be on Mac and Windows, so I can’t count on having bash.
5 Likes
2

Strongly support the thanks. I had add the link to this to lecture notes for our fall class. Pkg can be tough at the beginning and I recall how I was confused when I started to use it.

1 Like
3

  1. No, the release happens by storing any commit of your choice in the registry. This is what the JuliaRegistrator bot is for, once you’ve enabled it you just comment @JuliaRegistrator register on some commit in your repo. An example from Makie here: up versions (#2198) · JuliaPlots/Makie.jl@0252621 · GitHub

  2. dev tracks the code in the folder you specify, which means you can edit it and Revise will reload the code, etc. add copies the repository to some internal Julia folder where it sits in read-only mode. I almost never add local repositories.

  3. The manifest format that can specify the Julia version was only added in 1.6. If you have an older manifest, or if you have a newer manifest where the Julia version is specified, but you use an even newer Julia version, you will get such warnings. You can call Pkg.upgrade_manifest(), I think, to write the new Julia version you’re using into the manifest to silence that warning. In principle, a newer Julia version should not be a problem because no code for Julia 1.n should break in 1.n+1 (unless it uses internals of course…). If you use a lower Julia version, it could of course be that it’s no longer compatible.

2 Likes
4

I never use dev. If I am developing a package, I always work in that package environment, which seems OK if I modify the source code and test it in the REPL. I am confused what is dev used for?

1 Like
5

I believe dev is employed mainly to use your package alongside other packages which are not dependencies of your package.

6

Then why not just use add as a user? does use my package implicitly means that the user will not modify my package. Since the package is not going to be modified, it is not necessary to use dev.

And as a package developer, here is my workflow which can be done without dev too:

Suppose my package locates at /path/to/MyPackage. First cd into that dir and activate the MyPacakge env,

$ cd /path/to/MyPackage
$ julia --project

Within the REPL (with Revise loaded in startup.jl)

julia> using MyPackage

I can now modify the source code of MyPackage using any editor, and run its functions in the REPL.

julia> myfunc()
julia> MyPacakge.not_exported_func()

So my conclusion is that dev can be removed. It only bings up confusions.

2 Likes
7

If you only use the package environment and don’t dev your package into a different environment you can’t install separate packages that have their dependencies resolved correctly relative to your package’s compat settings.

1 Like
8

You can use dev to work on the code of some other package of which this package depends, at the same time.

9

I am not aware of that. Thanks for your clarification.

Regarding to this aspect, I have a workaround and I feel comfortable with it. I fix the version of the other dependencies also created by myself. Once I find there is a bug or I need new features I will switch to work on that particular package in the same way as I work on the current package. Once I fix the bug or add the new feature in that package and publish a minor or major version, I will then switch back to the current package. This way, I can make sure each version of my packages has a clear motivation. It also causes some issues for sure. For example, the version updates too frequently sometimes.

10

]dev is same to ]add except for the fact that ]dev allows you to install a package even when it is not version controlled via git, which may be convenient and flexible in early stage of development. Whereas, you cannot ]add a package if it is not version controlled. Also, when you ]dev a version controlled package, then the package is installed as is, meaning that uncomitted changes will take effect.

Other difference is that when you ]dev a package from remote site, then it is copied under your .julia/dev/ or pwd()/dev/ according to the options --shared or --local. Furthermore it is editable. You can make some changes as you wish. It is never copied to .julia/packages/ folder, which is unlike to ]add. The code in .julia/packages can’t be edited.

So, ]dev is very convenient.

2 Likes
11
  1. Is changing the version = line in the package Project.toml during some git commit the only thing I need to do to “release” a new version?

This is my understanding:
Experiment will reveal that the line version= is only for displaying when you ]st <a package>
A developer must keep the number aligned to the package’s actual content manually.

The real versioning is manifested by git tag to a commit. So for example a maintainer should git tag v0.1.1 -m"some description" before commit and push. Afterwards, if one does ]add <git-url-to-package>#v.0.1.1, it will clone the right version regardless that the version line in the project file still stays at 0.1.0 by mistake.

  1. How can I resolve Julia version conflicts with instantiate?
    When instantiating a Manifest.toml I got a warning that I am using a different Julia version than the manifest. Is there anything that can automatically fix that or am I going to have to do something fancy with juliaup? Users will be on Mac and Windows, so I can’t count on having bash.*

I think menifiest file is a generated file not intended for human editing. I suspect the compat section of the project.toml of interest may be incompatible with your current julia version. The compat section in a project file looks like:

[compat]
julia = “1.8”

1 Like