How to create a private package registry

Is there any documentation explaining how to create a package repository for the new package manager? I want to set one up for use at my company for various packages that can’t be shared publicly.

3 Likes

Just create the package as usual and then you have to manually set the origin to your repository. The repository has to be created first of course. If you are a compony, Github certainly wants you to buy them.

We have a self-hosted GitLab instance.

Also, I see that I didn’t word my question well. I’m not wondering how to host the repository for the source code of a package, but rather how to create my own private version of METADATA, or whatever it is called for the new package manager.

I think I should have said package registry, not package repository. Still trying to come up to speed on the new terminology.

ok. In the 0.6.* world you can simply Pkg.clone the package. A private version of METADATA seems to be too much effort in my opinion. I have several private packages and since I cannot rely on REQUIRE, I simply clone all private dependencies when using the package:

if !isdir(Pkg.dir("MyDependingPrivatePackage"))
  Pkg.clone("https://....../MyDependingPrivatePackage.jl.git")
end

OK, thanks.

What about in the 0.7 world, with Pkg3? I was thinking it would be able to more easily configure things so that Pkg.add would work, even for private packages.

1 Like

As far as I have understood in the recent posting the answer is “not yet”. I am also patient waiting to see how things will work in the future.

OK, thanks!

Registry: a source tree with a standard layout recording metadata about a registered set of packages, the tagged versions of them which are available, and which versions of packages are compatible or incompatible with each other. A registry is indexed by package name and UUID, and has a directory for each registered package providing the following metadata about it:

Are you sure you want to maintain a registry? Instead of making everything a package, we can work with independent projects via the new feature of Pkg3:

  1. generate a project
(v0.7) pkg> generate PriviteProject
Generating project PriviteProject:
    PriviteProject/Project.toml
    PriviteProject/src/PriviteProject.jl
  1. cd to the project folder
shell> cd PriviteProject/
/Users/gnimuc/PriviteProject
  1. add dependencies directly from url
(PriviteProject) pkg> add https://github.com/Gnimuc/GLTF.jl.git
  Updating git-repo `https://github.com/Gnimuc/GLTF.jl.git`
  Updating registry at `~/.julia/registries/Uncurated`
  Updating git-repo `https://github.com/JuliaRegistries/Uncurated.git`
 Resolving package versions...
  Updating `Project.toml`
  [0844153c] + GLTF v0.0.1 # [https://github.com/Gnimuc/GLTF.jl.git]
  Updating `Manifest.toml`
  [0844153c] + GLTF v0.0.1 # [https://github.com/Gnimuc/GLTF.jl.git]
  [2535ab7d] + JSON2 v0.1.3
  [ade2ca70] + Dates 
  [8dfed614] + Test 

here, I added an unregistered project which depends on some registered packages.

  1. init git repo and upload to your Gitlab server
shell> git init
shell> git add --all
shell> git commit -m "init"
shell> git remote add `your Gitlab repo url`

here, I published it to Github to mimic Gitlab.

  1. share and use the project locally
# on someone else's machine
➜  ~ git clone https://github.com/Gnimuc/PriviteProject.git 
Cloning into 'PriviteProject'...
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
Unpacking objects: 100% (6/6), done.
remote: Total 6 (delta 0), reused 6 (delta 0), pack-reused 0
➜  ~ cd PriviteProject 
➜  PriviteProject git:(master) JULIA_DEPOT_PATH=./test julia  # note that, here I changed DEPOT_PATH to mimic a clean Julia package environment, you don't have to do this.
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.7.0-alpha.59 (2018-06-09 17:04 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 2a45839ef9 (6 days old master)
|__/                   |  x86_64-apple-darwin17.6.0

(PriviteProject) pkg> instantiate
   Cloning default registries into /Users/gnimuc/PriviteProject/test/registries
   Cloning registry Uncurated from "https://github.com/JuliaRegistries/Uncurated.git"
  Updating registry at `test/registries/Uncurated`
  Updating git-repo `https://github.com/JuliaRegistries/Uncurated.git`
   Cloning git-repo `https://github.com/Gnimuc/GLTF.jl.git`
  Updating git-repo `https://github.com/Gnimuc/GLTF.jl.git`
Downloaded JSON2 ─ v0.1.3

julia> using PriviteProject

we just need to clone, cd and instantiate the project.

UPDATE: you might need to cd + activate instead of a single cd

8 Likes

Is the code that generates the general registry from METADATA public? How can I make my own registry from my own METADATA?

You can look at e.g. https://github.com/fredrikekre/Registry and clone it to .julia/registries.

3 Likes

That looks like a valid registry, but do you also have the code that generated it from a package directory?

No, I just made it. Some parts can be found here though: https://github.com/JuliaLang/Pkg.jl/tree/master/bin

Ok thanks!

How would you add JuAFEM using your registry?

I’ve tried the following.

(TestRegistry) pkg> registry add  https://github.com/fredrikekre/Registry.git
   Cloning registry from "https://github.com/fredrikekre/Registry.git"
     Added registry `Registry` to `~/.julia/registries/Registry`

(TestRegistry) pkg> add JuAFEM
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package JuAFEM [30d91d44]:
 JuAFEM [30d91d44] log:
 ├─JuAFEM [30d91d44] has no known versions!
 └─restricted to versions * by an explicit requirement — no versions left

I’m hoping to set up a private registry without needing to explicitly define a Versions.toml, which in my case would be over the top or prohibitive.

Edit

The following answers my original question.

(TestRegistry) pkg> add JuAFEM#master
   Cloning git-repo `https://github.com/KristofferC/JuAFEM.jl.git`