# Packages for "my own" code

**URL:** <https://discourse.julialang.org/t/packages-for-my-own-code/27129>\
**Category:** New to Julia\
**Created:** [August 3, 2019, 2:56pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129 "2019-08-03T14:56:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 3, 2019, 2:56pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/1 "2019-08-03T14:56:04Z")

</div>

I am developing code that I expect will be used only by myself for one or two projects (solving a specific model in economics). This depends on a few modules with reusable code (likely reusable mainly by myself for other projects).  
My question is: should I organize this code in packages?  
The potential benefit is automatic tracking of dependencies.  
But then, for different projects to track different versions of the reusable code, the packages would presumably have to be registered (?).  
I am reluctant to pollute the registry with many small packages that are likely of interest only to myself.  
If I don’t register the packages, how would my code track versions?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 3, 2019, 3:30pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/2 "2019-08-03T15:30:28Z")

</div>

> [@hendri54](#):
>
> But then, for different projects to track different versions of the reusable code, the packages would presumably have to be registered (?).

Not really, you can just `pkg> add` them directly from the URL. Just commit the `Manifest.toml` into the project.

---

<div class="post-metadata">

**Author:** ![pixel27](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pixel27/32/8902_2.png) [@pixel27](https://discourse.julialang.org/u/pixel27)\
**Post date:** [August 3, 2019, 3:50pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/3 "2019-08-03T15:50:48Z")

</div>

I have 2 “reusable” modules that I’m using between a few applications. Each of the reusable modules has it’s own GIT repository (currently just on my file system). For the application that uses a module I went into the package manager and did an: `add file:///home/pixel/julia/module1` which brought the module into my project.

When I make updates to module1, I commit them to git, then go back to my application, in the package manager do an update, that pulls in the newer version. (This is all under Atom so I then have to restart my julia console so it actually uses the new version, there are probably some commands to force this without the restart.)

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 3, 2019, 4:50pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/4 "2019-08-03T16:50:27Z")

</div>

Sorry - not quite following your reply.  
Are you suggesting to change the version number, commit to github, and then `Pkg` can pull the right (earlier) version when needed?  
Sorry if there is some documentation that spells all of this out that I am missing.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 3, 2019, 4:52pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/5 "2019-08-03T16:52:17Z")

</div>

Thanks - that’s pretty much what I am doing now (except via github rather than local storage).  
What I am looking for, though, is the ability to come back to the “main” project and have it pull the _old_ version of the “reusable” packages.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [August 3, 2019, 4:56pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/6 "2019-08-03T16:56:02Z")

</div>

AFAIK there is no way to version non-registered packages (in a way that Pkg is aware of the versions). You might be able to create your own registry though so that you’d register the packages there instead of Julia’s general one.

(Please, someone correct me if I am wrong)

---

<div class="post-metadata">

**Author:** ![pixel27](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pixel27/32/8902_2.png) [@pixel27](https://discourse.julialang.org/u/pixel27)\
**Post date:** [August 3, 2019, 6:18pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/7 "2019-08-03T18:18:02Z")

</div>

In [12. API Reference · Pkg.jl](https://julialang.github.io/Pkg.jl/v1/api/) Under `Pkg.add` it has:

```julia
Pkg.add(PackageSpec(url="https://github.com/JuliaLang/Example.jl", rev="master")) # From url to remote gitrepo

```

Which suggests if you have branches…maybe labels…you should be able to specify which version of the module you want.

PackageSpec has the following constructor:

```julia
PackageSpec(; name, url, path, rev, version, mode, level)

```

My uneducated guess is that version might reference a tag, if rev references a branch. The documentation doesn’t really describe how the values map to a git…

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 3, 2019, 6:57pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/8 "2019-08-03T18:57:09Z")

</div>

One option would be to create a personal package registry. @GunnarFarneback maintains a branch of Registrator.jl ([https://github.com/GunnarFarneback/Registrator.jl](https://github.com/GunnarFarneback/Registrator.jl)) that provides commands for managing a registry Git-repo locally. You can easily host such a registry on GitHub without running a Registrator server.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 3, 2019, 7:24pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/9 "2019-08-03T19:24:44Z")

</div>

Thank you for all the suggestions.  
Let me try to summarize the options as I understand them:

1. Create my own registry. Drawbacks: It is not clear to me from the documentation of [GitHub - GunnarFarneback/Registrator.jl](https://github.com/GunnarFarneback/Registrator.jl) how this would interact with the main registry. It is also not clear how I would access the registry on a remote machine (just `rsync` it?).
2. Create a local copy of each package each time it is reused. Then `add` the local path to each projects `Project.toml`. Seems workable but “inelegant.”
3. It _may_ be possible to specify non-registered package versions using `Pkg`, but it’s not clear whether / how that actually works.
4. Register each package in the general registry, which would create a large number of entries that are not meant for general reuse (each requiring a unique name and manual review for each update).

Am I missing something or this a _very_ common problem without a clear solution?

---

<div class="post-metadata">

**Author:** ![haberdashPI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/haberdashpi/32/26337_2.png) [@haberdashPI](https://discourse.julialang.org/u/haberdashPI)\
**Post date:** [August 3, 2019, 8:19pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/10 "2019-08-03T20:19:54Z")

</div>

I have found working with my own personal registry to work relatively seamlessly with the General registry. That is how I have dealt with this issue. Many thanks to @GunnarFarneback. To access the personal registry on a new machine you just `pkg"registry add [url]` it. That requires storing the registry somewhere (e.g. your github account). I think you could also just `rsync` it into the appropriate folder for registries, but I find it easier to have a place where the registry is synced from to make it easier to update later on.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 3, 2019, 9:52pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/11 "2019-08-03T21:52:06Z")

</div>

Thank you for that suggestion. Well, I now tried the personal registry, but without success.  
When I issues `register(MyPackage, registryPath)` I end up with an error:

```julia
HEAD is now at 5e6de9f update
ERROR: TOML.ParserError(150, 151, "unexpected character: `9`")

```

With enough effort, I’m sure one could track this down, but it’s not a promising start.

---

<div class="post-metadata">

**Author:** ![haberdashPI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/haberdashpi/32/26337_2.png) [@haberdashPI](https://discourse.julialang.org/u/haberdashPI)\
**Post date:** [August 4, 2019, 2:42am UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/12 "2019-08-04T02:42:17Z")

</div>

Are you sure it isn’t just a problem with your project’s files? I vaguely remember a similar issue and I think I fixed it by revising the project file for the package I was registering based on the line number in the error (in your case 150… I think?).

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 4, 2019, 11:43am UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/13 "2019-08-04T11:43:34Z")

</div>

I think 150 is a byte offset in `Registry.toml` (which is far shorter than 150 lines).  
I rebuilt the registry from scratch and get another parse error which suggests that the `UUID` is not valid.  
I don’t think I will try to debug this myself. Plan B for now is to see if I can make a private registry using `Pkg` (which the docs say is possible).  
Thanks for the suggestion, though.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 5, 2019, 11:05am UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/14 "2019-08-05T11:05:04Z")

</div>

By way of status update:  
I tried to make a private registry, but without success (“dirty registry” error; I probably have an inconsistency somewhere). This suggests to me that maintaining a registry by hand is too much work.  
The only approach that I can see then is to use `Pkg.develop` after each edit of the unregistered package dependencies to force a recompile.  
When it comes time to “freeze” the package versions of the dependencies, I suppose I could use `Pkg.develop` with the `local` option.  
Still hoping for a better solution…

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 5, 2019, 11:29am UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/15 "2019-08-05T11:29:32Z")

</div>

I have plenty of non-registered packages that are used by

```julia
] add https://github.com/PetrKryslUCSD/FinEtoolsDeforNonlinear.jl.git

```

No problem. That works smoothly.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 5, 2019, 2:55pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/16 "2019-08-05T14:55:26Z")

</div>

I have two problems with this approach:

1. If `A` uses `B`, how to get `A` to track changes in `B`? The only way I could find is `Pkg.develop(PackageSpec(B))`. Is there a better way?
2. At some point, `A` becomes a paper submitted for publications. Then I need to freeze the dependencies. How can I do this when `B` is not registered?

Thank you.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 5, 2019, 2:57pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/17 "2019-08-05T14:57:28Z")

</div>

To develop the package instead of adding it works fine.

If you keep the package up-to-date with the Project.toml and Manifest.toml, freezing the dependencies is not a problem. You simply freeze these two files.

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 5, 2019, 3:11pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/18 "2019-08-05T15:11:23Z")

</div>

When you say “freeze these two files” do you mean editing `[compat]` in `Manifest.toml` to fix the version of the dependency?  
The question is how `Pkg` can resolve versions when the package is not registered? I get the following error:

```julia
(TestPkg2LH) pkg> test
   Testing TestPkg2LH
 Resolving package versions...
ERROR: empty intersection between TestPkgLH@0.2.0 and project compatibility 0.1.8-0.1

```

---

<div class="post-metadata">

**Author:** ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)\
**Post date:** [August 5, 2019, 3:48pm UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/19 "2019-08-05T15:48:06Z")

</div>

Perhaps this is the way:

1. While editing the project, `develop` the unregistered dependencies. (I now realize that this changes the `url` in `Manifest.toml` to the local path.
2. To freeze the dependencies, remove and add them (which repoints `Manifest.toml` to a specific version on `github` with its commit `git-tree-sha1`).

Is this the way? Thanks again.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [August 6, 2019, 5:39am UTC](https://discourse.julialang.org/t/packages-for-my-own-code/27129/20 "2019-08-06T05:39:19Z")

</div>

You got it.

[Next page](https://discourse.julialang.org/t/packages-for-my-own-code/27129.md?page=2)