# More convenient way to develop dependent package?

**URL:** <https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206>\
**Category:** General Usage\
**Created:** [February 11, 2022, 3:55am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206 "2022-02-11T03:55:01Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 11, 2022, 3:55am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/1 "2022-02-11T03:55:01Z")

</div>

I’ve run into this problem alot over the years, and finally figured I’d ask if there’s not a solution I’m missing.

The setup is that I have some base package which I’m developing, call it `BasePackage`. Then I’m working on a project which uses it, call it `MainProject`. This is an active “top-level” project with some colleagues so I want everything exactly reproducible so this has a `Manifest.toml` checked into the repo so everyone can just `instantiate` and go.

As I work on `MainProject`, I may discover changes I need to make to `BasePackage`. When this happens, I have to `dev BasePackage` into my `MainProject` working directory, make the changes, commit them, then look up the commit hash for that commit, then `add BasePackage#that_hash` so the `Manifest.toml` in `MainProject` records the new desired commit.

Its this last set of steps that’s always a pain. Obviously as thing stabilize this happens less, but I’m tending to work in a fast developing code-base, so to me this is super frequent. Worse, constantly switching between `add`ing and `dev`ing `BasePackage` screws up `Revise` so I have to restart my session every time.

Is there maybe some better way to do this?

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [February 11, 2022, 4:19am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/2 "2022-02-11T04:19:09Z")

</div>

Is the repo for `BasePackage` public? Perhaps you could add `BasePackage#master` to your `Project.toml` file, and have users run `up` everytime they instantiate the project? This will update the `Manifest` accordingly and automatically.

(I think this should work, but I am no expert in package management).

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [February 11, 2022, 8:15am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/3 "2022-02-11T08:15:57Z")

</div>

I regularly have the same situation at work with the difference that `BasePackage` is registered in a private registry. My workflow is approximately the same:

1. `dev BasePackage` in `MainProject`.
2. Make changes in `BasePackage` and bump the version number in `Project.toml`.
3. Commit, push, and let CI register the new version of `BasePackage`. (\*)
4. Update to latest version of `BasePackage` in `MainProject`.

(\*) Before setting up CI for this I did it manually, typically by

```julia
$ cd BasePackage
$ julia
julia> using LocalRegistry # from global environment
julia> register()

```

At any rate I recommend setting up your own registry if you’re sharing one or more internal packages with colleagues. [LocalRegistry](https://github.com/GunnarFarneback/LocalRegistry.jl) is designed to make this easy.

---

<div class="post-metadata">

**Author:** ![melonedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/melonedo/32/15503_2.png) [@melonedo](https://discourse.julialang.org/u/melonedo)\
**Post date:** [February 11, 2022, 8:23am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/4 "2022-02-11T08:23:50Z")

</div>

Since you are developing based on Git repos, I suppose simply publishing your desired version of `BaseProject` under a `stable` branch, and let your `MainProject` depend on that branch could do the work.

You can have a local copy of `MainProject` which uses a `dev`ed version of `BaseProject` and also a copy of `MainProject` which tracks `BaseProject.git#stable`. The second copy is only for confirming whether your changes have taken place. In `BaseProject`, you can edit as you like, which will be immediately seen by any project that has `dev`ed it. When you are satisfied, commit your final result to the `stable` branch, so that your work will be visible to any project that tracks the `stable` branch after it is `update`d.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 11, 2022, 8:33am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/5 "2022-02-11T08:33:28Z")

</div>

Thanks, unfortunately the issue with tracking `BasePackage#stable` or `BasePackage#master` or something is that it ceases to be reproducible, since for any past commits in `MainProject` I won’t know what the right working version of `BasePackage` was. It also doesn’t really solve the issue that I need to keep switching between `add`ing and `dev`ing it, if I understand the suggestion (my situation is that upgrades to `BasePackage` are happening in unison with `MainProject`, so its not just an external dependency that occasionally is updating independently).

---

<div class="post-metadata">

**Author:** ![melonedo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/melonedo/32/15503_2.png) [@melonedo](https://discourse.julialang.org/u/melonedo)\
**Post date:** [February 11, 2022, 8:43am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/6 "2022-02-11T08:43:45Z")

</div>

> [@marius311](#):
>
> since for any past commits in `MainProject` I won’t know what the right working version of `BasePackage` was.

If you do `]add BaseProject#stable`, then Manifest.toml will record the commit that it picks. Upload it so that each version of MainProject records what commit it uses. As long as the git repo is accessible, any one can instantiate the environment with the Manifest.toml.

With two copies of MainProject, my suggested workflow would be:

1. Edit BaseProject, and test in the `dev`ed MainProject.
2. Commit the changes to `stable` branch.
3. `Pkg.update` in the MainProject which tracks `BaseProject.git#stable`. Now Manifest.toml records the commit in the last step.
4. Commit Manifest.toml of MainProject. Optionally test before you do so.

For any one who is working on MainProject, run `Pkg.instantiate` after they have checked a commit to ensure they use the right version of BaseProject.

---

<div class="post-metadata">

**Author:** ![lawless-m](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lawless-m/32/30869_2.png) [@lawless-m](https://discourse.julialang.org/u/lawless-m)\
**Post date:** [February 11, 2022, 11:02am UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/7 "2022-02-11T11:02:29Z")

</div>

If you do this

```julia
pkg> add "/path/to/Basepackage"

```

It gets added to the Project using the UUID

and written to the Manifest using the PATH

and then don’t share the Manifest

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 11, 2022, 6:43pm UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/8 "2022-02-11T18:43:30Z")

</div>

> [@melonedo](#):
>
> If you do `]add BaseProject#stable` , then Manifest.toml will record the commit that it picks.

Ah! This I had not appreciated, thanks, yea that can save me the looking up the hash by hand. But what happens if the user pulls a new version of `MainPackage` but forgets `instantiate`? Error message, or silently use the old version, or automatic update?

Your suggested workflow with two copies of `MainProject` is interesting. I guess the issue will be if I change files in `MainProject` simultaneously with `BasePackage`. The former I’d have to commit from the `dev`’ed copy (since that’s presumably my main working directory), and then in a separate commit I’d `Pkg.update` from the `#stable` copy? Still seems not totally ideal.

> [@GunnarFarneback](#):
>
> At any rate I recommend setting up your own registry

This for now is not an issue for me, they’re all public packages.

> [@lawless-m](#):
>
> and then don’t share the Manifest

Unfortunately this I think is a requirement since I want every commit fully reproducible.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [February 11, 2022, 6:53pm UTC](https://discourse.julialang.org/t/more-convenient-way-to-develop-dependent-package/76206/9 "2022-02-11T18:53:28Z")

</div>

> [@marius311](#):
>
> Ah! This I had not appreciated, thanks, yea that can save me the looking up the hash by hand. But what happens if the user pulls a new version of `MainPackage` but forgets `instantiate` ? Error message, or silently use the old version, or automatic update?

Don’t let your users do any `Pkg` operations at all and give them scripts which include instantiation. I start mine with

```julia
using Pkg
Pkg.activate(@ __DIR__ )
Pkg.instantiate()

```