# "pkg\> add ..." overwrites Manifest.toml changes made for developed packages

**URL:** <https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541>\
**Category:** General Usage\
**Created:** [June 8, 2018, 3:24pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541 "2018-06-08T15:24:32Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 8, 2018, 3:24pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/1 "2018-06-08T15:24:32Z")

</div>

I did `pkg> develop` for a couple of packages to work on making them compatible with 0.7. This included adding new dependencies (like say `Nullables`). However, doing a new `pkg> add ...` will wipe all the changes made to the developed packages deps. I suggest that `pkg> add` checks what packages are being developed and merges their `Manifest.toml` entries instead of overwriting them.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [June 9, 2018, 9:19am UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/2 "2018-06-09T09:19:25Z")

</div>

Could you elaborate a bit. It is not clear to me what is being overwritten and from where.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 9, 2018, 10:24am UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/3 "2018-06-09T10:24:09Z")

</div>

Sure. I’m working on developing a few of my packages, to upgrade them for v0.7. They have dependencies which have not yet been updated to support v0.7. Take for example `DateParser` which uses `Nullables`.

Remember that I don’t work on a project - I just want to get my packages to load into the Julia v0.7 REPL. So I start the REPL and do a `using Genie` and then debug the errors. Thus I’m talking about the global `~/.julia/environments/v0.7/Manifest.toml` file.

And let’s say I get an error about `Nullables` in `DateParser`. I go into `pkg>` and do `develop DateParser`. I then edit the source file of `DateParser` and add `using Nullables`. But I also have to add `Nullables` to the `deps` list in `Manifest.toml`.

So the corresponding section of `Manifest.toml` now looks like this:

```julia
[[DateParser]]
deps = ["Dates", "Nullables", "TimeZones"]
path = "/Users/adrian/.julia/dev/DateParser"
uuid = "c513aa4d-7626-5656-a882-4604be010eff"
version = "0.1.1+"

```

Then I happily get back to work - and at some point I realize that I need to add a new package. Say `OhMyREPL`. So I go back to the REPL and run:

```julia
(v0.7) pkg> add OhMyREPL
  Updating registry at `~/.julia/registries/Uncurated`
  Updating git-repo `https://github.com/JuliaRegistries/Uncurated.git`
 Resolving package versions...
  Updating `~/.julia/environments/v0.7/Project.toml`
  [5fb14364] + OhMyREPL v0.2.11
  Updating `~/.julia/environments/v0.7/Manifest.toml`

```

Now `Nullables` is gone from `DateParser`s list of `deps` and my code breaks:

```julia
[[DateParser]]
deps = ["Dates", "TimeZones"]
path = "/Users/adrian/.julia/dev/DateParser"
uuid = "c513aa4d-7626-5656-a882-4604be010eff"
version = "0.1.1+"

```

Note that I’ve also updated the `REQUIRE` file for `DateParser`, but that doesn’t seem to help either:

```julia
julia 0.7
TimeZones
Nullables

```

P.S. - thanks for all your hard work, I’m getting the hangs of Pkg3 and it’s pretty cool!

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 9, 2018, 10:30am UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/4 "2018-06-09T10:30:01Z")

</div>

Hmmm, sorry… I figured it out. Actually updating the `REQUIRE` does help – but I updated the wrong `REQUIRE`. Instead of updating the one from `~/.julia/dev/DateParser`, I edited the one in `~/.julia/packages/DateParser`.

Sorry for the false alarm. It takes a bit of getting used to the ways of Pkg3 and it’s more of a discovery process.

So I guess my only comment would be to have some sort of upgrade workflow in the docs and mention that the `REQUIRE` file also needs to be updated, in the developed package, until fully migrated to Pkg3? I can probably write that, I think I now understand how it should be done.

* * *

Or rather, when doing `pkg> develop` why doesn’t Pkg3 update the cloned package? By adding the corresponding `Manifest.toml` file, to force an upgrade and get people to use the new system?

* * *

Update:  
I probably still don’t get what’s the update package workflow. I just done `pkg> develop` on my own `Genie` as I’m working to migrate from `HttpServer` to `HTTP`. So I’ve done `pkg> add HTTP` in the `Genie` project (in `~/.julia/dev/Genie`) and added `using HTTP` in the files but the dependency is still not satisfied:

```julia
ERROR: LoadError: LoadError: LoadError: ArgumentError: Module HTTP not found in current path.
Run `Pkg.add("HTTP")` to install the HTTP package.
require(::Module, ::Symbol) at ./loading.jl:868
include at ./boot.jl:314 [inlined]
include_relative(::Module, ::String) at ./loading.jl:1071
include(::Module, ::String) at ./sysimg.jl:29
include(::String) at /Users/adrian/.julia/dev/Genie/src/Genie.jl:4
top-level scope
include at ./boot.jl:314 [inlined]
include_relative(::Module, ::String) at ./loading.jl:1071
_require(::Base.PkgId) at ./loading.jl:997
require(::Base.PkgId) at ./loading.jl:878
require(::Module, ::Symbol) at ./loading.jl:873
eval at ./boot.jl:316 [inlined]
(::getfield(Distributed, Symbol("##164#166")){Module,Expr})() at ./task.jl:254
in expression starting at /Users/adrian/.julia/dev/Genie/src/Cookies.jl:6
in expression starting at /Users/adrian/.julia/dev/Genie/src/Genie.jl:39
Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:221
 [2] macro expansion at ./task.jl:240 [inlined]
 [3] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Distributed/src/macros.jl:206
 [4] top-level scope at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Distributed/src/macros.jl:190
 [5] include at ./boot.jl:314 [inlined]
 [6] include_relative(::Module, ::String) at ./loading.jl:1071
 [7] include(::Module, ::String) at ./sysimg.jl:29
 [8] exec_options(::Base.JLOptions) at ./client.jl:251
 [9] _start() at ./client.jl:427
in expression starting at /Users/adrian/Dropbox/Projects/HotelHound/hotel-hound/genie.jl:190

```

* * *

Update 2  
I had to add `HTTP` as a dependency of `Genie` into the main `Manifest.toml` file to get it to work… Oh dear.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [June 9, 2018, 11:12am UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/5 "2018-06-09T11:12:41Z")

</div>

Don’t edit the Manifest by hand.

> [@essenciary](#):
>
> So I’ve done `pkg> add HTTP` in the `Genie` project (in `~/.julia/dev/Genie` ) and added `using HTTP` in the files but the dependency is still not satisfied:

At this point, the current Manifest.toml has not had a chance to update. In other words, the Manifest.toml which list all the dependencies are out of sync with the actual dependencies. You can do `dev Genie` again to cause the manifest to update, or something like `update --fixed`. This is a common confusion and we are thinking about better ways to do this.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 9, 2018, 11:27am UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/6 "2018-06-09T11:27:34Z")

</div>

That worked, but now something else has happened.

1 - I’ve initially done `pkg> develop Genie`. No `Project.toml` nor `Manifest.toml` files were created for `Genie` dev.  
2 - I’ve run `pkg> init` in `Genie` dev and got an empty `Project.toml`  
3 - I’ve added a package (say `Nullables`). The requirements were not imported from `REQUIRE` into `Project.toml` and `Manifest.toml` – only `Nullables` was added to `Genie` dev `Project.toml` and `Manifest.toml`.  
4 - now I’ve done `pkg> develop Genie` again to sync `Genie`’s dev deps with master `Manifest` and it wiped off all the dependencies that were initially imported from `REQUIRE` into the master `Manifest`.

* * *

So I think my previous comment is even more important here: when doing `pkg> develop` it should automatically create the `Project.toml` and `Manifest.toml` and set them up with the deps from `REQUIRE`. Then ideally remove the `REQUIRE` file so only one source of deps is left.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [June 9, 2018, 3:40pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/7 "2018-06-09T15:40:37Z")

</div>

> [@essenciary](#):
>
> I’ve initially done `pkg> develop Genie` . No `Project.toml` nor `Manifest.toml` files were created for `Genie` dev.

That’s not within the functionality of develop to do. Upgrading existing packages to have Project file is a wip (what is left is OS-specific deps and then everything is in place)

> [@essenciary](#):
>
> I’ve added a package (say `Nullables` ). The requirements were not imported from `REQUIRE` into `Project.toml` and `Manifest.toml` – only `Nullables` was added to `Genie` dev `Project.toml` and `Manifest.toml` .

That seems correct, what did you expect to happen?

> [@essenciary](#):
>
> now I’ve done `pkg> develop Genie` again to sync `Genie` 's dev deps with master `Manifest` and it wiped off all the dependencies that were initially imported from `REQUIRE` into the master `Manifest` .

Yes because you created a project file in Genie which take precedence over the REQUIRE file.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 9, 2018, 4:43pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/8 "2018-06-09T16:43:55Z")

</div>

> [@kristoffer.carlsson](#):
>
> That seems correct, what did you expect to happen?

I would’ve liked that:  
a. `Project.toml` and `Manifest.toml` to be created. Since we’re migrating to the new Pkg, what’s the value in _not_ creating them?  
b. after creating them, to import the dependencies from `REQUIRE` into `Project.toml` - `REQUIRE` is the current authoritative source for the package’s dependencies. What is the benefit in _not_ doing it?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [June 9, 2018, 6:47pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/9 "2018-06-09T18:47:26Z")

</div>

> [@essenciary](#):
>
> What is the benefit in _not_ doing it?

None, which is why I said that there is WIP to create them for you. In order to do this properly though we need to support in the Project file what the current package manager support. What is left is OS-specific dependencies. After that is in, the next task is to have some upgrade function that does exactly what you want. Create a Project file based on the REQUIRE + test/REQUIRE files.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 9, 2018, 8:09pm UTC](https://discourse.julialang.org/t/pkg-add-overwrites-manifest-toml-changes-made-for-developed-packages/11541/10 "2018-06-09T20:09:25Z")

</div>

Awesome, thank you!

P.S. - a lot is lost when communicating in writing, so it’s not like I’m “demanding” things, just providing feedback. I know you guys have looked at all the angles but the users end up employing the APIs in completely unexpected manners. I respect and appreciate your work and I’m looking forward to Pkg3 maturing.