# Travis CI script does not clone Project.toml of dependent packages

**URL:** <https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709>\
**Category:** General Usage\
**Created:** [October 10, 2019, 3:04am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709 "2019-10-10T03:04:39Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 10, 2019, 3:04am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/1 "2019-10-10T03:04:39Z")

</div>

I have two packages developed myself. Say they are `MyPkg` and `MySubPkg`. `MyPkg` uses `MySubPkg`, and both are pushed to GitHub.

Both packages have `.travis.yml` defined for running unit tests automatically by Travis CI when new code is pushed to GitHub. The default Travis CI script (which is called when the `script:` section is not defined in `.travis.yml`) runs fine for `MySubPkg`, but it generates the following error for `MyPkg`:

```julia
...
   Testing MyPkg
 Resolving package versions...
ERROR: could not find project file for MySubPkg at /home/travis/.julia/packages/MySubPkg/EgBdX

```

It was not clear to me what “`project file`” means in this error message, but I guessed it would be the `Project.toml` file of `MySubPkg`. Indeed, `Project.toml` of a dependent package does not seem cloned in some cases.

For example, if I remove `~/.julia` for a fresh start, start REPL in the directory of `MyPkg`, and perform `pkg> activate .` (with the period) followed by `(MyPkg) pkg> instantiate`, then it starts installing the dependent packages as defined in `Project.toml` of `MyPkg`. Because `MySubPkg` is one of the dependent packages, `MySubPkg` is cloned in the `~/.julia/packages/MySubPkg/(some random string)/` directory. However, this cloned directory does not contain `Project.toml` of `MySubPkg`.

In contrast, if I explicitly add `MySubPkg` in REPL by `pkg> add https://github.com/MY_GITHUB_ID/MySubPkg.jl`, then `Project.toml` of `MySubPkg` is correctly cloned in the `~/.julia/packages/MySubPkg/(some random string)/` directory.

Why is `Project.toml` not cloned in some cases? How can I make this file cloned while the Travis CI script runs? Or, is this not the cause of the above error? In that case, how can I resolve the above error?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [October 10, 2019, 6:07am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/2 "2019-10-10T06:07:47Z")

</div>

It would be much easier to help with this if you linked the actual repositories.

---

<div class="post-metadata">

**Author:** ![abulak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abulak/32/28314_2.png) [@abulak](https://discourse.julialang.org/u/abulak)\
**Post date:** [October 10, 2019, 8:28am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/3 "2019-10-10T08:28:20Z")

</div>

I workaround this issue by adding

```julia
script:
  - julia -e 'using Pkg; pkg"add https://github.com/MY_GITHUB_ID/MySubPkg.jl"; Pkg.build(); Pkg.test(coverage=true);'

```

to `.travis.yaml`, but i’d be glad to see a true solution

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [October 10, 2019, 8:32am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/4 "2019-10-10T08:32:18Z")

</div>

For now that is _the_ solution. Might change in the future though, see e.g. [Automatically detect unregistered dependencies by 00vareladavid · Pull Request #1088 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/pull/1088)

---

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 10, 2019, 12:31pm UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/5 "2019-10-10T12:31:09Z")

</div>

Thanks @abulak and @fredrikekre. I tried @abulak’s suggestion, and now I don’t get the error complaining about the missing project file. However, I get a new error:

```julia
ERROR: path /home/travis/.julia/packages/MySubPkg/(random string) for package MySubPkg no longer exists. Remove the package or `develop` it at a new path

```

So I tried to replace `pkg"add ..."` with `pkg"dev ..."`, but I still get the same error.

Any suggestions? Maybe related to [https://github.com/JuliaLang/Pkg.jl/issues/821](https://github.com/JuliaLang/Pkg.jl/issues/821)?

---

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 10, 2019, 4:52pm UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/6 "2019-10-10T16:52:00Z")

</div>

More information. In fact, I have three packages: `MyPkg` and `MySubPkg` and `HisSubPkg`. `MyPkg` uses the other two packages. None of these are registered packages. Now, my Travis CI script looks like

```julia
script:
  - julia --color=yes -e 'using Pkg; pkg"add https://github.com/MY_GITHUB_ID/MySubPkg.jl"'
  - julia --color=yes -e 'using Pkg; pkg"add https://github.com/HIS_GITHUB_ID/HisSubPkg.jl"'
  - julia --project --color=yes -e 'using Pkg; Pkg.build(); Pkg.test(coverage=true)'

```

In fact, this completes the unit tests. However, very strangely, the first line that adds `MySubPkg` generates an error about `HisSubPkg`:

`ERROR: path /home/travis/.julia/packages/HisSubPkg/(random string) for package HisSubPkg no longer exists. Remove the package or `develop` it at a new path`.

If I swap the first and second line of the scripts to install `HisSubPkg` first and `MySubPkg` next, then the new first line that add `HisSubPkg` generates an error about `MySubPkg`:

`ERROR: path /home/travis/.julia/packages/MySubPkg/(random string) for package MySubPkg no longer exists. Remove the package or `develop` it at a new path`.

It is very strange that the command adding one package complains about not the package it is adding, but about the other package. These two packages `MySubPkg` and `HisSubPkg` do not have interdependency: they are separate, independent packages.

To be more specific, these are actual packages:

- `MyPkg`: [https://github.com/wsshin/MaxwellFDM.jl](https://github.com/wsshin/MaxwellFDM.jl)
- `MySubPkg`: [https://github.com/wsshin/StaggeredGridCalculus.jl](https://github.com/wsshin/StaggeredGridCalculus.jl)
- `HisSubPkg`: [https://github.com/stevengj/GeometryPrimitives.jl](https://github.com/stevengj/GeometryPrimitives.jl)

and you can see the Travis CI job log here: [Travis CI - Test and Deploy Your Code with Confidence](https://travis-ci.org/wsshin/MaxwellFDM.jl/jobs/596203178).

Any help will be greatly appreciated!

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [October 10, 2019, 6:29pm UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/7 "2019-10-10T18:29:47Z")

</div>

Ah, you have commited the Manifest.toml I guess. You could just use `instantiate` then and everything should work normally.

---

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 10, 2019, 7:55pm UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/8 "2019-10-10T19:55:20Z")

</div>

I still get the same error even if I do `Pkg.instantiate()` before `Pkg.build()`.

However, I solved this issue by a different way. I noticed that AppVeyor’s default script succeeds without errors. Looking at AppVeyor’s job log, I was able to pick up its default scripts, which are very simply

```julia
julia -e 'using Pkg; Pkg.build()'
julia -e 'using Pkg; Pkg.test(coverage=true)'

```

By specifying these scripts in `.travis.yml`, I was able to make the Travis CI job succeeds. I’m not sure why the above is not the default scripts of Travis CI.

For other people who experience the same problem, this is the `script:` section of my `.travis.yml`:

```julia
script:
  - julia --color=yes -e 'using Pkg; Pkg.build()'
  - julia --color=yes -e 'using Pkg; Pkg.test(coverage=true)'

```

I have `Project.toml` and `Manifest.toml` in all the three unregistered packages involved: `MyPkg`, `MySubPkg`, and `HisSubPkg`, and I am testing only in the Julia nightly build (currently 1.4.0-DEV.285).

---

<div class="post-metadata">

**Author:** ![wsshin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wsshin/32/360_2.png) [@wsshin](https://discourse.julialang.org/u/wsshin)\
**Post date:** [October 11, 2019, 4:51am UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/9 "2019-10-11T04:51:01Z")

</div>

Weird. I just tested Travis CI’s default script by removing the `script: ` section in `.travis.yml`, and it works. The default script was not working at the point I asked the original question. I think I made some changes that made the default script work, and that is why AppVeyor’s default script started working above.

---

<div class="post-metadata">

**Author:** ![findmyway](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/findmyway/32/4946_2.png) [@findmyway](https://discourse.julialang.org/u/findmyway)\
**Post date:** [October 16, 2019, 5:45pm UTC](https://discourse.julialang.org/t/travis-ci-script-does-not-clone-project-toml-of-dependent-packages/29709/10 "2019-10-16T17:45:01Z")

</div>

> [@fredrikekre](#):
>
> You could just use `instantiate` then and everything should work normally.

This saved my day!