# Adding packages to Project.toml file using package URL

**URL:** <https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643>\
**Category:** General Usage\
**Tags:** question, package, registry\
**Created:** [April 30, 2024, 8:31am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643 "2024-04-30T08:31:00Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Name1](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@Name1](https://discourse.julialang.org/u/Name1)\
**Post date:** [April 30, 2024, 8:31am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/1 "2024-04-30T08:31:00Z")

</div>

Hi all,

I want to add a dependency (called MyDependency) that I have privately developed to my Project.toml file for my main module (called URLPack) using MyDependency’s URL. When I go to the package manager using the ‘]’ character in the Julia REPL and enter “add ” for MyDependency, that works. However, if I attempt to manually put in the Project.toml under the [deps] section “MyDependency = ”, using the same URL, I get an “ERROR: Malformed value for `MyDependency` in `deps` section.”.

For the sake of this post and also to test using a URL to add a commonly used package, I’ve tried adding DataFrames.jl using it’s URL (which I hopefully have correct) in my Project.toml. I get the same malformed value error. The structure of the files/folders in my environment is as such (I ran the command “generate URLPack” in Julia’s package manager to create the environment): on the top level there is the Project.toml file and /src/ folder, then within the src/ folder there is a file called URLPack.jl.

The Project.toml file looks like this:

```julia
name = "URLPack"
uuid = "426469bc-1635-46be-b45b-3f9bb9408c73"

[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
DataFrames = "https://github.com/JuliaData/DataFrames.jl.git"
JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"

```

The URLPack.jl file looks like this:

```julia
module URLPack

using CSV
using JSON3
using DataFrames

greet() = print("Hello World!")

end # module URLPack

```

I’m wondering what the correct way is to specify within the Project.toml file a dependency that comes directly from a URL. Also, if anything needs to be done differently when adding the dependency from a URL that requires a GitHub username and Personal Access Token (PAT) for authentication, please mention too if you can.

Hopefully my example recreates the error I’m getting and I’ve given enough info, if you need more info from me please feel free to ask.

Thank you 🙂

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [April 30, 2024, 9:03am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/2 "2024-04-30T09:03:13Z")

</div>

That’s not meant to be supported, Project.toml is not a registry. Perhaps you want to create a local registry? I’m not sure what problem are you trying to solve, though.

---

<div class="post-metadata">

**Author:** ![Name1](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@Name1](https://discourse.julialang.org/u/Name1)\
**Post date:** [May 1, 2024, 3:05am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/3 "2024-05-01T03:05:01Z")

</div>

Thanks for the reply. After playing around a bit more I’ve seen that in the Manifest.toml file it tracks the URL (pretend its `https://github.com/Me/MyDependency.git`) that I used to add MyDependency to the URLPack project using Julia’s package manager. The reason I’ve had problems with cloning URLPack into other environments before is because I had the Manifest.toml file (but not the Project.toml file) in the .gitignore file so the URL for MyDependency no longer had a record.

Is the Project.toml file capable of specifying the URL to grab a direct dependency from, or is that typically done in the Manifest.toml file because the Project.toml file isn’t meant to be a registry? I thought that the Project.toml file was supposed to specify all of a projects direct dependencies such that when the “instantiate” command was run in the project environment the Manifest.toml file could be built for the direct and indirect dependencies off of what the Project.toml file specified. My thinking is that if the URL for an unregistered package like MyDependency was in the Project.toml file, it could still build the Manifest.toml file with the “instantiate” command in Julia’s package manager rather than me going `add https://github.com/Me/MyDependency.git`, unless it’s not possible to specify URL’s in the Project.toml file (ChatGPT gave me hope but none of it’s suggestions for adding a URL to the Project.toml file worked).

Sorry to have not mentioned the .gitignore file at first, I figured it wouldn’t be relevant but after playing around now and seeing that my local .zip and .tar files (which would have included the Manifest.toml file) worked even with MyDependency as a dependency for URLPack but the clone from Git didn’t, I then remembered that the Manifest.toml file was ignored from my repo for the URLPack project.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [May 1, 2024, 4:28am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/4 "2024-05-01T04:28:25Z")

</div>

> [@Name1](#):
>
> Is the Project.toml file capable of specifying the URL to grab a direct dependency from

See my previous message.

> [@Name1](#):
>
> is that typically done in the Manifest.toml file

The Manifest.toml is not usually meant to shared in any way. It fully describes the environment of a package, though, so sharing it is good when complete reproducibility is necessary.

Anyway, as implied above, the proper “fix”, taking your requests at face value, would be to create a local registry, something that’s rarely done. I still don’t understand why is it you want this, so I suppose this is an instance of an XY problem:

> **[XY problem](https://en.wikipedia.org/wiki/XY_problem)**
>
> The XY problem is a communication problem encountered in help desk, technical support, software engineering, or customer service situations where the question is about an end user's attempted solution (X) rather than the root problem itself (Y or Why?).
> The XY problem obscures the real issues and may even introduce secondary problems that lead to miscommunication, resource mismanagement, and sub-par solutions. The solution for the support personnel is to ask probing questions as to why the infor...

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [May 1, 2024, 6:24am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/5 "2024-05-01T06:24:23Z")

</div>

It is not clear to me which problem you are trying to solve.

If you have a public package you can register it and the problem is solved.

If you have a private package you can add something like this to scripts that  
are using your private package:

```julia
using Pkg
if ! ("ControlPlots" ∈ keys(Pkg.project().dependencies))
    using TestEnv; TestEnv.activate()
    pkg"add https://github.com/aenarete/ControlPlots.jl"
end

```

This assumes that you have the package `TestEnv` in your global environment.

Perhaps you do not need TestEnv, just try it out.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [May 1, 2024, 6:49am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/6 "2024-05-01T06:49:03Z")

</div>

It has never been possible to write a URL in place of a UUID but I believe [Support a `[sources]` section in Project.toml for specifying paths and repo locations for dependencies by KristofferC · Pull Request #3783 · JuliaLang/Pkg.jl · GitHub](https://github.com/JuliaLang/Pkg.jl/pull/3783) will help you do what you want.

> [@Name1](#):
>
> I thought that the Project.toml file was supposed to specify all of a projects direct dependencies such that when the “instantiate” command was run in the project environment the Manifest.toml file could be built for the direct and indirect dependencies off of what the Project.toml file specified.

The role of `instantiate` is to download all packages as specified in `Manifest.toml`. To create a `Manifest.toml` from a `Project.toml` is a job for `resolve`. The latter usually has many possible solutions and may vary over time, so for reproducibility you need to ship `Manifest.toml`.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [May 1, 2024, 6:54am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/7 "2024-05-01T06:54:46Z")

</div>

> [@nsajko](#):
>
> would be to create a local registry, something that’s rarely done

I don’t know about rare but at any rate it’s not particularly difficult with my [LocalRegistry](https://github.com/GunnarFarneback/LocalRegistry.jl) package.

---

<div class="post-metadata">

**Author:** ![Name1](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@Name1](https://discourse.julialang.org/u/Name1)\
**Post date:** [May 2, 2024, 7:45am UTC](https://discourse.julialang.org/t/adding-packages-to-project-toml-file-using-package-url/113643/8 "2024-05-02T07:45:58Z")

</div>

Hi all,

Thank you for the replies. Removing the Manifest.toml from my .gitignore file has solved the problem because now the URL for MyDependency is tracked somewhere on the repo. If I want to put the Manifest.toml back into the .gitignore and find another solution, I’ll attempt to add MyDependency within a script in the codebase as was suggested in the post and see how that goes.

I narrowed too early in on specifying the URL in the Project.toml because I thought it was doable to use a URL and not a UUID off of what ChatGPT said. That’s my mistake for communicating the problem poorly and not understanding properly the difference between the Project.toml file v.s. the Manifest.toml file.

Thank you all!