I want to add a dependency (called MyDependency) that I have privately developed to my Project.toml file for my main module (called URLPack) using MyDependency’s URL. When I go to the package manager using the ‘]’ character in the Julia REPL and enter “add ” for MyDependency, that works. However, if I attempt to manually put in the Project.toml under the [deps] section “MyDependency = ”, using the same URL, I get an “ERROR: Malformed value for MyDependency in deps section.”.
For the sake of this post and also to test using a URL to add a commonly used package, I’ve tried adding DataFrames.jl using it’s URL (which I hopefully have correct) in my Project.toml. I get the same malformed value error. The structure of the files/folders in my environment is as such (I ran the command “generate URLPack” in Julia’s package manager to create the environment): on the top level there is the Project.toml file and /src/ folder, then within the src/ folder there is a file called URLPack.jl.
module URLPack
using CSV
using JSON3
using DataFrames
greet() = print("Hello World!")
end # module URLPack
I’m wondering what the correct way is to specify within the Project.toml file a dependency that comes directly from a URL. Also, if anything needs to be done differently when adding the dependency from a URL that requires a GitHub username and Personal Access Token (PAT) for authentication, please mention too if you can.
Hopefully my example recreates the error I’m getting and I’ve given enough info, if you need more info from me please feel free to ask.
That’s not meant to be supported, Project.toml is not a registry. Perhaps you want to create a local registry? I’m not sure what problem are you trying to solve, though.
Thanks for the reply. After playing around a bit more I’ve seen that in the Manifest.toml file it tracks the URL (pretend its https://github.com/Me/MyDependency.git) that I used to add MyDependency to the URLPack project using Julia’s package manager. The reason I’ve had problems with cloning URLPack into other environments before is because I had the Manifest.toml file (but not the Project.toml file) in the .gitignore file so the URL for MyDependency no longer had a record.
Is the Project.toml file capable of specifying the URL to grab a direct dependency from, or is that typically done in the Manifest.toml file because the Project.toml file isn’t meant to be a registry? I thought that the Project.toml file was supposed to specify all of a projects direct dependencies such that when the “instantiate” command was run in the project environment the Manifest.toml file could be built for the direct and indirect dependencies off of what the Project.toml file specified. My thinking is that if the URL for an unregistered package like MyDependency was in the Project.toml file, it could still build the Manifest.toml file with the “instantiate” command in Julia’s package manager rather than me going add https://github.com/Me/MyDependency.git, unless it’s not possible to specify URL’s in the Project.toml file (ChatGPT gave me hope but none of it’s suggestions for adding a URL to the Project.toml file worked).
Sorry to have not mentioned the .gitignore file at first, I figured it wouldn’t be relevant but after playing around now and seeing that my local .zip and .tar files (which would have included the Manifest.toml file) worked even with MyDependency as a dependency for URLPack but the clone from Git didn’t, I then remembered that the Manifest.toml file was ignored from my repo for the URLPack project.
The Manifest.toml is not usually meant to shared in any way. It fully describes the environment of a package, though, so sharing it is good when complete reproducibility is necessary.
Anyway, as implied above, the proper “fix”, taking your requests at face value, would be to create a local registry, something that’s rarely done. I still don’t understand why is it you want this, so I suppose this is an instance of an XY problem:
It is not clear to me which problem you are trying to solve.
If you have a public package you can register it and the problem is solved.
If you have a private package you can add something like this to scripts that
are using your private package:
using Pkg
if ! ("ControlPlots" ∈ keys(Pkg.project().dependencies))
using TestEnv; TestEnv.activate()
pkg"add https://github.com/aenarete/ControlPlots.jl"
end
This assumes that you have the package TestEnv in your global environment.
The role of instantiate is to download all packages as specified in Manifest.toml. To create a Manifest.toml from a Project.toml is a job for resolve. The latter usually has many possible solutions and may vary over time, so for reproducibility you need to ship Manifest.toml.
Thank you for the replies. Removing the Manifest.toml from my .gitignore file has solved the problem because now the URL for MyDependency is tracked somewhere on the repo. If I want to put the Manifest.toml back into the .gitignore and find another solution, I’ll attempt to add MyDependency within a script in the codebase as was suggested in the post and see how that goes.
I narrowed too early in on specifying the URL in the Project.toml because I thought it was doable to use a URL and not a UUID off of what ChatGPT said. That’s my mistake for communicating the problem poorly and not understanding properly the difference between the Project.toml file v.s. the Manifest.toml file.
