Can't get local package to allow inclusion of a new package

General Usage
1

I have a package locally. It is a git repo, but it is not shared to github. It is what S. Karpinski casually but usefully defined as an “application”: a collection of Julia source files to do some self-contained task, in contrast a building block to be used (potentially) broadly in many people’s work. This is useful distinction. As users of Julia, not developers or maintainers of Julia itself or the many great, crucial packages the community has made available, Pkg can be a complex tool that meets the needs of the latter group with essential capabilities and sometimes befuddles us mere “users”.

We little users mostly want to access packages and get the manifold benefits for our work. We might develop a package to contribute to the community, but more likely we develop a personal (or small group) package as the best way to wrap a module so it’s easy to use as our own local package(s). And Package works for this simpler need, too–but with complications.

So, I get this message because I added a new using <a package> to the module definition of my local package. But, it is hard to get that into project.toml and manifest.toml. The error message below recommends Pkg.resolve() and/or Pkg.instantiate(). I have had this work once but under circumstances I don’t understand. Twice I have had to go ahead and manually modify the project files, to add as a dependency with its uuid (which I lookup in my .julia directory), which I should really never do. (I don’t touch manifest.toml for my local package). But, it is the only way around the problem that I have found. Done carefully, it solves the problem.

But it is not the proper way and I’d rather do it the proper way if I knew what that was and i could get it to work. So–this is my question: what and how?

You are probably familiar with error I get, but here it is:

PkgPrecompileError: The following 1 direct dependency failed to precompile:

GeneralNN 

Failed to precompile GeneralNN [47e0c118-78ee-11ef-3e02-d9224f55ee60] to "/Users/lewis/.julia/compiled/v1.11/GeneralNN/jl_JOrcRG".
ERROR: LoadError: ArgumentError: Package GeneralNN does not have Colors in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have GeneralNN checked out for development and have
  added Colors as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with GeneralNN
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2299 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2271
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2260
 [7] include
   @ ./Base.jl:557 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base ./loading.jl:2881
 [9] top-level scope
   @ stdin:6
in expression starting at /Users/lewis/code/nn by hand/src/GeneralNN.jl:82
in expression starting at stdin:
2

Working with Pkg takes some getting used to for many people. If you have not done so already, I recommend reading the first 5 chapters of the Pkg manual. As far as I know, it should be possible to do all basic tasks (like adding and removing packages) from the Pkg REPL prompt and normally when resolve and instantiate are suggested, they will fix the problem.

It looks to me like you are working in the default environment, which is fine for beginners, but I think you are at the level where it would be advantageous for you to use separate environments for each project that you work on (you can read more at 2. Getting Started · Pkg.jl ). The advantage of this is that if you get problems like this, then you can often solve them by just deleting (or renaming) your projects Manifest.toml, making a few adjustments to the Project.toml (from the pkg REPL) and reinstantiating.

The intention is that your default environment should only contain tools like Plots, LocalRegistry, Revise etc and that real dependencies (like GeneralNN) should be in separate environments.

Since you are having all these problems, now might be a good time to start this habit: shut down julia, rename your /Users/lewis/.julia directory to something else and start julia again. You will now have a clean environment. Create a new environment by navigating to the directory where you want to save this environment and type ]activate . and then add all the packages that you need for your project. Starting from scratch like this usually solves problems like the one that you describe.

Edit:
Reading through you post again, it occurred to me that maybe you have not added the package to your GeneralNN Project.toml file. This can be fixed with:

cd("/Users/lewis/code/nn by hand/src/GeneralNN.jl")
import Pkg
pkg"activate ."
pkg"add WhateverThePackageIsCalled"

(the last 3 lines are effectively the same as entering pkg mode with ] and executing those two commands in the pkg REPL). To return to the default environment after this, execute pkg"activate"

1 Like
3

That’s been my experience as well. Plus when using a local package from its directory needed to preface the script with

using Pkg
Pkg.activate(".")
4

This is better written as

Pkg.activate(@__DIR__)

Then it finds the right environment even if you start the script from another directory.

1 Like