Thank you for your kind suggestions. And thank you for tolerating my confused anxiety. By my 3rd post I was beginning to understand what was going on. I did not realize that Dev should be used in lieu of Add and only needed to be done once in an environment, not in each file where the package being developed might be used. Using then accesses the package at it’s “Dev” location.
And, yes, changes are picked up immediately from the working files with no need to commit and then “up” the package. As designed.
So, all of your suggestions worked perfectly, of course. I have “using Revise” in both startup.jl and startup_ijulia.jl so I am covered there.
Your suggestions about the notebooks are also apt. I removed my notebook directory from the repo for now. One notebook is used primarily for exploration of approaches and debugging. Others provide highlighted outputs and documentation of how to use the package and some of its workings. I’ll move those back into the repo when everything is a bit more complete.
Stymied again. I had marked @Danielwe’s great suggestions as solution, but it can’t handle changes to the dependencies used in a package.
I have confirmed that project.toml and manifest.toml have always contained Tables. But, see the error: when I add “using Tables” to the module definition in a package that is “dev”, it refuses to pick up the changes.
[ Info: Precompiling CovidSim_ilm [c70e2dfc-8e06-4fce-bc57-ad774aa1cb0a]
ERROR: LoadError: ArgumentError: Package CovidSim_ilm does not have Tables in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
to ensure all packages in the environment are installed.
- Or, if you have CovidSim_ilm checked out for development and have
added Tables as a dependency but haven't updated your primary
environment's manifest file, try `Pkg.resolve()`.
It turned out that manifest.toml for the package DID contain an entry for Tables, but project.toml for the package did not!
I’ll do an rm and dev again and see if that fixes it. I can report that this doesn’t fix it. Since I am working in my base default environment @1.9 everything should be fine. But, it is not. Now I’ll try rm’ing and add’ing Tables. And then redoing rm and dev for my package.
Still doesn’t work. Seems unfixable. Bug in Pkg.Dev?
Truly stuck. Haven’t seen anything like it. I need to add Tables to the requirements for the package. How to force it?
I have done Pkg.resolve() but that shows no change because my default environment DID have Tables. Likewise Pkg.instantiate() All I did was add “using Tables” to the module of the package. It is never picked up. So, I think I need to do a commit and upgrade to fix this because Pkg.instantiate() and Pkg.resolve() don’t seem to fix it.
I have manually deleted the package directory in ~/.julia. I manually added Tables to project.toml for the package. Then re-dev’ed. Nothing fixes this problem.
What could be going wrong? It has to be possible to do this.
Can we see your Project.toml? I usually never manually edit it. I always use Pkg to do it.
Deleting the package directory in ~/.julia made it worse. That’s what it’s complaining about. The Manifest says the package is installed, but you deleted it. Pkg.instantitiate() should fix the issue as long as you added the dependency with the correct UUID to Project.toml.
I am going to use your steps, but the problem was never with the default environment. Julia complained that it needed Pkg.resolve(). Not clear where in your sequence of commands it belongs. Tables had been there from the day I installed 1.9 and then 1.9.1.
Which project.toml do you mean? I assume you mean the one in the dir of the package being developed:
[ Info: Precompiling CovidSim_ilm [c70e2dfc-8e06-4fce-bc57-ad774aa1cb0a]
ERROR: LoadError: ArgumentError: Package CovidSim_ilm does not have Tables in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
to ensure all packages in the environment are installed.
- Or, if you have CovidSim_ilm checked out for development and have
added Tables as a dependency but haven't updated your primary
environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with CovidSim_ilm
Stacktrace:
[1] macro expansion
@ ./loading.jl:1634 [inlined]
[2] macro expansion
@ ./lock.jl:267 [inlined]
[3] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1611
[4] include
@ ./Base.jl:457 [inlined]
[5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
@ Base ./loading.jl:2045
[6] top-level scope
@ stdin:3
in expression starting at /Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/src/CovidSim_ilm.jl:42
in expression starting at stdin:3
Failed to precompile CovidSim_ilm [c70e2dfc-8e06-4fce-bc57-ad774aa1cb0a] to "/Users/lewis/.julia/compiled/v1.9/CovidSim_ilm/jl_bPX86T".
Here is a minimalist demo from Windows Powershell.
First I create a directory “LewisDemo”. That directory contains “Project.toml” and the directory “src”. The directory “src” contains LewisDemo.jl". When I try to do using LewisDemo I get the same error message that you got.
PS C:\Users\mkitti> tree LewisDemo /F
Folder PATH listing for volume Windows
C:\USERS\MKITTI\LEWISDEMO
│ Project.toml
│
└───src
LewisDemo.jl
PS C:\Users\mkitti> cat .\LewisDemo\Project.toml
name = "LewisDemo"
uuid = "c01d0324-b1ab-4217-8193-3b6beef15e0f"
PS C:\Users\kittisopikulm> cat .\LewisDemo\src\LewisDemo.jl
module LewisDemo
using Tables
end
PS C:\Users\mkitti> julia --project=LewisDemo -e "using LewisDemo"
ERROR: LoadError: ArgumentError: Package LewisDemo does not have Tables in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
to ensure all packages in the environment are installed.
- Or, if you have LewisDemo checked out for development and have
added Tables as a dependency but haven't updated your primary
environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with LewisDemo
Stacktrace:
...
To resolve this, I set the project environment to that of the package, and add Tables to it via Pkg. This will manipulate my Project.toml.
Works when removing the double double-quotes around the package name. That must be some Powershell thing.
julia --project="/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM" -e 'using Pkg; Pkg.add("Tables")'
Resolving package versions...
No Changes to `~/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/Project.toml`
No Changes to `~/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/Manifest.toml`
But, as you can see no changes are made to project.toml or manifest.toml. It also remains strange that ] st shows the package that I “dev’ed”, but there is no package in my Julia installation. I can also do “using mypackage”, but of course the failed dependency error still occurs.
This feels like a delete everything and start over problem or some bug in Pkg.
Essentially, there are two project environments involved here:
Your package project enviornment. /Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/
Your default enviornment. /Users/lewis/.julia/environments/v1.9
Since you are buliding a package, we need to manipulate /Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/Project.toml via Pkg. This declares dependencies for the Covid-ILM package.
Can we see the contents of /Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM/Project.toml?
Next command. This creates a temporary project environment, to which we will dev the package into.
julia -e 'using Pkg; Pkg.activate(; temp=true); Pkg.develop(path="/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM"); using CovidSim_ilm)'
If that works, you can try adding the package to your default enviornment.
julia -e 'using Pkg; Pkg.instantiate(); Pkg.develop(path="/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM"); using CovidSim_ilm)'
julia -e 'using Pkg; Pkg.instantiate(); Pkg.develop(path="/Users/lewis/Library/CloudStorage/Dropbox/Covid Modeling/Covid-ILM"); using CovidSim_ilm'
Resolving package versions...
No Changes to `~/.julia/environments/v1.9/Project.toml`
No Changes to `~/.julia/environments/v1.9/Manifest.toml`
Good. You should be able to just do julia -e "using CovidSim_ilm" now and your issue should be resolved. You should also be able to start julia interactively and type using CovidSim_ilm at the julia prompt.
there is no pacakge directory for the dev’ed package in ~/.julia/packages. Is that typical? I had been successfully using Dev (I could make edit changes and simple re-run code and the changes were picked up) AND the package was in the .julia directory.
I just did using CovidSim_ilm in Julia repl successfully.
I am going to try the notebook(): Hooray! It works.
So, to make the long story short: what happened?
How did your fixes fix it?
What should I have done when adding a dependent package? (ordinary changes to my Julia source were working fine…)
Perhaps I had been using Pkg.develop() incorrectly. I passed the path to the directory of the project as the argument rather than using Path=“the path”.
I also looked for the folder in /var/folders but it was gone–guess that’s what temp=true means.