Odd disappearance of some packages on update

Tooling Package Management
1

I have a Julia package that uses Optim with autodiff. I believe because of that Calculus and DualNumbers have been part of my environment. I just did a package update and both Calculus and DualNumbers were removed, despite the fact that most other packages got minor bumps.

The update included some error messages, and so I am wondering if things just got mangled.

At any rate, I’m trying to figure out if I have a problem, and what to do about it.

Running Julia 1.7.2 with Revise autoloaded.

Here’s a partial transcript of the update; unfortunately the dynamic updates to the display wiped some stuff out, like the update command I issued (Julia REPL in VSCode):

(MyFirst) pkg> st
     Project MyFirst v0.1.0
      Status `J:\Users\rdboylan\Documents\KBD2\MyFirst\Project.toml`
  [324d7699] CategoricalArrays v0.10.5
  [a93c6f00] DataFrames v1.3.2
  [31c24e10] Distributions v0.25.53
  [f6369f11] ForwardDiff v0.10.25
  [38e38edf] GLM v1.6.2
  [d41bc354] NLSolversBase v7.8.2
  [86f7a689] NamedArrays v0.9.6
   Installed StatsFuns ─────── v0.9.18
   Installed Parsers ───────── v2.3.1
   Installed Preferences ───── v1.3.0
   Installed ForwardDiff ───── v0.10.27
   Installed StaticArrays ──── v1.4.4
   Installed StatsAPI ──────── v1.3.0
   Installed GLM ───────────── v1.7.0
   Installed LogExpFunctions ─ v0.3.13
   Installed Static ────────── v0.6.2
   Installed DataFrames ────── v1.3.3
   Installed Compat ────────── v3.43.0
   Installed Distributions ─── v0.25.54
    Updating ┌ Error: Error watching manifest
│   exception =
│    ArgumentError: invalid type for argument s in method definition for size at C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
│    Stacktrace:
│     [1] top-level scope
│       @ C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
│    Revise evaluation error at C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
│ 
│    Stacktrace:
│     [1] methods_by_execution!(recurse::Any, methodinfo::Revise.CodeTrackingMethodInfo, docexprs::Dict{Module, Vector{Expr}}, mod::Module, ex::Expr; mode::Symbol, disablebp::Bool, always_rethrow::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
│       @ Revise C:\Users\rdboylan\.julia\packages\Revise\VskYC\src\lowered.jl:227
└ @ Revise C:\Users\rdboylan\.julia\packages\Revise\VskYC\src\pkgs.jl:477
`J:\Users\rdboylan\Documents\KBD2\MyFirst\Project.toml`
  [a93c6f00] ↑ DataFrames v1.3.2 ⇒ v1.3.3
  [31c24e10] ↑ Distributions v0.25.53 ⇒ v0.25.54
  [f6369f11] ↑ ForwardDiff v0.10.25 ⇒ v0.10.27
  [38e38edf] ↑ GLM v1.6.2 ⇒ v1.7.0
    Updating `J:\Users\rdboylan\Documents\KBD2\MyFirst\Manifest.toml`
  [49dc2e85] - Calculus v0.5.1
  [34da2185] ↑ Compat v3.42.0 ⇒ v3.43.0
  [9a962f9c] ↑ DataAPI v1.9.0 ⇒ v1.10.0
  [a93c6f00] ↑ DataFrames v1.3.2 ⇒ v1.3.3
  [b552c78f] ↑ DiffRules v1.10.0 ⇒ v1.11.0
  [31c24e10] ↑ Distributions v0.25.53 ⇒ v0.25.54
  [fa6b7ba4] - DualNumbers v0.6.8
  [f6369f11] ↑ ForwardDiff v0.10.25 ⇒ v0.10.27
  [38e38edf] ↑ GLM v1.6.2 ⇒ v1.7.0
  [34004b35] - HypergeometricFunctions v0.3.8
  [2ab3a3ac] ↑ LogExpFunctions v0.3.10 ⇒ v0.3.13
  [90014a1f] ↑ PDMats v0.11.7 ⇒ v0.11.8
  [69de0a69] ↑ Parsers v2.2.4 ⇒ v2.3.1
  [21216c6a] ↑ Preferences v1.2.5 ⇒ v1.3.0
  [aedffcd0] ↑ Static v0.6.0 ⇒ v0.6.2
  [90137ffa] ↑ StaticArrays v1.4.3 ⇒ v1.4.4
  [82ae8749] ↑ StatsAPI v1.2.1 ⇒ v1.3.0
  [4c63d2b9] ↑ StatsFuns v0.9.17 ⇒ v0.9.18
Precompiling project...
  Progress [========================================>]  55/55
  ✓ Compat
  ✓ DataAPI
  ✓ StatsAPI
  ✓ Preferences
  ✓ Missings
  ✓ Tables
  ✓ PooledArrays
  ✓ JLLWrappers
  ✓ ChainRulesCore
  ✓ DataStructures
  ✓ CategoricalArrays
  ✓ Rmath_jll
  ✓ Compat
  ✓ DataAPI
  ✓ StatsAPI
  ✓ Preferences
  ✓ Missings
  ✓ Tables
  ✓ PooledArrays
  ✓ JLLWrappers
  ✓ ChainRulesCore
  ✓ DataStructures
  ✓ CategoricalArrays
  ✓ Rmath_jll
  ✓ OpenSpecFun_jll
  ✓ PrettyTables
  ✓ Rmath
  ✓ ChangesOfVariables
  ✓ LogExpFunctions
  ✓ SortingAlgorithms
  ✓ StatsBase
  ✓ SpecialFunctions
  ✓ DataFrames
  ✓ StatsFuns
  ✓ StatsModels
  ✓ RegressionFormulae
  55 dependencies successfully precompiled in 43 seconds (38 already precompiled)
  24 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions
[ Info: We haven't cleaned this depot up for a bit, running Pkg.gc()...
      Active manifest files: 3 found
      Active artifact files: 73 found
      Active scratchspaces: 7 found
     Deleted 157 package installations (29.334 MiB)
2

Calculus was removed from the dependencies of Optim in January 2020 (#770). If it’s no longer required explicitly by you or implicitly by one of your dependencies, it can safely be removed. If you need it, add it.

2 Likes
3

And DualNumbers in 2016 (#244)

1 Like
4

Thanks for the info. I wonder why this particular update was the one that triggered their removal.

So are the errors that appeared in the transcript harmless?
Ross

5

Ehm. No, the errors are probably not entirely harmless. But they might be transient. Do they appear consistently when you update packages?

If you run julia in an empty directory as julia --project=., and then try adding a few packages at a time, what happens?

It might simply be time to clear out your environment and add back the packages you need.

6

Not too much to go on yet, but I did another update, which did update a few packages, and there were no errors.

Similarly, I started julia in an empty directory as you advised and added a couple packages. One, GLM had a huge number of dependencies. No errors.

7

Then I would say you’re out of the woods.