Odd disappearance of some packages on update

I have a Julia package that uses Optim with autodiff. I believe because of that Calculus and DualNumbers have been part of my environment. I just did a package update and both Calculus and DualNumbers were removed, despite the fact that most other packages got minor bumps.

The update included some error messages, and so I am wondering if things just got mangled.

At any rate, I’m trying to figure out if I have a problem, and what to do about it.

Running Julia 1.7.2 with Revise autoloaded.

Here’s a partial transcript of the update; unfortunately the dynamic updates to the display wiped some stuff out, like the update command I issued (Julia REPL in VSCode):

(MyFirst) pkg> st
     Project MyFirst v0.1.0
      Status `J:\Users\rdboylan\Documents\KBD2\MyFirst\Project.toml`
  [324d7699] CategoricalArrays v0.10.5
  [a93c6f00] DataFrames v1.3.2
  [31c24e10] Distributions v0.25.53
  [f6369f11] ForwardDiff v0.10.25
  [38e38edf] GLM v1.6.2
  [d41bc354] NLSolversBase v7.8.2
  [86f7a689] NamedArrays v0.9.6
   Installed StatsFuns ─────── v0.9.18
   Installed Parsers ───────── v2.3.1
   Installed Preferences ───── v1.3.0
   Installed ForwardDiff ───── v0.10.27
   Installed StaticArrays ──── v1.4.4
   Installed StatsAPI ──────── v1.3.0
   Installed GLM ───────────── v1.7.0
   Installed LogExpFunctions ─ v0.3.13
   Installed Static ────────── v0.6.2
   Installed DataFrames ────── v1.3.3
   Installed Compat ────────── v3.43.0
   Installed Distributions ─── v0.25.54
    Updating β”Œ Error: Error watching manifest
β”‚   exception =
β”‚    ArgumentError: invalid type for argument s in method definition for size at C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
β”‚    Stacktrace:
β”‚     [1] top-level scope
β”‚       @ C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
β”‚    Revise evaluation error at C:\Users\rdboylan\.julia\packages\Compat\AuXFf\src\Compat.jl:1141
β”‚    Stacktrace:
β”‚     [1] methods_by_execution!(recurse::Any, methodinfo::Revise.CodeTrackingMethodInfo, docexprs::Dict{Module, Vector{Expr}}, mod::Module, ex::Expr; mode::Symbol, disablebp::Bool, always_rethrow::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
β”‚       @ Revise C:\Users\rdboylan\.julia\packages\Revise\VskYC\src\lowered.jl:227
β”” @ Revise C:\Users\rdboylan\.julia\packages\Revise\VskYC\src\pkgs.jl:477
  [a93c6f00] ↑ DataFrames v1.3.2 β‡’ v1.3.3
  [31c24e10] ↑ Distributions v0.25.53 β‡’ v0.25.54
  [f6369f11] ↑ ForwardDiff v0.10.25 β‡’ v0.10.27
  [38e38edf] ↑ GLM v1.6.2 β‡’ v1.7.0
    Updating `J:\Users\rdboylan\Documents\KBD2\MyFirst\Manifest.toml`
  [49dc2e85] - Calculus v0.5.1
  [34da2185] ↑ Compat v3.42.0 β‡’ v3.43.0
  [9a962f9c] ↑ DataAPI v1.9.0 β‡’ v1.10.0
  [a93c6f00] ↑ DataFrames v1.3.2 β‡’ v1.3.3
  [b552c78f] ↑ DiffRules v1.10.0 β‡’ v1.11.0
  [31c24e10] ↑ Distributions v0.25.53 β‡’ v0.25.54
  [fa6b7ba4] - DualNumbers v0.6.8
  [f6369f11] ↑ ForwardDiff v0.10.25 β‡’ v0.10.27
  [38e38edf] ↑ GLM v1.6.2 β‡’ v1.7.0
  [34004b35] - HypergeometricFunctions v0.3.8
  [2ab3a3ac] ↑ LogExpFunctions v0.3.10 β‡’ v0.3.13
  [90014a1f] ↑ PDMats v0.11.7 β‡’ v0.11.8
  [69de0a69] ↑ Parsers v2.2.4 β‡’ v2.3.1
  [21216c6a] ↑ Preferences v1.2.5 β‡’ v1.3.0
  [aedffcd0] ↑ Static v0.6.0 β‡’ v0.6.2
  [90137ffa] ↑ StaticArrays v1.4.3 β‡’ v1.4.4
  [82ae8749] ↑ StatsAPI v1.2.1 β‡’ v1.3.0
  [4c63d2b9] ↑ StatsFuns v0.9.17 β‡’ v0.9.18
Precompiling project...
  Progress [========================================>]  55/55
  βœ“ Compat
  βœ“ DataAPI
  βœ“ StatsAPI
  βœ“ Preferences
  βœ“ Missings
  βœ“ Tables
  βœ“ PooledArrays
  βœ“ JLLWrappers
  βœ“ ChainRulesCore
  βœ“ DataStructures
  βœ“ CategoricalArrays
  βœ“ Rmath_jll
  βœ“ Compat
  βœ“ DataAPI
  βœ“ StatsAPI
  βœ“ Preferences
  βœ“ Missings
  βœ“ Tables
  βœ“ PooledArrays
  βœ“ JLLWrappers
  βœ“ ChainRulesCore
  βœ“ DataStructures
  βœ“ CategoricalArrays
  βœ“ Rmath_jll
  βœ“ OpenSpecFun_jll
  βœ“ PrettyTables
  βœ“ Rmath
  βœ“ ChangesOfVariables
  βœ“ LogExpFunctions
  βœ“ SortingAlgorithms
  βœ“ StatsBase
  βœ“ SpecialFunctions
  βœ“ DataFrames
  βœ“ StatsFuns
  βœ“ StatsModels
  βœ“ RegressionFormulae
  55 dependencies successfully precompiled in 43 seconds (38 already precompiled)
  24 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions
[ Info: We haven't cleaned this depot up for a bit, running Pkg.gc()...
      Active manifest files: 3 found
      Active artifact files: 73 found
      Active scratchspaces: 7 found
     Deleted 157 package installations (29.334 MiB)

Calculus was removed from the dependencies of Optim in January 2020 (#770). If it’s no longer required explicitly by you or implicitly by one of your dependencies, it can safely be removed. If you need it, add it.


And DualNumbers in 2016 (#244)

1 Like

Thanks for the info. I wonder why this particular update was the one that triggered their removal.

So are the errors that appeared in the transcript harmless?

Ehm. No, the errors are probably not entirely harmless. But they might be transient. Do they appear consistently when you update packages?

If you run julia in an empty directory as julia --project=., and then try adding a few packages at a time, what happens?

It might simply be time to clear out your environment and add back the packages you need.

Not too much to go on yet, but I did another update, which did update a few packages, and there were no errors.

Similarly, I started julia in an empty directory as you advised and added a couple packages. One, GLM had a huge number of dependencies. No errors.

Then I would say you’re out of the woods.