# How to gracefully retire a package
**URL:** https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101
**Category:** General Usage
**Tags:** question, packages
**Created:** [October 9, 2024, 10:02am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101 "2024-10-09T10:02:22Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [October 9, 2024, 10:02am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/1 "2024-10-09T10:02:22Z")
Julia 1.11 has made [PushVectors.jl](https://github.com/tpapp/PushVectors.jl) obsolete. I wonder what the best workflow is to retire it.
I have read the previous threads on this, with various recommendations (eg [1](https://discourse.julialang.org/t/how-do-i-deprecate-a-package/14129), [2](https://discourse.julialang.org/t/juliahub-packages-feature-request-detect-and-mark-deprecated-or-archived-packages/97370)). I am still not sure how to proceed.
Specifically,
1. I don’t want to make the package uninstallable on \>=1.11, as it is a very harsh move (code is functional, and will work forever, just not needed after 1.11). So either releasing a new version that is capped, or capping it directly in the registry, is not a good solution, as it forces immediate work on others, instead of allowing them to pick a time for it.
2. I considered a gentle warning message, eg as
3. I will archive the repo, but AFAIK that currently does not do anything in Juliahub.
4. Since this package has 2 dependents, I can just notify them directly. But that does not scale.
I think that in the early days of Julia, retiring packages was not an immediate problem, so it is understandable that this was not a priority. But now that the ecosystem is mature this is an issue we have to deal with.
Also, note that I am not talking about “hard deprecation”, ie making the package impossible or difficult to install. One of the great things about Julia is _reproducibility_: once code works, you can dig it out 10 years later and it will still work. Once in the registry, packages should remain installable forever, and ideally work on all future Julia versions to make upgrading easier.
---
### Author:  [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)
#### Post date: [October 9, 2024, 10:12am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/2 "2024-10-09T10:12:25Z")
In this case I would just make the maintenance status and the migration path clear in the README, and possibly update docstrings with relevant information.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2024, 10:31am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/3 "2024-10-09T10:31:34Z")
> [@Tamas\_Papp](#):
>
> - I considered a gentle warning message, eg as
>
> ```julia
> function __init__ ()
> if VERSION ≥ v"1.11"
> @warn """
> The issue which required PushVectors.jl as a workaround has been fixed in
> Julia 1.11, so consider removing it. See README.md.
> """
> end
> end
>
> ```
>
> but having that show up constantly is a bit obnoxious.
A better way to do this IMO would be to emit the message during precompilation, that way you usually only get the message when you actually install the package. That’s what we went for when deprecating Formatting.jl, see this example: [Formatting.jl/src/Formatting.jl at master · JuliaIO/Formatting.jl · GitHub](https://github.com/JuliaIO/Formatting.jl/blob/master/src/Formatting.jl#L11-L25)
So in your case, I’d do
```julia
if ccall(:jl_generating_output, Cint, ()) == 1
@warn """
The issue which required PushVectors.jl as a workaround has been fixed in
Julia 1.11, so consider removing it. See README.md.
"""
end
end
```
instead of using ` __init__ `.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [October 9, 2024, 10:50am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/4 "2024-10-09T10:50:33Z")
Semi-related, but an edit of an old benchmark I had for `setindex!` vs `push!`-after-`sizehint!` corroborates the narrowing in performance advantage (`push!` used to be ~80μs), though `setindex!` is still far faster:
```julia
julia> benchmarkalloc3() # v1.11.0
push! after sizehint!:
1.096 μs (3 allocations: 128.06 KiB)
13.500 μs (0 allocations: 0 bytes)
setindex! an undef Array
1.040 μs (3 allocations: 128.06 KiB)
1.680 μs (0 allocations: 0 bytes)
push! PushVector with sizehint:
1.129 μs (4 allocations: 128.09 KiB)
11.300 μs (0 allocations: 0 bytes)
```
> **Benchmark code**
>
> ```julia
> using Benchmarktools, PushVectors
>
> function pushtest(v, m) # after sizehint!
> for i in 1:m push!(v, i) end
> empty!(v) # prevent memory leak, should keep capacity
> end
>
> function settest(v, m)
> for i in 1:m v[i] = i end
> v
> end
>
> function benchmarkalloc3()
> m = 2^14
>
> println("push! after sizehint!:")
> v = @btime sizehint!(Int64[], $m)
> @btime pushtest($v, $m)
>
> println("setindex! an undef Array")
> v2 = @btime Vector{Int64}(undef, $m)
> @btime settest($v2, $m)
>
> println("push! PushVector with sizehint:")
> v3 = @btime PushVector{Int64}($m)
> @btime pushtest($v3, $m)
> nothing
> end
>
> ```
It’s buried in a long Rust blog thread, but the reason for that benchmark was that Rust’s [analog of `push!`](https://github.com/rust-lang/rust/blob/4203c686136428ab10e2765a00886b7c2909a477/library/alloc/src/vec/mod.rs#L2400) checks the allocated capacity and thus possibly skips the vector-lengthening code to the analog of a `setindex!` and incrementing the semantic `length`, so the performance difference would be far less if an analog of `undef` vectors were used in Rust. At the time, the working hypothesis was the ccall overhead, but it was unclear because of the C implementation of `_growend!`. The new [`_growend!`](https://github.com/JuliaLang/julia/blob/ecf41b18cce0e41455a8144f220e9e6171987194/base/array.jl#L1102) and [`push!`](https://github.com/JuliaLang/julia/blob/ecf41b18cce0e41455a8144f220e9e6171987194/base/array.jl#L1264) seems to do something like the Rust approach, but I can’t tell why the improved `push!` still has a ways to approach `setindex!`, maybe the extra number crunching to check the capacity? Maybe PushVectors can pull this off and it won’t be obsolete for a bit longer.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [October 9, 2024, 12:54pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/5 "2024-10-09T12:54:43Z")
I think the biggest bottleneck with push is that the compiler doesn’t figure out how to vectorize it
---
### Author:  [@matthias314](https://discourse.julialang.org/u/matthias314)
#### Post date: [October 9, 2024, 1:04pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/6 "2024-10-09T13:04:53Z")
I thought the same, but the difference remains if you fill the vector with `rand(Int)`:
```julia
40.544 μs # pushtest
19.999 μs # settest
```
Below is the code. According to `@code_native`, `settest` is not vectorized in this case, contrary to @Benny’s original example.
```julia
function pushtest(v, m) # after sizehint!
for i in 1:m
push!(v, rand(Int))
end
empty!(v) # prevent memory leak, should keep capacity
end
function settest(v, m)
for i in 1:m
v[i] = rand(Int)
end
v
end
```
---
### Author:  [@matthias314](https://discourse.julialang.org/u/matthias314)
#### Post date: [October 9, 2024, 1:15pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/7 "2024-10-09T13:15:52Z")
It seems that the difference simply comes from updating the length of the vector. With the new data structure `PV` that includes the length I get
```julia
36.622 μs # pushtest
35.919 μs # settest for new type PV
```
> **Code**
>
> I’m only showing the new code. I’m using Chairmarks.jl.
>
> ```julia
> mutable struct PV{T}
> const v::Vector{T}
> n::Int
> end
>
> function settest(w::PV{T}, m) where T
> for i in 1:m
> w.v[i] = rand(T)
> w.n += 1
> end
> w
> end
>
> function benchmarkalloc3()
> m = 2^14
>
> println("push! after sizehint!:")
> v = sizehint!(Int64[], m)
> display(@b pushtest($v, $m))
>
> println("setindex! an undef PV:")
> v2 = Vector{Int64}(undef, m)
> w = PV(v2, 0)
> display(@b settest($w, $m))
> end
>
> ```
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2024, 1:34pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/8 "2024-10-09T13:34:51Z")
> [@matthias314](#):
>
> I thought the same, but the difference remains if you fill the vector with `rand(Int)`:
I think Oscar doesn’t mean that the arithmetic doesn’t vectorize, it’s that the compiler doesn’t realize it can vectorize `push!` calls themselves (presumably due to the branches inside that determine if the storage should be replaced or not).
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [October 9, 2024, 1:45pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/9 "2024-10-09T13:45:07Z")
> [@Benny](#):
>
> Maybe PushVectors can pull this off and it won’t be obsolete for a bit longer.
Honestly, the package was kind of a joke, I expected that the issue would be fixed much quicker. I am actually eager to retire it. Performance fixes should go to Base.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [October 9, 2024, 9:53pm UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/10 "2024-10-09T21:53:59Z")
If I’m reading this right, majority of it is whether the particular calls around `setindex!` are vectorizable, and the rest is the increment of the length (likely on the heap) taking a surprisingly (to me) significant portion of the time. IIRC Rust’s `Vec` has its metadata on the stack and the buffer on the heap, but it sounds like the same sort of work needs to happen for mutation in a method via a mutable reference. Rust wasn’t as easily tinkered with, so the steps weren’t profiled at the time, and the simple code examples were probably also vulnerable to compiler optimizations like SIMD throwing off comparisons.
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [October 10, 2024, 2:10am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/11 "2024-10-10T02:10:18Z")
> [@GunnarFarneback](#):
>
> In this case I would just make the maintenance status and the migration path clear in the README, and possibly update docstrings with relevant information
I think this is the right answer for now, and adding warnings on loading or pre compilation is a nice bonus, I think. If the question is what functionality would we like Pkg / registries to support, I have some thoughts.
There are a few different reasons a package might be deprecated or a package author might otherwise want to discourage people from using them:
1. Functionality isn’t needed anymore due to improvements to Julia (as in this example).
2. Functionality is better provided by other packages in the ecosystem (just had this come up with BioTools.jl).
3. The author just doesn’t plan to support it going forward.
In all of these cases, I totally agree that working code should be allowed to continue to work, so capping Julia versions is a bad idea. But it would be nice to be able to add a Pkg nudge if someone tries to add or load a deprecated package as a direct dependency, either by asking for confirmation, or requiring an extra flag (eg `] add PushVectors` errors, `] add PushVectors --force` works).
I wouldn’t want to add friction if it’s coming in as an indirect dependency - unless there’s a security issue, I never want to be forced to get involved in decisions package authors are making.
---
### Author:  [@kevbonham](https://discourse.julialang.org/u/kevbonham)
#### Post date: [October 10, 2024, 2:13am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/12 "2024-10-10T02:13:04Z")
Separately, a bunch of posts here should probably be split into a different discussion, but I didn’t know if I should use a flag - I associate that with bad behavior and I didn’t want to imply that…
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [October 10, 2024, 6:35am UTC](https://discourse.julialang.org/t/how-to-gracefully-retire-a-package/121101/13 "2024-10-10T06:35:55Z")
> [@kevbonham](#):
>
> But it would be nice to be able to add a Pkg nudge if someone tries to add or load a deprecated package as a direct dependency, either by asking for confirmation, or requiring an extra flag (eg `] add PushVectors` errors, `] add PushVectors --force` works).
Yes, I think that this is the right solution. Perhaps an extra field could be added to the Project.toml, indicating deprecation, possibly with a message that `Pkg.add` could display.
As for existing dependents, I think that they are best notified via opening an issue.