# Archival of discontinued packages in JuliaML
**URL:** https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375
**Category:** Machine Learning
**Tags:** package, machine-learning, organization
**Created:** [May 20, 2022, 2:22pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375 "2022-05-20T14:22:34Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [May 20, 2022, 2:22pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/1 "2022-05-20T14:22:34Z")
I am cleaning the house in JuliaML and will proceed with a sequence of archival operations to facilitate the lives of newcomers who come to Julia and are completely lost in the middle of packages that don’t even load anymore.
We are coordinating this effort in our Zulip machine learning stream:
> **[Chat for distributed teams](https://julialang.zulipchat.com/#narrow/stream/238249-machine-learning/topic/Archiving.20discontinued.20packages.20in.20JuliaML)**
>
> Zulip combines the immediacy of real-time chat with an email threading model. With Zulip, you can catch up on important conversations while ignoring irrelevant ones.
The list of packages that will be archived is listed below:
If you would like to inhibit the archival of any package listed above or want to do some final adjustment to its Project.toml before we archive it, please leave a message.
I will cross-post this thread on Slack as well so that we don’t miss anyone.
We also need help updating the JuliaML website. The website is referring to discontinued packages and certainly deserves a new version. Please join our Zulip stream if you want to help.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [May 20, 2022, 2:56pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/2 "2022-05-20T14:56:16Z")
Thanks for doing this! Could you explain what “archival” means here? Presumably you’re not yanking things from General?
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [May 20, 2022, 2:59pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/3 "2022-05-20T14:59:00Z")
We want to use GitHub archival feature to mark the repository as read-only. It will show a banner at the top but the URL will exist forever on the General registry.
The archival feature is to signal users and contributors that the package is no longer actively maintained, and that there are no plans of reviving these efforts. Some of these packages date back to pre-Project.toml era.
---
### Author:  [@iamed2](https://discourse.julialang.org/u/iamed2)
#### Post date: [May 20, 2022, 9:12pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/4 "2022-05-20T21:12:29Z")
We are currently depending on MLDataUtils, MLLabelUtils, and MLDataPattern.
Is there anything in those that’s not in MLUtils that we should know about? I imagine we just need to migrate to that.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [May 20, 2022, 9:34pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/5 "2022-05-20T21:34:43Z")
MLDataUtils.jl and MLDataPatterns.jl were absorbed in MLUtils.jl by @CarloLucibello , he can provide more details.
Regarding MLLabelUtils.jl, there are also plans to migrate parts of it to MLUtils.jl and @darsnack is probably leading that front.
Keep in mind that our ultimate goal is to reduce the number of packages in JuliaML and concentrate efforts around MLUtils.jl. Any help with this migration is highly appreciated.
---
### Author:  [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [May 20, 2022, 9:53pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/6 "2022-05-20T21:53:18Z")
MLUtils.jl should be almost a drop in replacement for MLDataUtils and MLDataPattern. MLLabelUtils has not been ported yet. @iamed2 can you point me to the specific methods that you are using from there or to the dependent package (if it is public)?
---
### Author:  [@devmotion](https://discourse.julialang.org/u/devmotion)
#### Post date: [May 20, 2022, 10:08pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/7 "2022-05-20T22:08:09Z")
Are there plans for a new interface package? It seems MLUtils is a lot heavier then LearnBase and so something more lightweight might be useful for packages that want e.g. to implement `getobs`. I think lightweight interface packages are quite useful for additional or optional functionality without pulling in too many dependencies or using Requires, as e.g., ChainRulesCore, StatsAPI, InverseFunctions etc. show.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [May 20, 2022, 10:21pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/8 "2022-05-20T22:21:36Z")
> [@devmotion](#):
>
> Are there plans for a new interface package?
Not at the moment. An effort to build an interface package is justified after we have a decent set of packages sharing these concepts. Right now JuliaML is not like that, unfortunately.
We should try to reuse existing interface packages like StatsAPI.jl as much as possible before deciding on yet another one. I feel that this is an effort that can be postponed to after we have cleaned up JuliaML and its dead orphan packages.
---
### Author:  [@darsnack](https://discourse.julialang.org/u/darsnack)
#### Post date: [May 20, 2022, 10:56pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/9 "2022-05-20T22:56:37Z")
With the addition of multi-threaded data loading, MLUtils.jl has grown larger than we initially planned. We are trying to trim back, though it might be time to introduce an interface package.
It’s worth noting that the default set of methods to implement are `Base.getindex` and `Base.length`. The only time you need to implement `getobs` or `numobs` is if these functions have a different meaning for your type than `getindex`/`length` (e.g. multi-dimensional arrays).
---
### Author:  [@Rory-Finnegan](https://discourse.julialang.org/u/Rory-Finnegan)
#### Post date: [May 26, 2022, 5:54am UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/10 "2022-05-26T05:54:32Z")
Assuming the `MLDataUtils.jl` and `MLDataPattern.jl` APIs are fully ported to `MLUtils.jl` with minimal API changes I don’t think this is a big concern for us. We do use `islabelenc` and `convertlabel` in 1 private package, but that’s the only case I could find and it wouldn’t be too hard to work around if necessary. I think our bigger concern is the dependency stack and size of `MLUtils.jl`. A lot of the time we just use it to call a single function like `kfolds` to reduce code size in small packages, but I’m not sure if the added dependencies on `FLoops`, `FoldThreads`, `ShowCases`, etc makes it worth it in those cases.
---
### Author:  [@Rory-Finnegan](https://discourse.julialang.org/u/Rory-Finnegan)
#### Post date: [May 26, 2022, 9:07pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/11 "2022-05-26T21:07:37Z")
Update: Looks like the MLDataPattern.jl functionality wasn’t directly ported. For example, a previously working call to `shuffleobs` now errors because it’s internally calling `length` on a dataframe.
```julia
MethodError: no method matching length(::DataFrame)
Closest candidates are:
length(::Union{Base.KeySet, Base.ValueIterator}) at /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/base/abstractdict.jl:58
length(::Union{DataStructures.OrderedRobinDict, DataStructures.RobinDict}) at ~/.julia/packages/DataStructures/izXYB/src/ordered_robin_dict.jl:86
length(::Union{DataStructures.SortedDict, DataStructures.SortedMultiDict, DataStructures.SortedSet}) at ~/.julia/packages/DataStructures/izXYB/src/container_loops.jl:322
...
Stacktrace:
[1] numobs(data::DataFrame)
@ MLUtils ~/.julia/packages/MLUtils/W3W0A/src/observation.jl:19
[2] _check_numobs(data::Tuple{DataFrame, Adjoint{Float64, Matrix{Float64}}})
@ MLUtils ~/.julia/packages/MLUtils/W3W0A/src/observation.jl:120
[3] numobs(data::Tuple{DataFrame, Adjoint{Float64, Matrix{Float64}}})
@ MLUtils ~/.julia/packages/MLUtils/W3W0A/src/observation.jl:129
[4] shuffleobs(rng::Random._GLOBAL_RNG, data::Tuple{DataFrame, Adjoint{Float64, Matrix{Float64}}})
@ MLUtils ~/.julia/packages/MLUtils/W3W0A/src/obstransform.jl:194
[5] shuffleobs(data::Tuple{DataFrame, Adjoint{Float64, Matrix{Float64}}})
@ MLUtils ~/.julia/packages/MLUtils/W3W0A/src/obstransform.jl:191
```
Having these breaking changes mixed into the porting process means we can’t simply rely on semver to stagger the work needed to resolve them. If the goal was to centralize the code in one package then my preference would have been to (1) copy the existing functionality, as is, into MLUtils.jl (2) tag a 1.0 release and (3) make the desired breaking changes for a v2 release…?
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [May 26, 2022, 9:27pm UTC](https://discourse.julialang.org/t/archival-of-discontinued-packages-in-juliaml/81375/12 "2022-05-26T21:27:08Z")
I am not following the specific details of this issue you raised, but notice that you have all the time in the world to migrate your projects to the new package. The archival just means that no one is gonna touch the old packages moving forward. They will continue to exist but are dead projects. If you want to receive updates from JuliaML maintainers, then you can migrate your packages slowly to MLUtils.jl.
As others already mentioned, LearnBase.jl, MLDataPattern.jl , … continue to exist on GitHub as frozen repos that no one is maintaining. We are concentrating efforts in a single repo now, that is MLUtils.jl
You can submit a PR to MLUtils.jl with improvements if you feel something should be done differently.