# State of the Eco-System
**URL:** https://discourse.julialang.org/t/state-of-the-eco-system/7285
**Category:** Meta Discussion
**Created:** [November 24, 2017, 11:43am UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285 "2017-11-24T11:43:49Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)
#### Post date: [November 24, 2017, 11:43am UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/1 "2017-11-24T11:43:49Z")
I want to open up a discussion regarding the state of the Eco-System.
from over 1000 packages only a subset is actually useful.
Some are just plain broken since Julia is still introducing breaking changes.
Some are way under documented and thus not so useful, some are too specific to be a package.
Should there be a review process for new packages?
---
### Author:  [@cormullion](https://discourse.julialang.org/u/cormullion)
#### Post date: [November 24, 2017, 11:58am UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/2 "2017-11-24T11:58:48Z")
Related discussions:
> [@Obsolete packages?](https://discourse.julialang.org/t/obsolete-packages/607):
>
> Looking through [pkg.julialang.org](http://pkg.julialang.org) I noticed quite a few packages that look to be obsolete (n years since being updated, all tests failing). I had a go at “adopting” one recently, and made a PR, but I suspect that the author has moved on to other things. Is there likely to be a clear out of obsolete packages some day? Perhaps some kind of “JuliaMorgue” org could be created where dead packages could be moved and preserved for posterity?
> [@Lost in transition: the current state of the Julia ecosystem?](https://discourse.julialang.org/t/lost-in-transition-the-current-state-of-the-julia-ecosystem/5515/10):
>
> Similarly, Gumbo itself is at v0.3.0 on Julia v0.6, despite the fact that [pkg.julialang.org](http://pkg.julialang.org) lists Julia v0.5: 0.3.0 (7 months ago) / Tests pass. Julia v0.6: 0.4.0 (a month ago) / Tests pass. How can this be possible on a brand new Julia v0.6 install? julia\> versioninfo() Julia Version 0.6.0 Commit 903644385b (2017-06-19 13:05 UTC) Platform Info: OS: macOS (x86\_64-apple-darwin13.4.0) CPU: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz WORD\_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC\_…
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [November 24, 2017, 12:16pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/3 "2017-11-24T12:16:35Z")
And also
> [@Should a sort of quality-assurance program introduced for Julia packages?](https://discourse.julialang.org/t/should-a-sort-of-quality-assurance-program-introduced-for-julia-packages/6346):
>
> Just a thought. The inclusion of a package in METADATA is currently conditional only to formal considerations… if the tag is in the right format, if the require file is present and so on… but no “judgment” on the quality of the code nor on the relevance of the package is made. While I appreciate this “open” approach, I also appreciate those when some sort of QA is employed. In particular I liked the experience I had to develop [Drupal](https://www.drupal.org) modules (Drupal is a web Content Management System). In that…
> [@Pruning and quality control for the package ecosystem](https://discourse.julialang.org/t/pruning-and-quality-control-for-the-package-ecosystem/7021):
>
> Related to [previous discussion](https://discourse.julialang.org/t/ann-higherprecision/6956) on starting new projects in place of abandonned ones (also see [this one about QA](https://discourse.julialang.org/t/should-a-sort-of-quality-assurance-program-introduced-for-julia-packages/6346/3)), I found [this short blog post](http://esr.ibiblio.org/?p=7303) where Eric Raymond discusses similar issues about Rust interesting. Some quotes: when the question is “where do I get feature X?” the answer “oh, there are 23 crates for that” is objectively terrifying […] As a potential Rust user, what I want to do is be able to go from a given feature requirement to one high-quality implementation with an implicit …
---
### Author:  [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [November 24, 2017, 12:39pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/4 "2017-11-24T12:39:56Z")
Pkg3 will bring namespaces, and we’ll likely have some kind of curated namespace for known-high-quality packages.
There are some real issues here around curation and discoverability of packages, but vetoing packages based on subjective worth isn’t the right solution, and it’s a bad route to go down. Some of those crappy packages will become the JuMP or DifferentialEquations.jl of the future, given a little encouragement.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [November 24, 2017, 1:00pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/5 "2017-11-24T13:00:52Z")
> [@MikeInnes](#):
>
> vetoing packages based on subjective worth isn’t the right solution
However, some (very permissive) _objective_ criteria could help, like removing packages from certain lists if the package has not passed unit tests for a year or more, or has not achieved some minimum level of test coverage (eg 50%). These could be automated, with advance notifications to the authors so they have time to react, etc. The point would be to remove abandonware, not to stifle work in progress.
---
### Author:  [@yakir12](https://discourse.julialang.org/u/yakir12)
#### Post date: [November 24, 2017, 1:20pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/6 "2017-11-24T13:20:52Z")
I totally agree.
In addition:
There are a number of high quality packages that are both small, self-contained, and abstract in nature. I’ve noticed a few cases where larger packages reimplement functionalities that these small packages already include (or where literally made for). This code duplication probably stems from the fact this ecosystem is so new (e.g. I think [IntervalSets.jl](https://github.com/JuliaMath/IntervalSets.jl) came _after_ [IntervalTrees.jl](https://github.com/BioJulia/IntervalTrees.jl)). But I feel like whenever one of these core packages pops up and matures a bit, package owners should reflect if they can make use of their functionalities.
This will allow us to have a slim Base and slim packages (because a bulk of the code will reside in those abstract packages).
- I’ve been meaning to compile a list of such packages, just as an example, but _life_.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [November 24, 2017, 2:09pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/7 "2017-11-24T14:09:49Z")
> [@MikeInnes](#):
>
> Pkg3 will bring namespaces, and we’ll likely have some kind of curated namespace for known-high-quality packages.
>
> There are some real issues here around curation and discoverability of packages, but vetoing packages based on subjective worth isn’t the right solution, and it’s a bad route to go down. Some of those crappy packages will become the JuMP or DifferentialEquations.jl of the future, given a little encouragement.
Yeah, this is the two issues. On one hand, you need to let everything be open so that the ecosystem has a chance to evolve. But on the other hand, curated libraries where someone has made some subjective decisions about what’s useful can make a much more concise and manageable library. We need both.
---
### Author:  [@jlperla](https://discourse.julialang.org/u/jlperla)
#### Post date: [November 24, 2017, 5:42pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/8 "2017-11-24T17:42:54Z")
I would be curious if there are a lot of libraries which are currently are outdated, but where the original authors had planned to restart post 1.0 (when they expect API stability and the general number of users to rapidly increase). Even if not, there may be people willing to take over the older libraries at that point. So having easy ways to find potentially outdated libraries is useful.
On another note, I spent a bunch of time over the last few weeks getting a sense of the ecosystem, and found it very hard to find **functionality** as opposed to just packages.
- One example I can think of is when I was looking for a `Kalman Filter` which I expect has been re-implemented a million times - mostly poorly - and with incompatible state-space model specifications. There were a few obvious places which I had a strong prior might have the functionality, but it much harder than it had to be.
- Another was looking for chebyshev polynomial nodes and quadrature weights. I had some guesses of repositories to look in, but consider how hard it is to look through something like [https://pkg.julialang.org/](https://pkg.julialang.org/) to guess which packages to look closer in.
To aid in this for the post-Pkg3 infrastructure, consider whether it is possible to have indexing of functionality below the level of package and its keywords? Maybe it is possible to have a bot to index docstrings in the repository, which could be used by a general package search engine from within the “curated” list? That way, if someone writes: "Algorithm taken from: Laub, “A Schur Method for Solving Algebraic Riccati Equations”, as in [https://github.com/JuliaControl/ControlSystems.jl/blob/0ecafbcc8c05a87c98f7d5b55cb4368d9ecbd10e/src/matrix\_comps.jl](https://github.com/JuliaControl/ControlSystems.jl/blob/0ecafbcc8c05a87c98f7d5b55cb4368d9ecbd10e/src/matrix_comps.jl), someone searching would finding it for “Riccati” or Laub.
Nothing too elaborate is necessary, but would help make reusable code easier to find, and encourage loosely coupled packages rather than monolithic ones with repetition.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [November 24, 2017, 5:53pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/9 "2017-11-24T17:53:03Z")
> [@jlperla](#):
>
> On another note, I spent a bunch of time over the last few weeks getting a sense of the ecosystem, and found it very hard to find functionality as opposed to just packages.
One way I like to search is through the big orgs. For example, for statistics I just go straight to JuliaStats and search there. Of course there’s good stuff not in orgs, but usually (the big) orgs have a testing and maintenance structure that keeps packages alive, while random single author packages tend to also be single maintainer (and single tester).
> [@jlperla](#):
>
> Another was looking for chebyshev polynomial nodes and quadrature weights. I had some guesses of repositories to look in, but consider how hard it is to look through something like [https://pkg.julialang.org/](https://pkg.julialang.org/) to guess which packages to look closer in.
FastGaussianQuadrature.jl or just use ApproxFun.jl? This is a good example of something that gets asked a lot but seems hard to find I guess.
> [@jlperla](#):
>
> To aid in this for the post-Pkg3 infrastructure, consider whether it is possible to have indexing of functionality below the level of package and its keywords? Maybe it is possible to have a bot to index docstrings in the repository, which could be used by a general package search engine from within the “curated” list? That way, if someone writes: "Algorithm taken from: Laub, “A Schur Method for Solving Algebraic Riccati Equations”, as in [https://github.com/JuliaControl/ControlSystems.jl/blob/0ecafbcc8c05a87c98f7d5b55cb4368d9ecbd10e/src/matrix\_comps.jl](https://github.com/JuliaControl/ControlSystems.jl/blob/0ecafbcc8c05a87c98f7d5b55cb4368d9ecbd10e/src/matrix_comps.jl), someone searching would finding it for “Riccati” or Laub.
[JuliaObserver.com](http://JuliaObserver.com) ?
---
### Author:  [@jlperla](https://discourse.julialang.org/u/jlperla)
#### Post date: [November 24, 2017, 6:42pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/10 "2017-11-24T18:42:50Z")
@ChrisRackauckas Thanks!
> One way I like to search is through the big orgs. For example, for statistics I just go straight to JuliaStats and search there. Of course there’s good stuff not in orgs, but usually (the big) orgs have a testing and maintenance structure that keeps packages alive, while random single author packages tend to also be single maintainer (and single tester).
Thanks. The problem is that github doesn’t support (or I can’t figure out!) how to search code within an organization rather than searching in the repository itself. Maybe I just missed it. I ended up code searching in
> FastGaussianQuadrature.jl or just use ApproxFun.jl? This is a good example of something that gets asked a lot but seems hard to find I guess.
Thanks. I didn’t find `FastGaussianQuadrature.jl` which is more directly focused on what I was getting at. I did figure out `ApproxFun`, but that was a perfect example of having a strong suspicion to look there because I knew about chebfun and suspected they would have to implement it internally… Too much knowledge required.
> [JuliaObserver.com](http://JuliaObserver.com)
> I didn’t know about this! If it is indexing the whole repositories, rather than just the README.md, then expanding, publicizing, and making it semi-official would really help people find packages.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [November 24, 2017, 6:49pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/11 "2017-11-24T18:49:45Z")
> [@jlperla](#):
>
> The problem is that github doesn’t support (or I can’t figure out!) how to search code within an organization rather than searching in the repository itself.
If you go to the org main page and search, it’ll search all repos of an org.
> [@jlperla](#):
>
> If it is indexing the whole repositories, rather than just the README.md, then expanding, publicizing, and making it semi-official would really help people find packages.
I really do think it should be at least semi-official.
---
### Author:  [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)
#### Post date: [November 24, 2017, 7:16pm UTC](https://discourse.julialang.org/t/state-of-the-eco-system/7285/12 "2017-11-24T19:16:18Z")
I think this should be no surprising, as time goes on, some packages will survive and become mainstream while the others will not. When I first started using R many years ago, I have to use a lot of different packages for a single project, but now my choice of tools have been quite narrowed down.
In addition, Julia has been incorporating some new features, some packages may not be updated at the same pace.