# Fixing Package Fragmentation

**URL:** <https://discourse.julialang.org/t/fixing-package-fragmentation/98712>\
**Category:** Community\
**Created:** [May 12, 2023, 2:13am UTC](https://discourse.julialang.org/t/fixing-package-fragmentation/98712 "2023-05-12T02:13:26Z")\
**Posts on this page:** 1\
**Showing post:** 110

<div class="post-metadata">

**Author:** ![Eric\_Sadit\_Tellez](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eric_sadit_tellez/32/33615_2.png) [@Eric\_Sadit\_Tellez](https://discourse.julialang.org/u/Eric_Sadit_Tellez)\
**Post date:** [May 25, 2023, 8:01pm UTC](https://discourse.julialang.org/t/fixing-package-fragmentation/98712/110 "2023-05-25T20:01:49Z")

</div>

Giving users tools to find and choose packages, use better the package’s API, organize them, etc., is fundamental. This can help avoid someone starting a new package and choosing another working package that requires a minor intervention to do whatever is needed, and in the same way, enable more effective use in general through documentation.

On this matter, I was on the idea of creating full-text search indexes for packages descriptions and docs,

> [@Searching and clustering julia packages by name, description, topics, and readme](https://discourse.julialang.org/t/searching-and-clustering-julia-packages-by-name-description-topics-and-readme/98813):
>
> Hi, This post is an early proposal to tackle some discoverability and fragmentation issues on the Julia package ecosystem with information tools. The idea is to complement the call of gathering efforts and standardization with discoverability strategies and tools that take advantage of this and specific text information in packages. The intuition is that providing these tools allows people to find the required package and avoid unnecessary package multiplication. I know that Juliahub has some …

For instance, several package managers support package searching and some categorization, but most will not support full-text search.

I kept working on the proof of concept. The new version uses web scrapping instead of GitHub API and includes support for packages hosted in Gitlab and possibly others.

[https://github.com/sadit/Search-Julia-Packages/blob/main/search-pkgs.ipynb](https://github.com/sadit/Search-Julia-Packages/blob/main/search-pkgs.ipynb)

It can search on package names (with errors) and by readme content (also topics and descriptions if they are in Github). It supports changing name and content weighting scores.

It also adds support for FT search on module documentation based on `Base.Docs` and uses the `Lunr` search index that `Documenter` produces (searching without installing).

[https://github.com/sadit/Search-Julia-Packages/blob/main/search-docs.ipynb](https://github.com/sadit/Search-Julia-Packages/blob/main/search-docs.ipynb)

It is pretty similar to packages but for documentation. It is in an early stage and requires a lot of work, but also just works.

In addition, visualization and a cluster are generated  
[https://github.com/sadit/Search-Julia-Packages/blob/main/clustering.ipynb](https://github.com/sadit/Search-Julia-Packages/blob/main/clustering.ipynb)  
These are based on package README files. This can help get a view of similar packages as a whole.

A small web API using the `Oxygen` package was created to see who to use the API. Additionally, the package database is slow to load per query, so if it is used someday, it should be as a web server or something like that.

There are some sites to search for packages (juliahub included), but most of them need a browser or install other kinds of tools. The proposal is to create tools that work in the REPL or any Julia process.

So, the idea is to gather people interested in going in this direction and produce some tools that can help the community using the information already on the packages. I am open to discussing and collaborating on these ideas (negative ideas that make me stop this effort are also welcome).

---

_[View the full topic](https://discourse.julialang.org/t/fixing-package-fragmentation/98712)._