How can we create a leaner ecosystem for Julia?

Amin_Yahyaabadi · January 5, 2020, 4:23pm

New redundant packages appear because it is usually hard to merge pull requests into current repositories. Also, people don’t reuse code because of the reasons that I will state in the following.

My personal experience offers that there are a couple of patterns that repeat regarding this topic

When a package’s development is stopped or not worth it:
E.g.: PackageCompiler and PackageCompilerX: the development of the PackageCompiler is very slow or stopped, and there is that JuliaCon youtube on how to rewrite PackageCompiler. That is why PackageCompilerX has emerged.
Many other examples fit under this.
When developers of a package don’t want extra features:
E.g.: Julia-Client and Juno-Plus: I submitted a PR to Julia-Client, however developers thought that Juno-Plus has too many extra features that should not be inside Julia-Client, and so the 2nd one got emerged.
When a package’s developer wants to minimize dependencies.
E.g.: PrettyTables and AcuteML: despite all of the benefits of using another advanced package, the developer thought using AcuteML as the HTML backend will increase the overall loading time of the package.
It is hard to become a member of an organization:
Requires contacting the main owner.
Lack of package tagging:
Many mentioned this already in this thread.
Lack of documentation:
Static Linter, Language Server are examples of packages with zero documentation. That is why no one knows how to use them anywhere else other than in VSCode.
Not allowing overriding a method:
Overriding a method in Julia will result in an incremental precompilation warning. If someone wants to use some code but wants to override a method, they should either write the code from scratch or make a fork of the original package and edit the function there. However forks are not supported in Project.toml, and Manifest.toml files aren’t as nice as Project.tomls.
When people want to make a portfolio:
In this case, people usually tend to write everything from scratch. And those, who have written a package on their own, tend to disagree with merging other’s people PRs.

A solution to some of these problems is allowing to have multiple packages in a repository:

github.com/JuliaLang/Pkg.jl

Feature request: Store multiple registered packages in a single Git repository

opened 04:36AM - 14 Jul 19 UTC

closed 02:42PM - 27 Apr 20 UTC

DilumAluthge

enhancement triage

Currently, a single Git repository can only contain one registered package, whic…h must live in the top folder of the Git repository. I would like to request the ability to store more than one registered package in the same Git repository. For example, I might have two packages `Foo` and `Bar` that I store in a single GitHub repository `DilumAluthge/FooBar.git`. In this case, the contents of `https://github.com/DilumAluthge/FooBar.git` might look something like this: ``` FooBar ├── Bar │ ├── Project.toml │ ├── src │ │ └── Bar.jl │ └── test │ └── runtests.jl └── Foo ├── Project.toml ├── src │ └── Foo.jl └── test └── runtests.jl 6 directories, 6 files ``` For example, the contents of `https://github.com/DilumAluthge/FooBar.git` might instead look something like this: ``` FooBar ├── myfolder │ └── Foo │ ├── Project.toml │ ├── src │ │ └── Foo.jl │ └── test │ └── runtests.jl └── otherfolder └── subfolder └── Bar ├── Project.toml ├── src │ └── Bar.jl └── test └── runtests.jl 9 directories, 6 files ``` --- ## Possibly related issues/pull requests: - #691: "Allow packages to live in a directory of repo" - This issue discusses relaxing the requirement that a package is located in the top folder of the Git repository. However, the issue doesn't discuss whether or not you would be allowed to have multiple registered packages in the same Git repository. - #1233: Proposal for "sub-projects" - This issue discusses sub-projects. However, a sub-project is an "incremental addition of packages to an already existing main-project". This is different from what I am proposing here, in which there is no "main project" from which the other projects derive.

or allowing to load a subset of a package