I’m considering splitting a large monolithic package into 3-4 smaller ones. The thought was they might be easier to maintain, migrate, test, etc. But I am a bit concerned that the “admin overhead” will actually increase. E.g. I’ve been watching the Optim packages a bit and it didn’t look all that st…

This may be useful if others depend on the package, but your “final” reexporting package will still depends on all the same packages. It has been somewhat useful to split functionality out from Optim in some sense, but really it’s a huge burden. All the package have to fit together anyway. That sai…

[image] cortner: concerned that the “admin overhead” will actually increase I would base this decision on whether there is a functional, self-contained collection of code that is potentially useful for others beside the rest of the original package. If yes, split; if not, or when in doubt, do…

[image] Tamas_Papp: I would base this decision on whether there is a functional, self-contained collection of code that is potentially useful for others beside the rest of the original package. If yes, split; if not, or when in doubt, don’t. Yes, this. With DiffEq, we got it wrong at first an…

Everybody : thank you for the very helpful responses. I take away from this that some good reasons to split a package are there is a clear case for independent use of a sub-module making life easier for contributors I’ll probably take some time now to reconsider this, or at least go much more cau…

[image] cortner: @ChrisRackauckas : a huge test set is one of my issues, how do you split it? Different repos or use Travis environments. For the latter, see: https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/.travis.yml#L9-L12 Gives three different tests: Travis CI - …

I would also favor splitting the packages, but under the following conditions: Packages are self-contained. For example, X.jl does one functionality or a suite for one purpose or provides one struct. Users or packages may just need/want a subset of the code. You could still use a master module tha…

How do you “group” the smaller packages though, a parent folder in ~/.julia/v0.6/, a github organization, other?

If by group you mean for development and maintaining probably through a Github org. For functionality, pipeline, battery or whatnot, have the main module call them. module MainModule using SmallPackageA, SmallPackageB, SmallPackageC # Code end

[image] Nosferican: If by group you mean for development and maintaining probably through a Github org. For functionality, pipeline, battery or whatnot, have the main module call them. Reexport.jl is nice for this. DifferentialEquations.jl is mostly an empty repo that just packages everything…

Large vs Small Packages

General Usage

chakravala March 2, 2018, 10:02pm 17

This topic has been discussed in detail already multiple times before

2 Likes

Topic		Replies	Views
Package size and scope? Internals & Design	25	4588	December 1, 2017
Fixing Package Fragmentation Community	71	6294	May 28, 2023
Advice on structuring larger codebases General Usage code-organization	19	2035	March 19, 2020
Dependency policy - should we avoid dependencies or embrace them? General Usage dependencies	20	2353	February 5, 2018
How can we create a leaner ecosystem for Julia? Statistics package , proposal , time-series , machine-learning	101	10837	October 15, 2020

Large vs Small Packages

Related topics