When to Use An Interface ("Base-like") Package

[astrobc1]

I’m uncertain as to at what point when developing a package it’s worth splitting things up into a “base” package containing the interface(s)/api + abstract types, and a “primary” package containing concrete types and implementations of the interface(s).

Some specific questions to keep in mind:

1. Is it a bad choice to define an interface and concrete implementations within the same package?
2. If my project uses sub-modules with “sub-interfaces”, should I define those modules within the base or primary package? In other words, is it common for base packages to have sub-modules?
3. Is it ok to define “utility” methods within the base package (e.g., methods that deal only with built in Julia types and may be useful across the package use cases)?
4. Within the base package, is it ok to define a concrete type which is expected to be sufficient for 99% of use cases, or should this still belong in the primary package?
5. What examples would you recommend to look over? I was looking at Optim.jl & OptimBase.jl, however there are strange choices I couldn’t make sense of, like OptimBase defining:

abstract type Optimizer end

but then Optim.jl defines as its base Optimizer type:

abstract type AbstractOptimizer end

I have read over this blog post but I’m still confused what best practices are. Thanks for any and all insight!

[ChrisRackauckas]

@astrobc1

• Is it a bad choice to define an interface and concrete implementations within the same package?

Is that said package the only package using the interface? Then no not only is that totally fine, that much better since it’s simpler. Keep things simple when you can. Interface packages are for the case where tons of packages are building off said interface, and you want to be able to share the high level code structure.

@astrobc1

If my project uses sub-modules with “sub-interfaces”, should I define those modules within the base or primary package? In other words, is it common for base packages to have sub-modules?

Usually not. Usually if you’re going into a level of detail like that, it’s starting to be implementation of the interface, and the implementation should be kept away from the interface.

@astrobc1

Is it ok to define “utility” methods within the base package (e.g., methods that deal only with built in Julia types and may be useful across the package use cases)?

Yes, that’s fine. Different Base packages have varying levels of detail in them.

@astrobc1

Within the base package, is it ok to define a concrete type which is expected to be sufficient for 99% of use cases, or should this still belong in the primary package?

Yes, totally fine. Even better to force everyone in the interface to use the same type. For example, in SciML there are like 10+ ODE solver packages, but all of them take in an ODEProblem and solve spits out an ODESolution. To a user, any differences between the solvers are fully abstracted away using this interface.

@astrobc1

1. What examples would you recommend to look over? I was looking at Optim.jl & OptimBase.jl, however there are strange choices I couldn’t make sense of, like OptimBase defining:

SciMLBase.jl is probably the most typical example (GitHub - SciML/SciMLBase.jl: The Base interface of the SciML ecosystem). MathOptInterface is another. I wouldn’t look at StatsBase though because that’s more of an atypical example and more of a functionality package at this point.