I am looking for a best-practice for the following case:
Say I am the writer of a package Foo.jl, which wants to use the functionality of Bar.jl. I thus add Bar as a dependency to Foo, and start using it.
Now Bar supports generic input types, as long as they implement a set of interface functions, which are defined in a separate light-weight package BarInterface, which is included in the dependencies of Bar.
My question is, if I want to use the functionality in Bar for my custom type FooType defined in Foo, is it considered best practice to either:
Add BarInterface as an explicit dependency and extend its interface functions
Import the interface functions via Bar and extend them this way, without ever requiring the BarInterface as dependency.
Similarly, if I were the writer of Bar and wanted to use this method of separating out my interface functions into a small package, would it be considered good practice to explicitly import these interface functions such that writers of Foo can easily extend them without having to depend on BarInterface if they wanted to, or does this defeat the purpose of having the separate BarInterface package in the first place?
In case a more concrete example is helpful: I am asking this in the context of KrylovKit.jl, which works for arbitrary vector types that implement VectorInterface.jl. Should a package that already explicitly depends on KrylovKit also explicitly depend on VectorInterface, or is it enough to import these functions via KrylovKit?
If there are any alternative constructions for organising these kinds of dependency chains, I’d love to hear them as well
It’s better to depend on the interface package directly, and import it as a normal package. Importing it “through” another package (like using KrylovKit.VectorInterface) is often actually using internal implementation details of the package (KrylovKit). Here, the assumption is that VectorInterface is in the namespace of the module KrylovKit - perhaps someday they could reorganize things so VectorInterface is only loaded in some submodule or package extension. In that case, your code would break if it assumes KrylovKit.VectorInterface exists.
Adding it as a direct dependency is also important so you can manage compatibility bounds. If VectorInterface has a breaking change, KrylovKit could update to accommodate the changes without itself making a breaking change. But your code might break. So it is better to set compatibility bounds with VectorInterface directly.
Lastly, there’s basically no cost to it. If the package would have to be loaded anyway (since KrylovKit needs it), you adding it as a direct dependency doesn’t add any load time etc.
Note: I don’t actually know anything about KrylovKit / VectorInterface specifically, I’m just using it as the example.
I agree that this sounds like the cleanest approach, but maybe let me add a different scenario that could also work:
If KrylovKit were to re-export VectorInterface, I guess this would then mean that VectorInterface becomes part of its public api, so it would have to consider VectorInterface breaking changes as breaking changes for KrylovKit. This effectively moves the responsibility for dealing with the compat bounds to KrylovKit, and the end-user would only need to think about compat entries for KrylovKit, and not KrylovKit and VectorInterface. This might be a desirable feature, as it could become quite a headache to figure out the correct compat bounds for both of these if you do not know much about the internals of both packages. I guess this is also the main cost associated to adding VectorInterface explicitly as a dependence - you now have to maintain the compat entries.
Yes, that’s true, I agree KrylovKit could make VectorInterface part of it’s public API, which would change things. In that case I agree you could avoid the direct dep. However, I don’t think it will actually shield you from very much complexity, since if VectorInterface changes its api or functionality, that will force KrylovKit to change its api or functionality, and you will be exposed to the change anyway.
Also, for compat bounds, I think they can be not-too-hard to maintain following a few simple steps:
set the compat bound to whatever version you are currently using and testing. You can do ] st -m VectorInterface to see that version, then just do VectorInterface = "0.5" or whatever it is. That will declare your package as compatible with v0.5 and any version semver-compatible with that (so v0.5.1 etc).
if VectorInterface has a breaking release (i.e. v0.5 to v0.6 or v1 to v2), check the release notes, update your compat bound to the new version, and test your package still works. Once you have your package updated, make a new release.
Of course it can be more of a headache than that (maybe some dependency hasn’t updated yet, maybe you want to try to support both versions for some reason, etc). But I think these simple rules can make managing compatibility easier, even if you don’t know a ton about the package.