Optional dependencies and name collisions

General Usage
1

Say I am developing package MyPkg that computes some stuff using an exported type TA. The package also contains an optional dependency on Makie, implemented with Requires.jl. After my computation using MyPkg, I do using Makie to plot the results. The package MyPkg extends the plot function from Makie with a method plot(::TA). But since Makie takes a bit to load, Makie is not loaded by MyPkg up front, only when needed (by employing Requires.jl).

Here comes the rub. Both package MyPkg and Makie export another function with a common name: transform!. The combination of optional dependency and name collision is giving me serious trouble. I do using MyPkg and then I can use transform!(::TA) defined by MyPkg. But as soon as I do using Makie I get

WARNING: both MyPkg and Makie export "transform!"; uses of it in module Main must be qualified

From that point on I cannot use transform! anymore, just MyPkg.transform! and Makie.transform!

If the dependency on Makie were not optional I would of course just extend transform! from Makie, but with an optional dependency I don’t know how to do that. Any idea?

2

Use import and prefix calls with module name.

Exporting a generic name like transform! is really asking for trouble…

1 Like
3

Thanks @kristoffer.carlsson. Renaming transform! might end up to be the best solution, but I actually thought it would be possible to merge the transform! method tables of both packages, as they act on types defined within the packages themselves (no piracy). Why is it not possible?

To be more specific, this is my use case:

module MyPkg

using Requires

export transform!, plot, compute

struct TA
    a
end

function compute()
    # ... heady stuff!
    # returns a ::TA
end

function transform!(::TA)
    # ... useful stuff
end

@require Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a" begin
    using Makie
    function plot(::TA)
        #plot code
    end
end

end

And my usage would be

using MyPkg
solution = compute()  # returns a ::TA
solution2 = transform!(solution)
using Makie # now plot(::TA) is loaded too
plot(solution2)
solution3 = tranform!(solution2)  # I cannot do this anymore! I need to say MyPkg.transform!(solution2)
4

I strongly suspect that I am actually doing something wrong here. I actually have the same problem with plot as with transform!, so this must certainly not be the way to go about this problem of optionally extending functions from other packages… It’s the first time I try.

5

Automatic method merging would prevent writing generic code since you have no idea anymore what functions mean.

In other words, a generic function is more than just a name. It has a higher-level purpose that can be documented and allow us to write generic code based on that.

By extending a function (e.g push!) you are agreeing to the meaning of that function, so in the push! case:

help?> push!

  push!(collection, items...) -> collection

  Insert one or more items at the end of collection.

We can now write generic code using push! because we know what push! does.

It is fine for another function called push! to exist which has a completely different meaning, and you chose which one of these you are using based on what you import or by module prefixing.

6 Likes
6

Aha, that’s a very good point. I’m beginning to get it. For unqualified use of a function one needs to be clear on who is the original owner, and who is extending it. It makes perfect sense. Then, in a case like mine, I have to make sure none of my exported functions are named like any exported functions of an optional dependency. And to extend plot I really need to do using Makie: plot import Makie: plot inside the @require so that I acknowledge Makie as the owner.
Good lesson!

1 Like
7

One implication of this genericism is that what a function means needs to be defined somewhere. A common way of organising julia packages is to have a “base” package that defines these function meanings and exports the function names. Specific packages implement those methods for their relevant domain. For instance, StatsBase.jl implements a function called coeftable. But the implementation in StatsBase is just an error saying it hasn’t been implemented. The various statistical packages then import that name, implement it as appropriate (e.g. in GLM lm for an lm, and here for a glm) and then export it. This way, groups of related packages can coordinate on the meaning of functions, and have a single meta package to import the names from.

5 Likes