Minimal working example that demonstrates the danger of type piracy

General Usage
,
1

I understand what type piracy is, but I’m still hazy on exactly what sort of problems it can cause. Can someone provide a minimal working example that demonstrates the unexpected behavior that can result from type piracy?

1 Like
2

How extreme do you want to go? The most extreme example I know of is Base.:(+)(::Int, ::Int) = "oops" :slight_smile: Though “working example” is a bit stretched here…

1 Like
3

Ha, well preferably something that doesn’t crash my Julia session.

Here’s the type of example that I’m looking for:

Base.size(::Pair{T, T}) where T = (2,)

Except I can’t think of anything bad that would result from that example of type piracy… :man_shrugging:

A good MWE would probably involve 2 or 3 modules with new functions and types like foo, bar, A, and B, rather than showing type piracy with Base functions and types. I imagine that such a MWE would demonstrate unexpected behavior resulting from type piracy, i.e. not a runtime error but wrong results.

4

Imagine doing this for DataFrame or something in a package you wrote, now DataFrames.jl may not use it, but a third package that uses size() blindly (doesn’t care about DataFrame for example), or a user’s function, now instead of a error they may be getting weird result, silently.

1 Like
5

size already works on a DataFrame, so I don’t see a reason why a user or package would want to define their own size(df::Dataframe) method. Below is a short example that demonstrates the kind of type piracy that might actually be useful. However, the example doesn’t show any ill-effects of the type piracy. Perhaps the example can be elaborated a bit to demonstrate an ill-effect or wrong result that occurs due to the type piracy.

module Package1
    struct A end
    foo(::A) = 42
    export foo, A
end

module Package2
    struct B end
    export B
end

module Package3
    using ..Package1, ..Package2
    Package1.foo(::B) = 1
    export foo
end

julia> using .Package1, .Package2, .Package3

julia> foo(A())
42

julia> foo(B())
1
6

The classic argument is that the type piracy can unexpectedly change how a package works by loading a seemingly unrelated package. I can’t imagine someone intentionally doing this though, unless upstream is unwilling to take a patch or something similar though.

1 Like
7

I know this is the classic argument, but I’m looking for an MWE here. :sweat_smile:

8

You could replace size with something that doesn’t work but “makes sense to an individual” so they decided to define it in their package, later causes issues.

idk what MWE you want because it’s hard to “just come up with real-life example” while keeping it simple/not contrived.

1 Like
9

The following is a type piracy of the signature Tuple{typeof(Base.findmin), Any, Any}:

Base.findmin(f, X) = mapfoldl(x -> (f(x), x), min, X)

This is almost harmless in Julia v1.6.2, and moreover is very useful as follows:

ts = range(0, 2; length=201)
findmin(sinpi, ts)

(1.0, 0.5)

rosenbrock2d((x, y),) = (1 - x)^2 + 100(y - x^2)^2
xs = ys = range(-5, 5; length=1001)
findmin(rosenbrock2d, Iterators.product(xs, ys))

(0.0, (1.0, 1.0))

However, in Julia v1.7.0-beta3 and above, it is destructive because Base.findmin(f, domain) is already and more carefully defined in a different specification. (See https://github.com/JuliaLang/julia/blob/master/base/reduce.jl#L862)

The following is not a type piracy, because its signature Tuple{typeof(Base.findmin), Foo, Any} contains the user-defined type Foo:

struct Foo{F} f::F end
Base.findmin(foo::Foo, X) = mapfoldl(x -> (foo.f(x), x), min, X)

rosenbrock2d((x, y),) = -(1 - x)^2 + 100(y - x^2)^2
xs = ys = range(-5, 5; length=1001)
findmin(Foo(rosenbrock2d), Iterators.product(xs, ys))

(0.0, (1.0, 1.0))

The following is also not a type piracy, because its signature Tuple{typeof(My.findmin), Any, Any} contains the type of the user-defined function My.findmin:

module My
findmin(f, X) = mapfoldl(x -> (f(x), x), min, X)
end

rosenbrock2d((x, y),) = (1 - x)^2 + 100(y - x^2)^2
xs = ys = range(-5, 5; length=1001)
My.findmin(rosenbrock2d, Iterators.product(xs, ys))

(0.0, (1.0, 1.0))

Another story

The following causes the error below in Julia v1.7.0-beta3 and above:

rosenbrock2d((x, y),) = (1 - x)^2 + 100(y - x^2)^2
xs = ys = range(-5, 5; length=1001)
findmin(rosenbrock2d, Iterators.product(xs, ys))

MethodError: no method matching keys(::Base.Iterators.ProductIterator{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base. TwicePrecision{Float64}}, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}}}})

The easiest way to solve this problem is a type piracy of the signature Tuple{typeof(Base.keys), Base.Iterators.ProductIterator} :sweat_smile::

Base.keys(itr::Iterators.ProductIterator) = Iterators.product(keys.(itr.iterators)...)
findmin(rosenbrock2d, Iterators.product(xs, ys))

(0.0, (601, 601))
2 Likes
10

Ok, here’s the MWE that I came up with. Comments and other MWEs are welcome.

julia> module Package1
           foo(::Integer) = 42
           export foo
       end
Main.Package1

julia> using .Package1

julia> foo(1)
42

julia> module Package2
           using ..Package1
           Package1.foo(::Int) = 100
           export foo
       end
Main.Package2

julia> using .Package2

julia> foo(1)
100

Here’s my take:

  • Type piracy of the kind in my previous MWE where Package3 added a foo(::B) method is usually not an issue since it turns code that would have been a MethodError into runnable code.
  • Type piracy is a problem when code that is runnable with Package1 returns a different result after loading Package2. This typically happens when Package2 adds a method specialized for type T (where T is a type not defined in Package2) to a function that already has a method in Package1 for a supertype of T.
1 Like