# Name this method extension technique, piracy evasion
**URL:** https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099
**Category:** General Usage
**Tags:** method-invalidation, methods
**Created:** [July 2, 2023, 11:52pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099 "2023-07-02T23:52:06Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 2, 2023, 11:52pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/1 "2023-07-02T23:52:07Z")
Let’s say I want to change how `Base.write` works with a Base type, `NTuple{N,UInt8}` from a package.
Currently, writing `NTuple` via `Base.write` errors.
```julia
julia> iob = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)
julia> data = (0x01,0x02,0x03)
(0x01, 0x02, 0x03)
julia> Base.write(iob, data)
ERROR: MethodError: no method matching write(::IOBuffer, ::Tuple{UInt8, UInt8, UInt8})
julia> typeof(data)
Tuple{UInt8, UInt8, UInt8}
julia> typeof(data) == NTuple{3,UInt8}
true
```
Under type piracy rules, I should not do the following because
1. I did not originate the types.
2. I did not orginate the method.
```julia
function Base.write(io::IO, data::NTuple{N,T}) where {N,T}
sum(map(1:N) do i
Base.write(io, data[i])
end)
end
julia> write(iob, data)
3
```
To evade commiting piracy, I will originate a new method called `Foo.write`.
```julia
module Foo
# Edit: comment out the export here. People should do this more explicitly.
# export write
# Fallback to Base implementation
write(args...; kwargs...) =
Base.write(args...; kwargs...)
function write(io::IO, data::NTuple{N,T}) where {N,T}
sum(map(1:N) do i
Base.write(io, data[i])
end)
end
end
```
This is not piracy because I did not violate both criteria I outlined above. I can use this as follows in a fresh REPL session.
```julia
# Explicitly indicate the use Foo.write instead of Base.write
julia> using .Foo: write
julia> @which write
Main.Foo
julia> iob = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)
julia> data = (0x01,0x02,0x03)
(0x01, 0x02, 0x03)
julia> write(iob, data)
3
julia> write(iob, 4)
8
julia> write(iob, 0x5)
1
julia> take!(iob)
12-element Vector{UInt8}:
0x01
0x02
0x03
0x04
0x00
0x00
0x00
0x00
0x00
0x00
0x00
0x05
```
One could also use a `baremodule` to not have `Base` imported.
Another important effect is that I do not invalidate any uses of `Base.write` and thus do not force recompilation for any other packages.
Using the language of object oriented programming, `Foo.write` is a submethod that inherits from `Base.write`. `Foo.write` does everything `Base.write` does but it also writes `NTuple`. Is there a preexisting name for this?
Can you think of a better name?
**Edit** : Commented out the `export write` from the example.
---
### Author:  [@jling](https://discourse.julialang.org/u/jling)
#### Post date: [July 3, 2023, 12:35am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/2 "2023-07-03T00:35:21Z")
Well, if someone else export write, now you have conflict? Basically Plots.plot and Makie.plot
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [July 3, 2023, 4:09am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/3 "2023-07-03T04:09:15Z")
> [@mkitti](#):
>
> `Foo.write` is a submethod that inherits from `Base.write`
That is not my understanding, `Foo.write` calls `Base.write` but functions/methods do not inherit from each other. A method in a subclass may override a method in a superclass, providing single dispatch. `Foo` is a separate module from `Base`, not a subclass inheriting from a superclass.
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 6:11am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/4 "2023-07-03T06:11:38Z")
> [@jling](#):
>
> Well, if someone else export write, now you have conflict? Basically Plots.plot and Makie.plot
This is distinct from that case. For `plot`the underlying implementations are from completely different origins. Here there is a shared origin. The case for wanting to use both simultaneously is much smaller. Here `Foo.write` does everything that `Base.write` does, but Foo adds a capability where Base errors. If you need Foo’s extra feature, then there is no need to access the Base version.
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 6:22am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/5 "2023-07-03T06:22:23Z")
> [@Benny](#):
>
> That is not my understanding, `Foo.write` calls `Base.write` but functions/methods do not inherit from each other. A method in a subclass may override a method in a superclass, providing single dispatch. `Foo` is a separate module from `Base`, not a subclass inheriting from a superclass
Formally, you are correct. There is no explicitly relationship between the two via the type system.
Effectively though by forwarding all arguments the `Foo.write` has inherited all of the functionality of `Base.write`
```julia
write(args...; kwargs...) =
Base.write(args...; kwargs...)
```
The analogy that I making is that the relationship between `Foo.write` and `Base.write` is similar to that between subclass and its parent class. Rather here we have a submethod and a method. Changing the method will affect the submethod. Changing the submethod will not affect the parent.
Effectively, what we have in Julia is method oriented programming enabled by multiple dispatch.
I am not claiming that any relationship exists between the modules.
---
### Author:  [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [July 3, 2023, 6:32am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/6 "2023-07-03T06:32:20Z")
Yes, from a type theoretic perspective, you are correct (though this has nothing to do with multiple dispatch). I wouldn’t bring object orientation/classes into this (there’s way too much baggage around that term) - what you’ve found is simply a case of behavioral subtyping (which admittedly originates in object orientation).
The issue that comes with your definition of type piracy though is that because you don’t own any argument types and you intend your `Foo.write` to otherwise behave exactly the same as `Base.write`, as soon as `Base.write` adds a method with the same signature, your code is either broken (it doesn’t do the same thing as `Base.write` anymore) or you’ve violated the contract of `Base.write` (not extending it with methods on types you don’t own). So while you’ve _technically_ skirted around your definition of piracy in the literal sense, I’d argue that you’re still comitting type piracy, because there’s an expectation from `Base.write` that it’s free to define methods for types it owns.
It gets a bit muddy mostly because you’ve called your function `write` too - if it were called differently, there wouldn’t be any issue, because the expectation that it would behave like `Base.write` wouldn’t exist.
* * *
Put differently, you’ve assigned your `Foo.write` a contract it can’t possibly fulfill - being exactly like `Base.write`, while doing some things potentially different from `Base.write`, all the while having no control over what `Base.write` may decide to (rightly so) do on types it owns.
---
### Author:  [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)
#### Post date: [July 3, 2023, 8:44am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/7 "2023-07-03T08:44:27Z")
> [@Sukera](#):
>
> So while you’ve _technically_ skirted around your definition of piracy in the literal sense, I’d argue that you’re still comitting type piracy
Not really – this isn’t type piracy even when Base adds a method with that signature. You’re just shadowing the original implementation with your own, which may of course have disastrous consequences, but at least they’re local (whereas actual type piracy has global consequences).
Of course, the non-global nature of this shadowing also limits its usefulness compared to actual type piracy (well, modifications to the global method table in general).
> [@Sukera](#):
>
> Put differently, you’ve assigned your `Foo.write` a contract it can’t possibly fulfill - being exactly like `Base.write`, while doing some things potentially different from `Base.write`, all the while having no control over what `Base.write` may decide to (rightly so) do on types it owns.
That contract _can_ be fulfilled at any given point of time. It just might not hold for future releases any more.
---
### Author:  [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [July 3, 2023, 8:58am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/8 "2023-07-03T08:58:41Z")
> [@pfitzseb](#):
>
> That contract _can_ be fulfilled at any given point of time. It just might not hold for future releases any more.
My point is that forward compatibility (with `Base.write` in this case) is part of the contract of any well-behaved stable API; that’s not something `Foo.write` can provide, because there’s no room for a transition if Base decides to make that change and the method differs from the one `Foo.write` does. The only option left is to drop the requirement to be equivalent to `Base.write`; that is, having this as the contract of `Foo.write`:
```julia
Foo.write(io::IO, args...; kwargs...)
Except for `Foo.write(io::IO, NTtuple)`, `Foo.write` passes its arguments to `Base.write`.
```
is perfectly fine & well behaved, because it explicitly states that it doesn’t care what `Base` does with `NTuple` arguments - that difference is the defining feature of `Foo.write` after all. What’s left is the easily confused, but intentionally chosen name - to me an indicator that this shouldn’t be done 🤷
---
### Author:  [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)
#### Post date: [July 3, 2023, 9:32am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/9 "2023-07-03T09:32:52Z")
> [@Sukera](#):
>
> What’s left is the easily confused, but intentionally chosen name - to me an indicator that this shouldn’t be done 🤷
Fair enough. Whether the name is confusing is debatable though, because `Foo.write` _still_ satisfies the “definition” (intention) of `Base.write`, specifically
> Write the canonical binary representation of a value to the given I/O stream or file. Return the number of bytes written into the stream
So in this specific case I’d argue that it’s impossible for Base to define a method for these input arguments that results in a different output to `Foo.write`.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [July 3, 2023, 10:36am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/10 "2023-07-03T10:36:51Z")
> [@pfitzseb](#):
>
> So in this specific case I’d argue that it’s impossible for Base to define a method for these input arguments that results in a different output to `Foo.write`.
The behavior of `Foo.write` may be faithful to the intention of `Base.write`, but they could do it in measurably different ways. `Foo.write` here specializes on every tuple type, `Base.write` could `@nospecialize` instead to avoid more compilation.
---
### Author:  [@jling](https://discourse.julialang.org/u/jling)
#### Post date: [July 3, 2023, 11:39am UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/11 "2023-07-03T11:39:48Z")
> [@mkitti](#):
>
> Here `Foo.write` does everything that `Base.write` does, but Foo adds a capability where Base errors. If you need Foo’s extra feature, then there is no need to access the Base version.
I’m saying another package can do the same thing as your `Foo`.
e.g. `Bar.write` might get exported and now users will see conflict
---
### Author:  [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)
#### Post date: [July 3, 2023, 1:45pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/12 "2023-07-03T13:45:15Z")
@jling, yes, a namespace conflict, not a method override conflict, I think it would be good to make this explicit. If someone does not employ `using PackageName`, and instead always `import` specific names this will not happen. And, in fact, any package can have this problem (i.e., any two packages can have exported functions with the same name and this may have nothing to do with _piracy evasion_ or delegating to `Base`, it may be just an unfortunate coincidence).
About the taxonomy, it may be that I am being pedant, but I think the terms would be `delegation` or `forwarding` instead of `inheritance`. Which one of the two is a very subtle detail: [Forwarding (object-oriented programming) - Wikipedia](https://en.wikipedia.org/wiki/Forwarding_(object-oriented_programming))
---
### Author:  [@jling](https://discourse.julialang.org/u/jling)
#### Post date: [July 3, 2023, 1:56pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/13 "2023-07-03T13:56:42Z")
exactly, I guess my point was that this does evade piracy, but it doesn’t actually solve underlying problem, it’s not a solution that can be adopted by ecosystem because it doesn’t scale.
If people replace their want-to piracy with exporting shadowing `Base.blah()`, it would be pretty annoying
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 3:44pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/14 "2023-07-03T15:44:15Z")
This does scale better than piracy itself. The `export` declaration is probably too much of a distraction here, and people should explicitly declare they are using it. I’m commenting out the `export`.
I could imagine an `ExtendedBase.jl` that shadows all of `Base` but allows for faster iteration than Julia itself. It would be a forward looking version of Compat.jl in a sense.
---
### Author:  [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)
#### Post date: [July 3, 2023, 5:18pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/15 "2023-07-03T17:18:53Z")
Yes, I think the main idea of this pattern is to allow for Quality of Life (QoL) improvements. Ideally, this could be a way for solve problems before Julia/`Base` releases a new version and then have the code adopted in `Base` if it seems like a good idea.
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 6:30pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/16 "2023-07-03T18:30:13Z")
> [@Henrique\_Becker](#):
>
> About the taxonomy, it may be that I am being pedant, but I think the terms would be `delegation` or `forwarding` instead of `inheritance`. Which one of the two is a very subtle detail: [Forwarding (object-oriented programming) - Wikipedia](https://en.wikipedia.org/wiki/Forwarding_(object-oriented_programming))
I like “forwarding”. That seems very intuitive. I’m forwarding all the positional and keyword arguments. I’m not sure if the difference between “delegation” and “forwarding” applies here since there is no wrapper object involved.
> [@Sukera](#):
>
> Put differently, you’ve assigned your `Foo.write` a contract it can’t possibly fulfill - being exactly like `Base.write`, while doing some things potentially different from `Base.write`, all the while having no control over what `Base.write` may decide to (rightly so) do on types it owns.
This issue exists in object oriented programming as well. If the parent base class changes, then the subclass will be affected.
I would document the contract as follows.
> This method _forwards_ to `Base.write` except for the following overloads.
>
> 1. `write(io::IO, x::NTuple{N,T}) where {N,T}` which normally errors for `Base.write`.".
> [@Henrique\_Becker](#):
>
> Yes, I think the main idea of this pattern is to allow for Quality of Life (QoL) improvements. Ideally, this could be a way for solve problems before Julia/`Base` releases a new version and then have the code adopted in `Base` if it seems like a good idea.
Yes. Perhaps this behavior, with the export now gone, is limited to within a module, so this could be seen as a form of private method overloading.
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 7:11pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/17 "2023-07-03T19:11:52Z")
I was thinking that a syntax `function write <: Base.write end` might be nice. This is currently a syntax error, so perhaps this could be valid later. For now we could use a macro to do enable the following syntax: `function write() <: Base.write end`. This also allows for `supertype(typeof(Foo.write))` to work.
```julia
module MethodForwarding
export @methodfwd
macro methodfwd(ex)
if ex.head == :function && ex.args[2].args[end].head == :<:
fn = esc(ex.args[1].args[1])
parent = esc(ex.args[2].args[end].args[1])
else
error("@methodfwd can only be applied to explicit `function` declarations that containing a unary `<:`")
end
quote
$fn(args...; kwargs...) = $parent(args...; kwargs...)
Base.supertype(::Type{typeof($fn)}) = typeof($parent)
end
end
end
```
We could then write `Foo` as follows.
```julia
module Foo
import ..MethodForwarding: @methodfwd
@methodfwd function write() <: Base.write end
function write(io::IO, data::NTuple{N,T}) where {N,T}
sum(map(1:N) do i
Base.write(io, data[i])
end)
end
end
```
Then do the following.
```julia
julia> supertype(typeof(Foo.write)) == typeof(Base.write)
true
julia> Foo.write == Base.write
false
```
---
### Author:  [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [July 3, 2023, 9:03pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/18 "2023-07-03T21:03:44Z")
I like the general idea of this, but only in combination with something like [Taking API surface seriously: Proposing Syntax for declaring API · Issue #49973 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/49973) or [this experiment](https://github.com/Seelengrab/RequiredInterfaces.jl) I plan to use for [PropCheck.jl](https://github.com/Seelengrab/PropCheck.jl) (the change integrating the experiment into the latter is already done, I’ve just got to finalize & push it to github, after which I’ll register both packages).
My one counterpoint to `function write <: Base.write end` is that what you’re actually claiming with such a declaration is that your `write` is behaviorally compatible with `Base.write` , which it (as mentioned above) cannot be, as `Base.write` is (currently) free to define whatever methods on Base types it pleases. Not defining `Base.write(::IO, ::NTuple)` simply isn’t part of the contract of `Base.write`.
I’m also not sure forwarding like that is very useful in practice - what problem does it solve, other than being a way to get around type piracy?
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [July 3, 2023, 9:27pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/19 "2023-07-03T21:27:45Z")
> [@Sukera](#):
>
> what problem does it solve, other than being a way to get around type piracy?
1. I encapsulate the method overload to the current module.
2. I avoid causing method invalidations for other packages
3. Others do not have rewrite their code to use my modified method in their packages. They can just import the version from my package.
The alternative is that I give my method a completely new name.
```julia
module NTupleFriendlyWrites
ntuple_friendly_write(args...; kwargs...) = Base.write(args...; kwargs...)
function ntuple_friendly_write(io::IO, data::NTuple{N,T}) where {N,T}
sum(map(1:N) do i
Base.write(io, data[i])
end)
end
end
```
I suppose people could just do `const write = NTupleFriendlyWrites.ntuple_friendly_write` though.
---
### Author:  [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [July 3, 2023, 9:30pm UTC](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099/20 "2023-07-03T21:30:30Z")
> [@mkitti](#):
>
> Others do not have rewrite their code to use my modified method in their packages. They can just import the version from my package.
As pointed out above though, as soon as people actually want to use this, they’ll get namespace conflicts with Base. Not to mention that punning on the name/shadowing the definition from Base means inconsistent expectations when people don’t use the package.
I think it’s generally better to try to upstream such additions to an API, instead of wrapping & punning on the name.
[Next page](https://discourse.julialang.org/t/name-this-method-extension-technique-piracy-evasion/101099.md?page=2)