# How to detect/avoid type piracy?
**URL:** https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588
**Category:** General Usage
**Created:** [May 22, 2024, 5:31pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588 "2024-05-22T17:31:49Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 22, 2024, 5:31pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/1 "2024-05-22T17:31:49Z")
Ouch, it seems I’ve been bitten by the [correctness issue](https://yuri.is/not-julia/) in its following manifestation:
1. My package overloads the `Base.split()` for (::AbstractVector, ::Integer).
Unit tests pass, scripts/REPL work correctly.
2. However, when using the package in Pluto, I started to get incorrect results, _silently_, no errors or warning.
3. Debugging revealed that Pluto quietly loads the Lazy.jl package which also overloads `Base.split(::AbstractVector, ::Any)` which ended up being used instead of my method.
A type piracy has been committed. Unintended and undetected.
Isn’t this scary? A package developer has no control over the combination of packages his package will be used in. No amount of testing would help here.
So, what can be done?
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [May 22, 2024, 5:36pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/2 "2024-05-22T17:36:34Z")
Aqua.jl can spot type piracy: [Type piracy · Aqua.jl](https://juliatesting.github.io/Aqua.jl/stable/piracies/)
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [May 22, 2024, 5:37pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/3 "2024-05-22T17:37:27Z")
I just downloaded Pluto and checked the manifest. There does not seem to be any dependency on Lazy.jl . This makes sense, because Lazy.jl is an unmaintained package (it’s last commit was 4 years ago).
So I think the solution is
1. Don’t overload methods on types you don’t own. That is, don’t commit type piracy.
2. Don’t use Lazy.jl, or other packages which commit type piracy. It’s not Pluto that’s causing a dependency on Lazy.jl, though. It’s something else.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [May 22, 2024, 5:52pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/4 "2024-05-22T17:52:37Z")
Odd, if Pluto loads Lazy then I’d expect that to occur before the user can load anything. Do you import anything after your package?
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 10:10am UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/5 "2024-05-23T10:10:34Z")
Thanks for the advice!
> 1. Don’t overload methods on types you don’t own. That is, don’t commit type piracy.
Yep. Just checking that an existing function like `Base.split()`, has no specialisation for some type `::T` (e.g., `::AbstractVector` in my case), isn’t enough.
> 1. Don’t use [Lazy.jl](https://juliahub.com/ui/Packages/General/Lazy), or other packages which commit type piracy.
But how to ensure it? Get every package checked buy Aqua.jl?
> It’s not Pluto that’s causing a dependency on [Lazy.jl](https://juliahub.com/ui/Packages/General/Lazy), though. It’s something else
Oh, my apology to Pluto.jl then for jumping to a conclusion here.
---
### Author:  [@screw\_dog](https://discourse.julialang.org/u/screw_dog)
#### Post date: [May 23, 2024, 12:00pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/6 "2024-05-23T12:00:57Z")
> [@vvbond](#):
>
> Just checking that an existing function like `Base.split()`, has no specialisation for some type `::T` (e.g., `::AbstractVector` in my case), isn’t enough.
This is exactly the definition of type piracy. You don’t own the type `AbstractVector` (or `Integer` as mentioned in your first post), nor the function `Base.split`. So defining a method of **that** function with **those** types is type piracy.
You need to either define a new function (not a new method of a function in `Base`!), or a new type (ie a subtype of `AbstractVector`) that you can define a method for.
It shouldn’t be surprising that defining a new method for a function you don’t own, with types you don’t own, causes unexpected behaviour.
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 12:39pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/7 "2024-05-23T12:39:04Z")
My bad: turns out it is not Pluto.jl that loads Lazy.jl but PlotlyJS.jl via Blink.jl:
```julia
PlotlyJS v0.18.13
├─ Blink v0.12.9
│ ├─ Pkg v1.10.0 (*)
│ ├─ Mux v1.0.2
│ │ ├─ Pkg v1.10.0 (*)
│ │ ├─ HTTP v1.10.8 (*)
│ │ ├─ Hiccup v0.2.2
│ │ │ └─ MacroTools v0.5.13 (*)
│ │ ├─ AssetRegistry v0.1.0 (*)
│ │ └─ MbedTLS v1.1.9 (*)
│ ├─ HTTP v1.10.8 (*)
│ ├─ JSExpr v0.5.4 (*)
│ ├─ Lazy v0.15.1
│ │ └─ MacroTools v0.5.13 (*)
│ ├─ JSON v0.21.4 (*)
.
.
.
```
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 1:09pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/8 "2024-05-23T13:09:35Z")
I agree. It may be not surprising in hindsight. But this “mistake” is very easy to make: both myself and the author of Lazy package, being carried away by Julia’s power of generic programming, committed the piracy with the intention to extend the Base function in a generic way, i.e., using an informal abstract interface like `AbstractVector`.
My concern is that this mistake occurs in silence. Shouldn’t be there a safe guarding mechanism that would warn about such ambiguities like methods clash?
---
### Author:  [@Dan](https://discourse.julialang.org/u/Dan)
#### Post date: [May 23, 2024, 1:25pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/9 "2024-05-23T13:25:43Z")
There are some mitigating circumstances if the pirated definition is very natural and as such is essentially the only way to define something (think of Robin Hood, he wasn’t really a pirate 😉).
The following is how the Lazy definition works:
```julia
julia> v = [8, 8, 4, 1, 8, 5, 9, 2, 8, 3, 10, 5, 1, 6, 2, 1, 7, 5, 1, 7];
julia> split(v,5)
4-element Vector{Vector{Int64}}:
[8, 8, 4, 1, 8]
[9, 2, 8, 3, 10]
[1, 6, 2, 1, 7]
[1, 7]
```
How does the definition in your package work?
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 1:34pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/10 "2024-05-23T13:34:46Z")
The second argument in my implementation is a set of _indices_ to perform splits at:
```julia
julia> split([1,2,3,4,5,6,7,8,9, 10], [3, 6, 8])
4-element Vector{Vector{Int64}}:
[1, 2, 3]
[4, 5, 6]
[7, 8]
[9, 10]
```
---
### Author:  [@mikmoore](https://discourse.julialang.org/u/mikmoore)
#### Post date: [May 23, 2024, 2:50pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/11 "2024-05-23T14:50:10Z")
`Base.split` is only defined and used for strings. Another piece of advice is to never specialize a method if you’re making it serve some other purpose. For example, you should not overload `Base.size(::TeeShirt)` for your custom `TeeShirt` type to return shirt size, because `Base.size` is about the dimension of arrays.
Instead, only overload a method if you need code _written by other people_ to operate on your custom type in a custom way (either because the existing version would not work or would be inefficient). For example, it’s common to extend `Base.getindex` to new matrix types because you need them to be _usable_ like matrices in other code (e.g., in matrix multiplication code).
In this case, since nothing else exists that already provides this functionality (`Iterators.partition` is the closest I can think of), I would just rename your function to something else that is entirely new.
---
### Author:  [@Dan](https://discourse.julialang.org/u/Dan)
#### Post date: [May 23, 2024, 2:54pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/12 "2024-05-23T14:54:14Z")
Also look at [Home · ChunkSplitters.jl](https://juliafolds2.github.io/ChunkSplitters.jl/dev/) which might already have the functionality you want.
Or, the following might help:
```julia
julia> indexchunks(v, idxs) =
(i==0 ? (firstindex(v):idxs[1]) :
i==length(idxs) ? (idxs[i]+1:lastindex(v)) :
(idxs[i]+1:idxs[i+1]) for i in 0:length(idxs))
indexchunks (generic function with 1 method)
julia> [[1:10...][chnk] for chnk in indexchunks(1:10, [3,6,8])]
4-element Vector{Vector{Int64}}:
[1, 2, 3]
[4, 5, 6]
[7, 8]
[9, 10]
```
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [May 23, 2024, 3:03pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/13 "2024-05-23T15:03:29Z")
[Pirate Hunter](https://discourse.julialang.org/t/pirate-hunter/20402) has a script to help find type piracy, though I’m not sure if it needs to be updated
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [May 23, 2024, 3:05pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/14 "2024-05-23T15:05:01Z")
Maybe having some specific Aqua tests could be requirement for auto-merging a package in the registry?
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [May 23, 2024, 3:09pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/15 "2024-05-23T15:09:47Z")
to me that sounds reasonable at a high-level, but I worry about the details. E.g. some packages are essentially allowed to pirate methods from another, since they are designed together to do so. Or packages like GenericLinearAlgebra that add linear algebra methods for bigfloats etc are piracy, but basically desired. So given there is some amount of desired piracy in the ecosystem, we wouldn’t want to necessarily require manual merges for every version of such packages. But then it gets more complicated to make an allowlist etc.
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 4:59pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/16 "2024-05-23T16:59:13Z")
Thanks for the pointer! Seems to be a useful package.
The code is in essence my implementation of `Base.split()`.
The consensus here seems to be that it would be better served by a different name.
---
### Author:  [@fatteneder](https://discourse.julialang.org/u/fatteneder)
#### Post date: [May 23, 2024, 5:17pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/17 "2024-05-23T17:17:59Z")
Another alternative is to not add your method to `Base.split`, but instead keep it local to your pkg.
```julia
julia> module MyPkg
split(::AbstractVector, ::Any) = ...
end
julia> using MyPkg
julia> MyPkg.split(...)
```
---
### Author:  [@croberts](https://discourse.julialang.org/u/croberts)
#### Post date: [May 23, 2024, 5:25pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/18 "2024-05-23T17:25:44Z")
> [@ericphanson](#):
>
> E.g. some packages are essentially allowed to pirate methods from another, since they are designed together to do so.
I wish there were a more explicit way to do this and report downstream piracy (privateering?) conflicts.
This is related to the traits issue highlighted in the viral instruction post (“what’s bad about Julia.”). Julia facilitates polymorphism through the workflow: (1) subtype some abstract type from another package (e.g. “AbstractArray”); (2) implement a list of methods for your new subtype (e.g. Base.size, Base.iterate, etc…). However, it is currently not at all clear what methods must be implemented if the downstream package’s struct is to work with other functions defined on the struct’s abstract supertype. There should be a way to declare method implementation requirements for subtypes of an abstract type (this would resemble the Holy Traits pattern) and it should be quarriable (e.g. with a function Base.methodrequirements(my\_type)::Vector{Function}); packages that implement a subtype of a supertype with declared method requirements would throw a warning upon loading if the package does not define a required method. Similarly, there should be a way to declare functions that may be pirated while reporting duplicated piracy in downstream packages.
---
### Author:  [@mikmoore](https://discourse.julialang.org/u/mikmoore)
#### Post date: [May 23, 2024, 5:40pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/19 "2024-05-23T17:40:27Z")
> [@croberts](#):
>
> There should be a way to declare method implementation requirements
I believe Interfaces.jl is working toward this goal.
---
### Author:  [@vvbond](https://discourse.julialang.org/u/vvbond)
#### Post date: [May 23, 2024, 5:47pm UTC](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588/20 "2024-05-23T17:47:02Z")
Hm, is it so? In my environment, `Base.size` counts 277 methods. I don’t think all of them are array related. E.g.,
- [size(p::FFTWPlan) = p.sz](https://github.com/JuliaMath/FFTW.jl/blob/9b330c6871af236f0fba9d3376f1f95a7a01e69e/src/fft.jl#L308C1-L308C25)
or
- [Base.size(o::CCIPCA) = (indim(o), outdim(o))](https://github.com/joshday/OnlineStats.jl/blob/ba04afa24df766b949ab46f9fd806a168acc4d32/src/stats/pca.jl#L78)
looks close to the `TeeShirt` counter example.
One of my favourite things about multiple dispatch was the freedom to choose those name for my function which other people are used to and are more likely to understand without help instructions. On this ground, I’d precisely overload `Base.size(::TeeShirt)` for my `TeeShirt’s, since the name is _natural_.
I was under impression that by allowing for that Julia community was creating a programming ecosystem governed rather by natural than a programming language.
Is my thinking completely wrong?
[Next page](https://discourse.julialang.org/t/how-to-detect-avoid-type-piracy/114588.md?page=2)