Actually, does this follow? Assuming we write a definition for S as opposed to aliasing a Union for example, typeof(S) would be DataType or UnionAll, which along with the parametric Type are definitely owned by Base. But if we own S, is Type{S} really owned by Base? The concepts of ownership and type piracy are intended to prevent independent packages from interfering with other’s method dispatch, e.g. *(::Symbol, ::Symbol) is a possible call signature without your package (specifically unimplemented in Base alone) so you shouldn’t implement it. However, convert(::Type{S}, ::Symbol) requires your package’s S definition. Moreover that seems like an idiomatic Base.convert method.
I think it does (but I’m no authority). When you implement a new type SomeType{T} and define functions to manipulate it, you need to know that these functions do what you wrote, also internally, not just at the surface exposed to the user.
Consider this example:
mutable struct S{T}
x::T
y::T
a::Union{Missing,T} # value to be cached
S(x::T,y::T) where T = new{T}(x, y, missing)
end
function get_a(s::S)
if ismissing(s.a)
s.a = expensive_function(s.x, s.y)
end
return s.a
end
function get_b(s::S)
get_a(s) # Make sure `a` is cached
return s.a * s.a;
end
I don’t want to document that get_b is going to call get_a to populate the cached value of a. This is all internal stuff, but I should be able to depend on it when I implement my own functions on my own types. Also in the case of parametric types. So it makes sense to me that users are not allowed to define get_a(s::S{CustomType}) even if they own CustomType.
Of course it’s fine if I document that it’s allowed (as in the convert case).
Whether or not defining these properties is a good idea (unlikely) I don’t see how defining methods on NotMyType{MyType} is piracy.
It might not be a good idea, mind you: people generally expect that NotMyType will behave in a consistent way, and it’s bad form to go against the grain that way.
But notmymethod(t::NotMyType{MyType}) can’t affect code you don’t own, which is the definition of type piracy. Any other notmymethod(t:NotMyType) is either less specific or concretely different from that signature, so this cannot change the method resolved in any other circumstance.
I could be missing something here, in which case I’ll need to change a couple methods on Vector{MyType} which I’ve defined while assuming that this isn’t piratical. So this is more of a motivated question than a definitive statement.
5 Likes
I tried to show in my example above that the owner of notmymethod and NotMyType might expect the call to have undocumented side effects. I think it’s quite reasonable if it’s all internal like in that example. So in this case type piracy is causing problems because of ways it doesn’t affect the code it doesn’t own…
So looks like we need to have two actors to talk about piracy: “I” and “others”. So the question is, is Julia also “others”? I think yes, right? Because if I overwrite, say, propertynames(::Type{MyType}) and get rid of :name, it does break things for Julia, even though it only affects code I own. That’s how I’m making sense of it anyway.
AFAIK, the concept of type piracy does not involve whether names are public or internal. Defining my custom method myfoo(::NotMyPackage.InternalType) is not type piracy. However it is not good general practice because minor revisions to NotMyPackage can break or remove InternalType, so I’d have to fix my own code every time that happens. If that code is a package, I’d have to make strict [compat] and couple my own package revisions to NotMyPackage.
I agree. But one of the ways in which type piracy can cause problems, is by interfering with the internal working of a package, like in my example. By breaking the assumptions made internally by the authors.
Piracy is pretty straightforward and has nothing to do with internals.
It’s all about the fact that method tables are a shared global resource. The package that defines the function can define its methods with any types it wants, from any part of the ecosystem. It’s defining the standard behaviors.
Other packages should only extend that function with types they own in accordance with the function’s expectations.
It’s the both situation that’s the problem: when you’re adding methods to a function you don’t own with types you don’t own. That’s piracy. Style Guide · The Julia Language
Of course, you can also just implement broken definitions that aren’t pirating that’ll crash internals because they’re plain old buggy.
8 Likes
I think we all agree on that, the discussion here is about what qualifies as “types you own”. In particular, does NotMyType{MyType} qualify or not…
1 Like
I don’t think extending a method with ::NotMyType{MyType} would be considered piracy in and of itself, no, but it’s probably a good idea to ensure that NotMyType expects its parameters to be depended upon in this manner. Often those are the sorts of situations where you need to be very careful about ensuring such a definition isn’t plain broken (or overly fragile).
3 Likes
I agree that messing with internals is brittle, I’m saying that is a separate bad practice from type piracy.
It does get weird to say it’s a type you “own” because it’s a NotMyType subtype and, strictly speaking, depends on names defined in multiple places, but yes you can claim ownership because the explicit MyType parameter isolates the method from any possible call signature lacking MyType. Sure we’ll be adding to a method table we didn’t create in notmyfoo(x::NotMyType{MyType}), but it would be a novel branch you have full developer control over. For example, notmyfoo(x::Ref{S}) and notmyfoo(x::S) are both not type piracy, though you should adhere to how notmyfoo treats Refs in general.
I think it’s important not to overfocus on method tables, or rather one should also consider what’s not in a method table. As disastrous as it is to insert a method into the middle of a method table and change another package’s method dispatch design, it is also disastrous to add methods that shouldn’t exist to a method table. *(::Symbol, ::Symbol) is all concrete types so it seems like a trivially novel branch that couldn’t usurp any existing method, but all the types belong to Base so that’s the only place you can have full control as a developer. For example, you can’t do anything about another Base method relying on the *(::Symbol, ::Symbol) call signature to lack any method.
Under no circumstance should type piracy be documented as allowed, and convert isn’t doing that.
Should this be split to a “Is notmyfoo(x::NotMyType{MyType}) type piracy” topic? It’s an important aspect of implementing custom properties but seems to be more of an independent tangent.
Taking a close look at the style guide, there’s a separate guideline don’t overload methods of base container types, which surely applies to doing weird things to getproperties(::Type{MyType}).
And I actually don’t follow that guideline religiously in my own code, and probably should. The type itself isn’t exported, but instances of it can end up in user code, and even though there’s no good reason for them to put those structs into a Vector, it would in fact do weird things were they to do so.
It’s kind of a pain in the butt to wrap a Vector in a struct and supply all the necessary methods for it, having been down that road I frequently find myself doing some Vector business which I didn’t account for, getting an error, and having to shift focus back to the type in question to implement the missing method. But I do see why taking the lazy route falls short of best practice.
No, because Type is not a container.
1 Like
“The trouble is that users will expect a well-known type like ::Type to behave in a certain way, and overly customizing its behavior can make it harder to work with.” To paraphrase the guide.
Perhaps this could be updated to read “Don’t overload methods of base parametric types”, that would nicely avoid this sort of quibble, because it’s clear that the same logic applies to all of them.
Again, Type is not a container type, so this is invalid paraphrasing of “The trouble is that users will expect a well-known type like Vector() to behave in a certain way”. Parametric type is not synonymous with container type.
1 Like
Again, the manual should be updated to prevent this kind of pointless rules-lawyering, because, in fact, the same logic applies to any Base parameter type.
The manual is already very clear about the principle applying only to container types in Base, and no further rewording can prevent misreadings and mistaken extrapolations.
@mnemnion The ::Type{MyType} case is a bit special. It’s kind of a special construct (called a “type selector”) that is mainly included in the language so that you can dispatch on a type object. To quote from the manual:
While
Typeis part of Julia’s type hierarchy like any other abstract parametric type, it is not commonly used outside method signatures except in some special cases.
Thus, it is perfectly normal to write a method like this:
Base.parse(::Type{MyType}, s::AbstractString)
(EDIT: But I do wonder if overloading getproperty and setproperty! on Type{MyType} would have weird consequences.)
3 Likes
Which is fine, because Base.parse(::Type{T}, ...) where {T} isn’t defined. It isn’t an overload of a base method on a base parametric type, so this wouldn’t be a problem for Vector either: defining method(::Vector{MyType}...) for a method which Base doesn’t define for Vector should be fine by the same logic.
I wouldn’t expect good results from a custom implementation of reinterpret, however. That would deviate from expected behavior, in the same way which the style guide recommends against.
I don’t see reasons why container types would work different from other parametric types this way. In either case one would want generic code using Base methods to work correctly for all concrete variants.
Consider this situation:
Author A writes:
struct NotMyType{T} end
obscure_function(::NotMyType) = true
is_NotMyType(t::NotMyType) = obscure_function(t)
Using Author A’s package, Author B writes:
struct MyType end
obscure_function(::NotMyType{MyType}) = nothing # bad, but is it piracy?
is_NotMyType(NotMyType{Bool}()) # true
is_NotMyType(NotMyType{MyType}()) # nothing
Author A has made certain assumptions when implementing is_NotMyType that are completely valid under their context, but Author B’s redefinition of obscure_function breaks it.
Is there another word for the bad thing that Author B did if not type piracy?