Question about methods defined for supertypes

General Usage
1

I am still new to Julia (about a year in and I am only starting to get it), so please bear with me.

So I was wondering what I could do with the type Base.KeySet which has supertype AbstractSet{K} where K. Currently, methods defined for Base.KeySet seems to be limited to

in(k, v::Base.KeySet) in Base at abstractdict.jl:63

Ok, fine. However, AbstractSet{K} where K has other methods defined for, such as:

setdiff!(s::AbstractSet, itr) in Base at abstractset.jl:167

In my head, I figured the methods defined for the supertype would also work for the methods defined for the subtype. However, the method above, does not work for type Base.KeySet.

I suppose I still have not quite gotten yet how types are meant to work in Julia, but I expected that Base.KeySet would behave as a AbstractSet{K} where K where behaviour is defined over methods, is this not correct?
Pedro

2

First, KeySet is an implementation detail, not an exposed interface (it is neither exported nor documented). It is used to implement keys. It is very unlikely that you need to be using it outside Base.

Second,

is not the right way to think about it. In absence of a more specific method, methods for the supertype can be called, but that does not mean they will work. Whether that is a bug depends on this behavior being part of the interface. Here, setdiff! fails because the first argument is not mutable (and it should not be).

If you want to modify a list of keys, use something like

k = Set(keys(a))

and work on that.

3

Thank you, for your reply. I am not sure what it means for Base.KeySet to be an implementation detail, but I do work with the instance of the type in my code, and for all intent and purposes, it’s just an instance of a type, like all others. Though we are talking specifically about Base.KeySet, we don’t need to, but for the sake of the question, if I wanted to write a function such as

function remove_default(xs::AbstractSet)::Nothing
  setdiff!(xs,(0,))
  return nothing
end

I would not be able to call remove_default(keys(Dict(0=>0,1=>1))). I get that for this specific case, you suggested I transform the instance into a new instance of type Set. But that’s not the point, my question is about the relationship between the subtype and the supertype. It would feel natural for the subtype to behave like the supertype, so we can write functions for the supertype, and pass in the subtype.

Now, to make matters worse, in this specific case, some methods of AbstractSet are defined for KeySet, so now, how do I consistently know what works, and what does not? I mean, if KeySet does not behave like a AbstractSet, in this specific case, why would it be a subtype of AbstractSet?

Except, of course, that I might not be getting this right as you suggested, that subtypes and supertypes are not meant to behave the same. But I would very much like to think that they should.

4

Why? I don’t see what prevents this. EDIT you would either need to make it use a functional implementation, or collect the argument though.

Not all subtypes of AbstractSet are mutable containers. Your code simply makes an erroneous assumption about the argument. Either the caller makes sure that the argument is mutable, or the function won’t work.

It is true that the interface for AbstractSet is not formally documented. But the real issue here whether some container is mutable or not. I am not aware of a way to query this in general (cf applicable though).

In this specific case, I would define a method using setdiff (note absence of !) for the generic case, and perhaps use setdiff! for some special cases I know that support it (Set comes to mind).

Also, you should be using ! for all function names that modify an argument.

5

Fair enough, redefining my function:

function remove_default!(xs::AbstractSet)::Nothing
  setdiff!(xs,(0,))
  return nothing
end

So the call remove_default!(keys(Dict(0=>0,1=>1))) would fail obviously because I used the method setdiff! defined for AbstractSet that does not work for KeySet as you noted that containers may or not may be mutable.

Let me give this one example, if I have the function signature as given below

function add_something(x::Number)

I would expect that I can pass any subtype of Number to the method above and get back a well-defined response. Agreed?

For AbstractSet, we cannot accomplish that, assuming this was intended to be this way. And I don’t see why these cases should be different.

If KeySet is an immutable container, then shouldn’t it be a subtype of some immutable abstract type? I expect Int to always behave like a Number, always. By the same token, I would also expect KeySet to always behave as a AbstractSet, always (if that is how a KeySet is supposed to behave like indeed).

Is this not the correct way to think about it? Would it not make sense for instance, in this specific case, to have a supertype ImmutableAbstractSet having KeySet as a subtype such that KeySet behaves like a ImmutableAbstractSet, always (assuming we are going into differences between mutable and immutable containers)?

At the moment, what I am walking away with, is that the supertype does not define behavior for the subtype, then I am not sure what the contract is between the subtype and the supertype.

6

By the way, the code makes no erroneous assumption about the argument, it passes a subtype of the type AbstractSet, no assumptions made (I shouldn’t need to know the internals of the function, I mean, you don’t go around the Base source code to inspect internals of methods before you call them, do you?). Example, you look up the documentation and find:

sort!(v::AbstractArray{T,1} where T, lo::Int64, hi::Int64, a::Base.Sort.MergeSortAlg, o::Base.Order.Ordering, t) in Base.Sort at sort.jl:544

Now, you know how to call the function right? You don’t need to go inspect the source code to know how it works. The types tell you what it’s acceptable, that’s the contract. So you can call the function without understanding its implementation details.

7

Your assumption is that the argument is mutable. Cf

julia> a = 1:2
1:2

julia> a[1] = 3
ERROR: setindex! not defined for UnitRange{Int64}
Stacktrace:
 [1] error(::String, ::Type) at ./error.jl:42
 [2] error_if_canonical_setindex(::IndexLinear, ::UnitRange{Int64}, ::Int64) at ./abstractarray.jl:1082
 [3] setindex!(::UnitRange{Int64}, ::Int64, ::Int64) at ./abstractarray.jl:1073
 [4] top-level scope at REPL[17]:1

even though

julia> typeof(a) <: AbstractVector
true

This could make sense, but I would very much prefer a trait for this. Branching each type would be tedious. OTOH, applicable would work as I suggested above (at no runtime cost).

8

Alright, yeah so the supertype does not tell the whole story. Oh well, trial and error it is then;)

9

I think you keep missing the suggestion about applicable.

10

Thank you for pointing that out again.

11

And that still leaves us with trial and error by the way.

12

I think you misunderstand. It gives you a way to query without trying, at compile time. Cf

abstract type A end
struct B <: A end
struct C <: A end
f(::A) = :general
f!(::C) = :specific
g(x::A) = _g(Val{applicable(f!, x)}(), x)
_g(::Val{false}, x) = f(x)
_g(::Val{true}, x) = f!(x)
g(B())                          # :general
g(C())                          # :specific
2 Likes
13

That’s pretty neat, thank you.
From a function designer’s perspective, someone providing the API, that’s great.
Now, from a user’s perspective, someone attempting to use the API, what you’re saying is that, when in doubt, we could cover both cases when a container may or may not be mutable? I mean, that’s certainly a solution. It feels like a bit more work than just trying applicable on the REPL for some specific use-case (trial and error) and moving on (as a user). Either way, we will still be trying either in an automated or manual fashion.

14

Well, from a user’s perspective, you should be relying on the documented API. This is what Tamas was pointing to up at the top when he said KeySet is an implementation detail and not documented or exported. The implication is that you should not necessarily expect it to be completely fleshed out. And crucially, you should not expect it to be maintained with the same functionality going forward.

Yes, in an ideal world, package authors would make sure that (documented and exported) things that inherit from abstract types have the behaviors associated with the parent type, but there’s nothing in principle stopping me from declaring

struct Foo <: AbstractVector end

Though basically no methods defined in AbstractVectors will work on it.

1 Like
15

Thanks, that does clear things up a bit on the expectation front. I was just a bit surprised given this one example, used at the beginning of the thread, is coming from Base (not an external package so I had a slightly different expectation of consistency for Base code), certainly much more so understandable from an external package’s perspective. None the less, I am still coming to terms that subtypes and supertypes are not required to behave the same. Maybe it’s just a wrong notion after all from other programming paradigms that I am trying to relay on top of Julia.

16

I think that’s a reasonable assumption in general, but undocumented is undocumented. All bets are off with something used internally. Julia doesn’t have the ability to declare “private” methods and types. But for things that are not exported, you should use at your own risk.

Perhaps. I think of abstract types less like a parent class in Python and more like an opportunity to make things generic. If I want a function that returns the square of the sum of two arguments, I can declare

squaresum(a::Real, b::Real) = (a+b)^2

Rather than having to write a method for Floats and Ints and everything else. Of course, someone might come along and declare a type that’s a subtype of Number but fail to define :+(), in which case there’s nothing my method can do.

Another way to think of it - I could have instead written

squaresum(a::Any, b::Any) = (a+b)^2

All things are subtypes of Any, would you expect every type to make sure it could be called by squaresum?

17

No. I would simply expect squaresum to work for all types subtyped by Any. Clearly, you defined it in a way that it won’t work by employing an operator which does not work for all types that subtype Any, that is a violation of the contract you just set forth for your method (a design error). Now, if your function definition used methods that are supposed to work on all types that subtype Any, and I pass an object with a type that subtypes Any and it fails, this is where we have the discrepancy which we have been discussing.

18

Right, this is the issue. And there shouldn’t be anything for your method to do, as long as you honored the contract of your method signature (the burden is not on your method).

19

I think you may be coming from a language with more formalized interfaces (C++?). Julia in theory could follow that path, but in practice most often it doesn’t: interfaces grow organically, may not have a formal definition until mature, and it is considered perfectly acceptable to just write a generic method and have it fail when called with the wrong types (cf duck typing).

You may want to just embrace this style of programming for a while in order to learn Julia better.

2 Likes
20

Thanks, I appreciate the feedback.