Composition and inheritance: the Julian way

Quite a while ago, I tried making SubDataFrame encapsulate an AbstractDataFrame. It worked fine, but I got an avalanche of method ambiguities. That’s handled better now, so it’d be great if someone gave this another shot. It’s the right approach.

I just want to clarify my suggested use of Mixers.jl, and when I would use it instead of aggregated composition.

I’m often working with multiple formulations of physiological processes that share a subset of parameters. The formulation is a method despatching on a type that holds the necessary parameters. But they don’t actually inherit any behaviours, they are just dispatched to run a particular version of a formulation, using some custom parameters, and some common parameters that represent the same physical properties - and have the same Parameters.jl defaults that I don’t want to duplicate.

They could be aggregated types but this would actually add non-existent interdependence between them, they would all need to access the same composed field in the method they dispatch on. It would also deepen the nesting, and the formulation methods would be harder to read. So I use mixins for those fields. It’s mostly for cleaning up inconsequential duplication, not organisation inheritance.

You could build concrete type inheritance with it as mixins can operate on mixins, and use holy traits for the dispatch hierarchy, which could even be automated. But I haven’t tried that, and it might be insane. But it would be more flexible than oop concrete type inheritance.

incidentally, while I am aware of the etymology, I am wondering how this comes across for newcomers to the community :wink:

2 Likes

I love it, I think he’s a patron saint of the Julia community, at least he is for my programming lately!
:pray::pray::pray:

1 Like

You don’t have to re-define all the methods accepting a Person object if you write those methods using functions instead of field access. You just have to define some basic functions

Changing your original example:

abstract type AbstractPerson end
# basic methods to define: name,age,set_age

#basic AbstractPerson
mutable struct Person <: AbstractPerson
    name::String
    age::Int 
end

# CONSTRUCTOR
function Person(name)
    return Person(name, 0)
end

#basic methods
name(a::Person) = a.name
age(a::Person) = a.age
set_age(a::Person,x::Integer) = (a.age = x; x)

# TYPE METHODS: always use `AbstractPerson` as input type...
import Base.display
function display(p::AbstractPerson)
    println("Person: ", name(p), " (age: ", age(p), ")")
end

function happybirthday(p::AbstractPerson)
    set_age(p, age(p) + 1)
    println(name(p), " is now ", age(p), " year(s) old")
end

function call(p::AbstractPerson)
    print_with_color(:red, uppercase(name(p)), "!")
end

#---------------------------------------------------------------------
# DERIVED TYPE : Citizen

# Use abstract type for the interface name, by convention prepend
# `Abstract` to the type name.
abstract type AbstractCitizen <: AbstractPerson end
# here you should think of basic methods for a AbstrctCitizen, such as nationality(c::AbstractCitizen).

# TYPE MEMBERS (composition of `Person` fields and new ones)
mutable struct Citizen <: AbstractCitizen
    person::Person
    nationality::String # new field (not present in Person)
end

#now would be a good time to use macros...
name(c::Citizen) = name(c.person)
age(c::Citizen) = age(c.person)
set_age(c::Citizen,x::Int) = set_age(c.person, x)

#basic abstractcitizen method
nationality(c::Citizen) = c.nationality

#Now everything defined for AbstractPerson should work for Citizen

#And you are not tied to field names anymore:

struct EternalBeing <: AbstractPerson end

name(e::EternalBeing) = "The One who Is"
age(e::EternalBeing) = typemax(Int)
set_age(e::EternalBeing,x) = nothing

const eternal = EternalBeing()

# All just work
display(eternal)
happybirthday(eternal)
call(eternal)
3 Likes

I think all that can be simplified a bit, if every type that uses Person and wishes to use it’s functions, simply had a person function.

Try the following:

abstract type AbstractPerson end
# basic methods to define: name,age,set_age

#basic AbstractPerson
mutable struct Person <: AbstractPerson
    name::String
    age::Int 
end

person(a::Person) = a

# CONSTRUCTOR
Person(name) = Person(name, 0)

#basic methods
name(a::AbstractPerson) = person(a).name
age(a::AbstractPerson)  = person(a).age
set_age(a::AbstractPerson,x::Integer) = (person(a).age = x; x)

# TYPE METHODS: always use `AbstractPerson` as input type...
Base.display(p::AbstractPerson) = println("Person: ", name(p), " (age: ", age(p), ")")

function happybirthday(p::AbstractPerson)
    set_age(p, age(p) + 1)
    println(name(p), " is now ", age(p), " year(s) old")
end

call(p::AbstractPerson) = (print_with_color(:red, uppercase(name(p)), "!"); println())

#---------------------------------------------------------------------
# DERIVED TYPE : Citizen

# Use abstract type for the interface name, by convention prepend
# `Abstract` to the type name.
abstract type AbstractCitizen <: AbstractPerson end

# here you should think of basic methods for a AbstractCitizen,
# such as nationality(c::AbstractCitizen).

# TYPE MEMBERS (composition of `Person` fields and new ones)
mutable struct Citizen <: AbstractCitizen
    person::Person
    nationality::String # new field (not present in Person)
end

person(c::Citizen) = c.person

#basic abstractcitizen method
nationality(c::Citizen) = c.nationality

#Now everything defined for AbstractPerson should work for Citizen

#And you are not tied to field names anymore:

struct EternalBeing <: AbstractPerson end

name(e::EternalBeing) = "The One who Is"
age(e::EternalBeing) = typemax(Int)
set_age(e::EternalBeing, x::Integer) = nothing

const eternal = EternalBeing()

# All just work
display(eternal)
happybirthday(eternal)
call(eternal)

println()

zulima = Citizen(Person("Zulima Martín García", 44), "Spain")
display(zulima)
happybirthday(zulima)
call(zulima)
13 Likes

To be fair though, your example is covered by getproperty overloading.

:+1: or getproperty overloading where field access now becomes a function call when necessary.

The key point here is the word just. For a simple example you’re definitely right. But for a real life application (such as wrapping a DataFrame object as we discussd above) this implies redefining ~200 methods. Of course a few of these (namely all those returning a DataFrame struct) have to be redefined anyway, but what remains is still quite a large number.

Of course this is feasible, either manually or through macros, but my first thought is that I should try to find an alternative method. This triggered all present discussion.

Yeah, clearly all what we say here applies only if a structure is involved.

Of course I don’t want to fight any train. I’m asking here because I recognize many here are as way more expert than me. Still: no one told me yes, you have to blindly forward those ~200 methods, and forget about inheritance.

Which is OK for me, but I didn’t saw this carved in stones, nor in the manual or in the discourse or anywhere else. Hence my doubt arose…

To answer @favba: in your proposal you are implementing the name, age and set_age for both Person and Citizen, which is exactly what I wanted to avoid (the famous ~200 methods…). Regardless of macro usage: the methods are always there: they do nothing, they flood the dispatch table and they have to be compiled.

2 Likes

Thanks @ScottPJones!!!
You’re solution is definitely a very interesting one, and I think it definitely solve the problem in the first discussion of this post. You’re great!! :+1::+1:

In brief, your solution uses composition of structures and avoid useless methods re-definition.
Clearly, as noted by @ChrisRackauckas, there are quite a lot of getproperty calls involved. However (correct me if I’m wrong) there are quite a lot of getproperty calls involved also in method forwarding isn’t it?

One apparently officially questionable way is discussed in https://discourse.julialang.org/t/getproperty-decorations-inheritance-in-0-7/11237.

No, you don’t have to. If your interface on the abstract type is done via accessor functions (or lazy properties), then the interface to implement is tiny and limited to defining those accessor functions. See something like the AbstractArray interface which takes

https://docs.julialang.org/en/v0.6.2/manual/interfaces/#man-interface-array-1

and then works in literally any function which takes arrays. This is because if you develop code for the abstract type based on the interface (and it doesn’t have to be an abstract type, for example the Iteration Interface is for anything which acts like an iterable) then any implementation of that interface will do. If all of Julia’s arrays can work with 5 methods plus some extra traits, then that shows there’s quite a lot you can do.

If your interface is 200+ methods, it doesn’t matter if it’s Julia or OOP, you’re doing it wrong!

Dataframes is a bad example because they never defined an interface for a Dataframe which is why it’s so hard to extend. If you want a table interface, look at DataStreams.jl or IterableTables.jl. These have only a handful of functions and any source/sink will then work as a table. For example, Dataframes implements the source while StatPlots.jl implements the sink, so those two packages work together. But DifferentialEquations.jl solutions are also a sink, so you can convert its solutions to Dataframes or use it just like a table due to the 3 methods implemented by David. So, 200 methods >> the 3 that are used in the real live actually working example :smile:.

Here’s the full code that makes every differential equation solution (ODE/SDE/DAE/DDE/jump/etc.) have a table interface and directly convert to DataFrames.

https://github.com/JuliaDiffEq/DiffEqBase.jl/blob/master/src/tabletraits.jl

1 Like

Now the bad news: despite the very clever suggestion by @ScottPJones this solution can not be applied in general, since it works only if the object you wish to extend/customize already uses the person(a::Person) = a method internally, which doesn’t apply to DataFrame

1 Like

It seems that there are two tasks, then for DataFrames, if you were to create a new type for metadata (which you don’t have to do! maintainers seem amenable to metadata living in DataFrames)

First, make sure DataFrames methods only take in AbstractDataFrame. Second, categorize which functions absolutely need to be defined toe AbstractDataFrame to work. Maybe there already exists a list somewhere? Or we could work on getting that list together and shortened. A good start would be to create a new MyDataFrame <: AbstractDataFrame type that is a mirror image of a DataFrame type and seeing how that works.

…and this provides a motivation for the second part of the question. thank you for clarifying! :+1:

Good to know. Actually, there already is a PR for this, although it has been assigned no reviewer yet…

Yeah, I can try to do it if some of the maintainers manifest their wish to go in that direction, so that there is some chance that I will not waste my time in a PR which will surely be rejected.

So it appears we’re finally figuring out a solution to the problem here. I will try to summarize:

  • the problem arose trying to extend/customize the behaviour of a Julia object (i.e. a structure+methods, call it Person) into another one (call it Citizen) in a way that it can be used exactly as the base type, except for a few twist (i.e. specialized methods);

  • we wanted to solve this problem minimizing as much as possible the number of methods redefined just to forward calls to the base type;

  • minimizing this number implies we had to implement an inheritance-like approach to the problem, and we identified a few rules to be followed (here and here);

  • the inheritance-like approach, however, turned out to be non convenitent as discussed by @ChrisRackauckas here and here. The best, and more Julian way to face this problem is by means of composition, regardless of the number of method to be re-defined;

  • @ScottPJones found what I think is a very nice approach here which uses both composition (in place of inheritance) and minimizes the number of method re-definitions. However, this method requires to re-write the methods in the base class;

  • The real interest for this discusion was the possibility to customize a DataFrame object (further discussion here) and refactor the whole package following @ScottPJones solution is not feasible;

  • @ChrisRackauckas provided the last piece of information: the DataFrame still lacks the definition of an interface which simplifies extending it. Here’s why we had all the problems which ultimately triggered this discussion.

The whole story was absolutely not clear to me when I first started this post, and I’m sorry if everything was obvious to many people here. I learned a lot and wish to thank all of you! :+1

14 Likes

I am curious if @ScottPJones’s method could be formalized. By methodology or by language (macros) support.

I am also curious if it will make negative performance impact comparing to “slavery” work with ~200 methods rewrite.

Scott’s method allow multiple inheritance too! (So it could also bring many problems which multiple inheritance use to bring)

1 Like

Dear all,
this topic has been inactive for a few months, but I just wanted to add that I implemented a package to solve the first problem posted, namely how to implement some form of inheritance mechanism in Julia.

Actually, the solution is to implement subtyping which, although not equivalent to inheritance (e.g. here), allows to quickly and elegantly solve the problem.

The trick is to unambiguously link an abstract type and a concrete one, realizing a kind of quasi-abstract type. Then we can use the latter as argument type for methods: it is both an abstract type (allowing the same method to be called on subtypes) and a concrete one (ensuring the method will receive the appropriate concrete structure). A macro can then takes care of copy/pasting the fields from one quasi-abstract* type to another, realizing a concrete subtyping pattern.

The details are discussed in the README.md file of the ReusePatterns package.

The implementation of concrete subtyping for the first example in this post is as follows:

using ReusePatterns

#---------------------------------------------------------------------
# BASE TYPE : Person
@quasiabstract mutable struct Person
    name::String
    age::Int
end
Person(name) = Person(name, 0)
Base.display(p::Person) = println("Person: ", p.name, " (age: ", p.age, ")")

function happybirthday(p::Person)
    p.age += 1
    println(p.name, " is now ", p.age, " year(s) old")
end

call(p::Person) = print(uppercase(p.name), "!")

son(p::Person) = Person(p.name * " junior", 0)

# EXAMPLES
p = Person("Giorgio")
happybirthday(p)
call(p)
call(son(p))


#---------------------------------------------------------------------
# DERIVED TYPE : Citizen
@quasiabstract mutable struct Citizen <: Person
    nationality::String # new field
end

# Note that the fields associated to `Citizen` automatically include those from `Person`
println(fieldnames(concretetype(Citizen)))

function call(p::Citizen)
    if p.nationality == "Italian"
        print(uppercase(p.name), " dove sei ?")
    elseif p.nationality == "UK"
        print(uppercase(p.name), " where are you ?")
    else
        invoke(call, Tuple{supertype(Citizen)}, p)
    end
end

Citizen(name, nationality) = Citizen(name, 0, nationality)

function son(p::Citizen)
    person = Person(getfield.(Ref(p), fieldnames(concretetype(Person)))...)
    return Citizen(person.name * " junior", 1, p.nationality)
end

p = Citizen("Giorgio", "Italian")
happybirthday(p)
call(p)
call(son(p))

p = Citizen("Kim", "UK")
happybirthday(p)
call(p)
call(son(p))

p = Citizen("Jean", "French")
happybirthday(p)
call(p)
call(son(p))
3 Likes

There are also the packages ConcreteAbstractions.jl and Classes.jl that do similar things.

1 Like