Composition and inheritance: the Julian way


#1

I am a relatively newbie and I am wondering what is the most appropriate Julian way to reuse the code.

I don’t want to trigger discussions on topics already discussed a million times. This post simply aims to lay down a clear list of rules to be followed to write good Julia code, and pose a couple of questions.

Let’s start with an example: I have a type Person

mutable struct Person <: AbstractPerson
    name::String
    age::Int
end

and a lot (hundreds) of methods dealing with it. Now I want to create a new type, say Citizen, which extend or specialize Person. Julia do not supports inheritance by design, hence we pursue code reuse by means of composition.

There are three viable approaches:

1. Encapsulate a Person struct within a Citizen one.

mutable struct Citizen
   person::Person
   nationality::String
end

Although straightforward, this approach implies that I have to redefine all the (undreds) methods working on Person, e.g.: display(p::Citizen) = display(p.person).

2. Very similar to 1, but we assume Citizen <: Abstractperson

mutable struct Citizen <: AbstractPerson
   person::Person
   nationality::String
end

This implies that all methods accepting a AbstractPerson may receive Citizen as input and may not work appropriately since the underlying data structure is completely different. As in approach 1, this means that all the Person methods should be overloaded;

3. Extend Person by composition:

mutable struct Citizen <: AbstractPerson
    name::String
    age::Int
    nationality::String
end

Although this involves a duplication of the struct field names and types, the methods accepting a AbstractPerson object will work seamlessly.

QUESTION 1:
In order to reuse the code in the Person methods as much as possible, approach 3 seems the best way. Is this correct?

In the following I will outline a complete implementation of both the Person and Citizen objects, in order to settle down all the minor details. The only “flaw” of the code is that it may requires the data structure to be copied back and forth (see the comment on the super method).

#---------------------------------------------------------------------
# BASE TYPE : Person

# Use abstract type for the interface name, by convention prepend
# `Abstract` to the type name.
abstract type AbstractPerson end

# TYPE MEMBERS
mutable struct Person <: AbstractPerson
    name::String
    age::Int
end

# CONSTRUCTOR
function Person(name)
    return Person(name, 0)
end

# TYPE METHODS: always use `AbstractPerson` as input type...
import Base.display
function display(p::AbstractPerson)
    println("Person: ", p.name, " (age: ", p.age, ")")
end

function happybirthday(p::AbstractPerson)
    p.age += 1
    println(p.name, " is now ", p.age, " year(s) old")
end

function call(p::AbstractPerson)
    print_with_color(:red, uppercase(p.name), "!")
end

# ...except in cases where the return type is a `Person` object: here
# we must use `Person` as input.
function son(p::Person)
    return Person(p.name * " junior")
end


# EXAMPLES
p = Person("Giorgio")
happybirthday(p)
call(p)
call(son(p))


#---------------------------------------------------------------------
# DERIVED TYPE : Citizen

# Use abstract type for the interface name, by convention prepend
# `Abstract` to the type name.
abstract type AbstractCitizen <: AbstractPerson end # NOTE: this inherits from AbstractPerson

# TYPE MEMBERS (composition of `Person` fields and new ones)
mutable struct Citizen <: AbstractCitizen
    name::String        # this must be the same as Person
    age::Int            # this must be the same as Person
    nationality::String # new field (not present in Person)
end

# Julia do not supports inheritance, hence `Person` and `Citizen`
# objects are completely unrelated entities and we need a function to
# obtain the `Person` object encapsulated in the `Citizen` one.  Note
# that this function introduces an overhead due to a copy.
super(p::Citizen) = Person(p.name, p.age)

# CONSTRUCTOR
function Citizen(name, nationality)
    p = Person(name) # Create parent object
    return Citizen(p.name, p.age, nationality)
end

# TYPE METHODS: always use `AbstractCitizen` as input type...

# Note that we are not overloading `display` and `happybirthday`: they
# work well for both `Person` and `Citizen` objects.

# Overload of the `call` method.  Although optional this new method
# allows us to specialize the behaviour on nationality.
function call(p::AbstractCitizen)
    if p.nationality == "Italian"
        print_with_color(:red, uppercase(p.name), " dove sei ?")
    elseif p.nationality == "UK"
        print_with_color(:red, uppercase(p.name), " where are you ?")
    else
        call(super(p))
    end
end

# Method returning a `Citizen` object must always be overloaded and
# input type (if any) must be a `Citizen`, not an `AbstractCitizen`.
function son(p::Citizen)
    return Citizen(p.name * " junior", p.nationality)
end


# EXAMPLES
p = Citizen("Giorgio", "Italian")
happybirthday(p)
call(p)
call(son(p))

p = Citizen("Kim", "UK")
happybirthday(p)
call(p)
call(son(p))

p = Citizen("Jean", "French")
happybirthday(p)
call(p)
call(son(p))

QUESTION 2:
In order to reuse the code in the Person methods (and possibly in the Citizen methods) as much as possible, the rules outlined above are the best solution. Is this correct?


#2

I think the way to do it is using functions for getting the name, age etc.
All AbstractPerson should share the same methods not necessarily the same fields.

I am on my phone right now. When I am on my pc I’ll write an example.


#3

I’ve found the @forward macro from Lazy.jl quite useful for this purpose: https://github.com/MikeInnes/Lazy.jl/blob/76dab7d30863e8375fc96b2444fdb2ead369bf4c/src/macros.jl#L248-L271

Essentially, you can define:

struct Person
  name::String
end

name(p::Person) = person.name

struct Citizen
  person::Person 
  nationality::String
end

@forward Citizen.person name

which will automatically make name(c::Citizen) = name(c.person)


#4

I don’t think there are rigid rules for every situation. I think of the type system as a set of building blocks, which I can use to design solutions that fit the problem well. I usually try to adhere to the following:

  1. abstract types are good for designating interfaces, you can also make a hierarchy of them, keeping substitutability in mind,
  2. slots should not be part of the interface, introduce accessor functions, even if trivial,
  3. composition is useful and convenient, just forward methods (some macros make this more convenient, eg Lazy.@forward).

For a pretty polished and well-maintained example, look at

which is similar to your problem: OffsetArrays contain arrays, yet they are also arrays, extending functionality.


How to add metadata info to a DataFrame?
#5

You can always make a macro to avoid boilerplate code, e.g.:

abstract type AbstractPerson end

mutable struct Person <: AbstractPerson
    name::String
    age::Int
end


macro inherit(name, base, fields)
    base_type = Core.eval(@__MODULE__, base)
    base_fieldnames = fieldnames(base_type)
    base_types = [t for t in base_type.types]
    base_fields = [:($f::$T) for (f, T) in zip(base_fieldnames, base_types)]
    res = :(mutable struct $name end)
    push!(res.args[end].args, base_fields...)
    push!(res.args[end].args, fields.args...)
    return res
end


@inherit Citizen Person begin
    nationality::String
end

But honestly, I know very few tasks where field inheritance is indeed the best approach.


#6

The @forward macro appears very useful. But the point here is whether the above example is the best possible way to reuse other code (regardless if it has been written by hand or generated by macros), under the assumption that the Person has hundreds of methods already implemented.


#7

Correct me if I’m wrong: in order to use @forward macro the two involved structures MUST NOT be substitutable.


#8

I am not sure how two stuctures could be substitutable, given that Julia does not have inheritance, only subtypes (of abstract types).


#9

Ah, I’m afraid I can’t say that it’s the best possible way, only that it has worked well for me in the past :slightly_smiling_face:


Getproperty, decorations, inheritance in 0.7
#10

Well, maybe substitutable is not the appropriate word (sorry I’m not a CS…), but Person and Citizen in my example are (in a sense) substitutable. Will this LSP reformulation:
the concrete istances of abstract types in a Julia program should be replaceable with concrete instances of their inherited abstract types without altering the correctness of that program.
apply ?

If so, the Person and Citizen structures are substitutable.


#11

If this is still about Lazy.@forward, I suggest you look at the macro and its documentation before further discussion. It is precisely for the composition example we are talking about.

The application of the substitution principle I was suggesting is much simpler: it simply states that if you have an abstract type T and some required interface for it, then

  1. all S<:T should implement this interface, and

  2. the programmer working with objects that are T should not need to care about the actual type S while relying on this interface, as long as it is <:T.

This almost sounds too trivial, but it is a powerful organizing principle. In the concrete example, the interface for AbstractPerson should be clarified first, then whether Citizen is implemented using composition is a detail that need not concern the user.


#12

The problem with the @forward macro is that you have to explicitly and painstakingly forward every method to the correct field. It would be tedious and verbose, and hard to even know all the methods to forward.

Would it be conceptually possible to create a macro that forwards all methods to the field with the composed object/type, unless the function is also declared for the encapsulating type (that would be hard for a macro, right)? Alternatively, would it be possible to support it in the language? And would it be a bad idea?

struct A
    x::B:::ForwardMethodsForThisTypeHere{B}
    y::Int
end

It just seems like the “composition over inheritance” argument would be easier to make if there were a way to efficiently cut all the boilerplate.


#13

In terms of macros to avoid boilerplate code using type composition, I wrote this recently:

It takes care of boilerplate fields, parametric types and macros in a fully automated way, you just write a struct with the @mix macro in front of it:

@mix @with_kw struct Foo{T} 
    bar:T = 100
end

Then use @Foo as a macro in front of other structs and voila. Fields, types, macros, default params, whatever. You can even chain them, and apply the @Foo macro to another @mix macro to merge them.


#14

@gcalderone In practise I’ve ended up using a mix of what you are calling composition and encapsulation, although I’m not used to them being described that way - I would call your encapsulation aggregation (which I thought was a subset of composition), and leave encapsulation to oop access restriction… but none of these terms really make sense to me any more in julia.

I use Mixers.jl when I have multiple types that share a subset of fields and macros.

I’m even thinking about using empty concrete types fully made with mixins, so custom types can be made later without boilerplate.


#15

If the list is long and the process is painstaking, you can

  1. consider simplifying the interface (too many verbs),

  2. write a macro (if you are doing it repeatedly).

This should not be hard to know: you forward the methods that are part of the relevant interface. Julia lacks formal specification for interfaces, so one should rely on documentation.


#16

I’m referring to a situation like this (pseudocode):

struct A
    x::Vector
    extrainfo::Int 
end

where I want A to be exactly like a Vector except for one small tweak. I’ll make a few special methods for A and everything else should be forwarded to the x field.

But everything that is part of the Vector interface should be part of the A interface. And I don’t even know what all those methods are.


#17

They are documented.


#18

That’s AbstractArray. Calling methodswith(Vector) spits out a longer list, and that’s without showparents.

I don’t mean to say this should be easy to fix, or even smart. I’m just trying to explain what I’m looking for, and that is “Something exactly like (a concrete) Vector, with a twist.” Right now, that means reimplementing or at least forwarding a lot of methods. Can this be automated?


#19

I struggled with this a lot when I started with Julia too. The approach I’ve come to find works best is the following.

  • Have Person and Citizen be <:AbstractPerson.
  • Define as many of your “100s of functions” on AbstractPerson instead of Citizen. This requires carefully thinking about what is generic behavior of AbstractPerson and what depends on implementation details such as which fields are in Person vs. Citizen. This might require introducing new functions, e.g. getters like get_name which can be specialized by Person vs. Citizen, but which then allow other many other functions to be remain generic.
  • For the functions which can’t be written generically on AbstractPerson, write specialized versions for Citizen and Person (if they truly can’t be written generically in step 2, this step you were always going to have to do anyway).

In general step 2 takes some thought and is not as quick and easy as in languages that have inheritance, say Python where its trivial to subclass and add a single new method or something, but I find that taking the time to do it this way leads to far better and more robust code. To the extent to which I’m familiar with the Julia standard library, this is the approach taken there as well.


#20

This sounds like a good approach. But it’s difficult to apply when the object you want to wrap is pre-existing, for example in Base.