Composition and inheritance: the Julian way

I don’t think there are rigid rules for every situation. I think of the type system as a set of building blocks, which I can use to design solutions that fit the problem well. I usually try to adhere to the following:

  1. abstract types are good for designating interfaces, you can also make a hierarchy of them, keeping substitutability in mind,
  2. slots should not be part of the interface, introduce accessor functions, even if trivial,
  3. composition is useful and convenient, just forward methods (some macros make this more convenient, eg Lazy.@forward).

For a pretty polished and well-maintained example, look at

which is similar to your problem: OffsetArrays contain arrays, yet they are also arrays, extending functionality.

6 Likes

You can always make a macro to avoid boilerplate code, e.g.:

abstract type AbstractPerson end

mutable struct Person <: AbstractPerson
    name::String
    age::Int
end


macro inherit(name, base, fields)
    base_type = Core.eval(@__MODULE__, base)
    base_fieldnames = fieldnames(base_type)
    base_types = [t for t in base_type.types]
    base_fields = [:($f::$T) for (f, T) in zip(base_fieldnames, base_types)]
    res = :(mutable struct $name end)
    push!(res.args[end].args, base_fields...)
    push!(res.args[end].args, fields.args...)
    return res
end


@inherit Citizen Person begin
    nationality::String
end

But honestly, I know very few tasks where field inheritance is indeed the best approach.

11 Likes

The @forward macro appears very useful. But the point here is whether the above example is the best possible way to reuse other code (regardless if it has been written by hand or generated by macros), under the assumption that the Person has hundreds of methods already implemented.

Correct me if I’m wrong: in order to use @forward macro the two involved structures MUST NOT be substitutable.

I am not sure how two stuctures could be substitutable, given that Julia does not have inheritance, only subtypes (of abstract types).

Ah, I’m afraid I can’t say that it’s the best possible way, only that it has worked well for me in the past :slightly_smiling_face:

Well, maybe substitutable is not the appropriate word (sorry I’m not a CS…), but Person and Citizen in my example are (in a sense) substitutable. Will this LSP reformulation:
the concrete istances of abstract types in a Julia program should be replaceable with concrete instances of their inherited abstract types without altering the correctness of that program.
apply ?

If so, the Person and Citizen structures are substitutable.

If this is still about Lazy.@forward, I suggest you look at the macro and its documentation before further discussion. It is precisely for the composition example we are talking about.

The application of the substitution principle I was suggesting is much simpler: it simply states that if you have an abstract type T and some required interface for it, then

  1. all S<:T should implement this interface, and

  2. the programmer working with objects that are T should not need to care about the actual type S while relying on this interface, as long as it is <:T.

This almost sounds too trivial, but it is a powerful organizing principle. In the concrete example, the interface for AbstractPerson should be clarified first, then whether Citizen is implemented using composition is a detail that need not concern the user.

The problem with the @forward macro is that you have to explicitly and painstakingly forward every method to the correct field. It would be tedious and verbose, and hard to even know all the methods to forward.

Would it be conceptually possible to create a macro that forwards all methods to the field with the composed object/type, unless the function is also declared for the encapsulating type (that would be hard for a macro, right)? Alternatively, would it be possible to support it in the language? And would it be a bad idea?

struct A
    x::B:::ForwardMethodsForThisTypeHere{B}
    y::Int
end

It just seems like the “composition over inheritance” argument would be easier to make if there were a way to efficiently cut all the boilerplate.

1 Like

In terms of macros to avoid boilerplate code using type composition, I wrote this recently:

It takes care of boilerplate fields, parametric types and macros in a fully automated way, you just write a struct with the @mix macro in front of it:

@mix @with_kw struct Foo{T} 
    bar:T = 100
end

Then use @Foo as a macro in front of other structs and voila. Fields, types, macros, default params, whatever. You can even chain them, and apply the @Foo macro to another @mix macro to merge them.

5 Likes

@gcalderone In practise I’ve ended up using a mix of what you are calling composition and encapsulation, although I’m not used to them being described that way - I would call your encapsulation aggregation (which I thought was a subset of composition), and leave encapsulation to oop access restriction… but none of these terms really make sense to me any more in julia.

I use Mixers.jl when I have multiple types that share a subset of fields and macros.

I’m even thinking about using empty concrete types fully made with mixins, so custom types can be made later without boilerplate.

If the list is long and the process is painstaking, you can

  1. consider simplifying the interface (too many verbs),

  2. write a macro (if you are doing it repeatedly).

This should not be hard to know: you forward the methods that are part of the relevant interface. Julia lacks formal specification for interfaces, so one should rely on documentation.

I’m referring to a situation like this (pseudocode):

struct A
    x::Vector
    extrainfo::Int 
end

where I want A to be exactly like a Vector except for one small tweak. I’ll make a few special methods for A and everything else should be forwarded to the x field.

But everything that is part of the Vector interface should be part of the A interface. And I don’t even know what all those methods are.

2 Likes

They are documented.

That’s AbstractArray. Calling methodswith(Vector) spits out a longer list, and that’s without showparents.

I don’t mean to say this should be easy to fix, or even smart. I’m just trying to explain what I’m looking for, and that is “Something exactly like (a concrete) Vector, with a twist.” Right now, that means reimplementing or at least forwarding a lot of methods. Can this be automated?

2 Likes

I struggled with this a lot when I started with Julia too. The approach I’ve come to find works best is the following.

  • Have Person and Citizen be <:AbstractPerson.
  • Define as many of your “100s of functions” on AbstractPerson instead of Citizen. This requires carefully thinking about what is generic behavior of AbstractPerson and what depends on implementation details such as which fields are in Person vs. Citizen. This might require introducing new functions, e.g. getters like get_name which can be specialized by Person vs. Citizen, but which then allow other many other functions to be remain generic.
  • For the functions which can’t be written generically on AbstractPerson, write specialized versions for Citizen and Person (if they truly can’t be written generically in step 2, this step you were always going to have to do anyway).

In general step 2 takes some thought and is not as quick and easy as in languages that have inheritance, say Python where its trivial to subclass and add a single new method or something, but I find that taking the time to do it this way leads to far better and more robust code. To the extent to which I’m familiar with the Julia standard library, this is the approach taken there as well.

16 Likes

This sounds like a good approach. But it’s difficult to apply when the object you want to wrap is pre-existing, for example in Base.

1 Like

I understand what you are trying to do, I am just arguing that it is not an approach that meshes well with Julia, or leads to good interface design.

First, I think that if you want an interface, you should define an abstract type to go with it. This is costless in terms of performance, and allows you do document the interface in the docstring of the abstract type.

Second, you should minimize the functions that need to be implemented for this interface, ideally by choosing a core and then some extra methods that use this core but can be overridden for performance/implementation reasons. This is what AbstractArray does, and Julia’s compilation model makes this costless (in terms of runtime) in most cases, because of specialization and inlining (and lately, constant propagation).

IMO an interface which violates these is not good design. Interfaces should not be associated primarily with concrete types, nor should they be very rich. Either will cause problems independently of forwarding methods. That said, designing good interfaces is an difficult iterative process; frequently they emerge from functions of concrete types, and similarly, get streamlined by refactoring.

This link seems broken now, this one works for me.

I’ve been using abstract types for broad classification (i.e. AbstractString & AbstractChar), but for the rest, I don’t make hierarchies of abstract types, I’ve found traits and parameterized concrete types much more useful for writing generic code and having a consistent API. Most of the things about strings and characters are orthogonal to one another (Mutable or not, validated or not, single- or multi- codeunit encoding, ASCII compatible or not, ISO compatible or not, Unicode subset, full Unicode, Unicode + invalids (i.e. Char), or not Unicode compatible, etc.)
Trying to make that into a hierarchical structure explodes the number of types to deal with.

Also, instead of forwarding everything, you can use a method to access the Person from whatever type you have that includes person, and then call the person specific methods directly on it, which to me is clearer.

struct Person
    name::String
    gender::String
    birthday::String
end
person(p::Person) = p

struct Citizen
    person::person 
    nationality::String
end
person(c::Citizen) = c.person

Then you don’t have to add methods for every field, just one, instead of name(c) you’d have person(c).name or name(person(c))

7 Likes