Composition and inheritance: the Julian way

Ah, I’m afraid I can’t say that it’s the best possible way, only that it has worked well for me in the past :slightly_smiling_face:

Well, maybe substitutable is not the appropriate word (sorry I’m not a CS…), but Person and Citizen in my example are (in a sense) substitutable. Will this LSP reformulation:
the concrete istances of abstract types in a Julia program should be replaceable with concrete instances of their inherited abstract types without altering the correctness of that program.
apply ?

If so, the Person and Citizen structures are substitutable.

If this is still about Lazy.@forward, I suggest you look at the macro and its documentation before further discussion. It is precisely for the composition example we are talking about.

The application of the substitution principle I was suggesting is much simpler: it simply states that if you have an abstract type T and some required interface for it, then

  1. all S<:T should implement this interface, and

  2. the programmer working with objects that are T should not need to care about the actual type S while relying on this interface, as long as it is <:T.

This almost sounds too trivial, but it is a powerful organizing principle. In the concrete example, the interface for AbstractPerson should be clarified first, then whether Citizen is implemented using composition is a detail that need not concern the user.

The problem with the @forward macro is that you have to explicitly and painstakingly forward every method to the correct field. It would be tedious and verbose, and hard to even know all the methods to forward.

Would it be conceptually possible to create a macro that forwards all methods to the field with the composed object/type, unless the function is also declared for the encapsulating type (that would be hard for a macro, right)? Alternatively, would it be possible to support it in the language? And would it be a bad idea?

struct A
    x::B:::ForwardMethodsForThisTypeHere{B}
    y::Int
end

It just seems like the “composition over inheritance” argument would be easier to make if there were a way to efficiently cut all the boilerplate.

1 Like

In terms of macros to avoid boilerplate code using type composition, I wrote this recently:

It takes care of boilerplate fields, parametric types and macros in a fully automated way, you just write a struct with the @mix macro in front of it:

@mix @with_kw struct Foo{T} 
    bar:T = 100
end

Then use @Foo as a macro in front of other structs and voila. Fields, types, macros, default params, whatever. You can even chain them, and apply the @Foo macro to another @mix macro to merge them.

4 Likes

@gcalderone In practise I’ve ended up using a mix of what you are calling composition and encapsulation, although I’m not used to them being described that way - I would call your encapsulation aggregation (which I thought was a subset of composition), and leave encapsulation to oop access restriction… but none of these terms really make sense to me any more in julia.

I use Mixers.jl when I have multiple types that share a subset of fields and macros.

I’m even thinking about using empty concrete types fully made with mixins, so custom types can be made later without boilerplate.

If the list is long and the process is painstaking, you can

  1. consider simplifying the interface (too many verbs),

  2. write a macro (if you are doing it repeatedly).

This should not be hard to know: you forward the methods that are part of the relevant interface. Julia lacks formal specification for interfaces, so one should rely on documentation.

I’m referring to a situation like this (pseudocode):

struct A
    x::Vector
    extrainfo::Int 
end

where I want A to be exactly like a Vector except for one small tweak. I’ll make a few special methods for A and everything else should be forwarded to the x field.

But everything that is part of the Vector interface should be part of the A interface. And I don’t even know what all those methods are.

3 Likes

They are documented.

That’s AbstractArray. Calling methodswith(Vector) spits out a longer list, and that’s without showparents.

I don’t mean to say this should be easy to fix, or even smart. I’m just trying to explain what I’m looking for, and that is “Something exactly like (a concrete) Vector, with a twist.” Right now, that means reimplementing or at least forwarding a lot of methods. Can this be automated?

3 Likes

I struggled with this a lot when I started with Julia too. The approach I’ve come to find works best is the following.

  • Have Person and Citizen be <:AbstractPerson.
  • Define as many of your “100s of functions” on AbstractPerson instead of Citizen. This requires carefully thinking about what is generic behavior of AbstractPerson and what depends on implementation details such as which fields are in Person vs. Citizen. This might require introducing new functions, e.g. getters like get_name which can be specialized by Person vs. Citizen, but which then allow other many other functions to be remain generic.
  • For the functions which can’t be written generically on AbstractPerson, write specialized versions for Citizen and Person (if they truly can’t be written generically in step 2, this step you were always going to have to do anyway).

In general step 2 takes some thought and is not as quick and easy as in languages that have inheritance, say Python where its trivial to subclass and add a single new method or something, but I find that taking the time to do it this way leads to far better and more robust code. To the extent to which I’m familiar with the Julia standard library, this is the approach taken there as well.

18 Likes

This sounds like a good approach. But it’s difficult to apply when the object you want to wrap is pre-existing, for example in Base.

3 Likes

I understand what you are trying to do, I am just arguing that it is not an approach that meshes well with Julia, or leads to good interface design.

First, I think that if you want an interface, you should define an abstract type to go with it. This is costless in terms of performance, and allows you do document the interface in the docstring of the abstract type.

Second, you should minimize the functions that need to be implemented for this interface, ideally by choosing a core and then some extra methods that use this core but can be overridden for performance/implementation reasons. This is what AbstractArray does, and Julia’s compilation model makes this costless (in terms of runtime) in most cases, because of specialization and inlining (and lately, constant propagation).

IMO an interface which violates these is not good design. Interfaces should not be associated primarily with concrete types, nor should they be very rich. Either will cause problems independently of forwarding methods. That said, designing good interfaces is an difficult iterative process; frequently they emerge from functions of concrete types, and similarly, get streamlined by refactoring.

This link seems broken now, this one works for me.

I’ve been using abstract types for broad classification (i.e. AbstractString & AbstractChar), but for the rest, I don’t make hierarchies of abstract types, I’ve found traits and parameterized concrete types much more useful for writing generic code and having a consistent API. Most of the things about strings and characters are orthogonal to one another (Mutable or not, validated or not, single- or multi- codeunit encoding, ASCII compatible or not, ISO compatible or not, Unicode subset, full Unicode, Unicode + invalids (i.e. Char), or not Unicode compatible, etc.)
Trying to make that into a hierarchical structure explodes the number of types to deal with.

Also, instead of forwarding everything, you can use a method to access the Person from whatever type you have that includes person, and then call the person specific methods directly on it, which to me is clearer.

struct Person
    name::String
    gender::String
    birthday::String
end
person(p::Person) = p

struct Citizen
    person::person 
    nationality::String
end
person(c::Citizen) = c.person

Then you don’t have to add methods for every field, just one, instead of name(c) you’d have person(c).name or name(person(c))

7 Likes

Sorry for the absence… I will comment on individual posts below:

Sorry but I don’t understand why it should be related to be composition example. The Lazy.@forward requires encapsulation (i.e. approach 1 and 2 in the first post), not composition (approach 3). You will forgive me if encapsulation and composition are not the right words (again, I’m not a CS…), I use them just to refer to the approach discussed in the first post.

Completely agreed. Besides, the boilerplate code may add lots of entries in the dispatch table, without actually adding any new functionality. Moreover, all the boilerplate code, even if automatically generated, must be compiled resulting in further waste of time.

Thank you @Raf for pointing out Mixers.jl, it quickly allows to add further fields to structures without listing again the previous ones. This goes exactly in the direction of aproach 3 in the first post.

You can’t. If you want to add a twist to (e.g.) Vector, you simply can’t because you didn’t wrote the interface.

Thank @marius311 for providing a list of rules. Following your example I will provide my proposal, which is actually very similar to yours (except for the AbstractCitizen type):

  1. define an abstract type as supertype for each struct, e.g. define, Person <: AbstractPerson, Citizen <: AbstractCitizen, etc. In OOP terminology, the structures will be the data members of a class, and the abstract types will be the interfaces;
  2. define the hierarchy among structures using abstract types (not concrete types), e.g. AbstractPerson <: AbstractCitizen. This will allow us to define hierarchy in the same way as we do it in OOP;
  3. define sub structures by repeating the fields of the fields of the parent structure, in the same order and with the same types. This step can be automatized with Mixers.jl by @Raf;
  4. all methods working on a structure must accept the corresponding abstract type, not the concrete one. In OOP terminology these are the methods associated to a class;
  5. the only exception to the rule 4 above are the constructors, and those methods returning a specific structure (either Person or Citizen in the example above). These methods must accept concrete types, not abstract ones;
  6. define a super method like super(p::Citizen) = Person(p.name, p.age) (the name is not mandatory…) to be used in all methods of the derived structures to access the corresponding method acting on the parent structure.

I would like to emphasize that the goal here is not to reproduce an OOP practice in Julia, I know it is not possible and will likely lead to troubles. Here we only want to simply and efficiently extend/customize/specialize (use the word you prefer…) the behaviour of a Julia object whose methods have been written by someone else.

In my opinion this simple list of rules allows to easily extend the functionality of any object, and use it (quoting @DNF) exactly like the concrete object, with a twist. Moreover, even if the object is not supposed to be extended it will not harm its development or performances, and I can’t think of any practical reason to avoid following them. Any counter example here is more than welcome.

Well, if we agree that the above rules are to be followed, the only feasible way is to ask the object developer to adapt its implementation according to the rules… :wink:

Two final comments:

  • the very fact that we are discussing what is the best way to wrap an object into another to slightly customize its behavour, without adding lot of boilerplate code (either written or generated with macros), implies there is not yet a standardized and commonly accepted way to do it. Or, at least, I am not aware of it (again, any advice here is very welcome);

  • many of us, certainly myself, are struggling to understand whether we can use Julia as the main tool for our daily work. Hence, we need to know whether Julia has some intrinsic limitation. The impossibility to customize the behaviour of an object is (in my opinion) a strong restriction.

4 Likes

I’m struggling to understand that it’s somehow impossible to customize the behavior of an objects.
I’ve been use only Julia (even for my low-level work) for over 3 years now, and haven’t had any problems customizing the behavior of anything. I’m partial to using traits these days, because they end up very efficient, and allow me to deal efficiently with many orthogonal traits, and be able to add new traits later on, and new types using those traits, instead of having to change some hierarchy of abstract types.

4 Likes

well, you know where all this discussion comes from :wink:. Extending the behavior of, say, a DataFrame object is not trivial at all…

It would be nice if you could provide an example which solves the Person/Citizen problem in the first post using traits. Please :pray: avoid encapsulation since on a real example (e.g. a DataFrame) this means re-defining hundreds of methods…

To be honest, if you are trying to solve the problem linked above (define a DataFrame with a twist), it seems that one issue is the lack of a “tabular data interface”. It’s hard to believe that the hundreds of methods defined for DataFrames are all necessary: I imagine it’d be possible to write most of them as a function of a much reduced “table interface” (see what e.g. Query does, getting everything to work for every iterable of named tuples).

A very good solution in my view would be finalizing this tabular interface and rewriting things in function of it (see for example here for an attempt at porting StatsModels to this design).

2 Likes

This should be fairly easy, as Julia does not support encapsulation as it is usually understood (= restricting access to some slots) :wink: