Idea: concrete interfaces

General Usage
,
1

While I’m generally extremely happy with Julia’s type system, I have sometimes found myself wishing it were easier to efficiently deal with heterogeneous collections. For example, if I have a Vector of elements of different concrete types, it’s much slower to call a method on all of those elements than it would be if they were all the same type.

FunctionWrappers.jl makes this possible by allowing you to store concretely-typed function wrappers instead of the objects themselves, but that package is very low-level. I’ve been building on top of FunctionWrappers and have come up with an approach that might be more user-friendly: https://github.com/rdeits/Interfaces.jl

The core of the package is the @interface macro, which is used like this:

@interface MyInterface{T}(self) begin
  foo(y::T)::T = self.x + 1
  bar()::T = self.x
end

This defines a new type, MyInterface which looks something like:

struct MyInterface{T}
  self::Any
  foo::FunctionWrapper{T, Tuple{T}}
  bar::FunctionWrapper{T, Tuple{}}

  MyInterface{T}(self) = new(self, y -> self.x + y, () -> self.x)
end

and also defines foo(::MyInterface{T}, ::T) and bar(::MyInterface) methods.

The result is that you can construct a MyInterface from any input self that you want, and all MyInterface{T} objects have the same concrete type regardless of the type of self. Calling the foo() and bar() methods on a MyInterface goes through the function wrappers, so there is no dynamic dispatch (even though self is untyped). That means that you can call y .= bar.(::Vector{MyInterface{T}}) with no memory allocation and much better performance than y .= bar.(::Vector{Any}).

I’ve put together a more complete demo here: https://github.com/rdeits/Interfaces.jl/blob/f729c534af987244348d08ea359c39fc47451885/demo.ipynb

I’m very curious if this is something that other people have wanted, and whether the design I’ve put together is one that would be useful. Please let me know!

4 Likes
2

At least for AbstractArrays, you can “cast” elements using MappedArrays:

julia> using MappedArrays

julia> a = Any[1, 1.0, 0x03]
3-element Array{Any,1}:
    1  
    1.0
 0x03  

julia> b = mappedarray(x->convert(Float64, x)::Float64, a)
3-element MappedArrays.ReadonlyMappedArray{Float64,1,Array{Any,1},##3#4}:
 1.0
 1.0
 3.0

julia> using Base.Test

julia> @inferred(a[1])
ERROR: return type Int64 does not match inferred return type Any
Stacktrace:
 [1] error(::String) at ./error.jl:21

julia> @inferred(b[1])
1.0

julia> @inferred(sum(b))
5.0
3

Oh cool! I didn’t know that was possible. But I think this only works if the elements of the array are all castable to some concrete type, right? I’m more interested in the case where the array elements are of different (and incompatible) concrete types, but with methods that have the same signature for each element.

4

You’re right this is much better at avoiding dynamic dispatch, so :+1:.

At least for bitstypes, 0.7 will be considerably better than 0.6:

julia> a = Union{Int32,Float64}[rand() < 0.5 ? Int32(x) : Float64(x) for x = rand(Int32, 10^6)];

julia> typeof(a)
Array{Union{Float64, Int32},1}

# after warmup (BenchmarkTools is broken on 0.7)
julia> @time sum(a)
  0.017148 seconds (2.00 k allocations: 31.391 KiB)
-8.18452016346e11

There are still a number of allocations, but nothing like what you get on 0.6, and the performance is roughly 5x better. But I’m not sure this is true for non-bitstypes, so Interfaces/FunctionWrappers is intriguing.

5

I definitely think that at the very least FunctionWrappers should be moved to Base. It provides some very important functionality and it would take quite a bit of specialized knowledge to reproduce from scratch.

2 Likes