Best way to call a function from another function with dispatch on first arg + various arguments

General Usage
1

OK I know my title doesn’t help but I don’t know how to explain my problem in a better way.

Here is the thing, I have a function fn1 that is called by another function fn2, and the dispatch is made on the type of the first argument in fn2 that is a subtype of a custom abstract type (“MyAbstractType”).

I want to implement different methods for fn1 in my package, possibly with different arguments, and some are computed in fn2 before calling fn1. The user could also implement its own method for fn1.

Here is the code of my abstract type for dispatch:

abstract type MyAbstractType end

Here is fn2:

function fn2(x1,x2::MyAbstractType)
    a = rand(1) + x1
    b = fn1(x2)
    return (b + a)
end

Here is one method for fn1:

struct Myfn1Struct <: MyAbstractType
    g::T
end

function fn1(x2::Myfn1Struct)
    x2.g+ rand(1)
end

OK now my issue is that I want fn1 methods to take any king of parameter (after x2) that is available from fn2 (fn2 computes a lot of variables, not just “a”). For example one could implement this method (adding the “a” parameter):

struct Myfn1Struct2 <: MyAbstractType
     g::T
end

function fn1(x2::Myfn1Struct2, a)
    x2 .g + rand(1) * a
end

The thing is this implementation of fn1 is not compatible with how we call it from fn2 anymore. fn1 is called like this in fn2:

b = fn1(x2)

But should be called like this when x2::Myfn1Struct2 :

b = fn1(x2,a)

How can I make the call more generic ?
I thought about adding all variables from fn2 that could be used by fn1 as arguments to fn1 so I can use the same call, and dispatch on the first argument, but I find it ugly.

I also thought about adding the variables computed in fn2 to a common struct that would be passed to fn1, but I feel passing that much information to a simple function is over-killed. Especially because I also need performance (those functions are called millions of times).

I could also implement several fn2 and dispatch on the call to fn2 considering the type of x2. But fn2 is complicated and has a lot of code, so it doesn’t feel right to copy paste a lot of this code.

Also I know I could break fn2 into several functions, but it represents a whole concept on its own and I want it to be easy to understand when reading it (it is already complicated enough as it is, I don’t want the reader to jump back and forth between functions).

2

I think the list of things you’ve considered mostly covers the available options. Ultimately you will either need to pass fn1 all of the things it might need (either as individual arguments or bundled up into a struct or tuple) or fn2 needs to be able to decide what to pass it based on the type of x2. That could either be done by having different methods of f2 or just a chain of if…elseif that hopefully gets resolved at compile time.

3

Passing all values as a NamedTuple should be okay, IIRC they are quite fast.
The second thing that comes to my mind is _... which looks weird but basically means ignore all positional arguments after that. You would then pass all values that you compute in fn2 to the call to fn1 and ignore all those that you don’t need in the definitions of fn1:

# I've abbreviated MyAbstractType to MAT and Myfn1Struct{n} to MS{n} here

julia> function fn2new(x1, x2::MAT)
         a = rand(1) + x1
         b = rand(1)
         c = sin.(a ./ b)
         d = x1 .* c
         # pass all the variables you may need to fn1(...)
         e = fn1(x2, a, b, c, d)
         return a+c+e
       end
fn2new (generic function with 1 method)

# as expected, this doesn't work with the original definitions

julia> fn2new([1.], MS1([2.]))
ERROR: MethodError: no method matching fn1(::MS1, ::Vector{Float64}, ::Vector{Float64}, ::Vector{Float64}, ::Vector{Float64})
Closest candidates are:
  fn1(::MS1) at REPL[4]:1
  fn1(::MS2, ::Any) at REPL[6]:1
[...]

julia> function fn1(x2::MS1, _...)
         # ... collects all further (positional) arguments
         # _ promises the compiler that you don't use this variable
         x2.g + rand(1)
       end
fn1 (generic function with 3 methods)

julia> fn2new([1.], MS1([2.])) # works now
1-element Vector{Float64}:
 3.9792069548588733

julia> function fn1(x2::MS2, a, _...) # take 1 
         x2.g .+ rand(1) .* a
       end
fn1 (generic function with 4 methods)

julia> fn2new([1.], MS2([2.]))
1-element Vector{Float64}:
 4.18615361885956

This works fine. But you have to document it in your API (if anyone should be able to make new subtypes with custom fn1 behaviour) and it gets rather ugly if you don’t need the first few additional variables but say, the first and fourth:

julia> struct MS3<:MAT
         g
       end

julia> function fn1(x2::MS3, a, _, _, d, _...) # could be worse, but imagine reading  a,_,_,s,_,_,_,_,x,y,z 
         # note that a trailing _... is completely harmless
         x2.g .+ rand(1) .* a ./ d
       end
fn1 (generic function with 6 methods)

julia> fn2new([1.], MS3([2.]))
1-element Vector{Float64}:
 5.7082930517576465
4

OK, thanks! Yeah I thought about using if statements, but it rapidly becomes ugly compared to multiple dispatch.

5

First time I see _...! It’s a very nice trick! OK I’ll consider using that if I don’t have too much methods for fn1 inside the package.

But maybe you’re right, a NamedTuple is the best solution after all. It’s more simple to read, and I’m afraid _... is a bit too cryptic for newcomers.

Thanks for the tip!

1 Like