High-order functions prototyping

New to Julia
1

I’m an old C/C++ programmer so please forgive me for addressing a problem from that point of view.

I have a function that takes, as an argument, another function that the user is expected to supply. A little bit like a sort function where you provide the comparison function.
I have strict requirements regarding the user-supplied function regarding the arguments it takes and what it returns. I would like to make sure this is validated at compile time.

For example, my function is called “foo” and the user-supplied function is “bar”
function foo( c::Int64, bar::function)::Float64

I want to make sure that “bar” has the following format:
function bar(d::Int64, String{})::Float64

How can I enforce this? At least, how can I validate that at run time before running into an error.

Thanks

2

In Julia, a Function usually has multiple methods, and each method has its own type signature. So the type of the function argument itself doesn’t carry the information that you need, you cannot dispatch on input or output type signatures of functions (like, allow all functions which conform to some signature).

So you will receive some unknown Function object in your function from the user. You can either just call the function and put an assert on the return type like result::Float64 = userfunc(...) and this will error if the output type is wrong. Another option is result = userfunc(...)::Float64 and this will convert the output value and error if that’s not possible.

But maybe you don’t want to call the function at all because it might have side effects or for whatever reason. You can make use of the internal function Base._return_type which can tell you what inference determined will be the output type of userfunc for a given set of input types. But there’s a whole debate whether one should rely on this or not, it definitely isn’t encouraged by the core developers. However, internally, Julia seems to use this itself for determining the containers holding the return values of broadcast operations, so you’re not alone with this problem. I personally think that if Base._return_type determines that the output type is exactly the concrete type you expect, it should be ok to use, because type inference should not get worse in future Julia versions. But of course type inference could return Any even if the function in practice always returns Float64 because the user has type instabilities in their code. So it’s a bit of a brittle approach.

Finally, you can have a look at GitHub - yuyichao/FunctionWrappers.jl and GitHub - chriselrod/FunctionWrappersWrappers.jl but these are also not “officially endorsed”.

It seems to me most people rely on duck typing in practice, or just confirming that the output always conforms to the signature that’s needed after the fact.

2 Likes
3

Easy solution but not efficient I guess.

Define a function bar with the parameter types and return type you want - that’s your only way to enforce it, so the user enters what you specified if they choose to use your function and expect a result you promised them. Then use the try...catch block to handle the errors, if they choose not to follow your instructions.

function bar(d::Int64, String{})::Float64
    
end

function foo(c::Int64, bar::function)::Float64
    try; bar(......)
    catch
        # do something about the error or not
    end
   # continue your program

Talking about enforcing a user to input only want you want is going to be hard as anyone can type what they want on their keyboard.

4

Many thanks Jules and uje

I definitively don’t want to enforce a function. The user may chose what he wants.

I guess I’ll have to resort to error catching at the beginning before starting a large computer intensive loop which I don’t want to include a try within.

5

Yeah, try..catch blocks, should not be frequently used as advised in the Julia documentation. So it’s best to just define one implementation of the bar function with its precise parameter types and return types, and then have the user use it or when he doesn’t catch the error earlier (if you choose to).

6

Checking that input arguments are valid is the easy part:

# Check that function `f` accepts arguments of given types
checkarguments(f, argtypes) = length(methods(f, argtypes)) > 0

# Example
foo(::Int, ::Float64) = 1

checkarguments(foo, (Int, Float64)) # true
checkarguments(foo, (String,))      # false

Inferring the return type is more problematic, though, as @jules pointed out.

Notice also that checkarguments may return different values for exactly the same inputs, if the table of methods for f is changed.

1 Like
7

Or just call hasmethod.

1 Like
8

Beautiful

  1. I check the arguments with checkarguments.
  2. Then I run the function with a dummy value that is in the acceptable set of the function (the interval [0,1] in my case) in a try/catch to see if it works
  3. Then I validate if the output is of type Float64 with typeof()

After that, I’m ready to start processing.

All set. Thanks to all for the very quick responses.

9

By the way, it’s probably also worth pointing out what you gain from the way Julia is set up, since I agree it’s awkward to enforce this kind of contract.

In C or C++, the most obvious solution to your problem would be to take a function pointer or a std::function with the appropriate signature. You’d then just call that function pointer within the loop, and your problem would be solved, right?

Maybe not: It turns out that can actually be a performance trap. If you take your input bar as a function pointer, then every time you call that function you are actually performing a function call, which has a measurable overhead even in C. This means that if your bar is some very cheap operation, then you’ll get much worse performance by passing it as a function pointer than if you’d just hard-coded that function in place. That’s because an opaque function pointer prevents the compiler from inlining that function call, which would eliminate that overhead otherwise.

For better performance in C++, we’d instead generally do something like:

template <typename F>
void foo(int c, const F& bar) {
};

and pass a C++ lambda or other C++ function directly as bar, without casting it to a function pointer at all. By making the exact type of bar visible to the compiler, we allow the compiler to inline bar if it chooses to, resulting in better performance. But now we’ve lost the ability to easily constrain the signature of bar (it’s not even necessarily a function anymore, just some thing we can call with ()). So then we start adding some unholy template metaprogramming std::result_of stuff, which nobody actually enjoys. In other words, our effort to write high-performance C++ has resulted in something that looks a lot more like Julia.

In Julia, you can think of every function as operating like a template function in C++. In particular, since every function in Julia has a distinct type, the compiler can generate specialized code for foo for each different bar you provide, potentially inlining that bar call if necessary.

If you want to reproduce the C-style function pointer approach (and associated slowness), you can do exactly that with GitHub - yuyichao/FunctionWrappers.jl , but I wouldn’t recommend it.

7 Likes
10

The usual practice is to make the function argument the first argument of your higher-order function to allow use of do block syntax.

4 Likes