Enforcing function signatures by both argument & return types

I’d like to pass a function (f) to another function (g) but g expects f to take some specific types of arguments and return a specific type. Is it possible to enforce that statically?

For example, g() expects f to take a single integer argument and return another integer. So the following code is valid:

julia> foo(x::Int) = x + 1
foo (generic function with 1 method)

julia> g(f, v::Int)::Int = f(v)
g (generic function with 1 method)

julia> g(foo, 10)
11

However, if I pass a function that doesn’t match the intended signature then I would like to see it fail to compile. The question is how do I write the signature of f in the definition of the g function?

3 Likes

I would love to see this too! Some previous discussion https://github.com/JuliaLang/julia/issues/17168 and Function Parameter Speculation #17168.

There’s a few challenges, the biggest of which is related to multiple dispatch–what happens if foo has multiple methods? (x::Int, y::Int) -> 1 + x + y and f(x::Int, y::Int) = 1 + x + y currently have different types in Julia. In fact, currently, a = (x, y) -> 1+x+y; b = (x, y) -> 1+x+y; typeof(a)==typeof(b) returns false. So there are many needed changes to the type system, including a return type foo(x::Int) --> Int

1 Like

Will this work instead:

julia> struct MyFunctor{T1,T2,S} end

julia> MyFunctor{T1,T2,S}(arg1::T1, arg2::T2) where {T1, T2, S} = S(arg1+arg2)

julia> myfunc = MyFunctor{Int,Int,Float64}
MyFunctor{Int64,Int64,Float64}

julia> myfunc(1,3)
4.0

julia> myfunc(1,3.)
ERROR: MethodError: no method matching MyFunctor{Int64,Int64,Float64}(::Int64, ::Float64)

julia> g(f::Type{MyFunctor{Int,Int,Float64}}, a::Int, b::Int) = f(a,b)::Float64
g (generic function with 1 method)

julia> g(MyFunctor{Int,Int,Float64}, 1, 2)
3.0

julia> g(MyFunctor{Int,Int,Int}, 1, 2)
ERROR: MethodError: no method matching g(::Type{MyFunctor{Int64,Int64,Int64}}, ::Int64, ::Int64)
Closest candidates are:
  g(::Type{MyFunctor{Int64,Int64,Float64}}, ::Int64, ::Int64) at REPL[6]:1

This is like defining your own function type and making methods for it, then dispatching on the argument and return types of the function.

You can also have an abstract type IntIntFloat64 and use f::Type{<:IntIntFloat64} instead when defining g, like this:

julia> abstract type TwoArgOneRetFunctor{T1, T2, S} end

julia> const IntIntFloat64 = TwoArgOneRetFunctor{Int,Int,Float64}
TwoArgOneRetFunctor{Int64,Int64,Float64}

julia> struct f1 <: IntIntFloat64 end

julia> f1(arg1::Int, arg2::Int) = Float64(arg1+arg2)
f1

julia> g(f::Type{<:IntIntFloat64}, a::Int, b::Int) = f(a,b)::Float64
g (generic function with 2 methods)

julia> g(f1, 1, 2)
3.0

julia> struct f2 <: IntIntFloat64 end

julia> f2(arg1::Int, arg2::Int) = Float64(arg1*arg2)
f2

julia> g(f2, 3, 2)
6.0
4 Likes

I don’t know if that approach would work in general but you can run type inference to get the return type of a method:

struct TypedFunction{T <: Function,I,O}
    f::T
    i::I
    o::O
end

function TypedFunction(f, method_index)
    
    m = collect(methods(f))[method_index]
    args = (m.sig.parameters[2:end]...)
    
    codeinfo = Base.code_typed(f,args)[1]
    ret = codeinfo.second
    
    TypedFunction(f,args,ret)
end

import Base: show, match
show(io::IO,f::TypedFunction) = write(io, string(tf.f,": $(tf.i) → $(tf.o)")) 
match(tf::TypedFunction,i,o) = tf.i == i && tf.o == o

Then you can check that the method signature matches the desired types (would be even nicer to use dispatch):

julia> f(x::Int,y::Float64) = x+y
f (generic function with 1 method)

julia> tf = TypedFunction(f, 1)
f: (Int64, Float64) → Float64

julia> match(tf,(Int64,Float64),Float64)
true
2 Likes

I think you’re looking for

There are usage examples in the tests.

3 Likes

You might also be interested in this trait-based approach which will work for normal functions for which you assert a certain argument-return types trait:

julia> struct IntIntFloat64 end

julia> f(x::Int, y::Int) = Float64(x+y)
f (generic function with 1 method)

julia> args_ret_trait(::Type{typeof(f)}) = IntIntFloat64()
args_ret_trait (generic function with 1 method)

julia> g(f::Function, ::IntIntFloat64, x, y) = f(x,y)
g (generic function with 1 method)

julia> g(f::F, x, y) where {F<:Function} = g(f, args_ret_trait(F), x, y)
g (generic function with 2 methods)

julia> g(f, 1, 2)
3.0
1 Like

Just saying, one of the wonderful things about Julia is that there are a number of performant ways of dealing with this sort of problem, courtesy of Jeff’s design of the type system.

2 Likes

I’m truly amazed with all of the solutions above. I was originally hoping for a one-liner just in the method signature though. I know I’m greedy. :wink:

Shouldn’t be too hard to use macros to make f(x::Int, y::Int)::Float64 = x+y syntactic sugar for:

if !isdefined(:IntIntFloat64)
     struct IntIntFloat64 end
end
f(x::Int, y::Int) = Float64(x+y)
args_ret_trait(::Type{typeof(f)}) = IntIntFloat64()

And g(f::Function{IntIntFloat64}, x, y) = f(x,y) syntactic sugar for:

g(f::Function, ::IntIntFloat64, x, y) = f(x,y)
g(f::F, x, y) where {F<:Function} = g(f, args_ret_trait(F), x, y)

Maybe you could try coding it up :wink:

1 Like

Haha… I keep forgetting the power of Julia’s meta programming ability. This is great! :+1:

1 Like

These are brilliant solutions! It would be great to see a solution so that a pre-existing function like abs(x::int) could be used. @jonathanBieler, any idea what it would take to get match(abs,(Int64),Int64) or match(abs,(Real), Real), to work as expected? I’m guessing that @mohamed82008’s solution would work if the appropriate traits were added to all functions, preferably automatically

I think the above solution would work only for one method functions. You can make it more clean and compoundable by defining a parametric trait instead of IntIntFloat64, similar to my first solution above.

But if you want to do the same thing without sacrificing multiple dispatch it would get a little complicated. I think in that case you would need a trait function that takes a function type, its desired input types and its desired return type then returns Val{true}() if a valid method was found to dispatch to with the desired input types and which returns the desired output type or Val{false}() otherwise. Then one can dispatch on this trait. This will make heavy use of dispatch and inference machinery though so it is beyond me at this point.

1 Like

It’s becoming a bit complicated but I think this works:

Now you can provide the signature you want to the constructor and it look through the function’s methods to find a match (returns an error otherwise)


f(z,x::Int,y::Int) = x+z
f(x::Int,y::Float64) = x+y
f(x::Int) = x
f() = 2

julia> tf1 = TypedFunction(f, (Int,Float64),(Float64))
f: Tuple{Int64,Float64} → Float64

You can use dispatch to put constrains on input/output types:

g(f::TypedFunction{T,I,O}) where {T,I<:Tuple{Number,Float64},O<:Number} = 1

julia> g(tf1)
1

Or use the function match_signature to check if there’s a method matching the signature:

match_signature(f,(Float64,Int,Int),(Float64),false)

The last argument determine if strict check is done (==) or subtyping (<:) which makes generic functions work too:

julia> match_signature(f,(Float64,Int,Int),(Float64),false)
1-element Array{Any,1}:
 f(z, x::Int64, y::Int64) 

There’s probably some issues I haven’t thought about.

2 Likes

Wow, this looks super promising! I tried match_signature(abs,Int64,Int64,false) and got a type UnionAll has no field parameters error. Looks like it comes from line 41, m.sig.parameters. I’m not sure why this would fail when your example for f works.

It’s because abs has parametric methods (e.g. abs(x::T)), it’s possible to handle those as well, but it requires a bit more work. I feel like I’m reinventing the wheel a bit here, since all this stuff is already done by the multiple dispatch system internally.

Given Julia 1.0 has been released since this topic was last updated, is there now an accepted way or does a user still have to decide between these disparate approaches?

5 Likes

Is there any ongoing work concerning this? Any plans to have built-in syntax for function types?

Right now it doesn’t seem possible to write out the full type of a function, like:

g(f::Function{Int, (Int, )}, v::Int)::Int = f(v)
  • FunctionWrappers.jl seems to be alive and well, but there’s no documentation, only tests
  • This RFC has been open and basically untouched since 2015
  • Answers to this post from 2019 say that it’s not possible to specify the type of a function argument which is supposed to be a function. The answers link to an entire PhD thesis and say that specifying the function signature won’t bring performance benefits. But the real benefit here is that it’ll make it easier for the programmer to understand what kind of arguments higher-order functions expect.

Why is there no built-in way of even expressing types of functions? Is this not an important feature? Specifically for a language that’s basically all about functions? AFAIK, Rust doesn’t have syntax to express types of (certain kinds of) closures, so there must be something difficult about that (I guess the difficult thing is the need to describe variables captured by the closure?), but Julia doesn’t seem to have syntax to express types of functions at all, which seems unnecessarily limiting. Is it way too difficult to implement?

I’m not an expert and couldn’t really keep up with this: Function Parameter Speculation #17168, but why is, according to this comment, “What is the type of a function?” a “hard question”? Isn’t it (Arg1Type, Arg2Type) -> ResultType? Or function(Type1, Type2)::RetType? Or Callable{RetType, (Type1, Type2)}?

Why not have something like function pointers, similar to C? Sure, function pointer types can get illegible really quickly, but at least they’re there if you need them.

Because it is difficult to give a type to a function like this one.

Are there sufficiently many such functions to care about them? Even Haskell (as far as I know, its type system is pretty advanced) has a non-obvious way of dealing with the Y combinator, so maybe just don’t allow such functions at all?

Comparing to C again - a lot of code is written in it, and nobody seems to complain about being unable to type Y combinators.

Regarding Julia as I understand it the semi dynamic nature of the language is another complication with regard to static type systems and I’m personally not too unhappy about the compromises.