[ANN] EnforcedTypeSignatureCallables.jl

Package Announcements
, , , ,
1

The EnforcedTypeSignatureCallables package is being registered: New package: EnforcedTypeSignatureCallables v1.0.0 by JuliaRegistrator · Pull Request #111416 · JuliaRegistries/General · GitHub

Once it’s registered, this link will point to its page on JuliaHub: EnforcedTypeSignatureCallables.jl

The Git repo is on Gitlab:

The package was motivated by the frequent requests by Julia newbies for expressing the type signatures for “functions” in the type system. This doesn’t really make sense as a problem in Julia, but this simple implementation is perhaps close to being a solution.

The package may also be useful for ensuring type stability/helping the Julia compiler’s type inference.

Related

The FunctionWrappers.jl package offers superficially similar functionality, however its goals are actually completely different. Unlike this package, FunctionWrappers doesn’t have a type parameter for the type of the wrapped callable. Thus the same FunctionWrappers type may wrap multiple functions, assuming the type signature is the same.

Also see this recently opened PR to Julia, aiming to make FunctionWrappers obsolete: RFC: Introduce `TypedCallable` by topolarity · Pull Request #55111 · JuliaLang/julia · GitHub

Request for comments

I’m not sure about the naming yet. Specifically, the name of the exported type is TypedCallable, the same as in the linked Julia PR. It’d probably be good if anyone can think of a better name before EnforcedTypeSignatureCallables is registered.

1 Like
2

I think this actually is a problem, it makes perfect sense, and it’s not specific to newbies. Simply rephrase the problem in terms of methods. Let the syntax

function my_map(fn::Function{T, U}, data::AbstractVector{T})::Vector{U} where {T,U}

mean: “fn is any function that has a method that accepts a single T and returns a U”. The usual “most concrete type first” rules apply: if eltype(data)===Float64, then fn(::Real)::U will be applied, not the more general fn(::T)::U, etc. Given some function like sin, find its most concrete method that satisfies the Function{T, U} constraint. There can be many functions that satisfy this criterion: one can call my_map(sin, [1,2,3]), my_map(sum, [1,2,3]), my_map(x -> "hello", [1,2,3]) and so on. No wrapper types required.

Julia’s documentation says:

…choose which of a function’s methods to call based on the number of arguments given, and on the types of all of the function’s arguments. Using all of a function’s arguments to choose which method should be invoked … is known as multiple dispatch.
…
The most specific method definition matching the number and types of the arguments will be executed when the function is applied.

Why not use multiple dispatch when passing a function as an argument? “Simply” pass the most specific method that match the parameter’s type as the argument. This approach seems very natural to me, very much in line with dynamic dispatch.

I think it’s strange that Julia doesn’t have syntax for function signatures (would be “method signatures” in Julia world, I guess), whereas other languages that have meaningful type annotations (i.e. C, Rust, Haskell, but not Python) do. Would such syntax not be useful? Is the idea of placing constraints like Function{T, U} on methods (not individual functions) wrong?

1 Like
3

Your messsage is interesting, but TBH it seems off-topic here. Could you move it to the Julia issue tracker on Github, as a feature request? I’m not able to change Julia’s dispatch behavior from a mere user package.

4

Why is this cascade of asserts required?

1 Like
5

It’s not :sweat_smile:. I included it “just in case”, however after checking with Core.Compiler.return_type it seems that none of the four type asserts in the cascade is useful. Deleting these four lines.

Julia is able to optimize out unnecessary type assertions, though, so I think they’re usually harmless.

1 Like
6

This technically already happens, just not in the way you think it can.

First, we have to throw out the U part because a specific call signature with a unique piece of compiled code may not have a fixed return type, so it’s not a useful part; you could make U abstract and leave it entirely up to the compiler, but that’s not controllable or stable. Second, 1 method (and method signature) can have an infinite number of call signatures (and thus compiled specializations) at once e.g. foo(x::Real) can be compiled for foo(::Int) and foo(::Float64). Type stability and control makes call signatures (like this very package!) better to specify, so you wouldn’t in fact want to dispatch on methods in any sense. Even in your example, T=Float64, and you make it clear no method fn(::Float64) exists, only fn(::Real). I see what you want, but the type system does not allow Blah{Float64} and Blah{Real} to share an instance. It also doesn’t allow multiple supertypes; T=Int in the example calls in fact, so sin, sum, etc need to subtype Function{Int} as well.

The good news is you don’t need any of that stuff. The first hint is that when a function call is dispatched, the types of the callable and the arguments are taken into account, so the types of functions like sin and sum only have to care about the callable. That’s how we can write methods for callable instances and vary positional arguments in multimethods, yet let the compiler infer specific compiled code so often.

So if we can already dispatch on all parts of a call, why the concept of FunctionWrappers in the first place? Sometimes we can’t narrow down a call site to 1 specialization because one position, often the callable, has to change types at runtime, and that normally needs possibly costly dynamic dispatch and ruins type inference downstream. So FunctionWrappers.jl wrapped specializations with input/output type conversions, putting that information in a concrete type (and parameters) so we can lump many callables together. EnforcedTypeSignatureCallables won’t be able to elide the dynamic dispatch part because most different functions will be represented by an abstract type parameter, but it does provide more inferrability and it’s not subject to many of FunctionWrappers’ constraints and dynamism shortcoming (example).

7

How feasible is it to incorporate keyword argument types from a call site into TypedCallable? Currently the call looks like:

function (tc::TypedCallable{A,R})(args::Vararg{Any,N}; kwargs...) where {A,R,N}
    args = args::A
    r = (tc.f)(args...; kwargs...)
    r::R
end

asserting that input types A result in output type R. But A is only asserted for positional arguments, while the return type will also vary with keyword arguments:

julia> foo(;a) = zero(a)
foo (generic function with 1 method)

julia> foo(;a=1), foo(;a=1.0)
(0, 0.0)

julia> Base.specializations(only(methods(var"#foo#1"))) # skipping some steps to find this symbol
Base.MethodSpecializations(svec(MethodInstance for var"#foo#1"(::Int64, ::typeof(foo)), MethodInstance for var"#foo#1"(::Float64, ::typeof(foo)), nothing, nothing, nothing, nothing, nothing))

If brevity is a concern, this could be a separate type for calls with keyword arguments, keep TypedCallable’s structure for the typical positional-only calls.

8

Possibly it’s doable, but certainly not trivial, which is why I gave up on it right away. Maybe make a PR?

Keyword arguments seem problematic, because they can come in any order. A call like f(; a = 1, b = true) will have a different type of named tuple for the kwargs than f(; b = true, a = 1).

1 Like
9

Exactly my concern over feasibility. Making a type depend on keyword argument order is not nonsensical because a target call site will have those keywords in a particular order, but that categorizes and restricts the callable in a fairly unnecessary way. The keyword argumented calls’ underlying calls sort the keywords into positional arguments, so they don’t appear to be specializing over arbitrary keyword order either.

1 Like
10

I’ve played around with things like this a few times, and I prefer to call them something like MethodWrappers or CallableMethods since they (ideally) would statically specify a single method of a function and make it invokeable.

Functions in Julia have no specific signature, only methods have signatures, so this concept is more related to methods than functions in my mind.

3 Likes