How to code-gen unrolled loop over a tuple?

I would like a function that accepts a tuple of functions, picks one at random based on provided relative weights, and passes it to a definite second function. Right now, I accomplish this as follows, for e.g. a tuple of 3 functions,

function pickfunc(data, funcs::Tuple{T1, T2, T3}, cumweights) where {T1, T2, T3}
    x = rand();
    x < cumweights[1] && (usefunc(data, funcs[1]); return)
    x < cumweights[2] && (usefunc(data, funcs[2]); return)
    usefunc(data, funcs[3]); return
end

(I forewent chained if-elses for succinctness). The iteration is explicitly unrolled to that the type of the f \in funcs that is chosen is known at the call site to usefunc, and type stability is maintained. For that reason I use a tuple so that the identities of the functions are known (and not, e.g. Array{Function}). This is all important because all the functions are very lightweight, but are called many times by usefunc, and so benefit significantly from inlining in its specialized methods.

I would like to programmatically support tuples of different sizes (say, up to 10). How can I generate the various methods for pickfunc with metaprogramming?

(Alternatively, I’m happy to take suggestions for other design patterns that accomplish the same goal. I initially considered tuples/arrays of FunctionWrapper, but my approach using these was about x10 slower.)

2 Likes

How about this? It seems that unrolling is not an issue here, rather only to ensure type stability…

function pickfunc(data, fncs, weigths)
    x = rand()
    i = findfirst( w -> w <= x, weigths)
    
    isnothing(i) && return fncs[end](data) 
    
    return fncs[i](data)
end
1 Like

Is cumweights a Tuple? If it’s not a Tuple, the implementation will possibly need to be more complicated.

What is the nature of the functions? I’m asking just in case this is an XY problem:

https://xyproblem.info

FTR

The return type here isn’t known.

… so this causes a run time dispatch.

1 Like

The return type is in a Union at least:
Base.getindex(fncs, i::Int64)::Union{var"#13#16", var"#14#17", var"#15#18"}

But you are right, it seems that is not optimized away, looking at llvm/native code :wink:

I suggest using recursion to unroll a tuple.
The general pattern would be something like:

function foo(tup1, tup2)
    h1 = first(tup1)
    h2 = first(tup2)
    if condition(h1, h2)
        return ba(h1, h2)
    else
        return foo(Base.tail(tup1), Base.tail(tup2))
    end
end

If cumweights isn’t a tuple but a vector, you can have an integer index and increment it for every recursive call, instead of calling first and tail.

Note that the compiler doesn’t always succeed in realizing recursions like this terminate.

5 Likes

@Elrod @nsajko Yes, you can assume cumweights is a a tuple (or even NTuple).

@nsajko The functions all have the same “signature” of two arguments and a 2-tuple output. For example, the identity ID(a,b) = (a,b), and the right-addition ADD(a,b) = (a, a+b) as possible members of funcs

More generally, the problem I am trying to solve is: given an array data, I want to Monte-Carlo sample the resulting array after throwing a large number (~100000) of functions sampled from the small weighted ensemble funcs. Each function acts on pairwise elements of the array, and they are applied in a spatial pattern that is implemented in usefunc, which is independent of the random function drawn at each time step.

1 Like

Here’s a solution that assumes cumweights is a tuple. It roughly follows the structure outlined by @Elrod. @nospecialize is used to try to prevent unnecessary compiler overhead. @inline is used to force unrolling.

function weighted_rand_pick_from_heterogeneous_nospecialize(
    rand::R,
    f::F,
    (@nospecialize heterogeneous::Tuple{Any,Vararg}),
    (@nospecialize weights::Tuple),
) where {R,F}
    a = heterogeneous[1]
    if heterogeneous isa Tuple{Any}
        f(a)
    else
        let heterogeneous = heterogeneous::Tuple{Any,Any,Vararg},
            weights = weights::Tuple{Any,Vararg},
            w = weights[1],
            x = rand()
            if x < w
                f(a)
            else
                let r_h = Base.tail(heterogeneous), r_w = Base.tail(weights)
                    @inline weighted_rand_pick_from_heterogeneous_nospecialize(rand, f, r_h, r_w)
                end::Nothing
            end
        end
    end
    nothing
end

function weighted_rand_pick_from_heterogeneous(
    r::R, f::F, h::Tuple{Any,Vararg}, w::Tuple
) where {R,F}
    weighted_rand_pick_from_heterogeneous_nospecialize(r, f, h, w)
end

function usefunc end

function pickfunc(data, funcs::Tuple{Any,Vararg}, cumweights::Tuple)
    f = let data = data, closure
        function closure(g::G) where {G}
            usefunc(data, g)
        end
    end
    weighted_rand_pick_from_heterogeneous(rand, f, funcs, cumweights)
end

I didn’t test this, so report back on whether it works. Will post a more involved solution a bit later.

NB: relating to Elrod’s remark about the compiler not being able to prove termination on its own, you might want to use something like Base.@assume_effects :terminates_globally at some point in the call chain, among the callers of pickfunc, assuming you’re sure about termination.

The metaprogramming solution is to use generated functions. This should do the trick:

@generated function pickfunc(data, funcs::NTuple{N}, cumweights::Tuple) where {N}
    ex = quote
        x = rand()
    end
    for i in 1:(N - 1)
        line = :(x < cumweights[$i] && (usefunc(data, funcs[$i]); return))
        push!(ex.args, line)
    end
    lastline = :(usefunc(data, funcs[$N]); return)
    push!(ex.args, lastline)
    return ex
end
1 Like
module TupleTail
    export tuple_tail
    function vararg_tail((@nospecialize unused), r...)
        r
    end
    # More or less equivalent to `Base.tail`
    function tuple_tail(@nospecialize t::Tuple{Any,Vararg})
        r = vararg_tail(t...)::Tuple
        if t isa NTuple
            r = r::NTuple
        end
        r
    end
end

# Taken from my in-development package HeterogeneousLists.jl
#
# Not all of the code in the module is necessary here.
module TypeDomainNaturalNumbers

export NonnegativeInteger, PositiveInteger, natural_successor, natural_predecessor

using ..TupleTail

abstract type AbstractNonnegativeInteger end

"""
    NonnegativeInteger

Nonnegative integers in the type domain.

The implementation is inspired by the Zermelo construction of the natural numbers.
"""
struct NonnegativeIntegerImpl{
    Predecessor<:Union{Nothing,AbstractNonnegativeInteger},
} <: AbstractNonnegativeInteger
    predecessor::Predecessor

    global const NonnegativeInteger = NonnegativeIntegerImpl{P} where {
        P<:Union{Nothing,NonnegativeIntegerImpl},
    }

    global function new_nonnegative_integer(p::P) where {P<:Union{Nothing,NonnegativeInteger}}
        t_p = P::DataType
        r = new{t_p}(p)
        r::NonnegativeInteger
    end
end

"""
    PositiveInteger

Positive integers in the type domain.
"""
const PositiveInteger = let t = NonnegativeInteger
    t{<:t}
end::Type{<:NonnegativeInteger}

"""
    natural_successor(::NonnegativeInteger)

Return the successor of a natural number.
"""
function natural_successor(o::NonnegativeInteger)
    new_nonnegative_integer(o)::PositiveInteger
end

"""
    natural_predecessor(::PositiveInteger)

Return the predecessor of a nonzero natural number.
"""
function natural_predecessor(o::PositiveInteger)
    o.predecessor::NonnegativeInteger
end

function Base.zero(::Type{NonnegativeInteger})
    new_nonnegative_integer(nothing)
end

Base.@assume_effects :foldable function to_int(@nospecialize o::NonnegativeInteger)
    if o isa PositiveInteger
        let p = natural_predecessor(o), t = @inline to_int(p)
            t::Int + 1
        end
    else
        0
    end::Int
end

function Base.convert(::Type{Int}, o::NonnegativeInteger)
    to_int(o)
end

const negative_error = ArgumentError("can't convert negative to natural")

Base.@assume_effects :foldable function from_val(::Val{N}) where {N}
    n = N::Int
    if n < 0
        throw(negative_error)
    end
    if n === 0
        zero(NonnegativeInteger)
    else
        let v = Val{n - 1}(), p = @inline from_val(v)
            natural_successor(p::NonnegativeInteger)
        end
    end::NonnegativeInteger
end

function from_int(n::Int)
    if n < 0
        throw(negative_error)
    end
    if n === 0
        zero(NonnegativeInteger)
    else
        from_val(Val{n}())
    end::NonnegativeInteger
end

function Base.convert(::Type{NonnegativeInteger}, n::Int)
    from_int(n)
end

Base.@assume_effects :foldable function subtracted((@nospecialize l::NonnegativeInteger), @nospecialize r::NonnegativeInteger)
    if r isa PositiveInteger
        let a = natural_predecessor(l), b = natural_predecessor(r)
            @inline subtracted(a, b)
        end
    else
        l
    end::NonnegativeInteger
end

Base.@assume_effects :foldable function added((@nospecialize l::NonnegativeInteger), @nospecialize r::NonnegativeInteger)
    if r isa PositiveInteger
        let a = natural_successor(l), b = natural_predecessor(r)
            @inline added(a, b)
        end
    else
        l
    end::NonnegativeInteger
end

function Base.:(-)((@nospecialize l::NonnegativeInteger), @nospecialize r::NonnegativeInteger)
    subtracted(l, r)
end


function Base.:(+)((@nospecialize l::NonnegativeInteger), @nospecialize r::NonnegativeInteger)
    added(l, r)
end

end

module TupleUtils

using ..TypeDomainNaturalNumbers

export natural_tuple_length, tuple_element_at_index

"""
    natural_tuple_length(::Tuple)

Return a nonnegative integer which is the length of the given tuple.
"""
Base.@assume_effects :foldable function natural_tuple_length(@nospecialize t::Tuple)
    if t === ()
        zero(NonnegativeInteger)
    else
        let a = tuple_tail(t), b = @inline natural_tuple_length(a)
            natural_successor(b::NonnegativeInteger)
        end
    end::NonnegativeInteger
end

function tuple_element_at_index(t::Tuple{Any,Vararg}, @nospecialize n::NonnegativeInteger)
    p = natural_successor(n)
    i = convert(Int, p)
    t[i]
end

end

module WeightedRandPickFromHeterogeneous

using ..TypeDomainNaturalNumbers, ..TupleUtils

export weighted_rand_pick_from_heterogeneous

function weighted_rand_pick_from_heterogeneous_recursive(
    rand::R,
    f::F,
    heterogeneous::Tuple{Any,Vararg},
    weights::Tuple,
    len::NonnegativeInteger,
    (@nospecialize n::NonnegativeInteger),
)
    i = len - n
    a = tuple_element_at_index(heterogeneous, i)
    if n isa PositiveInteger
        let heterogeneous = heterogeneous::Tuple{Any,Any,Vararg},
            weights = weights::Tuple{Any,Vararg},
            j = natural_predecessor(i),
            w = tuple_element_at_index(weights. j),
            x = rand()
            if x < w
                f(a)
            else
                let nm1 = natural_predecessor(n)
                    @inline weighted_rand_pick_from_heterogeneous_recursive(
                        rand, f, heterogeneous, weights, len, nm1,
                    )
                end::Nothing
            end
        end
    else
        f(a)
    end
    nothing
end

function weighted_rand_pick_from_heterogeneous(
    r::R, f::F, h::Tuple{Any,Vararg}, w::Tuple
) where {R,F}
    len = natural_tuple_length(w)
    weighted_rand_pick_from_heterogeneous_recursive(r, f, h, w, len)
end

end

function usefunc end

function pickfunc(data, funcs::Tuple{Any,Vararg}, cumweights::Tuple)
    f = let data = data, closure
        function closure(g::G) where {G}
            usefunc(data, g)
        end
    end
    weighted_rand_pick_from_heterogeneous(rand, f, funcs, cumweights)
end

Please don’t use metaprogramming when clearly not necessary.

Given the convoluted solutions provided, it think metaprogramming being “clearly unnecessary” is quite an overstatement. It is unnecessary, but the solution is not clear at all.

IMO the metaprogramming solution seems much more simpler and easy to understand, but that’s probably subjective.

1 Like

I set method.recursion_relation = Returns(true) to work around this, when necessary.
Maybe that’s a bad idea, but I don’t mind encouraging people to do it in case that leads to a less hacky solution.

Does the code really benefit from @nospecialize? I’d have expected it to all inline into unrolled code.

I think recursion is generally preferred over meta programming as the solution to this. I find it more readable.
However, meta programing can unfortunately be more reliable, i.e., less fighting compiler heuristics.

2 Likes

It would be nifty if it was possible to use typevars for the length of the Base.Cartesian unrollers (@nexprs, @nif, @nloops, …). Unfortunately, since they’re macros I’m not sure there’s any way of doing this (that doesn’t involve invoking @generated, at which point one can just do that). Since they require literal constants, I seldom find them usable.

You could use the trick that LV uses, which is punt to an @generated function.

@danielwe This seems like a really elegant solution! Would this work for funcs which is not an NTuple? The entire goal is to retain the individual type information of each element in funcs. If the signature contains funcs::Tuple will the compiler/preprocessor accept N = length(funcs) in the function body (which in principle is resolvable at compile time)?

Any Tuple can be regarded as subtyping some NTuple.
For example:

julia> Tuple{typeof(sin),typeof(cos),typeof(+)} <: NTuple{N,Function} where N
true

So, just use funcs::NTuple{N,Function} as the signature for the pickfunc function.

I thought NTuple{N} would do it, but it seems not:

julia> Tuple{typeof(sin),typeof(cos),typeof(+)} <: NTuple{N} where N
false

I don’t understand why…

NTuple{N} where N is an alias for NTuple{N,T} where {N,T}, so it always expects the same T. It doesn’t work for functions because each function has its own concrete type:

julia> typeof(sin)
typeof(sin) (singleton type of function sin, subtype of Function)

julia> typeof(cos)
typeof(cos) (singleton type of function cos, subtype of Function)

I guess the most generic signature you could use is NTuple{N,Any} where N, which will work for any tuple:

julia> Tuple{typeof(sin),typeof(cos),typeof(+),Float64} <: NTuple{N,Any} where N
true

This way you can also pass any callable object that doesn’t necessarily subtypes Function, but acts as one.

2 Likes

I actually thought NTuple{N} would match any Tuple of length N, but as @favba pointed out, you need NTuple{N,Any}. With that change you’ll be good to go.

In the body of a @generated function, argument names refer to their type, not their value, so funcs evaluates to something like Tuple{F1,F2,F3}, and length doesn’t apply to that. Matching NTuple{N,Any} is the way to go.

What’s more is, generated functions are a particularly dangerous form of metaprogramming available in Julia. They are basically reserved for power users with some awareness of how Julia is implemented, and certainly should not be promoted here on the forum. Some excerpts from the Metaprogramming page in the Manual:

Since the body of the generated function is non-deterministic, its behavior, and the behavior of all subsequent code is undefined.

Note that the set of operations that should not be attempted in a generated function is unbounded, and the runtime system can currently only detect a subset of the invalid operations. There are many other operations that will simply corrupt the runtime system without notification, usually in subtle ways not obviously connected to the bad definition. Because the function generator is run during inference, it must respect all of the limitations of that code.

TLDR: generated functions are not safe unless you’re on the level of a compiler dev

IMO, this is the real overstatement. The knowledge required to correctly do this goal with a @generated function is much less than the convoluted mess of doing it with Tuple recursion.

I find it especially ironic that you’re claiming that a regular user can’t safely use a generated function, but then you go on to talk about and advocating the use of @assume_effects invocations (which are in many ways, far more dangerous and easier to abuse).

@danielwe’s solution is a quite reasonable suggestion. Maybe using tuple recursion here is preferable, and it’s fine to debate what’s better, but it’s IMO not at all clear that we should be trying to stop people from even suggesting metaprogramming.

Metaprogramming is a completely legitimate and useful tool offered by Julia.

2 Likes