# CUDA kernel: how to pass an array of functions

**URL:** <https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784>\
**Category:** GPU\
**Tags:** cuda\
**Created:** [February 7, 2021, 12:29pm UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784 "2021-02-07T12:29:20Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shuhua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shuhua/32/27618_2.png) [@Shuhua](https://discourse.julialang.org/u/Shuhua)\
**Post date:** [February 7, 2021, 12:29pm UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/1 "2021-02-07T12:29:20Z")

</div>

I am working on [genetic programming](https://en.wikipedia.org/wiki/Genetic_programming), where I need to pick a function from a set of functions `fs` by an integer index in each node of the tree. An example demonstrating the difficulty I encountered is given below.

```julia
using CUDA

# a kernel
function pass_func_array(x::Float32, fs)
    for f in fs
        x = f(x)
    end
    @cuprintln("x = $x")
    nothing
end

```

where `fs` is an array of functions. However, the following way does not work since `Function` is not a bit type.

```julia
fs = CuArray([sin, cos])
LoadError: CuArray only supports bits types

fs = CuArray([CUDA.sin, CUDA.cos])
LoadError: CuArray only supports bits types

```

Any ideas on how to fix this?

I tried to pass symbols (i.e., function names), but a `Symbol` is not a bits type either.

---

<div class="post-metadata">

**Author:** ![maxfreu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maxfreu/32/17468_2.png) [@maxfreu](https://discourse.julialang.org/u/maxfreu)\
**Post date:** [February 7, 2021, 3:44pm UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/2 "2021-02-07T15:44:33Z")

</div>

Hi, dangerous half-knowledge here, but I think this is a use case for [metaprogramming](https://docs.julialang.org/en/v1/manual/metaprogramming/). As far as I know, everything you use in a kernel has to be hard-coded or must be inlineable. Maybe you can write a function which generates the entire kernel code for you, instead of passing stuff to a kernel.

---

<div class="post-metadata">

**Author:** ![Shuhua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shuhua/32/27618_2.png) [@Shuhua](https://discourse.julialang.org/u/Shuhua)\
**Post date:** [February 8, 2021, 2:30am UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/3 "2021-02-08T02:30:04Z")

</div>

Thanks. Maybe that is a solution, but I will first ask the expert @maleadt for help. 😄

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [February 8, 2021, 5:36am UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/4 "2021-02-08T05:36:32Z")

</div>

You might find the answer I gave [here](https://discourse.julialang.org/t/optimizing-cuda-jl-performance-for-small-array-operations/54808/4) helpful, although disclaimer I am not the expert 🙂 (and I mean that seriously, there might be better ways)

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 8, 2021, 7:49am UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/5 "2021-02-08T07:49:41Z")

</div>

We currently don’t support indirect device function calls, so you need something that’s fully analyzable by the Julia compiler. The solution by @marius311 will work, which basically boils down to using a tuple instead of an array and making sure Julia sees individual accesses of that tuple (instead of an iteration like your `for` loop):

```julia
julia> function pass_func_array(x::Float32, fs)
           x = fs[1](x)
           @cuprintln("x = $x")
           y = fs[2](x)
           @cuprintln("y = $y")
           nothing
       end
pass_func_array (generic function with 1 method)

julia> @cuda pass_func_array(1f0, (CUDA.sin, CUDA.cos));

x = 0.841471
y = 0.666367

```

---

<div class="post-metadata">

**Author:** ![Shuhua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shuhua/32/27618_2.png) [@Shuhua](https://discourse.julialang.org/u/Shuhua)\
**Post date:** [February 8, 2021, 10:14am UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/6 "2021-02-08T10:14:01Z")

</div>

Many thanks.

> [@maleadt](#):
>
> making sure Julia sees individual accesses of that tuple

Does it mean that only a constant index is allowed? The following example does not work either.

```julia
function pass_func_array2(x::Float32, order::CuArray, fs)
    for i in order
        x = fs[i](x)
        @cuprintln("x = $x")
    end
    nothing
end
@cuda pass_func_array2(3.14f0, CuArray([1, 2, 1]), (CUDA.sin, CUDA.cos))

```

The `order` above actually encodes an expression tree, where each integer represents an operation, and I try to _execute_ this tree on GPU. I don’t think it is proper to turn `order` into a tuple of functions directly since it may be very long (tens or hundreds). Is there any workaround to do what I want in CUDA.jl for now?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 8, 2021, 2:54pm UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/7 "2021-02-08T14:54:50Z")

</div>

> [@Shuhua](#):
>
> Is there any workaround to do what I want in CUDA.jl for now?

You could create a trampoline function, but generally no this pattern is not supported.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [February 8, 2021, 3:42pm UTC](https://discourse.julialang.org/t/cuda-kernel-how-to-pass-an-array-of-functions/54784/8 "2021-02-08T15:42:13Z")

</div>

Not sure if this is relevant, but recursion, `map` and `foldl` tend to produce better code than `for` loops for short tuples, so you may try things like:

```julia
foldl(|>, fs, init=x)

```

rather than

```julia
for f in fs
    x = f(x)
end

```

If `fs` is itself large, you may be better off replacing `fs[i](x)` with a unique function `f(x, i)` (if that’s possible in your use case).