# Cfunctions with local types on v0.7

**URL:** https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343
**Category:** General Usage
**Tags:** ffi
**Created:** [July 12, 2018, 8:51pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343 "2018-07-12T20:51:49Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 12, 2018, 8:51pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/1 "2018-07-12T20:51:49Z")

</div>

I am trying to get `cfunction` to work with local functions and types

[https://github.com/JuliaDiffEq/Sundials.jl/commit/54e7e18788fed1b10c2bb69fcc375f45f0de0177](https://github.com/JuliaDiffEq/Sundials.jl/commit/54e7e18788fed1b10c2bb69fcc375f45f0de0177)

I think I am having some GC segfaults because of this. @rdeits said he was playing around with this and functions can be interpolated into the macro:

```julia
julia> function local_cfunction(f)
         @cfunction($f, Float64, (Int,))
       end
local_cfunction (generic function with 1 method)

julia> local_cfunction(x -> x + 1.0)
Base.CFunction(Ptr{Nothing} @0x00007f41f46d9f80, getfield(Main, Symbol("##3#4"))(), Ptr{Nothing} @0x0000000000000000, Ptr{Nothing} @0x0000000000000000)

```

but the types cannot, and so those need to be handled via an `@generated` function:

```julia
julia> @generated function local_cfunction(f, ret, arg)
         quote
           @cfunction($(Expr(:$, :f)), $ret, ($arg,))
         end
       end
local_cfunction (generic function with 2 methods)

julia> local_cfunction(x -> x + 1.0, Float64, Int)
Base.CFunction(Ptr{Nothing} @0x00007f41df522f40, getfield(Main, Symbol("##6#7"))(), Ptr{Nothing} @0x0000000000000000, Ptr{Nothing} @0x0000000000000000)

```

It seems like that is a quite complicated solution though. Is there something simpler or is this `@generated` helper function required everywhere?

---

<div class="post-metadata">

### Author: ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)
#### Post date: [July 12, 2018, 8:54pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/2 "2018-07-12T20:54:25Z")

</div>

@tkoolen has also been taking a crack at this, x-ref: [Fixes for Julia 0.7. Drop 0.5 and 0.6. by tkoolen · Pull Request #7 · yuyichao/FunctionWrappers.jl · GitHub](https://github.com/yuyichao/FunctionWrappers.jl/pull/7)

---

<div class="post-metadata">

### Author: ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)
#### Post date: [July 12, 2018, 8:57pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/3 "2018-07-12T20:57:20Z")

</div>

> [@ChrisRackauckas](#):
>
> It seems like that is a quite complicated solution though. Is there something simpler or is this `@generated` helper function required everywhere?

I don’t quite understand the need for runtime local types. The only point of a `@cfunction` is to use it as a callback with an external C library, in which case the set of allowed argument types should be known statically.

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 12, 2018, 9:01pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/4 "2018-07-12T21:01:32Z")

</div>

> [@stevengj](#):
>
> The only point of a `@cfunction` is to use it as a callback with an external C library, in which case the set of allowed argument types should be known statically.

They are known statically. These types can be inferred by the compiler since everything is type-stable, and so the arguments are defined as things like `typeof(t)` or `Ref{typeof(userfun)})` since it’s a generic implementation. I don’t know what’s different between v0.6 `cfunction` and v0.7 `@cfunction` which makes this incompatible, and this incompatibility doesn’t seem to be mentioned in the NEWS or depwarn.

I can work around it in this specific case though.

---

<div class="post-metadata">

### Author: ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)
#### Post date: [July 12, 2018, 9:06pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/5 "2018-07-12T21:06:47Z")

</div>

> [@stevengj](#):
>
> The only point of a `@cfunction` is to use it as a callback with an external C library, in which case the set of allowed argument types should be known statically.

Yes, but knowing statically does not mean knowing locally. The usecase is writing helper functions that requires `cfunction` and the issue is that not allowing local type breaks the transparency wrapper functions.

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 13, 2018, 3:28am UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/6 "2018-07-13T03:28:44Z")

</div>

Notice that the generated function should be

```julia
@generated function local_cfunction(f, ret, arg)
   quote
     @cfunction($(Expr(:$, :f)), $ret, ($arg,)).ptr
   end
 end

```

i.e. it should return the `.ptr` to match v0.6

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 13, 2018, 5:56am UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/7 "2018-07-13T05:56:46Z")

</div>

I tried a bunch of different combinations:

```julia
# Good but depwarns
CVodeInit(mem, cfunction(cvodefun, Cint, (realtype, N_Vector, N_Vector, Ref{typeof(userfun)})), t[1], convert(N_Vector, y0nv))

# Segfaults
CVodeInit(mem, local_cfunction(cvodefun, Cint, (realtype, N_Vector, N_Vector, Ref{typeof(userfun)})), t[1], convert(N_Vector, y0nv))

# Doesn't compile because of local type
CVodeInit(mem, @cfunction(cvodefun, Cint, (realtype, N_Vector, N_Vector, Ref{typeof(userfun)})), t[1], convert(N_Vector, y0nv))

# Errors due to local type
funtype = Ref{typeof(userfun)}
CVodeInit(mem, @cfunction(cvodefun, Cint, (realtype, N_Vector, N_Vector, funtype)), t[1], convert(N_Vector, y0nv))

# Depwarn
funtype = Ref{typeof(userfun)}
CVodeInit(mem, cfunction(cvodefun, Cint, (realtype, N_Vector, N_Vector, funtype)), t[1], convert(N_Vector, y0nv))

```

So what exactly is the deprecation turning the `cfunction` call into? It’s not the same as the macro since the depwarn route works while the macro fails to compile, while the `local_cfunction` route causes segfaults. If I can just match what the deprecation route is doing then I’d be set.

---

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [July 13, 2018, 6:03am UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/8 "2018-07-13T06:03:09Z")

</div>

Following the depwarn logic, it seems that the way to handle this is:

```julia
 @noinline function old_cfunction(f, r, a)
   ccall(:jl_function_ptr, Ptr{Cvoid}, (Any, Any, Any), f, r, a)
 end
CVodeInit(mem, old_cfunction(cvodefun, Cint, Tuple{realtype, N_Vector, N_Vector, funtype}), t[1], convert(N_Vector, y0nv))

```

The `Tuple{...}` is from [https://github.com/JuliaLang/julia/pull/23066](https://github.com/JuliaLang/julia/pull/23066) which shows that the previous way to doing the types is a performance trap. But done like this the `old_cfunction` should be fine and doesn’t throw any depwarns, so that’s at least a viable solution. This really feels like a big regression from v0.6 though.

---

<div class="post-metadata">

### Author: ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)
#### Post date: [July 13, 2018, 7:07am UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/9 "2018-07-13T07:07:31Z")

</div>

> [@ChrisRackauckas](#):
>
> it should return the `.ptr` to match v0.6

I don’t think that’s right. The `CFunction` type is meant to keep all the necessary things from being garbage collected, so by just returning the `ptr` member and letting everything else go out of scope you’re circumventing that mechanism and running into segfaults. The `CFunction` docs (on master) state that

> it should be passed to `ccall` as a `Ptr{Cvoid}`, and will be converted automatically at  
> the call site to the appropriate type.

I wonder if this could be made safer by overloading `Base.getproperty` for `CFunction` to throw an error for naive field accesses (and then provide e.g. `.ptr_unsafe` as a way for internal use).

---

<div class="post-metadata">

### Author: ![jameson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jameson/32/23_2.png) [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [July 13, 2018, 9:36pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/10 "2018-07-13T21:36:01Z")

</div>

> [@ChrisRackauckas](#):
>
> These types can be inferred by the compiler since everything is type-stable, and so the arguments are defined as things like `typeof(t)` or `Ref{typeof(userfun)})` since it’s a generic implementation

This should be written as `(x::T) where {T} -> cfunction(f, Ref{T}, (Ref{T},))`. This ensures that lowering can see that everything is type-stable. (Basically just the same rules as `ccall`)

---

<div class="post-metadata">

### Author: ![jameson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jameson/32/23_2.png) [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [July 13, 2018, 9:41pm UTC](https://discourse.julialang.org/t/cfunctions-with-local-types-on-v0-7/12343/11 "2018-07-13T21:41:14Z")

</div>

Note that if you’re C library provide the ability to thread though a `void*` environment pointer (as all good libraries should, and as it looks like this one does), you should typically use that instead of allocating a new cfunction pointer object. The `qsort` example in the manual goes into more detail about how to do this.
