# Interpolation using $ with @cfunction in a macro reported as a closure, thus won't work on Apple Silicon

**URL:** <https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984>\
**Category:** General Usage\
**Tags:** question\
**Created:** [February 8, 2022, 1:12am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984 "2022-02-08T01:12:43Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![jackbeagley](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jackbeagley/32/30490_2.png) [@jackbeagley](https://discourse.julialang.org/u/jackbeagley)\
**Post date:** [February 8, 2022, 1:12am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/1 "2022-02-08T01:12:43Z")

</div>

I’m still learning about macros and only recently learnt about closures. I am trying to use the ArchGDAL package on my Apple Silicon Macbook, but it reported an error during precompilation: `ERROR: LoadError: cfunction: closures are not supported on this platform`.

I have managed to isolate a MWE:

```julia
macro cplprogress(progressfunc)
    @cfunction($(esc(progressfunc)), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
end

progressfunc = x -> x

@cplprogress(progressfunc)

```

This results in the following stacktrace:

```julia
ERROR: LoadError: cfunction: closures are not supported on this platform
Stacktrace:
 [1] var"@cplprogress"( __source__ ::LineNumberNode, __module__ ::Module, progressfunc::Any)
   @ Main ~/mwe.jl:2
in expression starting at /mwe.jl:7
in expression starting at /mwe.jl:7

```

From what I understand, the `$` in the macro is not in fact indicating a closure, but is actually just interpolation for the macro?  
Either way I would really appreciate some guidance to fixing it so the closure can be removed, or otherwise fixed!

Thanks in advance!

Julia: 1.7.1  
OS: MacOS Monterey 12.1  
CPU: M1 Pro  
Arch: darwin-aarch64

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 8, 2022, 1:58am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/2 "2022-02-08T01:58:11Z")

</div>

The ArchGDAL.jl package is apparently using this for the [`GDALProgressFunc`](https://gdal.org/api/cpl.html#_CPPv416GDALProgressFunc) callback API.

It seems like the solution is to change the Julia package to use the `void*` argument (`pProgressArg`) to pass the closure argument, rather than [forcing Julia to compile a new callback](https://julialang.org/blog/2013/05/callback/#the_problem_with_closures) for every `progressfunc` argument it wants to handle (which apparently isn’t supported on M1). That is, they should define something like:

```julia
function progressfunc_wrapper(dfComplete::Cdouble, pszMessage::Ptr{UInt8}, pProgressArg::Any)::Cint
    pszMessage == C_NULL && return pProgressArg(dfComplete)
    return pProgressArg(dfComplete, unsafe_string(pszMessage))
end
const progressfunc_wrapper_c = @cfunction(progressfunc_wrapper, Cint, (Cdouble, Cstring, Any))

```

and then, any place you actually want to pass a `progressfunc`, you instead pass `progressfunc_wrapper_c` as the callback function and `progressfunc` as the callback-data argument.

In APIs that support a callback-data argument, like this one, the above is the [approach I would recommend](https://julialang.org/blog/2013/05/callback/#passing_closures_via_pass-through_pointers). It puts a lot fewer demands on the compiler.

(Also, then their `progresscallback` functions are simpler and more Julian. They are passed a Julia `String` as their optional “message” argument rather than a pointer, and can return `true` or `false` instead of a `Cint`, since `progressfunc_wrapper` handles the conversions.)

---

<div class="post-metadata">

**Author:** ![jackbeagley](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jackbeagley/32/30490_2.png) [@jackbeagley](https://discourse.julialang.org/u/jackbeagley)\
**Post date:** [February 8, 2022, 2:07am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/3 "2022-02-08T02:07:11Z")

</div>

Thanks @stevengj . A wrapper function sounds like a good idea, I’ll see if I can get something like that working then hopefully submit a PR.

Do you know if the `$` in the original implementation is actually creating a closure though? Or is that just part of the macro syntax.

Cheers

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [February 8, 2022, 2:26am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/4 "2022-02-08T02:26:12Z")

</div>

> [@jackbeagley](#):
>
> Do you know if the `$` in the original implementation is actually creating a closure though? Or is that just part of the macro syntax.

Yes, because it’s splicing in a runtime closure [from the call site](https://github.com/yeesian/ArchGDAL.jl/blob/17b7e7d1200a6aead977a0e74ebd6e70d2283111/src/dataset.jl#L34).

Looking at how it’s used in ArchGDAL.jl, to be a drop-in replacement you might want a “dumber” `progressfunc_wrapper` than I wrote above, which just calls:

```julia
return pProgressArg(dfComplete, pszMessage)

```

to keep their existing `progressfunc` API. Then, when passing the callback to a `GDAL` function [like is done here](https://github.com/yeesian/ArchGDAL.jl/blob/17b7e7d1200a6aead977a0e74ebd6e70d2283111/src/dataset.jl#L106-L107), you would pass `progressfunc_wrapper_c` instead of `@cplprogress(progressfunc)`, and pass `(complete, msg) -> progressfunc(complete, msg, progressdata)` instead of `progressdata`. (The `@cplprogress` macro can be entirely deleted.)

In the longer run, I would re-think the `progressfunc` Julia API to make it more Julian (no raw-pointer or data arguments), but in the short run this will let you make a PR that keeps everything in the API the same and only improves the implementation.

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 8, 2022, 2:28am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/5 "2022-02-08T02:28:29Z")

</div>

> [@jackbeagley](#):
>
> Do you know if the `$` in the original implementation is actually creating a closure though? Or is that just part of the macro syntax.

Your MWE is actually a fairly weird looking macro because it’s invoking `@cfunction` as part of the macro expansion. So the `esc` here doesn’t really make sense.

Was this unintentional, and did you mean to quote the `@cfunction` syntax?

```julia
macro cplprogress(progressfunc)
    quote
        @cfunction($(esc(progressfunc)), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
    end
end

```

---

<div class="post-metadata">

**Author:** ![jackbeagley](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jackbeagley/32/30490_2.png) [@jackbeagley](https://discourse.julialang.org/u/jackbeagley)\
**Post date:** [February 8, 2022, 2:47am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/6 "2022-02-08T02:47:49Z")

</div>

This example was just taken verbatim from the `ArchGDAL` package. I tried it with the quote structure but got a invalid syntax error.

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 8, 2022, 4:13am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/7 "2022-02-08T04:13:25Z")

</div>

> [@jackbeagley](#):
>
> I tried it with the quote structure but got a invalid syntax error.

~~This seems like a bug in `Base`, in particular in the internals of `macroexpand` which doesn’t understand `Expr(:cfunction)`. The following should also work, but doesn’t.~~

```julia
macro cplprogress(progressfunc)
    quote
        f = $(esc(progressfunc))
        @cfunction(f, Cint, (Cdouble, Cstring, Ptr{Cvoid}))
    end
end

```

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 8, 2022, 4:36am UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/8 "2022-02-08T04:36:11Z")

</div>

> [@c42f](#):
>
> This seems like a bug in `Base`

Actually, no, I take that back. The problem here is that writing a macro to wrap a macro which recognizes `$` syntax is quite subtle. This is because normal syntax interpolation will see and remove the `$` before the inner quoted macro (the `@cfunction` in this case) gets to do so. I think the correct macro would be something like:

```julia
macro cplprogress(progressfunc)
    quote
        @cfunction($(Expr(:$, esc(progressfunc))), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
    end
end

```

Then we’d have:

```julia
julia> @macroexpand @cplprogress x->x
quote
    #= REPL[41]:3 =#
    $(Expr(:cfunction, Base.CFunction, :(x->begin
          #= REPL[43]:1 =#
          x
      end), :(Main.Cint), :(Core.svec(Main.Cdouble, Main.Cstring, Main.Ptr{Main.Cvoid})), :(:ccall)))
end

```

In any case, yes, the original macro is using `@cfunction` _outside_ a quote block, but _with_ a `$`. So this is using the closure syntax.

---

<div class="post-metadata">

**Author:** ![evetion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evetion/32/22679_2.png) [@evetion](https://discourse.julialang.org/u/evetion)\
**Post date:** [June 9, 2022, 8:44pm UTC](https://discourse.julialang.org/t/interpolation-using-with-cfunction-in-a-macro-reported-as-a-closure-thus-wont-work-on-apple-silicon/75984/9 "2022-06-09T20:44:08Z")

</div>

Sorry for bumping, just wanting to point out that we simply don’t support a custom callback function on Apple Silicon right now in ArchGDAL, which makes it work again. This fix is in v0.8.5.