I’m still learning about macros and only recently learnt about closures. I am trying to use the ArchGDAL package on my Apple Silicon Macbook, but it reported an error during precompilation: ERROR: LoadError: cfunction: closures are not supported on this platform
.
I have managed to isolate a MWE:
macro cplprogress(progressfunc)
@cfunction($(esc(progressfunc)), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
end
progressfunc = x -> x
@cplprogress(progressfunc)
This results in the following stacktrace:
ERROR: LoadError: cfunction: closures are not supported on this platform
Stacktrace:
[1] var"@cplprogress"(__source__::LineNumberNode, __module__::Module, progressfunc::Any)
@ Main ~/mwe.jl:2
in expression starting at /mwe.jl:7
in expression starting at /mwe.jl:7
From what I understand, the $
in the macro is not in fact indicating a closure, but is actually just interpolation for the macro?
Either way I would really appreciate some guidance to fixing it so the closure can be removed, or otherwise fixed!
Thanks in advance!
Julia: 1.7.1
OS: MacOS Monterey 12.1
CPU: M1 Pro
Arch: darwin-aarch64
The ArchGDAL.jl package is apparently using this for the GDALProgressFunc
callback API.
It seems like the solution is to change the Julia package to use the void*
argument (pProgressArg
) to pass the closure argument, rather than forcing Julia to compile a new callback for every progressfunc
argument it wants to handle (which apparently isn’t supported on M1). That is, they should define something like:
function progressfunc_wrapper(dfComplete::Cdouble, pszMessage::Ptr{UInt8}, pProgressArg::Any)::Cint
pszMessage == C_NULL && return pProgressArg(dfComplete)
return pProgressArg(dfComplete, unsafe_string(pszMessage))
end
const progressfunc_wrapper_c = @cfunction(progressfunc_wrapper, Cint, (Cdouble, Cstring, Any))
and then, any place you actually want to pass a progressfunc
, you instead pass progressfunc_wrapper_c
as the callback function and progressfunc
as the callback-data argument.
In APIs that support a callback-data argument, like this one, the above is the approach I would recommend. It puts a lot fewer demands on the compiler.
(Also, then their progresscallback
functions are simpler and more Julian. They are passed a Julia String
as their optional “message” argument rather than a pointer, and can return true
or false
instead of a Cint
, since progressfunc_wrapper
handles the conversions.)
2 Likes
Thanks @stevengj . A wrapper function sounds like a good idea, I’ll see if I can get something like that working then hopefully submit a PR.
Do you know if the $
in the original implementation is actually creating a closure though? Or is that just part of the macro syntax.
Cheers
Yes, because it’s splicing in a runtime closure from the call site.
Looking at how it’s used in ArchGDAL.jl, to be a drop-in replacement you might want a “dumber” progressfunc_wrapper
than I wrote above, which just calls:
return pProgressArg(dfComplete, pszMessage)
to keep their existing progressfunc
API. Then, when passing the callback to a GDAL
function like is done here, you would pass progressfunc_wrapper_c
instead of @cplprogress(progressfunc)
, and pass (complete, msg) -> progressfunc(complete, msg, progressdata)
instead of progressdata
. (The @cplprogress
macro can be entirely deleted.)
In the longer run, I would re-think the progressfunc
Julia API to make it more Julian (no raw-pointer or data arguments), but in the short run this will let you make a PR that keeps everything in the API the same and only improves the implementation.
1 Like
Your MWE is actually a fairly weird looking macro because it’s invoking @cfunction
as part of the macro expansion. So the esc
here doesn’t really make sense.
Was this unintentional, and did you mean to quote the @cfunction
syntax?
macro cplprogress(progressfunc)
quote
@cfunction($(esc(progressfunc)), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
end
end
This example was just taken verbatim from the ArchGDAL
package. I tried it with the quote structure but got a invalid syntax error.
This seems like a bug in Base
, in particular in the internals of macroexpand
which doesn’t understand Expr(:cfunction)
. The following should also work, but doesn’t.
macro cplprogress(progressfunc)
quote
f = $(esc(progressfunc))
@cfunction(f, Cint, (Cdouble, Cstring, Ptr{Cvoid}))
end
end
Actually, no, I take that back. The problem here is that writing a macro to wrap a macro which recognizes $
syntax is quite subtle. This is because normal syntax interpolation will see and remove the $
before the inner quoted macro (the @cfunction
in this case) gets to do so. I think the correct macro would be something like:
macro cplprogress(progressfunc)
quote
@cfunction($(Expr(:$, esc(progressfunc))), Cint, (Cdouble, Cstring, Ptr{Cvoid}))
end
end
Then we’d have:
julia> @macroexpand @cplprogress x->x
quote
#= REPL[41]:3 =#
$(Expr(:cfunction, Base.CFunction, :(x->begin
#= REPL[43]:1 =#
x
end), :(Main.Cint), :(Core.svec(Main.Cdouble, Main.Cstring, Main.Ptr{Main.Cvoid})), :(:ccall)))
end
In any case, yes, the original macro is using @cfunction
outside a quote block, but with a $
. So this is using the closure syntax.