# The following works fine:
mt = MyType{Int32,CTable(16)}()
# Define convenience ctor:
MyType{T}() where {T} = MyType{T,CTable(16)}()
# This throws a weird error:
MType{Int32}()
I maintain a Julia interface to a C library, and am working on some new features.
Here is a minimal working example of the problem I’ve encountered:
struct CTable
t::Ptr{Float64}
function CTable(n)
# In practice t is received from the C program
t = @ccall malloc(n::Csize_t)::Ptr{Float64}
return new(t)
end
end
mutable struct MyType{T,Table}
arr::Ptr{T}
function MyType{T,Table}() where {T,Table}
arr = @ccall jl_malloc(10::Csize_t)::Ptr{T}
m = new{T,Table}(arr)
f(m) = @ccall jl_free(m.arr::Ptr{Cvoid})::Cvoid
finalizer(f, m)
return m
end
end
# The following works fine:
mt = MyType{Int32,CTable(16)}()
# However this convenience constructor breaks the code:
MyType{T}() where {T} = MyType{T,CTable(16)}()
MyType{Int32}() # Error!!!
#=
ERROR: ccall return type Ptr should have an element type (not Ptr{<:T})
Stacktrace:
[1] MyType
@ ./REPL[2]:5 [inlined]
[2] (MyType{Int32})()
@ Main ./REPL[4]:1
[3] top-level scope
@ REPL[6]:1
=#
The C library basically creates one look-up table (let’s call this CTable), which is then referenced by many instances of another type (let’s call this MyType) throughout execution. One nice feature of this library though is that one can create a second CTable in the same execution with different features, and perform operations with MyType which point to this second table. However, MyTypes pointing to the first table must not operate with MyTypes pointing to a second table. For many more complicated reasons too, it is extremely useful to store the CTable as a type parameter of MyType. However creating another convenience constructor with a default CTable causes this (compiler?) bug.
Thanks for the response! While this does work, it is not the use case I am going for, and still doesn’t explain if this is truly a compiler bug (which it certainly appears to be)
The use case I am going for is like
julia> default::CTable = CTable(16)
CTable(Ptr{Float64} @0x000000002b47a8f0)
julia> MyType{T}() where {T} = MyType{T,default}()
julia> MyType{Int32}()
ERROR: ccall return type Ptr should have an element type (not Ptr{<:T})
Stacktrace:
[1] MyType
@ ./REPL[3]:5 [inlined]
[2] (MyType{Int32})()
@ Main ./REPL[10]:1
[3] top-level scope
@ REPL[11]:1
Where the default is a typed global that can be switched.
Note that if I change first @ccall of the inner constructor of MyType to return a type Ptr{Cvoid} and then unsafe_convert to Ptr{T}, it seems to work:
mutable struct MyType{T,Table}
arr::Ptr{T}
function MyType{T,Table}() where {T,Table}
# remove Ptr{T} in the @ccall and then unsafe_convert
arr = Base.unsafe_convert(Ptr{T}, @ccall jl_malloc(10::Csize_t)::Ptr{Cvoid})
m = new{T,Table}(arr)
f(m) = @ccall jl_free(m.arr::Ptr{Cvoid})::Cvoid
finalizer(f, m)
return m
end
end
MyType{T}() where {T} = MyType{T,default}()
MyType{Int32}() # works now:
#= Output:
MyType{Int32, CTable(Ptr{Float64} @0x000000002b47a8f0)}(Ptr{Int32} @0x000000002ffee460)
=#
This suggests to me one of two things
I am doing something “undefined” (in which case this should not be allowed or should be very well documented somewhere…)
Compiler bug
I’d like to know which before proceeding with any solution of course
I can also make a completely separate function, and it works:
mutable struct MyType{T,Table}
arr::Ptr{T}
function MyType{T,Table}() where {T,Table}
arr = @ccall jl_malloc(10::Csize_t)::Ptr{T}
m = new{T,Table}(arr)
f(m) = @ccall jl_free(m.arr::Ptr{Cvoid})::Cvoid
finalizer(f, m)
return m
end
function MyType{T}() where {T}
arr = @ccall jl_malloc(10::Csize_t)::Ptr{T}
m = new{T,default}(arr) # get global default here
f(m) = @ccall jl_free(m.arr::Ptr{Cvoid})::Cvoid
finalizer(f, m)
return m
end
end
MyType{Int32}() # works now
I think it’s just complaining about the unbounded type variable that’s being passed to ccall, i.e. the return type declaration as Ptr{T} instead of something fully typed. That would be consistent with it being impossible in julia to instantiate abstract types, i.e. I think you can’t unsafe_load from a Ptr{Integer} because instances are always of a concrete type.
I’m not 100% sure on this though, and I’d have to check the implementation that throws the error to be sure. The error message at least seems unique enough to make this findable.
T here though is a concrete type (Int32), and it works if I call the constructor directly instead of the convenience constructor. So I’m not sure that’s the problem, even though that is what the error message makes it sound like
No, T is T, though it would (should) be inferred as Int32. The error originates from here:
i.e. for some reason ccall does get this as non-Int32. My (unconfirmed) guess is that this might be a case of where julia avoids specialization, resulting in T === Any, which is no longer concrete. I guess this could be confirmed by compiling a custom build of Julia and logging the type that is actually encountered there. Or debugging with gdb, whichever is more convenient.
Ok, one quick look with Cthulhu.jl later and it seems like the return type passed to ccall is taken as a literal, i.e. does not participate in the usual TypeVar resolution?
This would be consistent (if mildly annoying) with the documentation of the Expr(:foreigncall):
args[2]::Type : RT
The (literal) return type, computed statically when the containing method was defined.
Type inference definitely infers the type of the result correctly, since it infers arr as Ptr{Int32} correctly, so this might just be a missed bit in the compiler. People don’t usually go around having inferred ccalls, since one C function can generally only have one return type.
Interesting! Thanks for looking at this, I would have never been able to figure this out on my own.
Weirdly enough though, I had been doing this already with no issues (see TPS type here. I only started having this problem once I added a second parametric type representing the CTable in this MWE (current dev branch here )
So it seems to me that this is OK behavior, but possibly a compiler bug. If that interpretation sounds correct, then I will use one of the workarounds above and submit an issue to julialang?