I have a function which loads some CSVs, and then constructs a large Struct, sometimes these CSVs contain information as Strings when they should be numerical. Currently When initialising a struct I get the unhelpful error message:
ERROR: MethodError: Cannot `convert` an object of type InlineStrings.String31 to an object of type Int64
Is it possible for Julia to tell me which variable is of an invalid type? For example
struct Foo
a::Int64
b::Int64
c::Int64
d::Int64
end
# all good
Foo(1,2,3,4)
Instead of
Foo(1,2,3, "a")
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64
Closest candidates are:
convert(::Type{T}, ::Ptr) where T<:Integer at ~/.julia/juliaup/julia-1.7.2+0~x64/share/julia/base/pointer.jl:23
convert(::Type{IT}, ::GeometryBasics.OffsetInteger) where IT<:Integer at ~/.julia/packages/GeometryBasics/3PqdK/src/offsetintegers.jl:40
convert(::Type{T}, ::Base.TwicePrecision) where T<:Number at ~/.julia/juliaup/julia-1.7.2+0~x64/share/julia/base/twiceprecision.jl:262
...
Stacktrace:
[1] Foo(a::Int64, b::Int64, c::Int64, d::String)
@ Main ./REPL[20]:2
[2] top-level scope
@ REPL[22]:1
Can I get something like this
# Invalid string
Foo(1,2,3, "a")
ERROR: "a" was provided for Foo.d, this cannot be converted to Int64
I’m not sure if there is any package for that (seems something nice to have). But you could do it by hand using this:
julia> struct Foo
a::Int
b::Int
function Foo(a,b)
@assert a isa Int "a is not an integer!"
@assert b isa Int "b is not an integer!"
new(a,b)
end
end
julia> Foo(1,2)
Foo(1, 2)
julia> Foo(1,"a")
ERROR: AssertionError: b is not an integer!
Stacktrace:
[1] Foo(a::Int64, b::String)
@ Main ./REPL[1]:6
[2] top-level scope
@ REPL[3]:1
But you do already get that information, as part of the stack trace?
This already says that you tried to call Foo with three Int and one String argument. It’s just that the MethodError can’t see that stacktrace, since the method that fails is when it tries to convert the String to the Int.
You can make it throw the MethodError by specifying the types in the constructor (no need for @assert, which is not guaranteed to actually be used and shouldn’t be relied upon for correctness of an algorithm, as documented):
though that will prevent something like Foo(0x0, 0x1, 0x2, 0x3) (i.e. passing all UInt8) as well, since now the default constructor that calls convert is overwritten.
If the purpose of the error message is to be clearer, particularly for users, I think that providing a better error message is nice anyway, these stack traces can be hard to follow.
Given the warning about the misuse of @assert there, you can use:
julia> struct Foo
a::Int
b::Int
function Foo(a,b)
a isa Int || error("a is not an integer!")
b isa Int || error("b is not an integer!")
new(a,b)
end
end
julia> Foo(1,"a")
ERROR: b is not an integer!
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] Foo(a::Int64, b::String)
@ Main ./REPL[1]:6
[3] top-level scope
@ REPL[2]:1
I’d check for a isa Integer, since new will still try to convert(Int, a). Additionally, ArgumentError is a little cleaner still than the generic error function and throwing explicitly also doesn’t clobber the stacktrace:
julia> struct Foo
a::Int64
b::Int64
function Foo(a,b)
a isa Integer || throw(ArgumentError("`a` is not an Integer!"))
b isa Integer || throw(ArgumentError("`b` is not an Integer!"))
new(a,b)
end
end
julia> Foo(0x0,"a")
ERROR: ArgumentError: `b` is not an Integer!
Stacktrace:
[1] Foo(a::UInt8, b::String)
@ Main ./REPL[2]:6
[2] top-level scope
@ REPL[3]:1
julia> Foo(0x0,0x1)
Foo(0, 1)
I’m still in the camp that reading a stacktrace & getting to grasps with what a MethodError means is a necessary skill when debugging julia, but yeah, giving a better error message is definitely nice.
julia> using ex
julia> read_files(true)
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Float64
Closest candidates are:
convert(::Type{T}, ::T) where T<:Number at ~/.julia/juliaup/julia-1.7.2+0~x64/share/julia/base/number.jl:6
convert(::Type{T}, ::Number) where T<:Number at ~/.julia/juliaup/julia-1.7.2+0~x64/share/julia/base/number.jl:7
convert(::Type{T}, ::Base.TwicePrecision) where T<:Number at ~/.julia/juliaup/julia-1.7.2+0~x64/share/julia/base/twiceprecision.jl:262
...
Stacktrace:
[1] setindex!
@ ./array.jl:905 [inlined]
[2] read_files(fail::Bool)
@ ex ~/expack/ex/src/loader.jl:6
[3] top-level scope
@ REPL[2]:1
The struct causing the problem is till Foo, but Foo isn’t in the stack trace
Thank you for your solution(s), the major downside of all the error variations is that they require code duplication. I’ve also just checked on the more complex minimal working example and this approach still doesn’t identify which argument is causing the problem, see the previously linked repo.
I didn’t run the test, but at least you missed the new(a,b,c,d) here:
using Parameters
using SparseArrays
@with_kw mutable struct Foo
a::Int64
b::Int64
c::Int64
d::SparseArrays.SparseMatrixCSC{Int64,Int64}
function Foo(a, b, c, d)
d isa SparseArrays.SparseMatrixCSC{Int64,Int64} || throw(ArgumentError("`b` is not an Integer!"))
new(a,b,c,d) # <<<< here
end
end
Apologies for the consistent mistakes. Reverting to the original question stacks are great, but for struts with a lot of fields and complex data types it is not trivial deciphering which is the problematic argument. When using something like pydantic you are notified of which arguments are problematic which is a huge help when debugging. Following from your responses it seems the only way to get this functionality is with a constructor which tests each arguments type, unfortunately this produces a lot of code repetition.
That clearly the place of a small macro/package. An experienced macro programmer probably can get that done in a few minutes. That would be useful, IMO.
edit: actually you don’t need a macro, but this probably doesn not play well with Parameters:
julia> struct Foo
a::Int
b::Float64
function Foo(args...)
names = collect(fieldnames(Foo))
types = collect(fieldtypes(Foo))
i = 0
for arg in args
i += 1
if typeof(arg) != types[i]
error("$(names[i]) is not of type $(types[i])")
end
end
new(args...)
end
end
julia> Foo(1,2.0)
Foo(1, 2.0)
julia> Foo(1,"a")
ERROR: b is not of type Float64
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] Foo(::Int64, ::Vararg{Any})
@ Main ./REPL[5]:11
[3] top-level scope
@ REPL[7]:1
As mentioned above, this case is exactly what MethodError is for (worse: it creates a method that pretends to work as usual, but then behaves unpredictably & breaks with the rest of the julia ecosystem. Should there be better error messages for MethodError in the future, your code won’t benefit from that). Everything here works as designed and the code proposed above just poorly emulates that behavior because it skips the possible convert entirely. You’d have to limit your arguments to ones that are convertible, which you as the author may not even know about.
Yes it may be harder to follow in some cases, but dealing with this problem generically is much harder and has more edge cases than you may think of initially.
I think this can be thought as a more pedestrian and simple problem. I don’t see how adding a type check for a custom structure can “behave unpredictably and break the julia ecosystem”.
Clearly I’m not claiming that I will add PR anywhere with that solution or any other, but I think something like that is perfectly fine for helping users of a specific package to deal with the interface that it exposes for data input.
Sorry, I may have worded this weirdly - I am not saying that it breaks the ecosystem as a whole, it breaks with the behavior of the rest of the ecosystem, in that it doesn’t throw a MethodError. More people adopting this hand-rolled approach means a lot of different error messages & implementations of dealing with this, which could be solved just as well by simply annotating the types in the constructor (which would result in the MethodError again). It’s core to how julia works, so hiding it/trying to get away from it would imo create more problems & confusion than it may initially solve, once users start to use other packages that just don’t use this same approach (a package doesn’t live in isolation after all).
Anecdotal evidence: I never had a problem with people using ArgumentError, but I (and other people that I tried to help on discourse) had problems with a more generic function/constructor existing but a more specific signature throwing a MethodError and everyone is confused and running in circles because “a MethodError could not have been thrown, there is at least one method that matches the arguments that is this generic one”.
But yes, I believe the most generic exception type should not be thrown, ArgumentError should be used instead.
Ah, I also recommend looking at the Base.Experimental.register_error_hint, which would probably be “the right” way to deal with the problem if it was not Experimental.
Apologies for the consistent mistakes. Reverting to the original question stacks are great, but for struts with a lot of fields and complex data types it is not trivial deciphering which is the problematic argument. When using something like