Stuck on a NamedTuple problem

General Usage
1

I think a NamedTuple can do what I want but I can’t quite figure out the syntax. Can someone help me with this bit of runnable code?

function nameparts(v)
    t = split(v,':')
    length(t) < 2 ? (nothing,t[1]) : (t[1],t[2])
end

struct QName{T<:AbstractString}
    pfx::Union{T,Nothing}
    nm::T
end
QName(nm::AbstractString) = QName(nameparts(nm)...)

struct ID{T<:AbstractString}
    id::T
end

attr = Dict("base"=>"xs:string")

atts = @NamedTuple{base::QName, id::Union{Nothing,ID}}

s = string.(fieldnames(atts))
vals = get.(Ref(attr), s, nothing) #vals will be strings or nothing
@show vals

# here I want to instantiate the types
# to create a result tuple that looks like
# `(QName{SubString{String}}("xs", "string"), nothing)`

maybe = atts(vals)

But it bombs with

ERROR: MethodError: Cannot convert an object of type
String to an object of type
QName
Closest candidates are:
convert(::Type{T}, ::T) where T at essentials.jl:171
QName(::AbstractString) at /Users/doug/dev/XSDJ/src/ex.jl:10
QName(::Union{Nothing, T}, ::T) where T<:AbstractString at /Users/doug/dev/XSDJ/src/ex.jl:7
Stacktrace:
[1] convert(::Type{Tuple{QName,Union{Nothing, ID}}}, ::Tuple{String,Nothing}) at ./essentials.jl:310
[2] Tuple{QName,Union{Nothing, ID}}(::Tuple{String,Nothing}) at ./tuple.jl:225
[3] NamedTuple{(:base, :id),Tuple{QName,Union{Nothing, ID}}}(::Tuple{String,Nothing}) at ./namedtuple.jl:90
[4] top-level scope at /Users/doug/dev/XSDJ/src/ex.jl:28
[5] include(::String) at ./client.jl:457
[6] top-level scope at REPL[13]:1

Do I need to implement a convert method?

2

For those of you who stumble on this post in the dying days of the solar system, the answer to my question is “Yes, I do.” Adding this solved the problem:

import Base: convert
Base.convert(::Type{QName}, s::String) = QName(s)

Cheers!

POST POST EDIT: It has been rightly pointed out below that my reckless use of convert amounts to type piracy. So don’t do this! Instead, see the rest of the thread for better solutions!

4 Likes
3

I usually take an error like

MethodError: Cannot `convert`...

as a signal that I’m doing something wrong, not as a signal that I need to add a convert method. If you can describe to us what you’re generally trying to accomplish, we might be able to recommend a different approach.

FWIW, in all the Julia code that I’ve written, I don’t think I’ve ever implemented a convert method.

2 Likes
4

While you can implement a convert method, I think it is better style to separate representation and an API entry point. Depending on the context (which we do not have), you may want to define something like a

process_attr(attr) = (QName = QName(first(attr)), ID = ...)

that does the conversion explicitly. That way, if the representation changes, and you want something else than NamedTuples (eg a struct), the callers won’t need to change anything.

1 Like
5

@Tamas_Papp Thank you for weighing in.

Here is some context, at length: I am building an XSD parser in Julia. The parser needs to know the set of all possible attribute names, with their types, that an element definition may legally contain according to the specification. A NamedTuple would fit the bill exactly:

legalatts = @NamedTuple{base::QName, id::Union{Nothing,ID}} # base must be present as a QName; id is optional.

When an actual XSD document is being parsed, the local attributes for an element are available from the StreamReader as a dictionary:

attr = Dict("base"=>"xs:string")

I want use the name from the legalatt NamedTuple to look up the attribute value in attr, returning a string value or nothing.

s = string.(fieldnames(legalatts))
vals = get.(Ref(attr), s, nothing) #vals will be strings or nothing

(“xs:string”, nothing)

Finally I want to instantiate each of the types contained in the Named Tuple for which a matching string value is present. So both QName and ID need to be creatable from a string. NamedTuples can be invoked by giving them a tuple, so I thought this would work:

legalatts(vals)

And it does work (for the above example) as long as I have a convert method for QName. But now the same code is failing when it tries to convert a String to a UInt64::

attr = Dict("base"=>"xs:string", **"value"=>"1"**)

atts = @NamedTuple{base::QName, id::Union{Nothing,ID}, **value::UInt**}

s = string.(fieldnames(atts))

vals = get.(Ref(attr), s, nothing) #vals will be strings or nothing
@show atts vals

atts = NamedTuple{(:base, :id, :value),Tuple{QName,Union{Nothing, ID},UInt64}}
vals = (“xs:string”, nothing, “1”)

But this blows up:

v = atts(vals)

ERROR: MethodError: Cannot convert an object of type String to an object of type UInt64
Closest candidates are:
convert(::Type{T}, ::T) where T<:Number at number.jl:6
convert(::Type{T}, ::Number) where T<:Number at number.jl:7
convert(::Type{T}, ::Ptr) where T<:Integer at pointer.jl:23

I realized just now that UInt64("1") will not work as desired. It needs to be parse(UInt64, "1"). So where do plug that in ??

Finally, I use the result tuple as arguments to create an instance of a complex Xsd… type that becomes part of the fully parsed schema.

Thanks for any suggestions,
doug

1 Like
6

You could add another convert method like this:

# Don't do this.
Base.convert(::Type{UInt64}, s::AbstractString) = parse(UInt64, s)

However, that would be type piracy, since you would be extending a method you don’t own (convert) on a type you don’t own (AbstractString). This type piracy is a hint that you’re headed down the wrong path.

Here’s roughly the approach that I would take. First, note that I’m using EzXML in my example, and EzXML doesn’t have a get method for element attributes, so I’ve implemented my own for this example:

using EzXML

function Base.get(n::EzXML.Node, key, default)
    try
        return n[key]
    catch
        return default
    end
end

In your example above, it appears that you’re trying to parse an XSD element representing a “restriction”, so that’s what I’m using in my example below. However, instead of using named tuples, I explicitly define a Restriction type.

abstract type XMLDataType end
struct XMLString <: XMLDataType end
struct XMLInteger <: XMLDataType end
# etc, define other XML data types

function parse_xml_data_type(s::AbstractString)
    if s == "xs:string"
        XMLString
    elseif s == "xs:integer"
        XMLInteger
    else
        throw(ArgumentError("oops"))
    end
end

abstract type XSDElement end

struct Restriction{T <: XMLDataType} <: XSDElement
    id::Union{Nothing, Int}
end

function Restriction(element::EzXML.Node)
    maybe_base = get(element, "base", nothing)
    if isnothing(maybe_base)
        throw(ArgumentError("Invalid XSD restriction element"))
    end
    base = parse_xml_data_type(maybe_base)

    maybe_id = get(element, "id", nothing)
    if isnothing(maybe_id)
        Restriction{base}(maybe_id)
    else
        id = parse(Int, maybe_id)
        Restriction{base}(id)
    end
end

elem1 = root(parsexml("""<restriction base="xs:string" id="42"/>"""))
elem2 = root(parsexml("""<restriction base="xs:integer"/>"""))
elem3 = root(parsexml("""<restriction asdf="xs:integer"/>"""))

Now we can see the Restriction constructor at work at the REPL:

julia> Restriction(elem1)
Restriction{XMLString}(42)

julia> Restriction(elem2)
Restriction{XMLInteger}(nothing)

julia> Restriction(elem3)
ERROR: ArgumentError: Invalid XSD restriction element

Note that generally you want to use constructors rather than convert methods.

1 Like
7

Possibly, but I still think you are coupling this too closely to the type conversion API of Julia — I was just suggesting an explicit instantiation interface instead of relying on T(x).

8

Thank you both. I have abandoned the approach involving the instantiation of NamedTuples that only worked with convert

@CameronBieganek : Thanks for pointing out the type piracy. I take a similar approach to yours but I am using a StreamParser (I found the DOM approach too complex given the vast trees of elements that are allowed under the XSD spec.). I prefer to push and pop things off a stack (SAX parser style).

@Tamas_Papp: I needed a way to create an Xsd complex type using only strings. I took your idea of a factory-like function and made the process methods below. Thanks for that!

abstract type XsdAnyType end
abstract type XsdAnySimpleType <: XsdAnyType end

function nameparts(v)
    t = split(v,':')
    length(t) < 2 ? (nothing,t[1]) : (t[1],t[2])
end

struct QName{T<:AbstractString} <: XsdAnySimpleType
    pfx::Union{T,Nothing}
    nm::T
end
QName(nm::AbstractString) = QName(nameparts(nm)...)

struct XsdNonNegativeInt{T<:UInt} <: XsdAnySimpleType
    v::T
end
XsdNonNegativeInt(s::AbstractString) = (parse(UInt,s) >= 0) ? XsdNonNegativeInt(parse(UInt,s)) : error("s must represent a non-negative integer. Got $(s)")

struct ID{T<:AbstractString}
    id::T
end

struct XsdRestriction
    base::QName
    id::Union{ID,Nothing}
    value::XsdNonNegativeInt
end
# No more convert...
# Base.convert(::Type{XsdNonNegativeInt}, s::String) = XsdNonNegativeInt(s)

# As many methods as necessary...
process(t::Type{UInt}, v) = t(parse(t,v))
process(::Nothing,v) = nothing
process(t::Type{<:XsdAnySimpleType}, v) = t(string(v))

This models a specification for XsdRestriction:

spec = Dict("restriction" => 
    (("base", QName, ""),
    ("id", Union{Nothing,ID}, nothing),
    ("value", XsdNonNegativeInt, 1)
    ))

The code below would be run during parsing of an XML schema document, specifically on XML such as

<xs:simpleType name=“privateDataType”>
<xs:restriction base=“xs:string”>
<xs:maxLength value=“2048”/>
</xs:restriction>
</xs:simpleType>

  1. Unpack the spec:
attr = Dict("base"=>"xs:string", "value"=>"1") # Come from attributes on the element
r = spec["restriction"]
nm = getindex.(r,1)
typ = getindex.(r,2)
defv = getindex.(r,3)
  1. Get actual attribute values
vals = get.(Ref(attr), nm, defv) #vals will be strings or nothing
@show vals

  1. Reduce type unions (I think this part can definitely be improved)
    deftypes = map(v->Nothing <: v ? Nothing() : v, typ)

  2. Build a tuple of sane values for XsdRestriction (probably can use broadcast here)

res=[]
for n in zip(deftypes, vals)
    v = process(n...)
    push!(res,v)
end

xr = XsdRestriction(res...)
# push xr onto the stack and move on....
2 Likes
9

Here’s a couple smaller comments that don’t pertain to the overall design.

1

It’s possible to overuse parameters for custom types. If you just want a field to be a String, then make it a String, rather than a T <: AbstractString. This is where you can rely on the implicit convert machinery that already exists. For example, if you try to assign a SubString to a String field, Julia will automatically convert the SubString to a String:

julia> struct ID
           id::String
       end

julia> ID(SubString("abcd", 2:3))
ID("bc")

2

The XsdRestriction type has non-concrete field types, which can lead to poor performance. (Maybe you just omitted some of the type details to simplify the example.)

struct XsdRestriction
    base::QName
    id::Union{ID,Nothing}
    value::XsdNonNegativeInt
end

Making QName, ID, and XsdNonNegativeInt non-parametric (in line with my first comment), would solve this. Something like

struct QName <: XsdAnySimpleType
    pfx::Union{String,Nothing}
    nm::String
end

struct XsdNonNegativeInt <: XsdAnySimpleType
    v::UInt
end

struct ID
    id::String
end
2 Likes
10

Good points, all. Thanks!