Abstract Types and StructTypes - error reading AbstractTypes into concrete with JSON3

General Usage
, , ,
1

Hi,

Believe I am running into a bit of a bug / unexpected behaviour that I can’t seem to work around when extracting an interface into a concrete type having defined more than one in a module.

e.g.: this works:


using StructTypes, Parameters, JSON3
abstract type ICompute end
StructTypes.StructType(::Type{ICompute}) = StructTypes.AbstractType()
StructTypes.subtypekey(::Type{ICompute}) = :computeName
function StructTypes.subtypes(::Type{ICompute})
(
    percentile = Percentile
)
end

struct Percentile <: ICompute
    computeName
    id
    percentiles :: Vector{Float64}
end
StructTypes.StructType(::Type{Percentile}) = StructTypes.Struct()

percentile = JSON3.read("""
{
    "computeName": "percentile",
    "id": "1",
    "percentiles": [10,20,30]
    }""", ICompute)

works, but trying to extend it to include more abstract types:

using StructTypes, Parameters, JSON3
abstract type ICompute end
StructTypes.StructType(::Type{ICompute}) = StructTypes.AbstractType()
StructTypes.subtypekey(::Type{ICompute}) = :computeName
function StructTypes.subtypes(::Type{ICompute})
(
    percentile = Percentile
)
end

struct Percentile <: ICompute
    computeName
    id
    percentiles :: Vector{Float64}
end
StructTypes.StructType(::Type{Percentile}) = StructTypes.Struct()


abstract type IQuery end
StructTypes.StructType(::Type{IQuery}) = StructTypes.AbstractType()
StructTypes.subtypekey(::Type{IQuery}) = :queryName
function StructTypes.subtypes(::Type{IQuery})
(
    dataDominance = DataDominance
)
end


struct DataDominance <: IQuery
    queryName
    id
    percentiles :: Vector{Float64}
end
StructTypes.StructType(::Type{DataDominance}) = StructTypes.Struct()

percentile = JSON3.read("""
{
    "computeName": "percentile",
    "id": "1",
    "percentiles": [10,20,30]
    }""", ICompute)

datadominance = JSON3.read("""
{
    "queryName": "dataDominance",
    "id": "1",
    "percentiles": [10,20,30]
    }""", IQuery)

fails with error:

ERROR: LoadError: MethodError: no method matching length(::Type{Main.GetMetricModels.DataDominance})
Closest candidates are:
  length(!Matched::Union{Base.KeySet, Base.ValueIterator}) at abstractdict.jl:58
  length(!Matched::Union{LinearAlgebra.Adjoint{T, S}, LinearAlgebra.Transpose{T, S}} where {T, S}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LinearAlgebra\src\adjtrans.jl:195
  length(!Matched::Union{DataStructures.OrderedRobinDict, DataStructures.RobinDict}) at C:\Users\doliver\.julia\packages\DataStructures\ixwFs\src\ordered_robin_dict.jl:86

version info:

image

Packages info:

image
image790×489 12.4 KB

Would someone mind trying to run the above and feeding back - I’ve tried myriad combinations and just not been successful.

Regards,

2

I could not run your code (Average not defined). It is a little bit odd to me the concept of writing and reading abstract types from a JSON file (how to know what the structure of the data written?) but I may be not aware of how that works.

Anyway, if you could provide a MWE, that will help.

3

Hi - thanks for responding - updated the above - I hadn’t removed the other subtypes from the listing.

As to the usage - it’s supported by Structypes.jl and I make use of it to facilitate multiple dispatch of computations (and filters) provided by users to an http endpoint on top of a calculation library.

i.e. Percentiles require percentile arguments but Average / Sum wouldn’t but they are both of type “ICompute”. The above snippet was / is working fine until I introduced “IQuery”, resulting in the error thrown.

Incidentally - I saw your contribution here and this StructTypes + Interface approach would potentially be applicable (I would post it if it worked…), since the OP is essentially making use of properties of a type rather than the type, with a default required for the generic: implying a conversion step on raw data (JSON?) to a type (or not) and an implementation of that method for the converted type to support multiple dispatch.

Like this:

#support calcs that allow streaming:
#default - fall through
streamingUpdate!(compute:: ICompute, value) = ()

function streamingUpdate!(ave:: Average, value) 
    
    if(ismissing(value)) return end
    ave.runningTotal += value
    ave.runningCount += 1

end

function streamingUpdate!(sum:: Sum, value) 
    
    if(ismissing(value)) return end
    sum.runningTotal += value

end

Regards,

4

Maybe you need

StructTypes.subtypekey that indicates which input key/field name should be used for identifying the appropriate concrete subtype.

defining “queryName” and “computeName”?

5

Thanks - unfortunately not as simple - I already define this in the script above

Regards

6

Sorry, I didn’t notice it.

1 Like
7

No problem at all.

would you be able to see if the original code snippets run for you? Still at a loss whats going on.

@lmiq did the edits allow you to execute the snippets?

Regards

8

Both snippets fail for me with:

ERROR: LoadError: MethodError: no method matching length(::Type{Percentile})
Closest candidates are:
  length(::Union{Base.KeySet, Base.ValueIterator}) at abstractdict.jl:58
  length(::Union{LinearAlgebra.Adjoint{T, S}, LinearAlgebra.Transpose{T, S}} where {T, S}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LinearAlgebra\src\adjtrans.jl:195
  length(::Base.LogicalIndex) at multidimensional.jl:732
  ...

My versioninfo() is the same as your except minor hardware details.

9

Yes, but I don’t know how to solve the problem :frowning:

1 Like
10

Many thanks to you both for trying - very odd that it works for @lmiq across both snippets but not for @Jeff_Emanuel at all…

Must be down to package and / or Julia versions - I can still get one scenario to work!

I’ll revert if I find the solution / issue.

Regards,

11

Oh, no, I meant that I can run the code, not that it works. I doesn’t work (I get the same error as you).

12

It should be

function StructTypes.subtypes(::Type{ICompute})
(
    percentile = Percentile,
)
end
1 Like
13

Thanks @Skoffer ! I was hoping it was something simple - very counter intuitive - is it something to do with Tuple definitions?

Thanks all for contributing.

Regards

14

Parentheses are also used to specify order of operations, so an explicit comma is required to disambiguate that you’re constructing a tuple.

1 Like
15

The way how it is written is slightly confusing, since tabs do not correspond to usual notations. If you rewrite your original code as

function StructTypes.subtypes(::Type{ICompute}) 
    (percentile = Percentile)
end

Or even better

StructTypes.subtypes(::Type{ICompute})  = (percentile = Percentile)

then issue is more obvious. In expression (percentile = Percentile) brackets are ignored, so it is equivalent to percentile = Percentile, i.e. it creates throaway variable percentile and returns the result of assignment operator which is equals to Percentile. So, instead of returning NamedTuple function was equivalent to

StructTypes.subtypes(::Type{ICompute})  = Percentile

And of course, since type is not the same as NamedTuple StructTypes was unable to process it.

If you want to create named tuple with single field, you can either use comma after expression or semicolon at the beginning. So both will do

StructTypes.subtypes(::Type{ICompute})  = (percentile = Percentile, )
# or
StructTypes.subtypes(::Type{ICompute})  = (; percentile = Percentile)
3 Likes