Retrieving the contents of C composite types

,

Hi!

I wrote a wrapper to a C function that defines its output as a structure, defined as follows in the associated .h file:

typedef struct {
  double lat;
  double lon;
} MTKt_GeoCoord;

typedef struct {
  MTKt_GeoCoord ctr;
} MTKt_GeoCenter;

typedef struct {
  double xlat;
  double ylon;
} MTKt_Extent;

typedef struct {
  MTKt_GeoCenter geo;
  MTKt_Extent hextent;
} MTKt_Region;

The function works correctly but returns both the output type (structure name) and value information:

julia> path = 37
37

julia> start_block = 32
32

julia> end_block = 40
40

julia> region = jMtkSetRegionByPathBlockRange(path, start_block, end_block)
JMtk15.MTKt_Region(JMtk15.MTKt_GeoCenter(JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839)), JMtk15.MTKt_Extent(633462.5, 307862.5))

julia> region.geo
JMtk15.MTKt_GeoCenter(JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839))

julia> region.geo.ctr
JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839)

How do I retrieve the vectors of values by themselves (e.g., (67.28628518822661, -95.22204196033839) rather than JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839)?

More generally, if a C function returns a complex structure, how do I retrieve the individual vectors without the type specification?

Thanks in advance for any suggestions.

Shouldn’t simply [region.geo.ctr.lat, region.geo.ctr.lon] give you the desired vector?

Not sure what you mean by “individual vectors” and “without the type specification” here. Perhaps absolutely not what you’re asking for but FWIW:

julia> vectorize(x) = [getfield(x, f) for f in fieldnames(typeof(x))]
vectorize (generic function with 1 method)

julia> struct MTKt_GeoCoord
           lat::Float64
           long::Float64
       end

julia> coord = MTKt_GeoCoord(rand(), rand())
MTKt_GeoCoord(0.5525629138140212, 0.05413709610407247)

julia> vectorize(coord)
2-element Vector{Float64}:
 0.5525629138140212
 0.05413709610407247
1 Like

Thanks, @carstenbauer. I guess, my question may sound strange out of context. I know how to retrieve individual values from a structure, but when testing the function, I got this:

julia> @test region.geo.ctr.lat == 67.28628518822661
Test Passed

julia> @test region.geo.ctr.lon == -95.22204196033839
Test Passed

julia> @test region.hextent.xlat == 633462.5
Test Passed

julia> @test region.hextent.ylon == 307862.5
Test Passed

julia> @test region.geo.ctr == (67.28628518822661, -95.22204196033839)
Test Failed at REPL[304]:1
  Expression: region.geo.ctr == (67.28628518822661, -95.22204196033839)
   Evaluated: JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839) == (67.28628518822661, -95.22204196033839)
ERROR: There was an error during testing

so my question was whether there is a native way to write the last test or whether I need to extract the vector using a function like the one you kindly provided to implement the testing.

Thanks again for your help.

That’s just how it’s printed.

Imagine you are in C and you had a MTKt_GeoCoord x with an array double y[2] … would you compare them with x == y? No, because they are different types. Same thing in Julia — you are comparing a MTKt_GeoCoord with a tuple, so == returns false by default because they are different types.

Of course, Julia is much more flexible in this regard than C because you can define lots of new operations on a type to make things like this work. For example, if you want to make a MTKt_GeoCoord act like a vector, you can do:

using StaticArrays # the StaticArrays.jl package

# define a struct that can also act as a 2-component vector:
struct MTKt_GeoCoord <: FieldVector{2,Float64}
    lat::Float64
    lon::Float64
end

in which case you can do things like:

julia> ctr = MTKt_GeoCoord(67.2, -95.3)
2-element MTKt_GeoCoord with indices SOneTo(2):
  67.2
 -95.3

julia> ctr == [67.2, -95.3]
true

julia> ctr .+ 1
2-element MTKt_GeoCoord with indices SOneTo(2):
  68.2
 -94.3
1 Like

Hi @stevengj. Thanks a lot: this is very interesting. Now, how could this process be generalized? Can I write

struct MTKt_GeoRegion <: FieldVector{5, FieldVector{2, Float64}}
    ulc::MTKt_GeoCoord
    urc::MTKt_GeoCoord
    ctr::MTKt_GeoCoord
    lrc::MTKt_GeoCoord
    llc::MTKt_GeoCoord
end

And how should structures containing object of different types be handled, such as

struct MTKt_GeoBlock
    block_number::Cint
    ulc::MTKt_GeoCoord
    urc::MTKt_GeoCoord
    ctr::MTKt_GeoCoord
    lrc::MTKt_GeoCoord
    llc::MTKt_GeoCoord
end

struct MTKt_BlockCorners
    path::Cint
    start_block::Cint
    end_block::Cint
    block::Vector{MTKt_GeoBlock}
end

Is there an equivalent “StaticTuples” package? Thanks again for your help.

I’m confused about what you are trying to accomplish. What data structure do you want? A struct, an array, or a tuple? This depends on what sort of operations you want to perform.

Sorry if this appears to be confusing… As indicated earlier, the wrapper to the C function works fine (it generates the correct values), but the outcome does not seem to be suitable for direct comparisons in @test procedures, as pointed out before.

In the meantime, I have implemented your suggestion, and redefined the structures as follows:

struct MTKt_GeoCenter <: FieldVector{1, FieldVector{2, Float64}}
    ctr::MTKt_GeoCoord
end

struct MTKt_Extent <: FieldVector{2, Float64}
    xlat::Cdouble
    ylon::Cdouble
end

struct MTKt_Region <: FieldVector{2, FieldVector{2, Float64}}
    geo::MTKt_GeoCenter
    hextent::MTKt_Extent
end

With those newly redefined types, the function outcome is

julia> region = jMtkSetRegionByPathBlockRange(path, start_block, end_block);

 julia> region.geo
1-element JMtk15.MTKt_GeoCenter with indices SOneTo(1):
 [67.28628518822661, -95.22204196033839]

julia> region.hextent
2-element JMtk15.MTKt_Extent with indices SOneTo(2):
 633462.5
 307862.5

and that, in turn, is suitable for testing:

julia> @test region.geo[1] == [67.28628518822661, -95.22204196033839]
Test Passed

julia> @test region.hextent == [633462.5, 307862.5]
Test Passed

So the next question is how to do the same when the structures contain different base types (those presumably cannot be mapped to a FieldVector), as in the last examples in my previous message…

Unless you truly need a nested arrays-of-arrays API for some reason, why not just construct instances of your structs for comparison in the tests and skip implementing a bunch of other stuff that it seems you want just so you can build tests? Nested arrays of arrays become hard to work with because there’s no information to identify what they mean.

@test region.geo.ctr ==MTKt_GeoCoord(67.2, -95.3)
1 Like

@Jeff_Emanuel: Thanks a lot for your input. Your approach also works:

julia> @test region.geo.ctr == JMtk15.MTKt_GeoCoord(67.28628518822661, -95.22204196033839)
Test Passed

though only if I explicitly include the prefix JMtk15 in front of MTKt_GeoCoord.

This puzzles me because the structure MTKt_GeoCoord is defined in a file included at the start of the project JMtk15, so why is it not defined after using JMtk15? Do I need to export the symbols defined in an included file? Apologies if this is a trivial question…

Yes, you should export the symbols from your module that should be available to users. Users can still qualify the unexported symbols with the module name. Those symbols might not be exported because they may be internal details or they may clash with common names. Essentials ¡ The Julia Language

Ok, thanks a lot @Jeff_Emanuel.

I have read the manual (Essentials · The Julia Language) multiple times and understood the first sentence “Evaluate the contents of the input source file in the global scope of module m” to mean that include("file_h.jl") instructions would bring the contents of file_h.jl into the current scope. Similarly, I always encountered export keywords in the context of import and using files.

So, just two more questions to close that argument:

  • what is the practical difference, if any, between include("file") and using file?

  • why did my dozens of wrapper functions work so far without the need to export such included definitions?

Thanks again for your inputs.

include basically just executes the included file in place of the include statement. It’s very coarse, like a #include in C. Modules provide a namespace for a unit of code. using loads the module and makes its exported symbols available in the calling namespace. using is somewhat like an import in Java or Go or other relatively recent (compared to C) languages. One noteworthy difference is that include can trample over existing names, but using will report a conflict. export is used to make a symbol in a module available to code that imports the module.

You don’t use a file, you use a module.

Do you mean within the module or the client code? A module has free access to its own symbols. If in the client code, I suspect that you may have directly included the source file defining those functions.

I am not sure what is meant by ‘client code’. Basically, I have a project with a unique module, called JMtk15. Within that project, I have a src directory containing Julia functions that are wrappers to the C functions, and a src/include subdirectory containing files with const, @enum and struct definitions such as the ones mentioned above (those statements are translated from the C .h files). So far, those files did not export any of the symbols, and were included at the start of the JMtk15 module.

When I execute, in the REPL, sequences of commands such as

using JMtk15

outputs = function(inputs)

or

using JMtk15

using Test

outputs = function(inputs)

@test outputs == expected_outputs

things work smoothly (again without any of the included definitions being explicitly exported). That’s why I always thought that those structure definitions were available as soon as using JMtk15 had been executed…

Client would be any code that uses your module. What you describe sounds proper except for the absence of export. I can’t explain why it worked without export.

Thanks for your contribution!