# DataFrames.jl (Tables.jl) is smart enough to consume my array of custom types...love it
**URL:** https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819
**Category:** General Usage
**Tags:** dataframes
**Created:** [August 4, 2021, 3:00pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819 "2021-08-04T15:00:49Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 3:00pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/1 "2021-08-04T15:00:49Z")
First, I want to say that I love that this just works:
```julia
using DataFrames
using HTTP
using JSON
using Unmarshal
const appid = # my appid
const url = "https://api.openweathermap.org/data/2.5/onecall"
params = Dict(
"appid" => appid,
"lat" => 30.33,
"lon" => -87.14,
"units" => "imperial",
"exclude" => "minutely,hourly,alerts,current"
)
struct Temp
min::Float64
max::Float64
end
struct DayWeather
temp::Temp
humidity::Int64
wind_speed::Float64
end
struct Response
lat::Float64
lon::Float64
daily::Vector{DayWeather}
end
function fetch_weather(url, params)
res = HTTP.get(
url,
["Content-Type" => "application/json"],
query=params
)
return unmarshal(Response, JSON.parse(String(res.body)))
end
julia> df = DataFrame(fetch_weather(url, params).daily)
8×3 DataFrame
Row │ temp humidity wind_speed
│ Temp Int64 Float64
─────┼──────────────────────────────────────────
1 │ Temp(78.42, 84.4) 76 9.91
2 │ Temp(78.04, 87.13) 57 10.71
3 │ Temp(79.18, 87.04) 58 13.42
4 │ Temp(80.85, 86.72) 61 11.41
5 │ Temp(81.84, 87.55) 61 12.06
6 │ Temp(82.51, 87.15) 63 9.51
7 │ Temp(82.36, 87.87) 58 11.32
8 │ Temp(82.51, 87.82) 60 10.42
```
DataFrames just knows how to consume my custom struct without me having to implement the Tables.jl interface… 🙂 👏
I do have a question though about how to design this code…I would like the `temp` field in my `DayWeather` struct to just hold the `max` temp from my `Temp` struct. Obviously I can’t define it like this:
```julia
struct Temp
min::Float64
max::Float64
end
struct DayWeather
temp::Temp.max # this won't work
humidity::Int64
wind_speed::Float64
end
```
so what’s the proper way to go about this?
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [August 4, 2021, 3:06pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/2 "2021-08-04T15:06:26Z")
I think what you want is
```julia
struct Temp
min::Float64
max::Float64
end
struct DayWeather
temp:Float64
humidity::Int64
wind_speed::Float64
end
DayWeather(t::Temp, h, w) = DayWeather (t.max, h, w)
```
---
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 3:14pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/3 "2021-08-04T15:14:54Z")
This doesn’t appear to work:
```julia
struct Temp
min::Float64
max::Float64
end
struct DayWeather
temp::Temp
humidity::Int64
wind_speed::Float64
end
DayWeather(t::Temp, h, w) = DayWeather(t.max, h, w)
julia> day = DayWeather(Temp(1.0,2.0), 1, 1.0)
DayWeather(Temp(1.0, 2.0), 1, 1.0)
```
EDIT: this does work:
```julia
struct Temp
min::Float64
max::Float64
end
struct DayWeather
temp::Float64 # <--- had to change this
humidity::Int64
wind_speed::Float64
end
DayWeather(t::Temp, h, w) = DayWeather(t.max, h, w)
julia> day = DayWeather(Temp(1.0,2.0), 1, 1.0)
DayWeather(2.0, 1, 1.0)
```
---
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 3:21pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/4 "2021-08-04T15:21:18Z")
Uh oh, this breaks the ability to unmarshal from the JSON response to my custom structs though:
```julia
julia> df = DataFrame(fetch_weather(url, params).daily)
ERROR: MethodError: no method matching Float64()
Closest candidates are:
(::Type{T})(::AbstractChar) where T<:Union{AbstractChar, Number} at char.jl:50
(::Type{T})(::Base.TwicePrecision) where T<:Number at twiceprecision.jl:243
(::Type{T})(::Complex) where T<:Real at complex.jl:37
...
Stacktrace:
[1] unmarshal(DT::Type, parsedJson::Dict{String, Any}, verbose::Bool, verboseLvl::Int64)
@ Unmarshal ~\.julia\packages\Unmarshal\pNr0C\src\Unmarshal.jl:135
```
😔
For now, I’ll just do this:
```julia
df = DataFrame(fetch_weather(url, params).daily)
df.temp = [df.temp[i].max for i in 1:length(df.temp)]
```
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [August 4, 2021, 3:33pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/5 "2021-08-04T15:33:11Z")
I would maybe make two structs, one for holding the stuff, which includes all the nesting, and another for what you actually want to work with, which just examines the day.
Also, I would question why you need `temp` to be a property in the struct, why not do `maxtemp(d::DayTemp) = max(d.temp)` or similar.
Is the issue you really want the automatic destructuring to work?
---
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 4:04pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/6 "2021-08-04T16:04:16Z")
> [@pdeffebach](#):
>
> Is the issue you really want the automatic destructuring to work?
Yes, was hoping to be able to automatically deserialize the JSON response to my custom types and the JSON response that I get back from the API is like this:
```json
{
.
.
.
"humidity": 76,
"dew_point": 74.16,
"wind_speed": 9.91,
"temp": {
"max": 87.5,
"min": 77.6
},
.
.
.
}
```
---
### Author:  [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [August 4, 2021, 7:05pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/7 "2021-08-04T19:05:15Z")
Is there any issue in doing it this way (inspired by [this post](https://discourse.julialang.org/t/array-of-structs-getindex-slicing-and-broadcasting-getfield/21887/3)):
```julia
DayWmax(t::DayWeather) = (tmax = t.temp.max, humidity = t.humidity, windspeed = t.wind_speed) # assessor function
df = DataFrame(DayWmax.(fetch_weather(url, params).daily))
8×3 DataFrame
Row │ tmax humidity windspeed
│ Float64 Int64 Float64
─────┼──────────────────────────────
1 │ 83.97 74 7.9
2 │ 86.41 58 9.42
3 │ 86.36 64 12.71
4 │ 84.34 67 11.34
5 │ 85.37 69 9.69
6 │ 86.45 66 9.51
7 │ 87.64 59 9.42
8 │ 87.93 60 10.85
```
---
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 7:07pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/8 "2021-08-04T19:07:06Z")
Besides not getting column names (which are easy enough to add), that’s a nice solution 🙂
---
### Author:  [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [August 4, 2021, 7:08pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/9 "2021-08-04T19:08:28Z")
Fixed above?
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [August 4, 2021, 7:20pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/10 "2021-08-04T19:20:18Z")
Maybe this works?
```julia
julia> struct B
bx
by
end;
julia> struct A
ax::Int
ay::B
end;
julia> nested_vec = [A(rand(1:5), B(rand(), rand())) for i in 1:10];
julia> df = DataFrame(nested_vec)
10×2 DataFrame
Row │ ax ay
│ Int64 B
─────┼─────────────────────────────────
1 │ 3 B(0.522192, 0.260377)
2 │ 4 B(0.956823, 0.148446)
3 │ 1 B(0.78614, 0.448788)
4 │ 2 B(0.806301, 0.886143)
5 │ 5 B(0.00277471, 0.0113921)
6 │ 3 B(0.268688, 0.267194)
7 │ 1 B(0.282506, 0.515099)
8 │ 1 B(0.380532, 0.981713)
9 │ 1 B(0.438347, 0.931338)
10 │ 4 B(0.88685, 0.947232)
julia> transform(df, :ay => Tables.columns => AsTable)
10×4 DataFrame
Row │ ax ay bx by
│ Int64 B Float64 Float64
─────┼────────────────────────────────────────────────────────
1 │ 3 B(0.522192, 0.260377) 0.522192 0.260377
2 │ 4 B(0.956823, 0.148446) 0.956823 0.148446
3 │ 1 B(0.78614, 0.448788) 0.78614 0.448788
4 │ 2 B(0.806301, 0.886143) 0.806301 0.886143
5 │ 5 B(0.00277471, 0.0113921) 0.00277471 0.0113921
6 │ 3 B(0.268688, 0.267194) 0.268688 0.267194
7 │ 1 B(0.282506, 0.515099) 0.282506 0.515099
8 │ 1 B(0.380532, 0.981713) 0.380532 0.981713
9 │ 1 B(0.438347, 0.931338) 0.438347 0.931338
10 │ 4 B(0.88685, 0.947232) 0.88685 0.947232
```
---
### Author:  [@cjdoris](https://discourse.julialang.org/u/cjdoris)
#### Post date: [August 4, 2021, 7:30pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/11 "2021-08-04T19:30:46Z")
To be fair, the credit for this behaviour goes to Tables.jl, which considers any vector of structs to be a table with columns being fields.
---
### Author:  [@mthelm85](https://discourse.julialang.org/u/mthelm85)
#### Post date: [August 4, 2021, 7:46pm UTC](https://discourse.julialang.org/t/dataframes-jl-tables-jl-is-smart-enough-to-consume-my-array-of-custom-types-love-it/65819/12 "2021-08-04T19:46:37Z")
That’s very nice. I assumed I would have to implement the Tables interface for my structs but before I went down that road I decided to just try passing my vector into `DataFrame` and I immediately felt as if I’d just stumbled upon a $100 bill walking down the street lol…was very happy that it just worked…I’m really glad the Tables.jl devs made that decision!