# Composite types versus lookup tables

**URL:** <https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534>\
**Category:** Offtopic\
**Tags:** question\
**Created:** [July 19, 2019, 8:06am UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534 "2019-07-19T08:06:16Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [July 19, 2019, 8:06am UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/1 "2019-07-19T08:06:16Z")

</div>

I’ve been fluctuating between holding my data in database-style tables versus composite types. Things like `join`ing in the DB version is very useful, but things tend to stay as floats and strings. Method dispatch and custom types are very useful in the composite type system. At the end I think it falls on if things have a one-to-one relationship or one/many-to-many: If everything is unique and has just one unique thing to be connected to, then a collection of such things would be best described as a composite type with two fields, one for each “thing”. If one thing can be related to multiple things then it seems like a DB table would be best.

Of course, nothing stoops me from mixing the two. It is however hard to know where to cut it, what should not be a composite type and instead be a DB-table entry, and what should be a composite type. I think I’m down the right track with the one-to-one versus one/many-to-many distinction “described” above, but I would love to hear what you have to say if any of this rambling sounds familiar…

Thanks!

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 19, 2019, 3:15pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/2 "2019-07-19T15:15:09Z")

</div>

Yes, this is a common design problem.

For data analysis, I try to start with a tables format (eg `DataFrame`) and work there unless there is a good reason to define `struct`s. Good reasons include:

1. ability to manage different representations with a single verb (ie multiple dispatch)
2. unit testing of complex operations
3. optimization/benchmarking

But as you said, you can freely mix the two. I usually define simple functions for the sole purpose of converting from one representation to another, to get the benefit of both worlds.

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [July 19, 2019, 3:18pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/3 "2019-07-19T15:18:11Z")

</div>

May I ask why DataFrames? Can you easily join DataFrames? I find JuliaDB api really really convenient. I kinda wish we had their api without necessarily all the other stuff (parallel, online stats, etc)…

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 19, 2019, 5:11pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/4 "2019-07-19T17:11:29Z")

</div>

No particular reason against JuliaDB, I am just OK with DataFrames.

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [July 19, 2019, 7:44pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/5 "2019-07-19T19:44:49Z")

</div>

But how do you do a join though? The reason I’m asking is that it’s mostly the joins that I find ultra useful when working with the table representation. Where an entry from one table relates to multiple entries in another table, so joining those two tables results in a product that moves further down the analysis pipeline.

I’ll mention that one of the things that pushed me away from the table representation was the way the data entered the pipeline. If before the initial state of the data was `.csv` and auxiliary files, now it’s fed more manually. When it’s entered entry by entry like that, I find it more useful to have types – types that by definition have the correct structure and values.

Thanks btw for this conversation, it’s super useful to know I’m not crazy 🙂

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 19, 2019, 8:12pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/6 "2019-07-19T20:12:35Z")

</div>

Personally, I typically combine DataFrames.jl with Query.jl which lets you [join](https://www.queryverse.org/Query.jl/stable/standalonequerycommands/#The-@join-command-1) ([alternative](https://www.queryverse.org/Query.jl/stable/linqquerycommands/#Joining-1)).

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [July 19, 2019, 8:14pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/7 "2019-07-19T20:14:06Z")

</div>

I have to start using Query…

---

<div class="post-metadata">

**Author:** ![js135005](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/js135005/32/8219_2.png) [@js135005](https://discourse.julialang.org/u/js135005)\
**Post date:** [July 22, 2019, 6:03pm UTC](https://discourse.julialang.org/t/composite-types-versus-lookup-tables/26534/8 "2019-07-22T18:03:47Z")

</div>

Also DataFramesMeta has useful capabilities.