# Proper structuring of an input data table

**URL:** <https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446>\
**Category:** Data\
**Created:** [August 24, 2020, 9:10am UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446 "2020-08-24T09:10:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mamo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mamo/32/25591_2.png) [@mamo](https://discourse.julialang.org/u/mamo)\
**Post date:** [August 24, 2020, 9:10am UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446/1 "2020-08-24T09:10:34Z")

</div>

How to build an input matrix (or a table) which will be sparse. For example:  
A is 4x4 matrix:  
A=[1 . . .; . . 2 3; . 5 . 1; 1 3 . .]  
If I use “missing” instead of dots, I have a problem with reading the data and identifying the type (String? Float64?). If I leave an empty place in an input table it will still identify it as a “missing”. Is it possible to obtain a table which will be of int or float type where empty fields are just ignored?  
Thank you!!

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [August 24, 2020, 10:35am UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446/2 "2020-08-24T10:35:14Z")

</div>

The ‘missing’ type is rather a good idea. Just have them as missing. You then know exactly what they are - missing.  
Not ‘white space’ or ‘undefined’

[https://docs.julialang.org/en/v1/manual/missing/](https://docs.julialang.org/en/v1/manual/missing/)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 24, 2020, 10:50am UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446/3 "2020-08-24T10:50:07Z")

</div>

Usually the other fields of what is called a “sparse matrix” are filled with `0`, and you can store just the non-zero elements with indexes. See the SparseArrays standard library.

> [@mamo](#):
>
> a table which will be of int or float type where empty fields are just ignored?

Ignored by whom? If the user of this table knows that these should be ignored, it can be set to arbitrary values. Otherwise, use `missing` or similar a @johnh suggested, or sentinel values, eg

> **[GitHub - JuliaData/SentinelArrays.jl: Array types that can use sentinel...](https://github.com/JuliaData/SentinelArrays.jl)**
>
> Array types that can use sentinel values of the element type for special values - GitHub - JuliaData/SentinelArrays.jl: Array types that can use sentinel values of the element type for special values

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [August 24, 2020, 1:59pm UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446/4 "2020-08-24T13:59:53Z")

</div>

Seems to me that what you want is in fact a `Dict` with `Tuple{Int, Int}` as keys, so the only entries that “really exist” are the ones you made explicit. Iterating over the keys/values will skip the ones you did not define (but will not go in order, unless you collect and sort the iterator). Trying to explicitly access a position you did not define will result in an error.

You may also use a Matrix of `Union{Number, Missing}` and use `skipmissing` but I think `skipmissing` will always iterate the data as it was a `Vector` (i.e., you will not be able to know exactly which field you are iterating over in cartesian coordinates).

---

<div class="post-metadata">

**Author:** ![mamo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mamo/32/25591_2.png) [@mamo](https://discourse.julialang.org/u/mamo)\
**Post date:** [August 24, 2020, 2:53pm UTC](https://discourse.julialang.org/t/proper-structuring-of-an-input-data-table/45446/5 "2020-08-24T14:53:52Z")

</div>

Thank you both. This later idea by Henrique looks good I will check it. Just to be clearer:  
PRT[o, f, p, j, t] == PRTint[o, f, p, j, t] \* bs[f, p]  
PRT and PRTint are variables, second is integer. bs is a matrix, filled with numbers only for specific pairs f,p. Since I introduced bs via a data frame table it reported load error and missmatch between formats.  
Thanks once again.