# DataViewer.jl supported data file formats
**URL:** https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569
**Category:** Visualization
**Created:** [October 30, 2023, 1:42pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569 "2023-10-30T13:42:30Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@rajmac](https://discourse.julialang.org/u/rajmac)
#### Post date: [October 30, 2023, 1:42pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/1 "2023-10-30T13:42:30Z")
I was interested in the recent post on DataViewer.jl and wondered if it could be used with csv files or whether it requires hf5/jdl2/json datafiles?
---
### Author:  [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [October 30, 2023, 2:00pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/2 "2023-10-30T14:00:01Z")
Why not give it a try?
An alternative would be Queryverse. However I am not sure how up to date this is
[Queryverse | Queryverse, Julia packages for data science.](https://www.queryverse.org/)
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [October 30, 2023, 2:06pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/3 "2023-10-30T14:06:23Z")
Try it out, and if it doesn’t work create an issue at [Issues · triscale-innov/DataViewer.jl · GitHub](https://github.com/triscale-innov/DataViewer.jl/issues) …
---
### Author:  [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [October 30, 2023, 2:08pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/4 "2023-10-30T14:08:44Z")
See also the package section on [extending DataViewer to support more data formats](https://triscale-innov.github.io/DataViewer.jl/dev/extend/).
---
### Author:  [@MJulia](https://discourse.julialang.org/u/MJulia)
#### Post date: [October 30, 2023, 2:12pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/5 "2023-10-30T14:12:06Z")
netcdf support would be very nice for geoscience community.
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [October 30, 2023, 4:35pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/6 "2023-10-30T16:35:48Z")
> [@johnh](#):
>
> However I am not sure how up to date this is
It should all work. The underlying Javascript Data Voyager hasn’t seen any updates in a long time though, not clear to me whether that particular project is still alive or not… But the existing functionality should just work.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [October 31, 2023, 8:15am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/7 "2023-10-31T08:15:30Z")
> [@rajmac](#):
>
> I was interested in the recent post on DataViewer.jl and wondered if it could be used with csv files or whether it requires hf5/jdl2/json datafiles?
Thanks for your interest! For now, `DataViewer` only supports `HDF5`, `JLD2` and `JSON` data files. But more fundamentally, it was designed with tree-like data structures in mind (think things like dictionaries of dictionaries of arrays of dictionaries).
It would probably not be difficult to add support for tabular/columnar file formats (like CSV, or Arrow which was mentioned in the other thread), but I’m not sure how we’d want to display them:
- like a dictionary of array-like columns? → _this would be already supported, probably the best option for now_
- like a vector of dict-like rows? → _this would also already be possible, but would it be useful?_
- like a spreadsheet? → _this does not currently exist in DataViewer and would probably be a bit more work to implement_
But (and here I might very well be wrong because I almost never work with such data), I’m under the impression that there already exist lots of tools which would be more suited to flat, tabular data. For example, in the QueryVerse (which has already been mentioned in this thread) I would expect the [`DataVoyager`](https://github.com/queryverse/DataVoyager.jl) UI or the [`ElectronDisplay`](https://www.queryverse.org/2019/02/13/electrondisplay/) “table display” feature to be particularly useful with columnar data coming from CSV files.
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [October 31, 2023, 9:27am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/8 "2023-10-31T09:27:30Z")
I would like to be able to browse a dictionary of transfer function, displaying them
as bode plots…
The function is stored as:
```julia
TransferFunction{Continuous, ControlSystemsBase.SisoRational{Float64}}
1.0s^3 + 0.24915934813931118s^2 + 0.01701570159733241s + 0.00018625726584153255
--------------------------------------------------------------------------------------
372.27171456943216s^3 + 92.88568061036781s^2 + 6.366403832359201s + 0.0695505145474942
```
---
### Author:  [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [October 31, 2023, 9:29am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/9 "2023-10-31T09:29:26Z")
One slightly off topic response from me. I am surprised that the .jld2 and .hdf5 file extension is being looked for.
Surely the file type can be found in the header?
Sorry if I am exposing my ignorance here…
Sure enough, FileTypes.jl does not cover these file types
[GitHub - JuliaIO/FileTypes.jl: Small and dependency-free Julia package to infer file and MIME type checking the magic numbers signature.](https://github.com/JuliaIO/FileTypes.jl)
---
### Author:  [@rajmac](https://discourse.julialang.org/u/rajmac)
#### Post date: [October 31, 2023, 9:51am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/10 "2023-10-31T09:51:46Z")
Thanks for all the resonses and for being pointed to Queryverse which may suit me better.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [October 31, 2023, 11:06am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/11 "2023-10-31T11:06:37Z")
> [@johnh](#):
>
> One slightly off topic response from me. I am surprised that the .jld2 and .hdf5 file extension is being looked for.
> Surely the file type can be found in the header?
That’s a very good point!
Lack of time was the main reason I did not do this, but I did consider it at one point, which lead me to the following remarks:
- I’m not sure whether it would be possible to reliably auto-detect text-based formats like `JSON` (or `CSV`, for that matter), so a file-extension-based mechanism might be needed anyway.
- Since the `JLD2` format is itself based on `HDF5`, I’m not sure a magic-number-based approach à la `FileTypes.jl` could reliably distinguish between the two. It might be possible to implement a two-stage approach, though: determine that the file is an `HDF5` container first, then look in it for specific meta-data signalling a `JLD2` file.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [October 31, 2023, 11:11am UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/12 "2023-10-31T11:11:50Z")
> [@ufechner7](#):
>
> The function is stored as:
>
> ```julia
> TransferFunction{Continuous, ControlSystemsBase.SisoRational{Float64}}
> 1.0s^3 + 0.24915934813931118s^2 + 0.01701570159733241s + 0.00018625726584153255
> --------------------------------------------------------------------------------------
> 372.27171456943216s^3 + 92.88568061036781s^2 + 6.366403832359201s + 0.0695505145474942
>
> ```
Is that what your file contains: `TransferFunction` instances? (In what type of file?)
One problem I see is that, in order to handle those, `DataViewer` would have to know about the `TransferFunction` type. That could probably be a nice extension that depends both on `ControlSystemsBase` and `DataViewer`.
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [October 31, 2023, 12:03pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/13 "2023-10-31T12:03:49Z")
> [@ffevotte](#):
>
> Is that what your file contains: `TransferFunction` instances? (In what type of file?)
Yes, indeed. Well, can be StateSpace or TransferFunction, they can easily be converted into each other. I have lots of these files (simple example):
```julia
julia> load("data/lin_turbine_0.95.jld2")
Dict{String, Any} with 9 entries:
"omega_max" => 0.11643
"Uest" => 6.88889
"ω" => 0.87478
"U0" => 7.0
"Pgc" => 1089328
"Γ" => 0.95
"linsys" => StateSpace{Continuous, Float64}…
"max_mag_db" => -43.3495
"phase_shift" => -238.909
```
---
### Author:  [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [October 31, 2023, 3:24pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/14 "2023-10-31T15:24:40Z")
JLD2 files have custom magic bytes at the beginning.
([Here’s](https://github.com/JuliaIO/JLD2.jl/blob/c87135ea5dc256f109cd15571064133eb40b441f/src/file_header.jl#L19) JLD2’s own header verification)
If required, I can help with JLD2 specific things.
EDIT: also, `FileIO` can already correctly identify JLD2 / HDF5
---
### Author:  [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [October 31, 2023, 3:30pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/15 "2023-10-31T15:30:30Z")
Note, that netcdf files are also just HDF5 with extra metadata strapped on it.
JLD2 can _read_ netcdf files already, with the caveat that using the metadata is not implemented.
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [October 31, 2023, 6:00pm UTC](https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats/105569/16 "2023-10-31T18:00:56Z")
> [@JonasIsensee](#):
>
> EDIT: also, `FileIO` can already correctly identify JLD2 / HDF5
Very good to know, thanks!
```julia-repl
julia> using FileIO
julia> query("sample.h5")
File{DataFormat{:HDF5}, String}("/home/francois/projets/DataViewer.jl/app/sample.h5")
julia> query("sample.jld2")
File{DataFormat{:JLD2}, String}("/home/francois/projets/DataViewer.jl/app/sample.jld2")
julia> query("sample.json")
File{DataFormat{:UNKNOWN}, String}("/home/francois/projets/DataViewer.jl/app/sample.json")
```
I guess I still need the extension-based mechanism for things like `JSON`, but still a very nice improvement for everything else! I probably won’t have any time to work on this soon, but filed an issue to remember about it
>
>
> From https://discourse.julialang.org/t/dataviewer-jl-supported-data-file-formats…/105569:
>
> \`\`\`julia
> julia\> using FileIO
>
> julia\> query("sample.h5")
> File{DataFormat{:HDF5}, String}("/home/francois/projets/DataViewer.jl/app/sample.h5")
>
> julia\> query("sample.jld2")
> File{DataFormat{:JLD2}, String}("/home/francois/projets/DataViewer.jl/app/sample.jld2")
>
> julia\> query("sample.json")
> File{DataFormat{:UNKNOWN}, String}("/home/francois/projets/DataViewer.jl/app/sample.json")
> \`\`\`