# Human-readable externalization for multi-dimensional arrays
**URL:** https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214
**Category:** General Usage
**Tags:** question
**Created:** [August 6, 2019, 11:20am UTC](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214 "2019-08-06T11:20:55Z")
**Posts on this page:** 4
**Page:** 2
### Author:  [@tbo](https://discourse.julialang.org/u/tbo)
#### Post date: [August 13, 2019, 2:11pm UTC](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214/21 "2019-08-13T14:11:48Z")
1. If you replace text editor with “some viewer” you should be fine. There are many other formats that are built on top of HDF5 (e.g. CGNS, NetCDF, Matlab’s standard .mat files,…), each bringing their own specific viewers, but each of these formats can be opened, read and debugged in hdfview, Python, Julia, …
2. Ok, this wont be possible for HDF5, since the feature set is so extensive. This is due to HDF5 not being a format but an IO library, with lots of file system specific tuning.
> I don’t have a particular gripe with HDF5, I am just concerned that if something breaks (which happens very commonly with software), I am not in a position to fix it.
I think HDF5 will be the lingua franca for scientific data for a long time. Even is predecessor HDF4 is still actively supported after 20 years on the market. The ecosystem and compatibility coming with HDF5 is so widespread that you should actually have good reasons to NOT use it, see [Wikipedia](https://en.wikipedia.org/wiki/Hierarchical_Data_Format). (yes I know I may sound like a salesman)
BTW: If you’re really forced to use plain text you could try [HDF5-JSON](https://hdf5-json.readthedocs.io/en/latest/), however without the advantage of ubiquitous reader support.
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2019, 3:15pm UTC](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214/22 "2019-08-13T15:15:41Z")
> [@tbo](#):
>
> If you replace text editor with “some viewer” you should be fine.
Sure, my problem is that these viewers mostly rely on a single C library, which is a relatively opaque piece of software implementing a huge spec maintained by a third party that technically publishes the source, but operates very differently from most contemporary open source projects.
Eg suppose that two years after writing some data, I cannot read it back (eg [this issue](https://github.com/JuliaIO/HDF5.jl/issues/522)). Even investigating whether this is a HDF5.jl or a libhdf5 bug is tricky. I could not even figure out where the bug tracker for HDF5 is, they just [ask that you send an e-mail](https://confluence.hdfgroup.org/display/knowledge/How+do+I+report+a+bug), so how should I learn if someone else had this issue, if there is a workaround, etc?
---
### Author:  [@ggggggggg](https://discourse.julialang.org/u/ggggggggg)
#### Post date: [August 13, 2019, 3:44pm UTC](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214/23 "2019-08-13T15:44:18Z")
Have you looked into ASDF? [ASDF - Advanced Scientific Data Format — asdf v2.13.1.dev13+gbf954a3](https://asdf.readthedocs.io/en/latest/)
It’s based on YAML and supports having arrays stored as binary. It’s planned to be used as the format for the James Webb Space Telescope, so I think it is likely to gain users over time.
There is [https://github.com/eschnett/ASDF.jl](https://github.com/eschnett/ASDF.jl) that uses PyCall to call the only full implementation in Python, but it looks pretty straightforward to implement in other languages.
---
### Author:  [@tbo](https://discourse.julialang.org/u/tbo)
#### Post date: [August 13, 2019, 3:53pm UTC](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214/24 "2019-08-13T15:53:05Z")
Access to the Jira bugtracker described right here:
> **[Jira Access - The HDF Group](https://www.hdfgroup.org/community/community-development/)**
>
> Community Ticket Access The HDF Group uses Jira to record bugs, issues, and feature requests. We’ve recently opened this up to the community. You can now login to The HDF Group’s Jira instance and view HDF bugs, issues, and feature requests....
I think this is becoming a discussion on personal preferences. Many of the big players put sufficient trust in HDF5 to make it their standard format.
Though not a hipster-ish project HDFGroup has been publishing robust, well-maintained, open-source software for the last decades and they will most certainly continue doing this for a long time.
[Previous page](https://discourse.julialang.org/t/human-readable-externalization-for-multi-dimensional-arrays/27214.md?page=1)