# JLD2: How to update existing dataset array on disk?
**URL:** https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103
**Category:** General Usage
**Tags:** jld2
**Created:** [March 3, 2024, 6:59pm UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103 "2024-03-03T18:59:52Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [March 3, 2024, 6:59pm UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/1 "2024-03-03T18:59:52Z")
I am trying to update parts of an array stored inside a .jld2 file. However, all the changes that I make are not actually stored to disk.
As an MWP, imagine doing
```julia
using JLD2
a = zeros(100,100)
jldsave("test.jld2"; a=a)
jldopen("test.jld2", "r+") do file
file["a"][:,1] .= 1
end
```
After that, I expect that loading the file again will give me an updated array “a” with the first column replaced by ones, however I get an array of purely zeros.
How to update an existing array in a .jld2 file?
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [March 4, 2024, 12:21am UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/2 "2024-03-04T00:21:43Z")
`file["a"][:,1] .= 1` is lowered to `file["a"]` reading the file to an array, then doing in-place broadcasting over that array, not the file. Setting indices or in-place broadcasting must be done with 1 bracket, so the only possible syntax is `file["a", :, 1] .= 1`, which seems unsupported because the docs don’t mention it. I’m not even sure if that syntax is feasible to implement. Maybe you could replace `file["a"][:,1] .= 1` with:
```julia
tempa = file["a"] # read file and allocate temporary array
tempa[:,1] .= 1 # update the temporary array
file["a"] = tempa # write temporary array to file
```
---
### Author:  [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [March 4, 2024, 12:05pm UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/3 "2024-03-04T12:05:32Z")
This gives an Error saying that the dataset named “a” already exists.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [March 4, 2024, 1:19pm UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/4 "2024-03-04T13:19:30Z")
Looked into that error a bit and it doesn’t look great: [overwriting existing data · Issue #450 · JuliaIO/JLD2.jl · GitHub](https://github.com/JuliaIO/JLD2.jl/issues/450). It’s a short enough read, but the takeaway:
> It is already possible to `delete(f, "dataset")` .
> This will delete your reference to the data and you can write a new Dataset with the same name. Note, that the data itself remains in the “gap”. So, if you do that many times, the file will grow.
I presume that is a typo and should be, in your case, `delete!(file, "a")` prior to `file["a"] = tempa`. The file keeping the obsolete data is concerning, especially with how much larger arrays can get. If I’m reading it right, this could be resolved with [Allow reading/modifying arrays with Mmap · Issue #235 · JuliaIO/JLD2.jl · GitHub](https://github.com/JuliaIO/JLD2.jl/issues/235), but #450 might be alluding to something else and also mentions that editing files generally risks corruption due to program crashes or file system blunders outside of your control, which you don’t have to worry about with arrays in RAM.
---
### Author:  [@Gregstrq](https://discourse.julialang.org/u/Gregstrq)
#### Post date: [March 4, 2024, 3:47pm UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/5 "2024-03-04T15:47:25Z")
To be honest, it all looks very strange to me.
If I use HDF5 instead, the same update works fine (if I replace broadcasting with assignment). I mean, if I do
```julia
using HDF5
h5open("test.jld2", "r+") do file
file["a"][:,1] = ones(size(a,1))
end
```
on the same file created by JLD2, the array is updated without any problems.
Given that JLD2 essentially realizes a subset of HDF5, it seems like a bug.
---
### Author:  [@Benny](https://discourse.julialang.org/u/Benny)
#### Post date: [March 5, 2024, 2:27am UTC](https://discourse.julialang.org/t/jld2-how-to-update-existing-dataset-array-on-disk/111103/6 "2024-03-05T02:27:21Z")
IIRC, JLD2.jl does include HDF5 compatibility as a goal but not support of HDF5 in its entirety. Also bear in mind that HDF5.jl is a wrapper of the reference implementation of HDF5 in C, and what `file["a"]` and thus `file["a"][:, 1] = ...` does is part of the wrapping, not the HDF5 model. JLD2.jl is not related to HDF5.jl, so how they do things can differ.