# Read multi .nc files along new dimension

**URL:** <https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696>\
**Category:** Geo\
**Created:** [October 31, 2021, 5:44pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696 "2021-10-31T17:44:53Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Inna](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@Inna](https://discourse.julialang.org/u/Inna)\
**Post date:** [October 31, 2021, 5:44pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/1 "2021-10-31T17:44:53Z")

</div>

Dear all,

I have some netcdf files containg temperature and precipitation, with size 360 x 180. There isn’t inside the .nc file the dimension of time, but it is on the name of each file (2012.01.nc, 2012.02.nc, … ). I want to open all files and read all variables along the dimension of time.

I already read this issue [https://github.com/Alexander-Barth/NCDatasets.jl/issues/124](https://github.com/Alexander-Barth/NCDatasets.jl/issues/124) (the same) but it is not currently possible to set aggdim a new dimension that do not exist in files (xarray can do it).

Here [How to initialize multidimensional arrays for using Julia's ncread() on the same variable in different files? - Stack Overflow](https://stackoverflow.com/questions/46920935/how-to-initialize-multidimensional-arrays-for-using-julias-ncread-on-the-same) i found a way to do it for each variable each time.

Did you find any more general way to achive this multi-read?

Cheers,  
Inna

---

<div class="post-metadata">

**Author:** ![visr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/visr/32/17204_2.png) [@visr](https://discourse.julialang.org/u/visr)\
**Post date:** [October 31, 2021, 6:50pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/2 "2021-10-31T18:50:37Z")

</div>

Is it possible for you to edit the files? Then you could write a function that opens a netCDF called “2012.01.nc”, and use that name to create a new variable called “time” with the value 2012-01-01, and save the file. If you run such a function on all files then you should be able to use `aggdim="time"` afterwards.

---

<div class="post-metadata">

**Author:** ![Inna](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@Inna](https://discourse.julialang.org/u/Inna)\
**Post date:** [October 31, 2021, 8:14pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/3 "2021-10-31T20:14:54Z")

</div>

Hi, thanks.  
It is possible to edit, but temperature and precipitation has only Lon & Lat as dimensions. When i define the time variable and set `aggdim="time"`, can’t merge in time my 2 variables.

However, if i firstly define a new variable “aod”, with dimensions Lon x Lat x Time, then i can merge. So the new task is how can i append a new dimension to an already existed variable with already defined dimensions ? ( I want to edit the dimensions of temperature as Lon x Lat x Time without rewriting again the whole netcdf file)

---

<div class="post-metadata">

**Author:** ![visr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/visr/32/17204_2.png) [@visr](https://discourse.julialang.org/u/visr)\
**Post date:** [October 31, 2021, 9:32pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/4 "2021-10-31T21:32:51Z")

</div>

Ah ok. I assumed that it could use scalar values variables called “time”. I updated the NCDatasets issue, because that would be quite helpful already.

> So the new task is how can i append a new dimension to an already existed variable with already defined dimensions ?

I don’t think you can add a dimension in place. You could create a variable based on what you have, but with the extra dimension, and then just copy over the data.

---

<div class="post-metadata">

**Author:** ![Fliks](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fliks/32/2494_2.png) [@Fliks](https://discourse.julialang.org/u/Fliks)\
**Post date:** [October 31, 2021, 10:39pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/5 "2021-10-31T22:39:31Z")

</div>

You can use [YAXArrays.jl](https://github.com/meggart/YAXArrays.jl). When you know that your files have all the same dimensions and sizes you can do the following:

```julia
using YAXArrays, NetCDF, Dates

"""
   getdate(x,reg = r"[0-9]{8}T[0-9]{6}", df = dateformat"yyyymmddTHHMMSS")
Return a DateTime object from a string where the time stamp is found by `reg`
and it is parsed according to the `df` dateformat.
"""
function getdate(x,reg = r"[0-9]{8}T[0-9]{6}", df = dateformat"yyyymmddTHHMMSS")
   m = match(reg,x).match
   date =DateTime(m,df)
end

filelist = readdir(pwd()) # This gets you a list of the files in the current directory. 
timestamps = getdate.(filelist)

cubelist = Cube.(filelist)

timeaxis = RangeAxis("Time", timestamps)
 cube = concatenatecubes(cubelist, timeaxis)

```

With this you would get a YAXArray with which you can then do your data analysis. Or you can use `savecube(cube, "pathtosave.nc")` to store the data on a single netcdf on disk.

---

<div class="post-metadata">

**Author:** ![Inna](https://avatars.discourse-cdn.com/v4/letter/i/8e8cbc/32.png) [@Inna](https://discourse.julialang.org/u/Inna)\
**Post date:** [November 4, 2021, 6:44pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/6 "2021-11-04T18:44:11Z")

</div>

> [@Fliks](#):
>
> ```julia
> function getdate(x,reg = r"[0-9]{8}T[0-9]{6}", df = dateformat"yyyymmddTHHMMSS")
> m = match(reg,x).match
> date =DateTime(m,df)
> end
> 
> filelist = readdir(pwd()) # This gets you a list of the files in the current directory. 
> timestamps = getdate.(filelist)
> 
> cubelist = Cube.(filelist)
> 
> timeaxis = RangeAxis("Time", timestamps)
> cube = concatenatecubes(cubelist, timeaxis)
> 
> ```

Hi,interesting way with YAXArrays.jl but when i `Cube.(filelist)` , i am getting the error:

```julia
ERROR: NetCDF file /home/inna/Desktop/tes.2018.06.nc does not have a variable named lon 

```

I found a way :

```julia
using NCDatasets, Glob
paths = sort(glob("home/inna/Desktop/tes*.nc","/"))
a=[Dataset(i) for i in paths] # an array with many datasets
a[1]["tmp"][:,:,:] # the first dataset with one specific variable

```

Thanks for your suggestion.

---

<div class="post-metadata">

**Author:** ![Fliks](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fliks/32/2494_2.png) [@Fliks](https://discourse.julialang.org/u/Fliks)\
**Post date:** [November 4, 2021, 8:45pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/7 "2021-11-04T20:45:31Z")

</div>

Can you post the full error message?  
You can also try to load the netcdf file as a Dataset from YAXArray by:

```julia
ds = open_dataset("pathtofile")

```

It would be interesting to see also this result.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [November 21, 2021, 9:45am UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/8 "2021-11-21T09:45:14Z")

</div>

Late to the party, but `RasterSeries` in [Rasters.jl](https://github.com/rafaqz/Rasters.jl) is designed for this use-case. Use the regex from @Fliks, then:

```julia
using Rasters
series = RasterSeries(filelist, Ti(timestamps))
# Retreive something
series[Ti=At(DateTime(2001, 1, 1)]

```

If you need to join the files into a single `Raster` variable with a time dimension, use `Rasters.combine`:

```julia
combined_raster = combine(series, Ti)
write("combined.nc", combined_raster)

```

You can do the same thing with a series of `tif` or whatever, and still `write` to netcdf.

---

<div class="post-metadata">

**Author:** ![tcarion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tcarion/32/27830_2.png) [@tcarion](https://discourse.julialang.org/u/tcarion)\
**Post date:** [December 21, 2021, 2:54pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/9 "2021-12-21T14:54:56Z")

</div>

Hi!

Is that solution supposed to work with an arbitrary dimension? Let’s say I have 2 files `member0.nc` and `member1.nc` that I want to combine along the `mydim = Dim{:member}([0, 1])`. I’m doing:

```julia
series = RasterSeries(files, mydim)
combined_raster = Rasters.combine(series, Dim{:member})

```

When doing so, I’m getting a strange

```julia
LoadError: BoundsError: attempt to access NTuple{7, Int64} at index [8]

```

Is it supposed to work and if so, would you have any idea what I’m doing wrong? Thanks!

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [December 21, 2021, 3:12pm UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/10 "2021-12-21T15:12:53Z")

</div>

Hi @tcarion, I’m not sure what is happening there, and don’t really have enough information to reproduce. Please make a github issue for this for Rasters.jl, and include a link to the files and full stack trace of the error, and I’ll help you resolve it.

If your MWE includes all of this as a working script (including downloading the file) I’ll be able to help more easily.

---

<div class="post-metadata">

**Author:** ![tcarion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tcarion/32/27830_2.png) [@tcarion](https://discourse.julialang.org/u/tcarion)\
**Post date:** [December 22, 2021, 7:52am UTC](https://discourse.julialang.org/t/read-multi-nc-files-along-new-dimension/70696/11 "2021-12-22T07:52:37Z")

</div>

Hi @Raf, thank you, the issue is open:  
[https://github.com/rafaqz/Rasters.jl/issues/234](https://github.com/rafaqz/Rasters.jl/issues/234)