I’m not sure if you are correct. I believe the dims argument for fft, rfft is intended for exactly that case:
rfft performs a length(dims)-dimensional Fourier transform over the dims dimensions of the N-dimensional array.
rfft(A [, dims])
Multidimensional FFT of a real array A, exploiting the fact that the transform has conjugate symmetry in order to save
roughly half the computational time and storage costs compared with fft. If A has size (n_1, ..., n_d), the result has
size (div(n_1,2)+1, ..., n_d).
The optional dims argument specifies an iterable subset of one or more dimensions of A to transform, similar to fft.
Instead of (roughly) halving the first dimension of A in the result, the dims[1] dimension is (roughly) halved in the
same way.