The irfft call internally creates a plan and thus is also not thread safe. You can just create one forward and one inverse plan and share them between threads.
(On a separate note, for performance I would recommend preallocating the output array (per thread) and executing the plan in-place with mul!.)