I found this thread and I think it’s not accurate. First of all Flux which uses NNLib dispatches to both 2D and 3D API exposed by CUDNN (that’s the case since long time). The 3D version is just not documented but you can pass three dimensional array through LSTM/GRU or any RNN. It’s something CUDNN supports and that is correctly dispatched to. I don’t know about non CUDAs applications.
Secondly, it states in documentation not to use map or broadcast dot operator with reccurent layers. Because order is not guaranteed and you want to send your input in sequencial order.