Elegant way to implement a rotating convolution filter without using `rot90` etc

General Usage
1

I am looking to implement a simple rotational convolutional filter to be used with Flux.jl see post describing the filter deep learning - Does a rotational convolutional filter exist in neural networks? - Data Science Stack Exchange

Basically, instead of sweeping across left to right and up and down like a normal convolutional filter, I want to instead rotate the filter as well. So I can apply it to a 4x4 grid and end up with a 2x2 grid. The 4x4 grid is intended to be the game 2048.

using Flux
bb = rand(Int8, 4, 4) # the 2048 game board
weights = rand(Float32, 2, 2)

function transpose_conv(bb)    
    sum(rotl90(@view(bb[1:2, 3:4])) .* weights) # see if I can differentiate through the top right corner
end

gs = gradient(()->transpose_conv(bb), params(weights))

gs[weights]

The above will give error as ERROR: Mutating arrays is not supported -- called setindex!(::Matrix{Int8},

So seems like Zygote.jl doesn’t like rotl90 as well.

Is there a better way to implement this? The best I can think of is to just write it out in full. But I wish to implement group CNN more generally which can do rotation.