Case study: Speeding up a logistic regression with RHS prior (Turing vs Numpyro) - any tricks I'm missing?

I’m probably not the most qualified person to answer this, but I think it’s well described here, because the Slack conversation has disappeared.
My take is that it’s possible thanks to amazing @sethaxen and his snippet (see the link above), but it’s non-trivial (because Bijectors require the same dimensions, so we need to use TransformedVariables.jl and go to a lower-level)