If the difference is so substantial, it might be interesting to add it.
I will add this into the mix for testing speed.
SimpleChains.jl
This seems like a good option. If I can just design layers in a similar way, with the only limitation being GPU support and I can still optimise then this would be ideal. I will test this, are the docs the best place for picking this up.
Broadcasting identity is not considered a no-op, nor is broadcasting adding false.
Personally not sure what this means? How should one use these bits of information?