Well, that’s part of the whack-a-mole dynamic I mentioned. Here it affects mostly compile times, (probably because a static block of code is guaranteed in cudnn, but I haven’t specifically checked), but that’s just an incidental benefit of this codepath which is a best case scenario.
My broader point is that this is not really the “fault” (I don’t like that word because it can connote a moral valence which doesn’t exist here), of the flux devs, but is something to be expected with the language design and problem domain.
Further regarding the benefits of dynamism in Julia, I was very kindly pointed to the following excerpts from Jeff’s thesis:
