For the MWE above, I would strongly consider V = exp ∘ sin ∘ cos (or V = ∘(exp, sin, cos)) if it’s just a line you’re writing and changing manually. But ∘ won’t work well for long or dynamic compositions.
For long or dynamic compositions, I’ll recommend either accepting instability (the cost of dynamic dispatch once per layer is modest compared to the calculation within a layer of decent size) or using FunctionWrappers.jl.
For example, if you define your activation functions like f = FunctionWrapper{Float64, Tuple{Float64}}(tanh) they will all have the common type FunctionWrapper{Float64, Tuple{Float64}} (notice: unrelated to tanh) and so your entire array will be made of identically-typed layer{FunctionWrapper{Float64, Tuple{Float64}} so entirely stable.
Initially, I would experiment with just accepting the instability and benchmarking to see if the cost is actually significant. You can probably mitigate a lot of the cost with function barriers or the occasional type annotation, if it’s relevant. FunctionWrappers looks like it would solve your problem, but is a little involved so is not my initial recommendation.