Yes, I now better-see the structure that’s happening here. Your F0...1 functions slightly resemble applications of the chain rule. Is there some algebraic simplification that can be made? I’m assuming not but figured I’d remark. Perhaps there isn’t anything clever to be done regarding the differentiation itself.
It looks as if you might be calling the same functions with repeated arguments many times? If that’s the case, there are several packages for performing memoization (I can’t speak to the comparative advantages of each, so won’t recommend one specifically). You could also restructure your code to simply save these results rather than call them repeatedly, but that might be less flexible and extensible.
You might at least be able to cut down some of the allocation/gc costs with a strategic use of StaticArrays in only some positions (maybe just for x and F0? Maybe force the return value of one of the higher order functions into a normal Array by wrapping the result in a call to Matrix(...) – assuming that the issue was that SMatrix tries to propagate to too-high orders. But I’m not certain this will provide much benefit.
In any case, I’m just about out of ideas. Perhaps someone else will have something clever to offer.