@dlfivefifty, I didn’t know about Dual! Thanks for the information. For my future reference, here is a way to get the derivative using Dual:
julia> using DualNumbers # ForwardDiff not needed
julia> z₀ = rand(ComplexF64); # arbitrary complex number
julia> f(Dual(z₀,1)).epsilon ≈ f′hand(z₀)
true
As a separate note, it seems that ForwardDiff.jl already has the capability to differentiate holomorphic functions via ForwardDiff.Dual. Then I am not sure why ForwardDiff.derivative() does not support complex functions yet…
In any case, developing the workaround gave me a great opportunity to think about complex analysis again. Hope other people enjoy it!