Does automatic differentiation work for function with discrete argument?

But the function you are dealing with is f(\theta) = g(H(X \mid \theta), X) which in general will not be continuous(ly differentiable) if H isn’t, even if g is.

Depending on how bad things are, stochastic (eg simulated annealing) or robust deterministic derivative-free optimization methods are used in these cases. See

1 Like