I don’t know mathematical optimization, and this might be a tremendously dumb idea, but I couldn’t get it out of my head after working on a project where I was fitting a surrogate model to the outputs from a very expensive simulation. So I made this thing for fun:

Not going to register it or anything, but I’m happy for feedback/discussions/links to related papers.

Basically, if fitting, taking the gradient of your model with respect to its parameters and optimizing on that gradient are all orders-of-magnitude cheaper operations than sampling, it might make sense to use your current model to inform the choice of sample points. Big drawback: It requires you to already be fairly confident in the general quality of your model, so it probably doesn’t hold up for science.