How exactly do optimizers reject infinity loss?

[ChrisRackauckas]

@fanny1

I wonder which optimizers will do that? and how exactly do they “reject the steps”? (e.g.: do they simply skip the current step? or do they randomly initialize a new set of parameters?)

They skip the current step and do some line search thing. I know the Optim.jl algorithms support this. I need to dig around to see which others do it well.