Training Neural SDEs with Mutating Arrays

Specific Domains Machine Learning
1

I am writing a training function that requires a loop as my loss is the difference between call prices which I need to calculate for various strikes using a loop. However, this results in a mutating array, how do I get around this?

function predict_neuralsde(p, u = S₀) 
  return Array(Pre_NSDE(u, p)) # Returns an array of stock prices/SDE solutions at each time point. 
end

function loss_neuralsde(p; n = 10000)
u = repeat(reshape(S₀, :, 1), 1, n)
stock = predict_neuralsde(p,u)[1,:,end]
    for i = 1:length(K)
NSDE_call[i] = exp(-r*tspan[2]).*mean(maximum([terminal_values .- K[i] zeros(size(terminal_values))], dims = 2))
end 
NSDE_call
    
loss = sum(abs2, NSDE_call - BS_Price')  
 return loss , NSDE_call  
end  

opt = ADAM(0.025)

adtype = Optimization.AutoZygote()
optf = Optimization.OptimizationFunction((x,p) -> loss_neuralsde(x, n=10000), adtype)
optprob = Optimization.OptimizationProblem(optf, Pre_NSDE.p)
result1 = Optimization.solve(optprob, opt, callback = callback, maxiters = 200)  # 200 iterations of training.

The error message is as follows:

Mutating arrays is not supported -- called setindex!(Matrix{Float64}, ...)
This error occurs when you ask Zygote to differentiate operations that change
the elements of arrays in place (e.g. setting values with x .= ...)
2

Unfortunately it’s difficult to help without a MWE, see Please read: make it easier to help you for more.

One thing to test while you’re working on that MWE: instead of using a loop here and assigning to some array, just use map or an array comprehension over the range. It should be just as fast and much more AD-friendly to boot.

3

Quadruplicate of Training with Mutating Arrays - #2 by ChrisRackauckas, answered there. Please don’t quadruple post (Training Neural SDEs with Mutating Arrays · Issue #785 · SciML/DiffEqFlux.jl · GitHub).

1 Like