I’m training a neural network with flux and outputting a probability distribution (ie. probability bins for the value of a continuous variable). The output layer is outputting the values for the bins which I then normalize (sum to one, or softmax). That much is working. Now, I want to smooth it before calculating the loss during training, which happens on the gpu (so avoid scalar indexing). Are there common strategies for smoothing like that?

In particular, I want it to “fill in the 0’s” effectively. So, for a distribution like [0,1,0,3,0,5], it would smooth to something resembling [0,1,2,3,4,5] (not exactly, just trying to illustrate the “fill in the 0’s” part).

I thought maybe a convolution with a flat vector (ie. [.2,.2,.2,.2,.2]) would do that. The convolution in Flux appears to be image specific (the doc seems to assume multiple dimensions). When I tried the conv function from DSP.jl, it appears to be incompatible with Zygote:

`Compiling Tuple{typeof(lock), CUDA.APIUtils.var"#10#13"{CUDA.APIUtils.var"#9#12", CUDA.APIUtils.HandleCache{Tuple{CUDA.CuContext, CUDA.CUFFT.cufftType_t, Tuple{Vararg{Int64, N}} where N, Any}, Int32}, Tuple{CUDA.CuContext, CUDA.CUFFT.cufftType_t, Tuple{Int64, Int64}, Tuple{Int64, Int64}}}, ReentrantLock}: try/catch is not supported.`

Any suggestions?