I adapted Flux.jl to calculate gradinets using multiple threads. The approach is relatively simple, as I divide the minibatch in approximately equal sizes and let each thread calculate the gradient on the minibatch and then I average the gradients on the end. Yet, I have a strong suspicion that there is some memory leaking.
I do not know, how to at the moment reproduce the problem. Can anyone suggest, how to track down the source of the problem?
Does anyone have similar problems?
Thanks for any help in advance.