Knet vs MXNet for programmer new to ML

[MikeInnes]

@osofr

using Flux on GPU with a custom loss that takes something to a power > 2.0, i.e., x^3.0 or x^5.0. Good luck with that. How about a loss that requires you to simulate from some non-uniform distribution, i.e., normal? Good luck with that as well. How about using Float32 instead of Float64 (gives a big boost in performance on GPUs)? Nope, no can do.

I’d love to have issues for all these things, if you have time. We know there’s plenty of work to do in many of these cases, but it always helps to have people poke these things so we can track and prioritise. Some of them may also be simple setup issues that we can fix easily; e.g. Float32 support should certainly not be a problem. At this stage, all the foundational stuff should work well and be reasonably performant on GPUs, and if it’s not that’s a bug.

@osofr

take a look at the Flux implementation of backprop with Adam optimizer – there is so much Julia magic there it hurts.

Optimisers are actually the part of Flux’s interface that I’m least happy with right now (which is largely why they are not documented beyond basic usage). I’m not happy that they are magical and want to redesign something that’s simple and powerful.