Can you explain what is AD?
AD is automatic differentation.
ForwardDiff.jl is robust, while Zygote.jl is new and exciting – much faster for large dimensional inputs, and also solves one of
ForwardDiff's major downsides: that it requires code to have been written in a generic fashion.
Eventually, we’ll have Capstan.jl too. It will probably be the default.
The last major ones that springs to mind are AutoGrad.jl, which is used by Knet.jl, and ReverseDiffSparse which has now been integrated into JuMP.jl. There are other libraries.
I encourage you to give it a try!
Definitely feels like magic (to me).
What is a difference between JIT and AOT (ahead of time compiler?)? Or maybe it is wrong place for discussion on this topic?
JIT is like Java, where a compiler can use runtime information to optimize “hot” code.
Julia’s compilation is more like
C++, except Julia’s compiler procrastinates until the first time you call a function with the given combination of argument input types. When you do that, it compiles a version of the function specialized for those argument types in more or less the same way
C++ would using LLVM.