# \[ANN\] Flux v0.5
**URL:** https://discourse.julialang.org/t/ann-flux-v0-5/9620
**Category:** Machine Learning
**Created:** [March 9, 2018, 8:00pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620 "2018-03-09T20:00:22Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [March 9, 2018, 8:00pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/1 "2018-03-09T20:00:22Z")
Hey all,
I’m pleased to announce the 0.5 release of the [Flux](https://fluxml.github.io/) machine learning library. Here’s an incomplete list of things that have changed since the last announcement.
- Experimental **JIT compilation** work for models, applying optimisations such as pre-allocating memory.
- **Run models in the browser** via [Flux.JS](https://github.com/FluxML/FluxJS.jl)
- Among many GPU performance improvements, **CUDNN integration for RNNs** – RNNs will now be much faster, no changes to your code needed!
- A new and improved **N-dimensional API for convolutions** , whose CPU versions now have **pure-Julia implementations**.
- **Regularisation** of model weights
- Stable APIs for **saving and loading models**
- **Tracked scalars** for more advanced AD use cases
- Much **new functionality and layers** , including the GRU RNN, numerically-stable log versions of softmax and sigmoid, binary cross entropy, permutedims, kronecker product, and much more.
- Many **more models** in the [model zoo](https://github.com/FluxML/model-zoo/)
Thanks to all contributors to Flux!
– Mike
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [March 12, 2018, 5:01pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/2 "2018-03-12T17:01:34Z")
This looks great!
Both tracked scalars and derivatives of `kron` will be useful for something I was trying to do, not involving neural networks at all. So maybe this is a good place to ask: What are your thoughts on using `Flux.Tracker` as a way to do AD more generally?
I’ve learned quite a bit by taking it apart, and it seems good at what I want, right now (once I fix a few bugs!). But I presume that the fact that it’s not its own package implies that I should be wary about its future. I have heard about Cassette.jl don’t understand much about either what this would replace, or when.
---
### Author:  [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [March 12, 2018, 5:20pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/3 "2018-03-12T17:20:16Z")
Please do! It should be pretty effective for any array code in Julia – that’s all ML models are anyway. And any improvements you can make will certainly improve things for everyone.
I’d even be open to splitting it into it’s own package if there’s a good reason to, though I’d hope that there’s no real downside to just importing Flux directly.
If all goes well with Cassette/Capstan, we’d like to replace Flux’s AD with that, but that’s a long way off (perhaps a year or more). For the time being, you can consider the AD to be part of the supported Flux interface and rely on it.
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [March 12, 2018, 6:02pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/4 "2018-03-12T18:02:43Z")
OK, that’s good to hear.
The new `Base.kron` is much neater than my implementation! I’ll see if there are any other bits to contribute back.
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [March 14, 2018, 2:27pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/5 "2018-03-14T14:27:15Z")
One more question. Suppose I want to provide (an approximation to) the gradient of some big slow function `f`, as below. Most of the work of calculating this `f∇(x)` is just calculating `f(x)`. I think this has already been done on the forward pass… is there any way that I can access it, or save it for use by some `f∇(x, fval)`?
```julia
using Flux.Tracker: back, track, grad, TrackedArray
f(x::TrackedArray) = track(f, x) ## scalar output
Flux.Tracker.back(::typeof(f), Δ, x) = back(x, Δ * f∇(x))
xp = param(rand(3,3))
fxp = f(xp)
Flux.back!(fxp)
grad(xp)
```
---
### Author:  [@amellnik](https://discourse.julialang.org/u/amellnik)
#### Post date: [March 14, 2018, 5:05pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/6 "2018-03-14T17:05:53Z")
There’s [https://github.com/simonster/Memoize.jl](https://github.com/simonster/Memoize.jl) and similar but I don’t know if using it could cause issues with AD.
---
### Author:  [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [March 14, 2018, 5:11pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/7 "2018-03-14T17:11:35Z")
Yup, easy peasy, just overload [this method](https://github.com/FluxML/Flux.jl/blob/72f13834f59e4ca0fbbeb5b853a3802a35ecd250/src/tracker/back.jl#L24) instead.
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [March 14, 2018, 5:27pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/8 "2018-03-14T17:27:44Z")
Perfect, thanks!
And so that I don’t forget: `Flux.Tracker.back_(::typeof(f), fval, Δ, x) = back(x, Δ * f∇(x,fval))`
---
### Author:  [@datnamer](https://discourse.julialang.org/u/datnamer)
#### Post date: [March 14, 2018, 10:53pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/9 "2018-03-14T22:53:06Z")
Can I use for loops and arbitrary Julia constructs with flux? How about autodiff with functions from packages not written with this in mind? What are the constraints with doing so?
---
### Author:  [@joshualeond](https://discourse.julialang.org/u/joshualeond)
#### Post date: [March 15, 2018, 2:06am UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/10 "2018-03-15T02:06:12Z")
This package is really great. It’s amazing how close the julia code in the [optimisers](https://github.com/FluxML/Flux.jl/blob/master/src/optimise/optimisers.jl#L52) file is to their arxiv [papers](https://arxiv.org/pdf/1412.6980v8.pdf)! The [regularisation](https://fluxml.github.io/Flux.jl/stable/models/regularisation.html) part of the documentation is great for the same reason. The code is just really welcoming to a beginner.
Just curious what the plans are for parallel CPU training?
---
### Author:  [@improbable22](https://discourse.julialang.org/u/improbable22)
#### Post date: [March 15, 2018, 1:03pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/11 "2018-03-15T13:03:51Z")
Not an expert but loops and branches in what operations you perform on a tracked array should be no problem.
But creating a new array by iteration over elements (e.g. writing your own matrix multiplication with loops) will be problematic. If there’s one step which can’t be handled it is easy to splice a `ForwardDiff.gradient` there, or a hand-written derivative.
---
### Author:  [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [March 15, 2018, 3:53pm UTC](https://discourse.julialang.org/t/ann-flux-v0-5/9620/12 "2018-03-15T15:53:08Z")
Right. Basically, the AD works in terms of high-level array operations (e.g. broadcast, reduce, or linear algebra). As long as a you stick to those it’ll work (even in package code), even if it has crazy control flow, or uses recursion, or whatever. The [treebank](https://github.com/FluxML/model-zoo/blob/1c2c28be223d02e9bb28331805cdc1aaf803864a/treebank/recursive.jl#L16) model is a good example of writing a model as a recursive function.
As @improbable22 said, you’ll have an issue if a function is “lower level” and implements works by looping over array elements, or similar, but in that case you can just tell Flux what the gradient is directly.