[ANN] Flux v0.10

We are delighted to announce the release of Flux v0.10. This is a huge release including the work of many contributors; the major development is that Flux now uses Zygote as its default backend for gradients, removing many warts from Flux’s interface and making a much larger class of Julia programs differentiable by default. See the news file for more information on other changes.

To try it out, get Julia 1.3 and ]add Flux (or ]up Flux if you have a previous version installed already) from a REPL. Since this is such a big change, some breakage and regressions are inevitable; some users may want to ]add Flux@0.9 to keep to the stable current release. We’d love to have feedback on the experience through the usual channels.

We would like to thank everyone involved in helping with getting this far, through testing, issues and patches. Flux’s users have been doing some really cool things with differentiable programming already, and we’d love to hear what folks are up to. Enjoy!

– Dhairya, Mike, and the Flux community

53 Likes

I’m trying out the second example but the gradient doesn’t change if I change the loss function, seems a bit weird:

julia> gs[W]
2Γ—5 Array{Float64,2}:
 0.567762  0.753178  0.520799  1.05333  0.337904
 3.84924   5.21601   3.50305   7.42851  2.15487

julia> function loss(x, y)
           yΜ‚ = predict(x)
           1
       end
loss (generic function with 1 method)

julia> gs = gradient(() -> loss(x, y), params(W, b))
Grads(...)

julia> gs[W]
2Γ—5 Array{Float64,2}:
 0.567762  0.753178  0.520799  1.05333  0.337904
 3.84924   5.21601   3.50305   7.42851  2.15487
1 Like

Forgot to mention in the original post; we strongly recommend getting Julia 1.3 in order to use this release of Flux. Although we support earlier Julia versions, 1.3 has a fix for this exact issue, which can cause some confusion otherwise.

Thanks for the reminder @jonathanBieler!

8 Likes

Why not put a lower bound on the Julia version, or some warning and workaround in the docs at least ? Otherwise people are gonna get in trouble.

4 Likes

Since Julia 1.0 is the LTS release, we’re going to continue supporting it as far as possible; and this is really a usability issue rather than a functional one.

There is a note on the Zygote readme about this with a workaround for earlier versions, but I agree we should add a note to the docs.

5 Likes

Using Flux throws some CUDA related errors for me. I have Julia 1.3 on a Windows 10 machine without GPU.

REPL Summary
julia> versioninfo()
Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-4310M CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)

(v1.3) pkg> add Flux
  Updating registry at `C:\Users\Y2CC\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed URIParser ──────────── v0.4.0
 Installed CUDAapi ────────────── v2.0.0
 Installed Requires ───────────── v0.5.2
 Installed FillArrays ─────────── v0.8.2
 Installed DiffRules ──────────── v0.1.0
 Installed BinDeps ────────────── v0.8.10
 Installed Colors ─────────────── v0.9.6
 Installed Flux ───────────────── v0.10.0
 Installed Media ──────────────── v0.5.0
 Installed TimerOutputs ───────── v0.5.3
 Installed MacroTools ─────────── v0.5.2
 Installed NNlib ──────────────── v0.6.0
 Installed Parsers ────────────── v0.3.10
 Installed CategoricalArrays ──── v0.7.3
 Installed ForwardDiff ────────── v0.10.7
 Installed CUDAnative ─────────── v2.5.5
 Installed AbstractTrees ──────── v0.2.1
 Installed DiffResults ────────── v0.0.4
 Installed VersionParsing ─────── v1.1.3
 Installed ColorTypes ─────────── v0.8.0
 Installed AbstractFFTs ───────── v0.4.1
 Installed CodecZlib ──────────── v0.6.0
 Installed NaNMath ────────────── v0.3.3
 Installed LLVM ───────────────── v1.3.2
 Installed Conda ──────────────── v1.3.0
 Installed CuArrays ───────────── v1.4.7
 Installed ZygoteRules ────────── v0.2.0
 Installed FFTW ───────────────── v1.0.1
 Installed CEnum ──────────────── v0.2.0
 Installed SpecialFunctions ───── v0.8.0
 Installed BinaryProvider ─────── v0.5.8
 Installed StaticArrays ───────── v0.12.1
 Installed CommonSubexpressions ─ v0.2.0
 Installed FixedPointNumbers ──── v0.6.1
 Installed Juno ───────────────── v0.7.2
 Installed CUDAdrv ────────────── v4.0.4
 Installed IRTools ────────────── v0.3.0
 Installed Adapt ──────────────── v1.0.0
 Installed GPUArrays ──────────── v2.0.0
 Installed TranscodingStreams ─── v0.9.5
 Installed Zygote ─────────────── v0.4.1
 Installed ZipFile ────────────── v0.8.3
 Installed DataStructures ─────── v0.17.6
  Updating `C:\Users\Y2CC\.julia\environments\v1.3\Project.toml`
  [587475ba] + Flux v0.10.0
  Updating `C:\Users\Y2CC\.julia\environments\v1.3\Manifest.toml`
  [621f4979] + AbstractFFTs v0.4.1
  [1520ce14] + AbstractTrees v0.2.1
  [79e6a3ab] + Adapt v1.0.0
  [9e28174c] + BinDeps v0.8.10
  [b99e7846] ↑ BinaryProvider v0.5.7 β‡’ v0.5.8
  [fa961155] + CEnum v0.2.0
  [3895d2a7] + CUDAapi v2.0.0
  [c5f51814] + CUDAdrv v4.0.4
  [be33ccc6] + CUDAnative v2.5.5
  [324d7699] ↑ CategoricalArrays v0.7.1 β‡’ v0.7.3
  [944b1d66] + CodecZlib v0.6.0
  [3da002f7] + ColorTypes v0.8.0
  [5ae59095] + Colors v0.9.6
  [bbf7d656] + CommonSubexpressions v0.2.0
  [8f4d0f93] + Conda v1.3.0
  [3a865a2d] + CuArrays v1.4.7
  [864edb3b] ↑ DataStructures v0.17.5 β‡’ v0.17.6
  [163ba53b] + DiffResults v0.0.4
  [b552c78f] + DiffRules v0.1.0
  [7a1cc6ca] + FFTW v1.0.1
  [1a297f60] + FillArrays v0.8.2
  [53c48c17] + FixedPointNumbers v0.6.1
  [587475ba] + Flux v0.10.0
  [f6369f11] + ForwardDiff v0.10.7
  [0c68f7d7] + GPUArrays v2.0.0
  [7869d1d1] + IRTools v0.3.0
  [e5e0dc1b] + Juno v0.7.2
  [929cbde3] + LLVM v1.3.2
  [1914dd2f] + MacroTools v0.5.2
  [e89f7d12] + Media v0.5.0
  [872c559c] + NNlib v0.6.0
  [77ba4419] + NaNMath v0.3.3
  [69de0a69] ↑ Parsers v0.3.7 β‡’ v0.3.10
  [ae029012] + Requires v0.5.2
  [276daf66] + SpecialFunctions v0.8.0
  [90137ffa] + StaticArrays v0.12.1
  [a759f4b9] + TimerOutputs v0.5.3
  [3bb67fe8] + TranscodingStreams v0.9.5
  [30578b45] + URIParser v0.4.0
  [81def892] + VersionParsing v1.1.3
  [a5390f91] + ZipFile v0.8.3
  [e88e6eb3] + Zygote v0.4.1
  [700de1a5] + ZygoteRules v0.2.0
  [9abbd945] + Profile
  Building CodecZlib ───────→ `C:\Users\Y2CC\.julia\packages\CodecZlib\5t9zO\deps\build.log`
  Building Conda ───────────→ `C:\Users\Y2CC\.julia\packages\Conda\kLXeC\deps\build.log`
  Building FFTW ────────────→ `C:\Users\Y2CC\.julia\packages\FFTW\MJ7kl\deps\build.log`
  Building SpecialFunctions β†’ `C:\Users\Y2CC\.julia\packages\SpecialFunctions\ne2iw\deps\build.log`
  Building ZipFile ─────────→ `C:\Users\Y2CC\.julia\packages\ZipFile\oD4uG\deps\build.log`

julia> using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
ERROR: LoadError: LoadError: Could not find libcublas
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] top-level scope at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\blas\CUBLAS.jl:23
 [3] include at .\boot.jl:328 [inlined]
 [4] include_relative(::Module, ::String) at .\loading.jl:1105
 [5] include at .\Base.jl:31 [inlined]
 [6] include(::String) at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:1
 [7] top-level scope at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:39
 [8] include at .\boot.jl:328 [inlined]
 [9] include_relative(::Module, ::String) at .\loading.jl:1105
 [10] include(::Module, ::String) at .\Base.jl:31
 [11] top-level scope at none:2
 [12] eval at .\boot.jl:330 [inlined]
 [13] eval(::Expr) at .\client.jl:425
 [14] top-level scope at .\none:3
in expression starting at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\blas\CUBLAS.jl:16
in expression starting at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:39
ERROR: LoadError: Failed to precompile CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae] to C:\Users\Y2CC\.julia\compiled\v1.3\CuArrays\7YFE0_48WYT.ji.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1283
 [3] _require(::Base.PkgId) at .\loading.jl:1024
 [4] require(::Base.PkgId) at .\loading.jl:922
 [5] require(::Module, ::Symbol) at .\loading.jl:917
 [6] include at .\boot.jl:328 [inlined]
 [7] include_relative(::Module, ::String) at .\loading.jl:1105
 [8] include(::Module, ::String) at .\Base.jl:31
 [9] top-level scope at none:2
 [10] eval at .\boot.jl:330 [inlined]
 [11] eval(::Expr) at .\client.jl:425
 [12] top-level scope at .\none:3
in expression starting at C:\Users\Y2CC\.julia\packages\Flux\oX9Pi\src\Flux.jl:24
ERROR: Failed to precompile Flux [587475ba-b771-5e3f-ad9e-33799f191a9c] to C:\Users\Y2CC\.julia\compiled\v1.3\Flux\QdkVy_48WYT.ji.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1283
 [3] _require(::Base.PkgId) at .\loading.jl:1024
 [4] require(::Base.PkgId) at .\loading.jl:922
 [5] require(::Module, ::Symbol) at .\loading.jl:917
2 Likes

Thank you! This is very exciting. :rocket:

I wonder why you chose 0.10 as the name of the release instead of 1. :smile: Usually after 0.9, 1 comes not 0.10 (=0.1).

2 Likes

Known issue with the latest CuArrays on Windows, new version with a fix is incoming: https://github.com/JuliaRegistries/General/pull/6025
Another Windows fix also incoming: https://github.com/JuliaGPU/CUDAnative.jl/pull/507

5 Likes

0.10 is not the same as 0.1, they just look similar (thanks to base-10 arithmetic). 1.0 should come when it’s the right time, not just because it just happens to look like there are no more numbers left :slightly_smiling_face:

6 Likes

Yep, it’s version numbers not decimals:

julia> 0.10>0.1
false

julia> v"0.10">v"0.1"
true
3 Likes

This at least an ignorant question, maybe stupid - in Julia 1.3, I type ]up Flux and the output info looks as I’d expect.

However when typing status I still see [587475ba] Flux v0.9.0 listed - so I guess it is not really updated? Am I missing something? I also tried removing and then adding Flux again and it still list version 0.9.

Also, is there a way to have Flux print its current version number?

Do you have, by chance, any packages installed that require an older version of FFTW? That was the problem for me, since that was holding back Zygote, which held back Flux. You can try ]add Flux@0.10, then you should get an error message, telling you which packages cause problems. It’s not a great experience right now, debugging these problems, we really need a better solution in Pkg for this.

2 Likes

Thanks - the version specification (add Flux@0.10) is useful to know. I have the same issue as you with FFTW and Zygote.

As I understand your comment, there is some package that is requiring an old version of FFTW - is there a trick to find out what package that is?

It looks like FFTW is on version 1.1.0, but my installed version is 0.3.

Could you post the output you’re getting? As far as I know, there’s really no convenient way to check this, so for me it was more or less trial and error.

Here is the output when I try to add current Flux:

(v1.3) pkg> add Flux@0.10
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Zygote [e88e6eb3]:
 Zygote [e88e6eb3] log:
 β”œβ”€possible versions are: [0.1.0, 0.2.0, 0.3.0-0.3.4, 0.4.0-0.4.1] or uninstalled
 β”œβ”€restricted by compatibility requirements with Flux [587475ba] to versions: 0.4.0-0.4.1
 β”‚ └─Flux [587475ba] log:
 β”‚   β”œβ”€possible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0] or uninstalled
 β”‚   └─restricted to versions 0.10 by an explicit requirement, leaving only versions 0.10.0
 └─restricted by compatibility requirements with FFTW [7a1cc6ca] to versions: [0.1.0, 0.2.0, 0.3.0-0.3.4] or uninstalled β€” no versions left
   └─FFTW [7a1cc6ca] log:
     β”œβ”€possible versions are: [0.1.0-0.1.2, 0.2.0-0.2.4, 0.3.0, 1.0.0-1.0.1, 1.1.0] or uninstalled
     └─restricted to versions 0.3.0 by an explicit requirement, leaving only versions 0.3.0

Try upgrading FFTW first or try run add Flux@0.10 FFTW

I did try to update FFTW, but it is stuck at 0.3. I just tried add Flux@0.10 FFTW and it gives the same output as I posted before :slightly_frowning_face:

Ah, it took me a while but I just tried add FFTW@1.1.0 and it tells me now that ArrayFire is the culprit.

I can delete ArrayFire for now, as I’m not using it too actively, but it was helpful for some cases (I’m testing a lot on a MacBook with Radeon Pro 555X not nVidia).

any good suggestions for simple access to GPU to replace ArrayFire?

I did https://gist.github.com/KristofferC/7c8048e2ee9aaa465966caeb98c1446a#file-packages_holding_back-md a while ago and indeed, ArrayFire is shown to hold back FFTW.