We are delighted to announce the release of Flux v0.10. This is a huge release including the work of many contributors; the major development is that Flux now uses Zygote as its default backend for gradients, removing many warts from Fluxβs interface and making a much larger class of Julia programs differentiable by default. See the news file for more information on other changes.
To try it out, get Julia 1.3 and ]add Flux (or ]up Flux if you have a previous version installed already) from a REPL. Since this is such a big change, some breakage and regressions are inevitable; some users may want to ]add Flux@0.9 to keep to the stable current release. Weβd love to have feedback on the experience through the usual channels.
We would like to thank everyone involved in helping with getting this far, through testing, issues and patches. Fluxβs users have been doing some really cool things with differentiable programming already, and weβd love to hear what folks are up to. Enjoy!
β Dhairya, Mike, and the Flux community
58 Likes
Iβm trying out the second example but the gradient doesnβt change if I change the loss function, seems a bit weird:
julia> gs[W]
2Γ5 Array{Float64,2}:
0.567762 0.753178 0.520799 1.05333 0.337904
3.84924 5.21601 3.50305 7.42851 2.15487
julia> function loss(x, y)
yΜ = predict(x)
1
end
loss (generic function with 1 method)
julia> gs = gradient(() -> loss(x, y), params(W, b))
Grads(...)
julia> gs[W]
2Γ5 Array{Float64,2}:
0.567762 0.753178 0.520799 1.05333 0.337904
3.84924 5.21601 3.50305 7.42851 2.15487
1 Like
Forgot to mention in the original post; we strongly recommend getting Julia 1.3 in order to use this release of Flux. Although we support earlier Julia versions, 1.3 has a fix for this exact issue, which can cause some confusion otherwise.
Thanks for the reminder @jonathanBieler !
8 Likes
Why not put a lower bound on the Julia version, or some warning and workaround in the docs at least ? Otherwise people are gonna get in trouble.
4 Likes
Since Julia 1.0 is the LTS release, weβre going to continue supporting it as far as possible; and this is really a usability issue rather than a functional one.
There is a note on the Zygote readme about this with a workaround for earlier versions, but I agree we should add a note to the docs.
6 Likes
Using Flux throws some CUDA related errors for me. I have Julia 1.3 on a Windows 10 machine without GPU.
REPL Summary
julia> versioninfo()
Julia Version 1.3.0
Commit 46ce4d7933 (2019-11-26 06:09 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i5-4310M CPU @ 2.70GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
(v1.3) pkg> add Flux
Updating registry at `C:\Users\Y2CC\.julia\registries\General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions...
Installed URIParser ββββββββββββ v0.4.0
Installed CUDAapi ββββββββββββββ v2.0.0
Installed Requires βββββββββββββ v0.5.2
Installed FillArrays βββββββββββ v0.8.2
Installed DiffRules ββββββββββββ v0.1.0
Installed BinDeps ββββββββββββββ v0.8.10
Installed Colors βββββββββββββββ v0.9.6
Installed Flux βββββββββββββββββ v0.10.0
Installed Media ββββββββββββββββ v0.5.0
Installed TimerOutputs βββββββββ v0.5.3
Installed MacroTools βββββββββββ v0.5.2
Installed NNlib ββββββββββββββββ v0.6.0
Installed Parsers ββββββββββββββ v0.3.10
Installed CategoricalArrays ββββ v0.7.3
Installed ForwardDiff ββββββββββ v0.10.7
Installed CUDAnative βββββββββββ v2.5.5
Installed AbstractTrees ββββββββ v0.2.1
Installed DiffResults ββββββββββ v0.0.4
Installed VersionParsing βββββββ v1.1.3
Installed ColorTypes βββββββββββ v0.8.0
Installed AbstractFFTs βββββββββ v0.4.1
Installed CodecZlib ββββββββββββ v0.6.0
Installed NaNMath ββββββββββββββ v0.3.3
Installed LLVM βββββββββββββββββ v1.3.2
Installed Conda ββββββββββββββββ v1.3.0
Installed CuArrays βββββββββββββ v1.4.7
Installed ZygoteRules ββββββββββ v0.2.0
Installed FFTW βββββββββββββββββ v1.0.1
Installed CEnum ββββββββββββββββ v0.2.0
Installed SpecialFunctions βββββ v0.8.0
Installed BinaryProvider βββββββ v0.5.8
Installed StaticArrays βββββββββ v0.12.1
Installed CommonSubexpressions β v0.2.0
Installed FixedPointNumbers ββββ v0.6.1
Installed Juno βββββββββββββββββ v0.7.2
Installed CUDAdrv ββββββββββββββ v4.0.4
Installed IRTools ββββββββββββββ v0.3.0
Installed Adapt ββββββββββββββββ v1.0.0
Installed GPUArrays ββββββββββββ v2.0.0
Installed TranscodingStreams βββ v0.9.5
Installed Zygote βββββββββββββββ v0.4.1
Installed ZipFile ββββββββββββββ v0.8.3
Installed DataStructures βββββββ v0.17.6
Updating `C:\Users\Y2CC\.julia\environments\v1.3\Project.toml`
[587475ba] + Flux v0.10.0
Updating `C:\Users\Y2CC\.julia\environments\v1.3\Manifest.toml`
[621f4979] + AbstractFFTs v0.4.1
[1520ce14] + AbstractTrees v0.2.1
[79e6a3ab] + Adapt v1.0.0
[9e28174c] + BinDeps v0.8.10
[b99e7846] β BinaryProvider v0.5.7 β v0.5.8
[fa961155] + CEnum v0.2.0
[3895d2a7] + CUDAapi v2.0.0
[c5f51814] + CUDAdrv v4.0.4
[be33ccc6] + CUDAnative v2.5.5
[324d7699] β CategoricalArrays v0.7.1 β v0.7.3
[944b1d66] + CodecZlib v0.6.0
[3da002f7] + ColorTypes v0.8.0
[5ae59095] + Colors v0.9.6
[bbf7d656] + CommonSubexpressions v0.2.0
[8f4d0f93] + Conda v1.3.0
[3a865a2d] + CuArrays v1.4.7
[864edb3b] β DataStructures v0.17.5 β v0.17.6
[163ba53b] + DiffResults v0.0.4
[b552c78f] + DiffRules v0.1.0
[7a1cc6ca] + FFTW v1.0.1
[1a297f60] + FillArrays v0.8.2
[53c48c17] + FixedPointNumbers v0.6.1
[587475ba] + Flux v0.10.0
[f6369f11] + ForwardDiff v0.10.7
[0c68f7d7] + GPUArrays v2.0.0
[7869d1d1] + IRTools v0.3.0
[e5e0dc1b] + Juno v0.7.2
[929cbde3] + LLVM v1.3.2
[1914dd2f] + MacroTools v0.5.2
[e89f7d12] + Media v0.5.0
[872c559c] + NNlib v0.6.0
[77ba4419] + NaNMath v0.3.3
[69de0a69] β Parsers v0.3.7 β v0.3.10
[ae029012] + Requires v0.5.2
[276daf66] + SpecialFunctions v0.8.0
[90137ffa] + StaticArrays v0.12.1
[a759f4b9] + TimerOutputs v0.5.3
[3bb67fe8] + TranscodingStreams v0.9.5
[30578b45] + URIParser v0.4.0
[81def892] + VersionParsing v1.1.3
[a5390f91] + ZipFile v0.8.3
[e88e6eb3] + Zygote v0.4.1
[700de1a5] + ZygoteRules v0.2.0
[9abbd945] + Profile
Building CodecZlib ββββββββ `C:\Users\Y2CC\.julia\packages\CodecZlib\5t9zO\deps\build.log`
Building Conda ββββββββββββ `C:\Users\Y2CC\.julia\packages\Conda\kLXeC\deps\build.log`
Building FFTW βββββββββββββ `C:\Users\Y2CC\.julia\packages\FFTW\MJ7kl\deps\build.log`
Building SpecialFunctions β `C:\Users\Y2CC\.julia\packages\SpecialFunctions\ne2iw\deps\build.log`
Building ZipFile ββββββββββ `C:\Users\Y2CC\.julia\packages\ZipFile\oD4uG\deps\build.log`
julia> using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
ERROR: LoadError: LoadError: Could not find libcublas
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] top-level scope at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\blas\CUBLAS.jl:23
[3] include at .\boot.jl:328 [inlined]
[4] include_relative(::Module, ::String) at .\loading.jl:1105
[5] include at .\Base.jl:31 [inlined]
[6] include(::String) at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:1
[7] top-level scope at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:39
[8] include at .\boot.jl:328 [inlined]
[9] include_relative(::Module, ::String) at .\loading.jl:1105
[10] include(::Module, ::String) at .\Base.jl:31
[11] top-level scope at none:2
[12] eval at .\boot.jl:330 [inlined]
[13] eval(::Expr) at .\client.jl:425
[14] top-level scope at .\none:3
in expression starting at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\blas\CUBLAS.jl:16
in expression starting at C:\Users\Y2CC\.julia\packages\CuArrays\7z7MV\src\CuArrays.jl:39
ERROR: LoadError: Failed to precompile CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae] to C:\Users\Y2CC\.julia\compiled\v1.3\CuArrays\7YFE0_48WYT.ji.
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1283
[3] _require(::Base.PkgId) at .\loading.jl:1024
[4] require(::Base.PkgId) at .\loading.jl:922
[5] require(::Module, ::Symbol) at .\loading.jl:917
[6] include at .\boot.jl:328 [inlined]
[7] include_relative(::Module, ::String) at .\loading.jl:1105
[8] include(::Module, ::String) at .\Base.jl:31
[9] top-level scope at none:2
[10] eval at .\boot.jl:330 [inlined]
[11] eval(::Expr) at .\client.jl:425
[12] top-level scope at .\none:3
in expression starting at C:\Users\Y2CC\.julia\packages\Flux\oX9Pi\src\Flux.jl:24
ERROR: Failed to precompile Flux [587475ba-b771-5e3f-ad9e-33799f191a9c] to C:\Users\Y2CC\.julia\compiled\v1.3\Flux\QdkVy_48WYT.ji.
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1283
[3] _require(::Base.PkgId) at .\loading.jl:1024
[4] require(::Base.PkgId) at .\loading.jl:922
[5] require(::Module, ::Symbol) at .\loading.jl:917
2 Likes
Thank you! This is very exciting.
I wonder why you chose 0.10 as the name of the release instead of 1.
2 Likes
maleadt
November 29, 2019, 10:00pm
8
5 Likes
0.10 is not the same as 0.1, they just look similar (thanks to base-10 arithmetic). 1.0 should come when itβs the right time, not just because it just happens to look like there are no more numbers left
8 Likes
mauro3
November 30, 2019, 10:43am
10
Yep, itβs version numbers not decimals:
julia> 0.10>0.1
false
julia> v"0.10">v"0.1"
true
5 Likes
This at least an ignorant question, maybe stupid - in Julia 1.3, I type ]up Flux and the output info looks as Iβd expect.
However when typing status I still see [587475ba] Flux v0.9.0 listed - so I guess it is not really updated? Am I missing something? I also tried removing and then adding Flux again and it still list version 0.9 .
Also, is there a way to have Flux print its current version number?
Do you have, by chance, any packages installed that require an older version of FFTW? That was the problem for me, since that was holding back Zygote, which held back Flux. You can try ]add Flux@0.10, then you should get an error message, telling you which packages cause problems. Itβs not a great experience right now, debugging these problems, we really need a better solution in Pkg for this.
3 Likes
Thanks - the version specification (add Flux@0.10) is useful to know. I have the same issue as you with FFTW and Zygote.
As I understand your comment, there is some package that is requiring an old version of FFTW - is there a trick to find out what package that is?
It looks like FFTW is on version 1.1.0, but my installed version is 0.3.
Could you post the output youβre getting? As far as I know, thereβs really no convenient way to check this, so for me it was more or less trial and error.
Here is the output when I try to add current Flux:
(v1.3) pkg> add Flux@0.10
Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Zygote [e88e6eb3]:
Zygote [e88e6eb3] log:
ββpossible versions are: [0.1.0, 0.2.0, 0.3.0-0.3.4, 0.4.0-0.4.1] or uninstalled
ββrestricted by compatibility requirements with Flux [587475ba] to versions: 0.4.0-0.4.1
β ββFlux [587475ba] log:
β ββpossible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0] or uninstalled
β ββrestricted to versions 0.10 by an explicit requirement, leaving only versions 0.10.0
ββrestricted by compatibility requirements with FFTW [7a1cc6ca] to versions: [0.1.0, 0.2.0, 0.3.0-0.3.4] or uninstalled β no versions left
ββFFTW [7a1cc6ca] log:
ββpossible versions are: [0.1.0-0.1.2, 0.2.0-0.2.4, 0.3.0, 1.0.0-1.0.1, 1.1.0] or uninstalled
ββrestricted to versions 0.3.0 by an explicit requirement, leaving only versions 0.3.0
Try upgrading FFTW first or try run add Flux@0.10 FFTW
I did try to update FFTW, but it is stuck at 0.3. I just tried add Flux@0.10 FFTW and it gives the same output as I posted before
Ah, it took me a while but I just tried add FFTW@1.1.0 and it tells me now that ArrayFire is the culprit.
I can delete ArrayFire for now, as Iβm not using it too actively, but it was helpful for some cases (Iβm testing a lot on a MacBook with Radeon Pro 555X not nVidia).
any good suggestions for simple access to GPU to replace ArrayFire?
Usually after 0.9, 1 comes not 0.10 (=0.1).
