[ANN] Flux v0.10

Ah, it took me a while but I just tried add FFTW@1.1.0 and it tells me now that ArrayFire is the culprit.

I can delete ArrayFire for now, as I’m not using it too actively, but it was helpful for some cases (I’m testing a lot on a MacBook with Radeon Pro 555X not nVidia).

any good suggestions for simple access to GPU to replace ArrayFire?

I did https://gist.github.com/KristofferC/7c8048e2ee9aaa465966caeb98c1446a#file-packages_holding_back-md a while ago and indeed, ArrayFire is shown to hold back FFTW.

Flux is already a helpful package, thanks for that!

I am currently playing with a Flux script that uses Flux 0.8.3 and everything is running to my satisfaction. Now the current Flux version is 0.10 and uses Zygote under Julia 1.3. This is a clear concept change, but o.k…

To test the effects, I loaded the above script and tried to execute it. I get the following error message:

ERROR: LoadError: Can't differentiate = expression
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] loss at C:\Users\guent\documents\Julia Projekte\Nano Transfer\Flux_Rotwein.jl:83 [inlined]
 [3] (::typeof(∂(loss)))(::Float32) at C:\Users\guent\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [4] (::Zygote.var"#153#154"{typeof(∂(loss)),Tuple{Tuple{Nothing,Nothing}}})(::Float32) at C:\Users\guent\.julia\packages\Zygote\8dVxG\src\lib\lib.jl:142....

The model is called in the script …

Flux.train!(loss, params(Modell), Epochen_Iterator, Optimierer)

… and it’s clear, I have to adapt the script! Just what? Is that described somewhere? Because the Flux documentation doesn’t seem to be adapted yet:

If I’m wrong here, has the documentary been adjusted? I think in the documentary I’ll find examples of how models under “Zgote” need to be adapted, right?

I am glad about every hint, thank you!

3 Likes

Awful update, it just broke my code. It even doesn’t work on it’s own example https://raw.githubusercontent.com/FluxML/model-zoo/master/vision/mnist/conv.jl

It says "Can’t differentiate foreigncall expression in expression starting at … ". How can I rely on Flux now?

Are you setting a global variable within your model? If so, no problem, we can add support for this in Zygote. I’m sure we can fix something so that your model works here.

1 Like

As mentioned above, this update is meant to be broadly but not 100% compatible, and if it breaks your code you can (a) tell us and (b) roll back temporarily to make sure you don’t lose work.

We’re aware of the model zoo issues and have updates incoming. If something else is breaking we can help you figure out if it’s a Zygote issue or just something that needs tweaking in your code.

7 Likes

Thank you for your answer! I can’t work on the model at the moment, but I know for sure that I haven’t set a global variable. I use tracked variables as required under 0.8.3. Using Zygote is certainly the right way to avoid duplicate work. It would be ideal, of course, if the Flux user didn’t even notice! :wink:

To be fair pytorch and tensorflow and many other frameworks break things all the time. I would suggest you version control your software and setup properly with fixed version numbers on every piece of software you use. This is the only way you can be sure to experience no breakage. Docker is super useful in this respect. :+1:t2:

1 Like

And to be fair, this is a bump of a major version, i.e. it’s a release that says it will break code.

8 Likes

Are there any performance benchmarks now that Zygote is the back-end?

I hadn’t been following it all that closely, but I know that performance has been a major concern with Flux in the past. I had assumed there wouldn’t be a serious focus on performance until Zygote was ready.

7 Likes

Does Metalhead work with Flux v0.10?

I’ve got:

(v1.3) pkg> status
Status ~/.julia/environments/v1.3/Project.toml
[3a865a2d] CuArrays v1.5.0
[7a1cc6ca] FFTW v1.1.0
[587475ba] Flux v0.10.0
[7869d1d1] IRTools v0.3.0
[6a3955dd] ImageFiltering v0.6.6
[916415d5] Images v0.19.0
[e88e6eb3] Zygote v0.4.1
[700de1a5] ZygoteRules v0.2.0

And when I try:

(v1.3) pkg> add Metalhead
Resolving package versions…
ERROR: Unsatisfiable requirements detected for package Flux [587475ba]:
Flux [587475ba] log:
├─possible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0] or uninstalled
├─restricted to versions 0.10.0 by an explicit requirement, leaving only versions 0.10.0
└─restricted by compatibility requirements with Metalhead [dbeba491] to versions: [0.8.0-0.8.3, 0.9.0] — no versions left
└─Metalhead [dbeba491] log:
├─possible versions are: [0.3.0, 0.4.0] or uninstalled
├─restricted to versions * by an explicit requirement, leaving only versions [0.3.0, 0.4.0]
└─restricted by compatibility requirements with Images [916415d5] to versions: 0.4.0 or uninstalled, leaving only versions: 0.4.0
└─Images [916415d5] log:
├─possible versions are: [0.17.3, 0.18.0, 0.19.0] or uninstalled
└─restricted to versions 0.19.0 by an explicit requirement, leaving only versions 0.19.0

I don’t know if Metalhead would work, but it was blocked from installing with this update recently::

It’s simple to make a PR in the GitHub interface, but I would want to check first, if it would work locally. Is there a simple way to check out overriding the package manager? I couldn’t find how in the docs, no matter how I tried, including what I thought applies:

pkg> develop Metalhead

Sorry for my late reply! I looked at the Flux Docu 0.10, but couldn’t find a clue about “global variables”! I also can’t see what I have to do differently…!
Can you give me a hint?

It’s a tad obscured but this page here links to the Julia performance tips where avoiding global state is described.

Thank you for your answer! I have looked at the site intensively, but unfortunately - also no latent - hint found. Maybe I don’t understand the described concept either.

It is not about performance optimization, but about an abort of the function call. Under Flux 0.8.3 everything still runs to my satisfaction and as written, I can’t see what I have to do differently.

In view of the fact that I understand the documentation correctly, I don’t see any changes for my dense model (similar to the Model Zoo’s iris example). If a “global” variable is expected, I would expect the same in the docu. Or for example in the iris model.

I still get the error message described at the beginning!

2 Likes

@MikeInnes, is there anything new on the subject? As written, I looked at the Iris example of the Model Zoo, but couldn’t find any clue. Am I overseeing anything?

Hello,
Thanks the Flux community for the great job about v0.10.
I have a question:
I programed with Flux.jl v0.9: weight = data.(params(model)) to remove tracking. With the v0.10 it’s still work, however a warning appears with REPL:

Warning: `data(x)` is deprecated, use `x` instead.

What can I do ?

Regards

Just delete all data function calls. It’s not needed anymore.

2 Likes

Just thought I’d post a quick note here. I did some testing on Colab with FluxML and actually got what I think was a reasonably notable workflow going. Not bad for v0.10. I posted it in a tutorial here: https://github.com/GdMacmillan/ml_flux_tutorial/blob/master/ML_Flux_Tutorial.ipynb

6 Likes