# Need help getting started with FluxML

**URL:** <https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [November 17, 2019, 10:55pm UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196 "2019-11-17T22:55:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![cgo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@cgo](https://discourse.julialang.org/u/cgo)\
**Post date:** [November 17, 2019, 10:55pm UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/1 "2019-11-17T22:55:24Z")

</div>

Hi all,

I am just getting started with Flux.  
While the documentation is present, I have a hard time finding some information.  
In particular, these are some questions that came up:

- am I to use Zygote or the Tracker module? Which one is used for the latest Flux, has Flux already transitioned to Zygote?
- are there examples for writing my own “layer”? More precisely, I would like to use a SVD and a derivative that I found in some paper.
- Somewhat more detailed question: I have an operation Diagonal(w)) \* A, where w is an Array and A is an Array (a matrix). This works fine without Flux, but if w is containing tracked values, Julia complains that it does not have an implementation for the multiplication operator (\*).  
I’m sure I’m doing something wrong that is obvious for the Flux expert.

… I know how to get this done in PyTorch, but I would love to try out Flux.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [November 17, 2019, 11:20pm UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/2 "2019-11-17T23:20:08Z")

</div>

> [@cgo](#):
>
> - am I to use Zygote or the Tracker module? Which one is used for the latest Flux, has Flux already transitioned to Zygote?

The current stable release of Flux uses Tracker.  
The master branch of Flux in development uses Zygote.  
I would not recommend trying to use that right now, for someone just starting out.  
(Its just kinda fiddly to setup)

> [@cgo](#):
>
> are there examples for writing my own “layer”? More precisely, I would like to use a SVD and a derivative that I found in some paper.

Docs on that are here:  
[https://fluxml.ai/Flux.jl/stable/models/basics/#Building-Layers-1](https://fluxml.ai/Flux.jl/stable/models/basics/#Building-Layers-1)

> [@cgo](#):
>
> Somewhat more detailed question: I have an operation Diagonal(w)) \* A, where w is an Array and A is an Array (a matrix). This works fine without Flux, but if w is containing tracked values, Julia complains that it does not have an implementation for the multiplication operator (\*).  
> I’m sure I’m doing something wrong that is obvious for the Flux expert.

Can you write up a minimum verifiable example,  
and include that and the full error message in your post.

---

<div class="post-metadata">

**Author:** ![cgo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@cgo](https://discourse.julialang.org/u/cgo)\
**Post date:** [November 18, 2019, 5:26am UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/3 "2019-11-18T05:26:13Z")

</div>

Thanks for answering!  
I saw the basics/building layers documentation; what is unclear to me is how I can supply my own implementation of a derivative. Is that somehow linked to the forward() function? Sorry if this is obvious to some 🙂 I’m really new to Flux and Julia.

As for a an example, I’ll try to de-solder the necessary code from my (messy) experiments.

---

<div class="post-metadata">

**Author:** ![cgo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@cgo](https://discourse.julialang.org/u/cgo)\
**Post date:** [November 18, 2019, 6:34am UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/4 "2019-11-18T06:34:21Z")

</div>

Ok looks like I figured out the part with the Diagonal.  
I was actually using a function somewhere that was explicitly creating an Array{Float32,2} and trying to use tracked float values to fill it. There was no conversion function from tracked to untracked, which I assume makes sense.  
I fixed my function.

Now the actual problem seems to be that I want to write my own layer, where the function itself returns the SVD (or a value computed from it), and the gradient is something that I found in the literature. To put it simply, I need to implement a function and an accompanying derivative so that the AD system (Tracker?) can make use of it when computing gradients.  
How does that work in Tracker, and where would I look to find information about that?

Once again, thanks a lot for your help!

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [November 18, 2019, 9:23am UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/5 "2019-11-18T09:23:26Z")

</div>

> [@cgo](#):
>
> I saw the basics/building layers documentation; what is unclear to me is how I can supply my own implementation of a derivative. Is that somehow linked to the forward() function? Sorry if this is obvious to some 🙂 I’m really new to Flux and Julia.

Its here: [https://fluxml.ai/Flux.jl/stable/internals/tracker/#Custom-Gradients-1](https://fluxml.ai/Flux.jl/stable/internals/tracker/#Custom-Gradients-1)

* * *

Aside: i am super hype about the general idea of putting SVD in NNs,.

---

<div class="post-metadata">

**Author:** ![cgo](https://avatars.discourse-cdn.com/v4/letter/c/b19c9b/32.png) [@cgo](https://discourse.julialang.org/u/cgo)\
**Post date:** [November 25, 2019, 9:47pm UTC](https://discourse.julialang.org/t/need-help-getting-started-with-fluxml/31196/6 "2019-11-25T21:47:53Z")

</div>

Cool thanks; that helped a lot already.

I´m still having some difficulties getting it to work though.  
I figured a few things out, but I can’t find out how  
to concatenate tracked Float numbers into a tracked array.  
Is that possible?  
I have a function that computes scalars which are tracked floats, but I want to fill a tracked array with those numbers.

Another things I was wondering about is this: Is it possible in Flux to concatenate tracked arrays and regular arrays into a new array  
that would then have to be partly tracked and partly untracked?

Cheers  
Christian