# \[ANN\] Lux.jl: Explicitly Parameterized Neural Networks in Julia

**URL:** <https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689>\
**Category:** Package Announcements\
**Tags:** package, announcement, machine-learning\
**Created:** [May 25, 2022, 11:24pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689 "2022-05-25T23:24:01Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 21, 2023, 3:48pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/21 "2023-08-21T15:48:08Z")

</div>

I did a major overhaul of the Lux documentation [https://lux.csail.mit.edu/](https://lux.csail.mit.edu/). I would appreciate feedback regarding the new look and the unified approach. Some improvements:

- Lists in the previous docs were broken due to how the generated markdown was printed (mkdocs seems to have a slightly different way of handling new lines). That is fixed now.
- There were complaints regarding how each docstring was not separated and hence hard to read. Now the docstrings have proper bounding boxes  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/d/ad6489b89b1401b1d7638cd4a109ac3b82dcef7e.png)
- Finally all the docs are in the same place which means they are easy to search!
- The versioning is at least visible now. I couldn’t work it out for mkdocs material previously. Though the versioning is not great (vitepress doesn’t natively support versioning)

Known Issues:

- Only `dev` is live. This is a problem from TagBot which I am looking into.
- Some docstrings have `!!! warning` instead of nice warnings/info, etc blocks. I need to go through all the packages and update their docstrings

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [August 21, 2023, 6:33pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/22 "2023-08-21T18:33:29Z")

</div>

Wow, it looks great!

There is an import error in the first example of this section: [https://lux.csail.mit.edu/dev/tutorials/beginner/Basics/main#Automatic%20Differentiation](https://lux.csail.mit.edu/dev/tutorials/beginner/Basics/main#Automatic%20Differentiation)

 ![Screenshot 2023-08-21 at 20.33.16](https://global.discourse-cdn.com/julialang/original/3X/9/e/9e8e613cc6776da39938aa7a8ff02c7dd0b418d9.png)

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 21, 2023, 10:44pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/23 "2023-08-21T22:44:07Z")

</div>

These look amazing! The homepage is particularly impressive.

One minor complaint I have is that the “Getting Started” page is quite sparse - aside from just getting Julia + Lux installed, we should also show some examples of how to use Lux in practice; if you need to be shown how to install Julia or a package, you probably also need to be shown how to really use that package 🙂

We could show a simple example of creating and training a really basic NN, and then provide links to more advanced or domain-specific examples for a variety of use cases.

---

<div class="post-metadata">

**Author:** ![lazarusA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lazarusa/32/6571_2.png) [@lazarusA](https://discourse.julialang.org/u/lazarusA)\
**Post date:** [August 22, 2023, 7:51am UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/24 "2023-08-22T07:51:21Z")

</div>

Great work! Always inspired by your amazing work. These docs should be the new standard :D.

As for the looks, so far I only have one suggestion:

it would be good to highlight in which section we are currently at, on the right panel. Current behaviour doesn’t show anything.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [August 22, 2023, 8:17am UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/25 "2023-08-22T08:17:58Z")

</div>

![image|50%](https://global.discourse-cdn.com/julialang/original/3X/0/6/062bce381a2cf2f389bb17b0a13dbe2b3cdb97a6.png)

On the tutorials page of this whole box only the `</>` is clickable, it’s hard to even notice that bit.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [August 22, 2023, 12:57pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/26 "2023-08-22T12:57:54Z")

</div>

Can you elaborate on the tools you used to build the Lux website and docs?

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 22, 2023, 3:32pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/27 "2023-08-22T15:32:05Z")

</div>

Thanks for all the suggestions. I have aggregated them in an issue [Documentation Enhancement Suggestions · Issue #387 · LuxDL/Lux.jl · GitHub](https://github.com/LuxDL/Lux.jl/issues/387).

> Can you elaborate on the tools you used to build the Lux website and docs?

It uses

- Literate.jl for generating the tutorials
- a small patch over [GitHub - JuliaDocs/DocumenterMarkdown.jl: Documenter's Markdown / MkDocs backend](https://github.com/JuliaDocs/DocumenterMarkdown.jl) (to generate vitepress friendly markdown see [DocumenterVitepress module into package · Issue #385 · LuxDL/Lux.jl · GitHub](https://github.com/LuxDL/Lux.jl/issues/385))
- Documenter.jl to do all the preprocessing of julia specific documentation components
- Finally [https://vitepress.dev/](https://vitepress.dev/) for generating the static website.

---

<div class="post-metadata">

**Author:** ![lazarusA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lazarusa/32/6571_2.png) [@lazarusA](https://discourse.julialang.org/u/lazarusA)\
**Post date:** [August 22, 2023, 4:22pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/28 "2023-08-22T16:22:42Z")

</div>

why vite? how the future looks like for this option, it’s only 3 years old. It’s definitely nicer at first glance, but beyond that? Just curious about the motivation for such change, if there is any 😃 .

---

<div class="post-metadata">

**Author:** ![Dale\_James\_Black](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dale_james_black/32/11595_2.png) [@Dale\_James\_Black](https://discourse.julialang.org/u/Dale_James_Black)\
**Post date:** [August 22, 2023, 4:29pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/29 "2023-08-22T16:29:13Z")

</div>

This might be overkill, but a brief mention of the `f0` and how it relates to Float32 might be helpful in [this line](https://lux.csail.mit.edu/dev/tutorials/beginner/Basics/main#:~:text=%2C%203%5D-,0.58720636f0,-This%20means%20get)

Amazing docs though, this template is fantastic!

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 22, 2023, 5:20pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/30 "2023-08-22T17:20:04Z")

</div>

(My entire JS knowledge comes from 2 lab assignments I had to do in my undergrad, so take everything I say with a grain of salt)

I was already kind of familiar with Vuejs, and it seems vuepress is in maintenance mode ([GitHub - vuejs/vuepress: 📝 Minimalistic Vue-powered static site generator](https://github.com/vuejs/vuepress#status)), and the current effort has shifted towards vitepress (with it being hosted in the official vuejs organization).

The other option was docusaurus, but I did not have any examples in the julia ecosystem to go off of here. There’s [https://expronicon.rogerluo.dev/](https://expronicon.rogerluo.dev/) which has been using vitepress for quite some time now.

---

<div class="post-metadata">

**Author:** ![Craig\_Hamel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/craig_hamel/32/202434_2.png) [@Craig\_Hamel](https://discourse.julialang.org/u/Craig_Hamel)\
**Post date:** [August 25, 2023, 3:09pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/31 "2023-08-25T15:09:05Z")

</div>

Is there anyway in Lux to use Float64 for the networks parameters like in Flux? I couldnt quite figure this out from the documentation.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 25, 2023, 3:18pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/32 "2023-08-25T15:18:53Z")

</div>

Just change the weights to Float64.

---

<div class="post-metadata">

**Author:** ![cortner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cortner/32/204_2.png) [@cortner](https://discourse.julialang.org/u/cortner)\
**Post date:** [August 25, 2023, 5:32pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/33 "2023-08-25T17:32:19Z")

</div>

I also found that surprisingly inconvenient to do actually. (Unless it’s changed recently)

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 25, 2023, 7:23pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/34 "2023-08-25T19:23:45Z")

</div>

I see; I can add a function like `f64/f32/f16` to automatically perform this conversion.

---

<div class="post-metadata">

**Author:** ![cortner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cortner/32/204_2.png) [@cortner](https://discourse.julialang.org/u/cortner)\
**Post date:** [August 25, 2023, 7:34pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/35 "2023-08-25T19:34:42Z")

</div>

Or just a kwarg when constructing the layer

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 25, 2023, 7:42pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/36 "2023-08-25T19:42:26Z")

</div>

Having a kwarg for every layer becomes infeasible very quickly. FWIW, in the latest release, constructs the parameters with the correct types if you do `init_weight=glorot_uniform(Float64)` and similar for different layers.

---

<div class="post-metadata">

**Author:** ![cortner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cortner/32/204_2.png) [@cortner](https://discourse.julialang.org/u/cortner)\
**Post date:** [August 25, 2023, 8:08pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/37 "2023-08-25T20:08:04Z")

</div>

oh that’s all I need, thank you. I think that didn’t exist a few versions ago, correct?

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [August 25, 2023, 8:14pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/38 "2023-08-25T20:14:43Z")

</div>

Yeah pretty much. It got merged yesterday [Allow specifying the return eltype of the arrays by avik-pal · Pull Request #4 · LuxDL/WeightInitializers.jl · GitHub](https://github.com/LuxDL/WeightInitializers.jl/pull/4)

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [August 25, 2023, 8:25pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/39 "2023-08-25T20:25:32Z")

</div>

> Having a kwarg for every layer becomes infeasible very quickly.

I’ve been experimenting with a “generative context” to pass defaults for device, numeric precision and RNG through a computation:

> <https://github.com/oschulz/HeterogeneousComputing.jl/blob/main/src/gen_context.jl>

Maybe something like that?

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [August 25, 2023, 9:16pm UTC](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689/40 "2023-08-25T21:16:29Z")

</div>

Using `ComponentArrays` this should do the trick

```julia
ps, st = Lux.setup(rng, model)
ps = Float64.(ComponentVector(ps))

```

[Previous page](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689.md?page=1)

[Next page](https://discourse.julialang.org/t/ann-lux-jl-explicitly-parameterized-neural-networks-in-julia/81689.md?page=3)