Lux is a new Julia deep learning framework that decouples models and parameterization using deeply nested named tuples. Functional Layer API – Pure Functions and Deterministic Function Calls. No more implicit parameterization Compiler and AD-friendly Neural Networks using Lux, Random, Optimisers,…

Just change the weights to Float64.

I also found that surprisingly inconvenient to do actually. (Unless it’s changed recently)

I see; I can add a function like f64/f32/f16 to automatically perform this conversion.

Or just a kwarg when constructing the layer

Having a kwarg for every layer becomes infeasible very quickly. FWIW, in the latest release, constructs the parameters with the correct types if you do init_weight=glorot_uniform(Float64) and similar for different layers.

oh that’s all I need, thank you. I think that didn’t exist a few versions ago, correct?

Yeah pretty much. It got merged yesterday Allow specifying the return eltype of the arrays by avik-pal · Pull Request #4 · LuxDL/WeightInitializers.jl · GitHub

Having a kwarg for every layer becomes infeasible very quickly. I’ve been experimenting with a “generative context” to pass defaults for device, numeric precision and RNG through a computation: oschulz/HeterogeneousComputing.jl/blob/main/src/gen_context.jl # This file is a part o…

Using ComponentArrays this should do the trick ps, st = Lux.setup(rng, model) ps = Float64.(ComponentVector(ps))

[image] bertschi: ps, st = Lux.setup(rng, model) ps = Float64.(ComponentVector(ps)) Wound it not be sufficient to do ps, st = Lux.setup(rng, model) ps = Float64.(ps) ?

[ANN] Lux.jl: Explicitly Parameterized Neural Networks in Julia

Package Announcements

Topic		Replies	Views
Deep learning in Julia Machine Learning	35	12830	April 22, 2024
Questions on NeuralPDE.jl Modelling & Simulations	5	798	July 7, 2022
Lux.jl demo with Lotka-Voltera with UODE General Usage	25	2550	January 29, 2023
Slow LSTM on GPU in Flux Machine Learning gpu , flux , pytorch	21	2296	February 15, 2024
Flux.jl versus Lux.jl General Usage	5	2071	May 10, 2023

[ANN] Lux.jl: Explicitly Parameterized Neural Networks in Julia

Related topics