I’m been wondering about the general status of the deep learning ecosystem in Julia. We obviously have the big project Flux.jl , as well as a similar but mutable version in Lux.jl . I bring this up because it would be FANTASTIC to be able to mess around with bits of language models in Julia (we have …

Also: this discussion of Lux.jl is great and topical. [image] [ANN] Lux.jl: Explicitly Parameterized Neural Networks in Julia Package Announcements Lux is a new Julia deep learning framework that decouples models and parameterization using deeply nested named tuples. Fu…

Also related to ML more broadly: [image] State of machine learning in Julia Machine Learning After the Twitter space Q&A @logankilpatrick hosted yesterday on “The future of machine learning and why it looks a lot like Julia,” I thought it would be useful to accumulate som…

A very thorough analysis of nice-to-haves: [image] State of machine learning in Julia Machine Learning I’ll offer a perspective from someone who (as a conscious choice) primarily uses Python over Julia. I work with, and maintain libraries for, all of PyTorch, JAX, and Jul…

[image] cpfiffer: If you wrote a new library, from scratch, with no time or resource constraints, how would you do it, and what would be new or different? It would be Lux. Back in 2020 it was clear that was the direction to go, it fixed the SciML examples and got things shipping. @avikpal rea…

Okay I guess nerd snipe, let me give some detail. Here’s a few PRs to look into: Fix Tests and Cleanup by avik-pal · Pull Request #873 · SciML/DiffEqFlux.jl · GitHub Remove Flux Support by sathvikbhagavan · Pull Request #789 · SciML/NeuralPDE.jl · GitHub End of Year Housekeeping by avik-pal · Pull…

I would never recommend using Lux over Flux, I find Flux much simpler and more elegant. Having parameters and states as part of the model as Flux (and pytorch!) does instead of juggling them around feels much more natural. Consider for instance the forward pass of a hypothetical TransformerBlock in…

[image] ChrisRackauckas: I’ll keep this just to the deep learning frameworks and not AD, since AD has its own topics which go into detail on these points. In my opinion, one of the key reasons why deep learning in Julia is light years behind PyTorch/JAX is the performance and convenience of A…

I have been wondering whether there is some sort of feature tracker for things missing e.g. compared to PyTorch similar to what Rust has at Are we learning yet . This would be helpful in finding the right areas to focus work on, and would make certain things more concrete. In this thread we have opin…

[image] CarloLucibello: Maybe the explicit parameter style is more convenient is some sciml use cases, I don’t know, but we are talking about very niche applications within the deep learning world. :100: agree. Not everything is SciML. Plus, the Julia community is doing many other things wit…

[image] CarloLucibello: but we are talking about very niche applications within the deep learning world. SciML is one area in which Julia is in a dominant position. Many would agree that a new product doesn’t compete with old products starting with a worse version of the product to a large au…

Deep learning in Julia

Specific Domains Machine Learning

cpfiffer April 11, 2024, 8:05pm 2

Also: this discussion of Lux.jl is great and topical.

Topic		Replies	Views
State of machine learning in Julia Machine Learning	60	67212	August 26, 2022
[ANN] Lux.jl: Explicitly Parameterized Neural Networks in Julia Package Announcements package , announcement , machine-learning	49	12218	April 16, 2024
Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet? Machine Learning	52	25822	January 11, 2021
Machine Learning using Julia - Aim/Idealogy of Flux.jl to for simplicity over compexity for programmers Machine Learning question , flux , machine-learning	11	2098	February 8, 2022
Flux ready for a beginner deep learning project? Machine Learning flux	31	8949	June 20, 2019

Deep learning in Julia

Related topics