Hey all! I’m interested in finding out who in the community is using Dagger, or has used Dagger in the past. If so, I would appreciate if you could provide answers to any of the following questions: What kind of problem(s) are you solving with Dagger? Are you seeing the performance you would hope …

I have tried JuliaDB on and off several times but never got it working. I thought JuliaDB and Dagger are no longer maintained.

I picked up Dagger maintenance about 2 years ago, when I joined the JuliaLab; previous to that, it was basically unmaintained, except for minor fixes for JuliaDB. Today, Dagger is very actively maintained by myself and @krynju . JuliaDB, on the other hand, is still struggling to stay maintained, alt…

[image] jpsamaroo: But I would be hard-pressed to say that JuliaDB is actively maintained right now. Being able to work with out-of-memory data is a huge deal-breaker in data science. I still don’t understand why JuliaDB gets such limited attention.

I haven’t used JuliaDB in a while (used to use Dagger via JuliaDB), but it’s nice to see that Dagger is actively maintained! Even though I’m not an active user / developer, here are my two cents. I imagine that map, filter, reduce and groupby already cover a fair amount of usecases of distributed …

I agree with more docs for end users. i would suggest documentation similar to Ray (dagger seems to be somewhere between dask and ray): Welcome to the Ray documentation — Ray 2.0.0 specifically, I think these would be helpful: instead of “usage” being the first page, i’d suggest “gentle introduct…

I’m a happy Dagger user although I mostly use it though the excellent FileTrees for quite simple and boring tasks compared to what it is designed for. I like that I can use the convenience of FileTrees regardless of problem size and I use the three first scales you mentioned (laptop, server and clu…

I guess I count as a pretty heavy Dagger user due to the fact that I based the DTable implementation fully on it. To me Dagger does the heavy lifting of the parallelization and memory management that projects like the DTable require. Without it we’d be reinventing the wheel every time someone wants…

Is Dagger.jl composable with Julia’s multi-threading features, where Dagger handles parallelization between nodes whereas threads handle that within each node?

Folks at Beacon are quite interested in seeing if Dagger is well-suited for some our distributed-Julia-on-K8s use cases :slight_smile: We don’t quite have the bandwidth to immediately undertake some of the internal experimentation/testing we’ve planned out, but are aiming to carve out some time in t…

Dagger actually directly utilizes Julia’s multithreading automatically; tasks launched via Dagger will be scheduled on a thread that Dagger chooses, on whatever worker Dagger chooses (of course, you can force Dagger to pick a given thread/worker, detailed in the docs). The scheduler attempts to be s…

Who is using Dagger?

Specific Domains Julia at Scale

Topic		Replies	Views
Dagger use cases? General Usage dagger	6	1379	September 6, 2022
Dagger Development and Roadmap Community announcement , parallel , dagger	4	1027	July 23, 2023
Online Meetup: Distributed computing with Dagger.jl Meetups announcement , dagger	15	1282	December 2, 2021
Not seeing speed-up with Dagger.jl. Am I doing something wrong? Julia at Scale	8	718	December 14, 2023
Python Ray Project Alternative in Julia Community question , distributed , machine-learning	6	3316	February 28, 2023

Who is using Dagger?

Related topics