Thanks for the quick reply and forgive my (mostly) uninformed thoughts.
I’ll have to check out JuliaDB. So as for GPU I’m not really sure what works so much faster on that vs CPU as I’m someone with basically no computer science training. I’ve taught myself R and now python. In my limited usage however a few essential things that would be needed are functions like groupby functions, joining, splitting, means sums other similar simple calculations. Pretty much any basic function supported by pandas or dplyr-like functions from R. Unfortunately, I’ve already run into bugs with dask-cuDF (it’s still pretty new) so I didn’t get to see just how much of an increase it would have given me but from the few functions I was able to get working I was seeing 6-20x speed increases over my 16 thread CPU. Also it looks like the dask people are trying to implement a lot of the pandas functions with cuDF so I would imagine that a lot of it would be faster on GPU.
It looks like there are some people doing some work with Julia on GPUs (https://github.com/JuliaGPU). Again I don’t really know what I’m doing but maybe you can use some of their code and integrate it into JuliaDB to get GPU access to users who have one.
I do have another question for you, are there any plans to integrate arrow based columnar data handling in JuliaDB? Thanks again.