Does anyone else have issues loading large CSV files into Julia on the new Apple chips? I have to load smaller files sequentially first, otherwise the loading process never finishes?
I imagine you are using the native build of julia.
Does it work in rosetta?
Can you disable threading?
Also can you share one of those large files so that people can test?
It might be this https://github.com/JuliaLang/julia/issues/41820 .
Another thing you can do is limit the number of julia threads to just the performance cores, so 4 on the normal m1. Dunno about the others.
Are you just running out of ram? If so, you should probably not be reading the whole file at once.
@Oscar_Smith: I don’t think so - I’m uncertain how to test it? It’s only 20,000 rows and my old Mac intel with 8gb of Ram read the file without issues. Either I receive an error like the screenshot above, or I don’t at all.
Thanks for your help - sorry about the late reply.
Try adding export export JULIA_NUM_THREADS=4 to your .zshrc. The economy cores don’t play well with julia. I imagine you are on a normal M1 mac.