>64 HW threads under windows

I haven’t checked in a while but has Julia fixed the problem with only using 1/2 or 1/4 of the available cpu threads on high-end (either >64 HW threads or multi-socket) windows systems?

To be clear the issue isn’t with Julia but libuv. I fixed it myself a while back and sent them the fix but got a little frustrated with conforming to their formatting standard so didn’t follow up much after and kept it for my personal use. But now I’m thinking I should follow through unless it’s been fixed.

I believe the issue isn’t libuv, but that windows doesn’t actually support more than 64 cores reasonably (but I’m not 100% usre). If you have that many cores you should be using Linux since it actually has proper schedulers and stuff IO system to work for big systems.

1 Like

No it supports 128 cores or more but you need a little tiny special codein your thread creation because of 64 bit thread masks. Ive fixed it in libuv and other libs plus my personal codebase. When i run with my modified libuv, i can max out any number of hw threads and get speedups past 64 threads (obviously its hard to get LINEAR speedups at that thread count :). Here’s an article i wrote about it with (slightly incomplete) sample code:

1 Like

And heres the libuv fix: Fix for win32 processor groups issue · libuv/libuv · Discussion #3457 · GitHub

I’m confused about why the code is stuffed into a Github discussion rather than as a pull request. It is very difficult to read and parse that way, especially since I would mostly be interested in the diff.

Note that there is a Julia fork of libuv:

I’m aware of what you speak since I have a multisocket Windows machine at work. My usual solution is to run an individual Julia process for each socket, especially since I usually want to be explicit about memory transfers between NUMA nodes.


Like i said git and github aren’t really a thing in my work and i was very unfamiliar - perforce has a more dominant position in aaa game dev.

I have since released some code (threading extensions to eigen GitHub - ValveSoftware/eigen: Fork of Eigen release version 3.4. Adds ability to use alternate threading systems from Open MP.) to github and am a little familiar now, another reason to follow up. Seems kind of selfish of me to be running the only julia on windows without this problem :). Plus it’s a pain to have to build my own libuv.
Ill see if i can put together a fixed version of the julia fork of libuv and submit a patch properly

1 Like

Can you zip or tar up the modified source code and post it somewhere?

Give me a little bit, I need to update to the latest and merge+test the changes.

If you would like, but you seem to be having a lof of challenges with git.