New to julia and experimenting with threads. I can’t seem to figure out how Julia determines JULIA_NUM_THREADS
I have two Intel xeon 2699 v3 in my hp z840
So what should JULIA_NUM_THREADS end up at please? remember there are TWO 2699’s in the machine.
thank you
JULIA_NUM_THREADS is an environment variable as noted in the manual. Instructions for setting it vary according to your operating system, but methods for common OSes are noted in the manual. You can also use Julia’s built-in autodetection of the number of threads by starting Julia with
julia -t auto
As for number of threads: remember that hyperthreading mostly serves to hide memory access latency by interleaving execution. If your application involves predictable instructions & memory access patterns, there’ll likely be little benefit to hyperthreading, so JULIA_NUM_THREADS = 36. Otherwise, try JULIA_NUM_THREADS = 72.
hi there
thank you for your reply. I am running linux mint 20.2.
I had tried starting julia with
julia -t auto
but when I ran Threads.nthreads() I was expected 72 but I got 36 which led me to believe that julia only saw 1 cpu.
julia> Threads.nthreads()
36
Right now I am just trying to experiment with julia multi-threading and following along with Prof Edelman ( MIT) code from the youtube below.
I am trying to figure out what to set the following to given I have dual xeon e5-2699 v3.
$installkernel( "Julia (4 threads)" , env = Dict( JULIA_NUM_THREADS" => "4"))
here is the video
Hi. If you have 2 socket Haswell each with 18 physical cores and multithreading, you should be able to use 72 threads. It does not always mean that it is optimal. AFAIK, there are cases when it is better to use only physical cores. I have never had any problems with correct detection of logical cores and I am usually using two socket systems, however on Haswell I have been using machines with only 1 socket, so no direct experience for this particular processors.
As for “installkernel”, it only applies when you want to use Jupyter notebooks. Otherwise you start julia with -t auto or in your case with -t 72 flag and it should work. If you insist on Jupyter notebook, assuming you have correctly working Jupyter, you should change 4 into 72 in the code provided by you. I guess you should also be able to follow all the examples provided in the video without Jupyter by using REPL in VS Code. This might be slightly less complicated and would be my recommendation if you consider yourself absolute beginner. Anyway:
# Some info about multithreding with Jupyter - this is from my notes - it is based on info from Internet - I am really sorry - I do not have link to the source - hope the author does not mind.
# If you want to do multithreading in Julia with Jupyter, you have to create a new kernelspec.
using IJulia
IJulia.installkernel("Julia 72 Threads", env=Dict(
"JULIA_NUM_THREADS" => "72",
))
# Then you can create a new notebook using that kernel (or change the kernel of an existing notebook in the Kernel > Change Kernel menu) and run code as you expect.
Threads.@threads for i in 1:72
println(Threads.threadid())
end
If the problem still persists, you may try to provide some additional information about your configuration. Not sure if I be able to help, however, maybe somebody will spot something unusual. Please do not forget to provide info on how you started Julia.
versioninfo()
@info "Sys.BINDIR" Sys.BINDIR
@info "Sys.STDLIB" Sys.STDLIB
@info "Sys.CPU_NAME" Sys.CPU_NAME
@info "Number of threads" Threads.nthreads()
using Pkg
Pkg.add("Hwloc")
using Hwloc
@info "Hwloc.topology_info()" Hwloc.topology_info()
println()
@info "Hwloc.topology" Hwloc.topology()
Have fun coding in Julia. 
thank you so much @j_u I appreciate you taking the time to look into this. I’ve no interest in using Jupyter as I like the look of Pluto, seems to be more my type of interface. I don’t really want to use vs code as text editor or a return to emacs will be my chosen companions on my new adventure coding in Julia. I’m going to take your advice and work through Prof Edelman’s code.
The machine I just started running julia on has this config.
dave@deepthought:~$ inxi -Fxxxrz
System:
Kernel: 5.4.0-90-generic x86_64 bits: 64 compiler: gcc v: 9.3.0
Console: tty 0 dm: GDM3 3.36.3, LightDM 1.30.0 Distro: Linux Mint 20.2 Uma
base: Ubuntu 20.04 focal
Machine:
Type: Desktop System: Hewlett-Packard product: HP Z840 Workstation v: N/A
serial: <filter> Chassis: type: 6 serial: <filter>
Mobo: Hewlett-Packard model: 2129 v: 1.01 serial: <filter>
UEFI: Hewlett-Packard v: M60 v02.32 date: 05/02/2017
CPU:
Topology: 2x 18-Core model: Intel Xeon E5-2699 v3 bits: 64 type: MCP SMP
arch: Haswell rev: 2 L2 cache: 90.0 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
bogomips: 165246
Speed: 1198 MHz min/max: 1200/3600 MHz Core speeds (MHz): 1: 1197 2: 1198
3: 1198 4: 1197 5: 1197 6: 1397 7: 1197 8: 1197 9: 1198 10: 1197 11: 1197
12: 1198 13: 1197 14: 1198 15: 1197 16: 1197 17: 1197 18: 1197 19: 1198
20: 1197 21: 1197 22: 1198 23: 1198 24: 1198 25: 1198 26: 1198 27: 1197
28: 1198 29: 1197 30: 1197 31: 1197 32: 1197 33: 1198 34: 1198 35: 1297
36: 2295
Graphics:
Device-1: NVIDIA GK104GL [Quadro K5000] vendor: Hewlett-Packard
driver: nvidia v: 470.82.00 bus ID: 04:00.0 chip ID: 10de:11ba
Display: server: X.org 1.20.11 driver: nvidia
unloaded: fbdev,modesetting,nouveau,vesa tty: 80x24
Message: Advanced graphics data unavailable in console. Try -G --display
Audio:
Device-1: Intel C610/X99 series HD Audio vendor: Hewlett-Packard
driver: snd_hda_intel v: kernel bus ID: 00:1b.0 chip ID: 8086:8d20
Device-2: NVIDIA GK104 HDMI Audio vendor: Hewlett-Packard
driver: snd_hda_intel v: kernel bus ID: 04:00.1 chip ID: 10de:0e0a
Sound Server: ALSA v: k5.4.0-90-generic
Network:
Device-1: Intel Ethernet I218-LM vendor: Hewlett-Packard driver: e1000e
v: 3.2.6-k port: 4040 bus ID: 00:19.0 chip ID: 8086:15a0
IF: eno1 state: down mac: <filter>
Device-2: Intel I210 Gigabit Network vendor: Hewlett-Packard driver: igb
v: 5.6.0-k port: 1000 bus ID: 05:00.0 chip ID: 8086:1533
IF: ens8191 state: up speed: 1000 Mbps duplex: full mac: <filter>
Thanks again for such a great reply.
I’ll start julia with
$julia -t 72
Hi, thanks! Pluto looks and works great, sometimes giving a machine a real spin. Hope all is working correctly and you are getting full utilization. Sorry, I am not able to precisely answer your question about how Julia detects the maximum number of threads available. Not even sure if it varies between Julia versions. Again, I personally did not have any problems with it, however, I know that some concerns regarding the correct detection were raised by very knowledgeable persons in GitHub posts I was reading. Never investigated it deeply. If there were problems with auto detection even if starting Julia with -t 72 flag when running some intensive computations I would double check in top to be sure all is fine (all cores utilized). BTW, I can hardly believe that with emacs you are an absolute beginner. Once again, have fun and hope to stay in touch.
I’m looking forward to using Pluto, they seem to be getting a real approach working for us. They are considering a SCRATCH like approach to development. I am a BIG fan of that.
I’ll stipulate the number of threads to Julia as I am not a fan of trusting autodetect anything.
As to emacs, I used to use it decades ago when I was working on multithreading in a language called Tal on Tandem fault tolerant systems. On those you had to configure the processing environment. It was pretty neat actually and I used to enjoy the whole micromessage processing. I’m hoping Julia will give me the same thrill.
thanks again for all your GREAT help and I agree, we should stay in touch
I am interested in Julia due to a hobby project. My current focus is on AI/ML/DL/RL. I started early on Atari and had all first issues of major computer magazines published in my country. I am not a pro developer. This is not my field of expertise. I am reading the posts at this discussion list with interest. I am currently usually on vim and vs code. I am glad I was able to help. Lets stay in touch.
My understanding is that julia -t auto will always give you one thread per physical core, so this behavior is expected, and not a failure of the detection scheme.
so if the core ( as in my case) has 2 processors each with 18 cores AND 36 threads why does the -t ( number of THREADS) only give 1 thread per core when there are 2? Don’t see why this isn’t a failure in the detection system?
I’m using julia as an upgrade to python. I programmed our ( my wife and I’s) trading augmentation tool in Python and it was FUN!! The downside was the asynch capabilities, they exist but seemed to be a tad flaky. Julia seems to be learning from the past and trying to make asynch coding as seamless as possible. I’m learning a LOT from youtube videos and a book called “Julia High performance 2nd ed by Avik Sengupta” I think it’s a tad out of date but it’s still a great read.
In general, the Julian solution to async is to just use threads. Since Julia has actual threading, you can just spawn new threads that will run asynchronously.
I get that. The issue here is how to determine the actual maximum number of THREADS available. In my case it’s 72 so it follows that I could spawn 72 threads. But if -t only assigns 1 thread per core then what happens?
So in my program I want to stipulate max_num_threads based on -t auto I’d get 36 which is half my capabilities. I could set the number as a parameter but HOW would I detect that if I ran the code on a machine with a different number of max threads?
You might be conflating two ideas here. The -t option sets the number of Julia threads. You can pass whatever number you want, irrespective of how many cores you actually have (try e.g. julia -t 128 -E 'Threads.nthreads()'). An entirely separate question is where, that is on which cores, these Julia threads will run. By default, Julia doesn’t do any thread pinning. That means that these threads can and will run on whatever cores the OS deems appropriate. So Julia doesn’t “assign 1 thread per core”. If you would want one thread per core you could use JULIA_EXCLUSIVE=1 or, for example, a tool like ThreadPinning.jl to pin threads to specific cores (only works on Linux).
First of all, to be clear, Julia currently doesn’t really allow you to add threads dynamically. That means that you’ll generally have to specify the desired number of threads upfront. The only “workaround” is to choose a “upper bound value” and then, potentially, only use a subset of the threads in your application.
If your question is simply how to find out how many physical and virtual cores a machine has, then the answer is that there are many ways. You could, for example, use Hwloc.jl. Example on my 6-core machine (12 hardware threads with hyperthreading):
julia> using Hwloc
julia> num_physical_cores()
6
julia> num_virtual_cores()
12
If you only care about the number of available hardware threads (i.e. including hyperthreads), than you don’t even need a package:
julia> Sys.CPU_THREADS
12
BTW, I’m confused that julia -t auto gives you only number of physical cores many threads. For me it matches the value of Sys.CPU_THREADS, i.e. all available threads, both on Linux and macOS.
➜ cbauer@influx1 ~ julia -t auto
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.0-beta3.0 (2021-07-07)
_/ |\__'_|_|_|\__'_| | release-1.7/e76c9dad42 (fork: 68 commits, 162 days)
|__/ |
julia> using Hwloc
julia> num_physical_cores()
20
julia> num_virtual_cores()
40
julia> Threads.nthreads()
40
@carstenbauer WOW! now that is what I call a great reply!! You have saved me a LOT of grief. I am going to dig into ThreadPinning.jl as I do want to experiment with this in my approach. I only use Linux ( right now mint 20.2 but happy to use any of the variants) . Thank you for taking the time and giving such a wonderful overview of the issues in my new adventure
Glad that I could clear things up a bit!
here’s what I get ( see m/c spec earlier in thread )
dave@deepthought:~$ julia -t auto
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.6.3 (2021-09-23)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> Threads.nthreads()
36Hm, perhaps hyperthreading is disabled? What’s the value of Sys.CPU_THREADS? What does Hwloc.jl give you for the number of virtual cores?
Side comment for the curious: To answer your question in the title of this thread, i.e. how Julia detects the max number of threads when it is started with julia -t auto. I guess the relevant bit is https://github.com/JuliaLang/julia/blob/master/src/sys.c#L591 and below.
IIUC, on Linux it essentially uses sysconf(_SC_NPROCESSORS_ONLN).