Benchmarking and threads

General Usage Performance
, ,
1

In the bench workspace, I have this function:

function kaftorTime(textLen::Integer,keyLen::Integer) # in nanoseconds
  text=fill(0x69,textLen)
  key=fill(0x96,keyLen)
  trial=@benchmark kaftorEncrypt!($text,$key)
  median(trial).time
end

I have two Julias running, one julia --project -t auto (which is 12) and one julia --project -t 1. (This is on my old tower; my new laptop has 32 threads and is significantly faster.) With 12 threads:

julia> kaftorTime(59049,32)/59049
637.9849531744823

With 1 thread:

julia> kaftorTime(59049,32)/59049
337.4873240867754

I don’t see how that can happen unless @bench is measuring total CPU time on all cores.

I’d like to tune the parallelized parts of the algorithm, with the number of threads depending on the message size, to minimize the elapsed time without greatly increasing the task switching overhead. Can I measure both elapsed time and total time of all threads? Also, is there a way to specify the number of threads that @threads uses?

2

Did you try with 2, 3, 4 … threads?

Most CPUs now have fast and slow cores. Using the slow cores is often useless and can slow things down, depending on your algorithm.

Threads can slow things down by:

  • causing more GC pressure
  • causing communication/ coordination overhead
3

The tower has only one kind of core. The laptop has two.

The algorithm does three things per round: jumble!, shufflePairs!, and mix3PartsSeq!; all of which operate on a vector in place, so there shouldn’t be any garbage collection. jumble! is already parallelized, with a worker function, so I can easily change the number of threads. shufflePairs! is parallelized with @threads; this produced a slight speedup. Replacing mix3PartsSeq! with mix3PartsPar! slowed it down.

Profiling it results in most of the time spent in task_done_hook, which doesn’t help me see what’s going on.

4

There are a lot of ways this can happen.

If your program is abusing the Task system and spawns too many short-running tasks, then you will see this behavior: When single-threaded, the entire task-switching / scheduling apparatus is faster.

You can also see this if your parallelization sucks, i.e. if you have lots of contention, irregardless of whether false sharing (different cores access different parts of the same cache-line) or true sharing (e.g. locks / contested atomics or data races). In such cases single-threaded is much faster.

5

It appears, from looking at the source, that @bench is measuring elapsed time. How can I measure total CPU time?

Is there a way to specify the number of threads that @threads uses? Or do I have to create a worker function?

How can I measure false sharing? There is no true sharing in this program; each iteration of a loop operates on a different pair or triple of bytes.

6

How can I measure total CPU time?

Standard tool to measure total cpu time is getrusage(2). We can do this in Julia as follows:

function cputime()
    buf = zeros(Int64, 18)  # struct rusage: 2 timevals + 14 longs = 144 bytes
    ccall(:getrusage, Cint, (Cint, Ptr{Cvoid}), 0, buf)  # 0 = RUSAGE_SELF
    (buf[1] + buf[3]) + (buf[2] + buf[4]) / 1e6  # (utime.sec+stime.sec) + usec/1e6
end

function kaftorTimes(textLen::Integer, keyLen::Integer; samples=100)
    text0 = fill(0x69, textLen)
    key = fill(0x96, keyLen)
    kaftorEncrypt!(copy(text0), key)  # warm up
    elapsed = Float64[]; cpu = Float64[]
    for _ in 1:samples
        text = copy(text0)
        t0 = time_ns(); c0 = cputime()
        kaftorEncrypt!(text, key)
        push!(elapsed, (time_ns() - t0) / 1e9)
        push!(cpu, cputime() - c0)
    end
    (median(elapsed), median(cpu))
end

Running this with -t 12 gives for me:

julia> kaftorTimes(59049,32)
(0.030032722, 0.11667699999999925)

Is there a way to specify the number of threads that @threads uses? Or do I have to create a worker function?

No there isn’t. You need a worker function, or use a package such as OhMyThreads.

How can I measure false sharing?

Standard tool is perf-c2c(1). I ran it, and there isn’t any false sharing happening.

The slowdown you see is just the overhead of using multiple threads, confirmed by the hits found by perf-c2c(1), that it’s Julia’s own task scheduler.

The algorithm does three things per round: jumble!, shufflePairs!, and mix3PartsSeq!; all of which operate on a vector in place, so there shouldn’t be any garbage collection. jumble! is already parallelized, with a worker function, so I can easily change the number of threads. shufflePairs! is parallelized with @threads; this produced a slight speedup. Replacing mix3PartsSeq! with mix3PartsPar! slowed it down.

That tracks with what perf c2c showed: mix3PartsPar! spawns nthreads() tasks unconditionally, no size check like jumble! has.

I did the following changes:
I gave it the same kind of gate jumble! uses. Additionally, I changed so if the task count was 1 to not do @spawn+wait, since it gives an overhead.

With both fixes in, swapping mix3PartsSeq! for mix3PartsPar! is a win. I now get roughly same timings on the 59KB case with 12 threads as with a single thread, and over 2 times faster on 4MB.

PR

7

You can launch Julia with the -t flag set to the number of threads you’d like. But once started that can’t be changed.
If you want fine grained control you’ll have to start working with tasks, but those don’t map exactly to threads unless you pin them.

8

What do you mean by “gate”?

I put perThread in after starting this discussion. I’m planning to do the same for mix3Parts! and probably shufflePairs! (with a worker function) as well. I adjusted perThread by benchmarking it with message sizes up to 3^17, which is close to 2^27.

9

Did you see the PR I made? But just like in your jumple! function I added

aInc=max(1,min(len÷mix3PerThread,nthreads()))
10

I have not looked at the repo yet, nor worked on the program. You posted when it was Sabbath here. I normally sleep after lunch on Saturdays, but there was a capacitation at church, so I went back to church in the afternoon and got home tired. Also I’ve been working on another project called Propolis, which is in C++.