Using Julia with @parallel pmap or blank makes no difference in speed.

Specific Domains Julia at Scale
1

Hi,

the follwoing three test examples are minmal examples of my code.

I do not understand why the speed is always the same.

Moreover, Julia seems to use only one core.

The whole CPU usage is only around 30%.

If I use the minimal example from Julia docs, this works perfectly.

I get 90-100% CPU usage.

pmap(svd, collection of some matrices)

Can somebody help me?

function test1(a)
    for i in 1:a
        randn(10^8)
    end
end

@everywhere function test2(a)
    @parallel for i in 1:a
        randn(10^8)
    end
end

@everywhere function test3(a)
    for i in 1:a
        randn(10^8)
    end
end

@time test1(10)

@time test2(10)

@time pmap(test3, [1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

Kind regards,
Till

2

How did you start up your Julia process? julia has the -p option, which is used to start multiple processes for parallel computing. For example, if you have four physical cores, you may start four Julia workers with julia -p 4.

3

SVD is already multithreaded. It’s probably better to just use it.

1 Like
4

Thank you for your suggestions.

The answer is:

addprocs()

function bigfunc(iterator)
function smallfunc(huhu)
for i in 1:3
randn(10^8);
end
end
wp = CachingPool(workers())
pmap(wp, smallfunc, iterator)
end

@time bigfunc(1:10)

Kind regards,
Till