Parallel version of code not giving the right answer

New to Julia
1

I’m new to parallel computing (and Julia as well for that matter). I have a very simple code that works perfectly in the regular un-parallel set-up, but that doesn’t work when distributed across several workers. That is : the program performs without any errors being raised, but the yielding result is erroneous (to be more precise : it blows up instead of converging). The (mathematical) problem is very simple : I run Monte-Carlo simulation on a grid of some function depending on a set of weights (i.e. the array \nu). Depending on the result of this simulation, I update the weights and so on, until the weights converge. Here’s the code:

using Distributed

@everywhere begin
using LinearAlgebra
using SharedArrays
const N = 5
const β = 1.0 
const M = 20 
const Δ = N/(2M) 


ν = SharedArray{Float64,1}(M)

W(r) = - (r >= 0.5) * r + (r <= -0.5) * r + (r >= -0.5)*(r <= 0.5)*(-r^2 - 1/4)
U(r) = Δ^2 * dot(ν, map(t -> W(r/Δ - (t - 1/2)) + W(r/Δ + (t - 1/2)), 1:M))

end

function ρ_ν(r, niter = 15_000)
    res = @sync @distributed (+) for _ = 1:niter
        sample = N * (rand(N)-1/2*ones(N)); sample[1] = r
        ρ = prod(map(t->exp(β*U(t)),sample))
        for k = 1:N-1 for q = k+1:N
                ρ *= exp(β * abs(sample[k] - sample[q]))
        end end
        ρ
    end
    N^(N-1) * res/niter
end



function PGD(ε)
    err = ε + 1
    iter = 1
    λ(t) = 10/t^(1/4)
    while err > ε
        err = 0
        ρ = map(t->ρ_ν((t-1/2)*Δ),1:M)
        ρ = ρ./(2*Δ*sum(ρ))
        for i = 1:M
            η = Δ^3 * sum(map(t->W((t - 1/2) - (i - 1/2)) + W(-(t-1/2)-(i-1/2)), 1:M).*(ones(M).- N*ρ))
            print("\n")
            print("eta =", η)
            print("\n")
            ν[i] += λ(iter) * η
            err += abs(η)
        end
        iter += 1
        print(ν, '\n')
    end

end

When I run the optimizing function PGD in Juno (without adding workers or whatsoever), everything works perfectly fine. Now, when I launch the program in a parallel set-up, it doesn’t work (even with just one processor, that “Julia -p 1 …”). In the parallel setting, \eta is always positive, making the weights blow up…

2

I think calling the SharedArray constructor on each worker creates a fresh copy everywhere, you only want the SharedArray to be created on the master and then pass a reference to each worker.

Does this work more like you expect?

using Distributed
using SharedArrays

@everywhere begin
using LinearAlgebra
const N = 5
const β = 1.0 
const M = 20 
const Δ = N/(2M) 

W(r) = - (r >= 0.5) * r + (r <= -0.5) * r + (r >= -0.5)*(r <= 0.5)*(-r^2 - 1/4)
U(ν, r) = Δ^2 * dot(ν, map(t -> W(r/Δ - (t - 1/2)) + W(r/Δ + (t - 1/2)), 1:M))

end

function ρ_ν(ν, r, niter = 15_000)
    res = @sync @distributed (+) for _ = 1:niter
        sample = N * (rand(N)-1/2*ones(N)); sample[1] = r
        ρ = prod(map(t->exp(β*U(ν, t)),sample))
        for k = 1:N-1 for q = k+1:N
                ρ *= exp(β * abs(sample[k] - sample[q]))
        end end
        ρ
    end
    N^(N-1) * res/niter
end


function PGD(ε)
    err = ε + 1
    iter = 1
    λ(t) = 10/t^(1/4)
    ν = SharedArray{Float64,1}(M)
    while err > ε
        err = 0
        ρ = map(t->ρ_ν(ν, (t-1/2)*Δ),1:M)
        ρ = ρ./(2*Δ*sum(ρ))
        for i = 1:M
            η = Δ^3 * sum(map(t->W((t - 1/2) - (i - 1/2)) + W(-(t-1/2)-(i-1/2)), 1:M).*(ones(M).- N*ρ))
            print("\n")
            print("eta =", η)
            print("\n")
            ν[i] += λ(iter) * η
            err += abs(η)
        end
        iter += 1
        print(ν, '\n')
    end
end
1 Like
3

Yes it does seem to be working much better ! Thanks a lot :slight_smile: