Suggestion of workflow to modify composite object in parallel

Specific Domains Julia at Scale
1

Hello,

I have an object composed of other objects inside, (Graph, Nodes). I want to run an operation on the nodes that modifies the node, adjacent nodes, and the graph. What is the best workflow to do it in parallel. I tried pmap, and I think it moves the objects to the workers all the time. Here is a minimal example

Module

module GraphModule

export MyGraph, MyNode, addnode, addedge

mutable struct MyGraph
    numnodes
    nodes
    edges
    MyGraph() = new(0,[],[])
end

mutable struct MyNode
    label
    childlabel
    parentlabel
end

function addnode(g::MyGraph)
    n = g.numnodes += 1
    node = MyNode("MyNode$n","","")
    push!(g.nodes, node)
    return node
end

function addedge(g::MyGraph,n1::MyNode,n2::MyNode)
    n1.childlabel = n2.label
    push!(g.edges,(n1.label,n2.label))
end

function setparentlabel(g::MyGraph,n::MyNode)
    for (n1,n2) in g.edges
        if n == n1
            n2.parent = n1
        end
    end
end

end

Code

push!(LOAD_PATH,".")
addprocs()

using GraphModule

g = MyGraph()

n1 = addnode(g)
n2 = addnode(g)
n3 = addnode(g)

addedge(g,n1,n2)
addedge(g,n2,n3)

n1.parentlabel

# Apply setparentlabel function to every node in parallel. Note that the function modifies an adjacent node

I’ve read the documentation around 5 times and I am very confused on what should I do.

thanks!

1 Like
2

You don’t specify what type your parentlabel is, but if you want to do this in parallel you should probably use a SharedVector. This will allow you to use pmap or @parallel to do what you want.

Note, though, that unless you’re doing this many, many times, and labeling is computationally expensive, it’s probably not worth the extra overhead to label nodes in parallel.

Also note that MetaGraphs.jl might do what you want already (though not in parallel).

3

Hello. Thank you for the quick response.

The Graph was just a simple example. In my application parentlabels represents an array of custom objects, and the operation performed is much more involved than just setting the label. It is optimizing a model in the node and posting results to other nodes. Can I have SharedArrays of custom types?. Should I use something like channels or remoterefs?.

4

As far as I understand, no: SharedArrays have to be of bitstype.