Dear Julia community,
Together with @ffevotte , @LaurentPlagne , and Vincent Sivadon, we have put together a package (DataFlowTasks) for automatically inferring implicit Task dependencies from explicitly annotated data dependencies.
The following example, taken from the documentation, illustrates the API:
using DataFlowTasks
A = Vector{Float64}(undef, 4)
result = let
@dspawn fill!(@W(A), 0) # task 1: accesses everything
@dspawn @RW(view(A, 1:2)) .+= 2 # task 2: modifies the first half
@dspawn @RW(view(A, 3:4)) .+= 3 # task 3: modifies the second half
@dspawn @R(A) # task 4: get the result
end
fetch(result)
In the example above the last task (created with @dspawn) will wait for the second and third, which in turn will wait for the first one, because of a detected data conflict. The second and third tasks can be executed in parallel however.
More interesting examples where we believe the constant reuse of data (together with a nontrivial access pattern) makes this data flow approach more natural than say a functional approach can be found in the examples section of the docs where we showcase e.g. a pure Julia threaded cholesky factorization inpar with MKL (for large matrices)!
We are considering registering the package if others may find it useful, and would be happy to take into account feedback from the community.
