Hi! The problem sounds very simple on the surface: compute something like
f1(f2(f3(a3), a2), a1) on a grid of values for
a1, a2, a2, and save results into array of corresponding shape. Of course, it can be done with
((a1, a2, a3) -> f1(f2(f3(a3), a2), a1)).(product(1:n1, 1:n2, 1:n3), but this is really inefficient because it recomputes
f2 for the same parameter values over and over. A for-loop is efficient is this sense:
for a3 in 1:n3 f3_val = f3(a3) for a2 in 1:n2 f2_val = f2(f3_val, a2) for a1 in 1:n1 val = f1(f2_val, a1) end end end
however the array creation and keeping track of its indices needs to be done manually. Also this is definitely not scalable with respect to adding new parameters or changing them, and parallelization using
pmap does not fit well into the for-loop code (but it is easily aplicable to
product example above). Loop comprehension has the same issue with parallelization.
Another drawback in this solution is that when
f3 is type-unstable, types in inner loops are not inferred as well, and performance degrades a lot. Function barrier would solve this, but it increases and complicates the code even more.
Any suggestions on how to implement this in an easy to use and efficient way? The ideal way I can imagine this would be something along these lines:
@pipeline begin @for a3 in 1:n3, parallel=true f3_val = f3(a3) @for a2 in 1:n2 f2_val = f2(f3_val, a2) @for a1 in 1:n1 @collect f1(f2_val, a1) end
which returns the final array of results (or even better,
AxisArray with correctly named and valued axes). But not sure if this is possible or feasible.