I am currently implementing a numerical simulation code on multiple GPUs using ParallelStencil+ImplicitGlobalGrid.
I wanted to use higher order numerical differentiation schemes which I could easily do by extending the the example on ImplicitGlobalGrid’s page: @views d_xa(A) = A[2:end , : , : ] .- A[1:end-1, : , : ];
I wanted to make sure that the @update_halo routine updates the appropriate boundary elements. Say, for a 6th order differentiation scheme for example, the boundary layer would have 3 elements on either side along each dimension.
I believe(not sure) the @update_halo macro works for updating single boundary elements along each dimension. Is there a method to extend this to a boundary layer with arbitrary width?
I assume this could be done using the @hide_communication macro in ParallelStencil but I was not sure about how I would go about using it.
I would appreciate if somebody has attempted to do so and could give any suggestions or advice.
The thickness of the halo is currently fixed to one cell in ImplicitGlobalGrid. It is on the road map to remove this limitation; however it will for sure not be feasible before the JuliaCon 2023 conference.
If you badly and urgently need it and you are not working with staggered grid, it might be possible to cast the arrays you pass to update_halo to CellArrays with cells as big has the required halo thickness and the global grid would have to be initialized accordingly. This would however also need to update ImplicitGlobalGrid to accept CellArrays. Then, I’m still not sure if it would work out of the box, and if yes, if the performance would be good out of the box. I think this would be the fastest track to get something working; a proper implementation will certainly take more time.
I wanted to use higher order numerical differentiation schemes which I could easily do by extending the the example on ImplicitGlobalGrid’s page: @views d_xa(A) = A[2:end , : , : ] .- A[1:end-1, : , : ];
This is an example without ParallelStencil. With ParallelStencil you can do something as discussed here: