I’m working on a (not very sophisticated) finite volume solver for my research, and I’ve run into a perplexing problem around storing the mesh in such a way that minimizes allocations.
The basic idea is that every cell also “knows” about its neighboring cells, and knows where the face between itself and each neighbor is. From this, it’s possible to calculate interfacial areas, etc, etc.
In order to improve the performance of the solver, I’ve decided to partition the mesh among threads and have the threads apply the FVM update rule. However, after partitioning, there’s no correlation between the a given cell’s ID and its index in an array of cells. I’ve tried using dictionaries with sizehint!, but I’m wasting a lot of time on GC by occasionally have to create large dictionaries.
Is there an existing data structure package that allows for fast lookup (like a dictionary), but also lets me pre-allocate a given buffer size so that I can re-use the buffers between time steps?
You could use a tree datastructure and store the tree nodes just in a flat array. Then you can reuse that memory and have rather fast O(\log N ) lookup. Generally the choice of datastructure will depend on the element it contains, e.g. for small lookup tables (10-20 elements) a simple list with linear search will beat most sophisticated data structures IIRC.
Is there any penalty to sharing a buffer between threads, provided no thread writes to the same location in the buffer?
What if each thread only reads from the shared buffer? I don’t really know how Julia’s thread system works internally.
I am not aware of any overhead associated with concurrent reading of an array. Julia’s arrays are rather thing wrapper over just plain memory (think C arrays). So the overhead, if there is one, should be the same as in C.
Julia’s threads are regular system threads (provided by libpthread IIRC). I am also not sure whether just writing to an array makes a large difference because there is no built-in synchronization (unless you use atomics).
1 Like