Afaiu, the GC will never ever become compacting, for reasons of C interoparability.
Consider pulling a deepcopy of your data. This is, by the way, not just for fragmentation but will also allocate such that any operation that traverses your structure in deepcopy-order is more cache-friendly.