I created a PR to limit the single-threading to the one call where it is currently necessary.
I also created a Julia issue for fixing the last place where single-threading is needed. The latter one needs some in-depth mutex knowledge, I assume. But we have a commit which probably introduced the problem and we have a test case, so it sounds possible to hunt that bug down.
Please have a look if you know something about these things.