Note that pop! for Set (which is a Dict) is not thread safe at the moment, and it unlikely to ever be. If I wanted to consume an unordered collection by threads, I would just design an appropriate data structure for this (with locks etc).
In any case, I think that this use case (mutation of unordered containers by multiple threads) is a red herring, and just distracts from the main API.
Personally, I would prefer a clean, simple API, with some clear contracts, even if it does not support all possible combinations one can think of — in fact, I would consider not supporting some methods an advantage when it cannot be done in a consistent way.