I fond what Guy Steele mentioned in the last half of this talk interesting:
IIRC there is a type of SRNG that can be split in such a way that the distribution of resulting streams combined is equivalent to the original stream (or something like that). It would be nice to look into (probably some follow up studies of):
http://supertech.lcs.mit.edu/papers/dprng.pdf
which were mentioned in the talk.
My understanding is that this would guarantee that parallel computation result with SRNG to be deterministic even when the scheduler is dynamic (as in Julia), if you construct the computation as some combination of map/filter/reduce/etc. (edit: hmm… wait, does it work for any computation?)