Random number generation is not guaranteed to be identical between different Julia versions; that would effectively stop any improvements from happening in that area. This is documented at Random Numbers · The Julia Language :
Warning
Because the precise way in which random numbers are generated is considered an implementation detail, bug fixes and speed improvements may change the stream of numbers that are generated after a version change. Relying on a specific seed or generated stream of numbers during unit testing is thus discouraged - consider testing properties of the methods in question instead.
If you want to have fixed random number sequences in your tests you should use a generator from the StableRNGs package and set a fixed seed.