# Julia 1.11.1 gives different results from 1.10.5

**URL:** <https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433>\
**Category:** General Usage\
**Created:** [December 4, 2024, 2:34am UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433 "2024-12-04T02:34:39Z")\
**Posts on this page:** 17\
**Page:** 3

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [December 5, 2024, 3:50am UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/42 "2024-12-05T03:50:44Z")

</div>

Fascinating! Glad I’m not a betting person. It depends on what you’re hashing; but this isn’t even guaranteed to be the same between restarts, let alone versions ([ref](https://docs.julialang.org/en/v1/base/base/#Base.hash)). You could try using an [ordered dictionary](https://juliacollections.github.io/OrderedCollections.jl/latest/ordered_containers.html).

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [December 5, 2024, 9:55am UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/43 "2024-12-05T09:55:01Z")

</div>

Sounds like your code could try different random partitioning schemes to see which one works best, like a Monte Carlo search.

---

<div class="post-metadata">

**Author:** ![world-peace](https://avatars.discourse-cdn.com/v4/letter/w/9f8e36/32.png) [@world-peace](https://discourse.julialang.org/u/world-peace)\
**Post date:** [December 5, 2024, 12:07pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/44 "2024-12-05T12:07:14Z")

</div>

> [@Palli](#):
>
> Not so.
> 
> You only need a working solution, in 1.10.5 here seemingly, then a non-working (needs not be minimal) code with (_deterministically_) wrong/other solution, and then you can bisect to the problem automatically.
> 
> I’ve never used `git bisect` personally but it’s a rather neat feature. Though ok, it might be construed to be _debugging_, without a debugger [program/activity].

Not so.

You always need to debug.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [December 5, 2024, 12:30pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/45 "2024-12-05T12:30:47Z")

</div>

You refer to:

> The hash value may change when a new Julia process is started.

It’s allowed, though I do not see it actually changing across restarts or Julia versions.

> [@PetrKryslUCSD](#):
>
> The partitioning is very different in the two julia versions. I am guessing that dictionaries (and/or sets)

I didn’t look into to confirm for Dicts, only the docs:

> Dict{K,V}() constructs a hash table

So it’s implied that hash is used and the order could change. Maybe the docs should be explicit on that, and recommend OrderedDict.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [December 5, 2024, 12:36pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/46 "2024-12-05T12:36:17Z")

</div>

> [@Palli](#):
>
> It’s allowed, though I do not see it actually changing across restarts or Julia versions.

You’re assuming what you see if part of the contract, despite the contract telling you that’s not the case, which is a great way to become a victim of the [Hyrum’s Law](https://www.hyrumslaw.com/).

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [December 5, 2024, 2:06pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/47 "2024-12-05T14:06:15Z")

</div>

> [@giordano](#):
>
> You’re assuming what you see if part of the contract, despite the contract telling you

No, I’m not _assuming_, I’m just stating I didn’t see it happening (yet), also I looked at the code, and can’t see it happening (at least for the types, like Int, I checked), I was just _implying_ this isn’t the cause. @jameson I see this maybe has never happened: [Make type hashing `:total` by Seelengrab · Pull Request #52427 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/52427)

> > I thought the hash of data types [could change between runs](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation)

> That was fixed on master recently, so we should remove that documentation

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [December 5, 2024, 2:19pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/48 "2024-12-05T14:19:02Z")

</div>

@Palli, the amount of unsubstantiated conjecture is just out of control. This does happen — and does so with regularity. Those quotes are only about hashing of `Type`s themselves. Here are two really simple objects that show the hash changing between restart and version:

```sh
▶ julia +1.10 -E 'hash(Ref(1)), hash(+)'
(0xbdcfb76821d76c23, 0x76519d57b640328b)

▶ julia +1.10 -E 'hash(Ref(1)), hash(+)'
(0xe4446a14eb42a20e, 0x76519d57b640328b)

▶ julia +1.11 -E 'hash(Ref(1)), hash(+)'
(0xc665e625b972c4a7, 0x86bb13ad0fb74bdb)

▶ julia +1.11 -E 'hash(Ref(1)), hash(+)'
(0xe122b7a2ac285a53, 0x86bb13ad0fb74bdb)

```

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [December 5, 2024, 3:00pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/49 "2024-12-05T15:00:02Z")

</div>

If this was a case of hashing different across restarts, then @PetrKryslUCSD would likely have hit the bug already in the older version?! I’m only trying to help here (as always), I can easily just stop doing that, maybe I will, or let it be up to him.

I never said hashing would for sure be stable, let alone for pointers, that would of course never happen i.e. for _different_ pointers, as you showed. [I don’t think hash(+) is showing anything very meaningful.]

> <https://stackoverflow.com/questions/14956313/why-is-dictionary-ordering-non-deterministic>

> I recently switched from Python 2.7 to Python 3.3, and it seems that while in Python 2 the ordering of dictionary keys was arbitrary but consistent

What we apparently need is more randomization (and a way to turn it off), like in Python:

> **[1. Command line and environment](https://docs.python.org/3/using/cmdline.html)**
>
> The CPython interpreter scans the command line and the environment for various settings. CPython implementation detail: Other implementations’ command line schemes may differ. See Alternate Impleme...

> -R
> 
> Turn on hash randomization. This option only has an effect if the [`PYTHONHASHSEED`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHASHSEED) environment variable is set to `0`, since hash randomization is enabled by default.  
> […]
> 
> Hash randomization is intended to provide protection against a denial-of-service caused by carefully chosen inputs that exploit the worst case performance of a dict construction, _O_(_n_^2) complexity. See [oCERT archive](http://ocert.org/advisories/ocert-2011-003.html) for details.

[Note, Python has changed to ordered dicts by default, what I’ve also proposed as a default for Julia… so then there’s no randomization in order. I assume _O_(_n_^2) denial-of-service attack still not possible.]

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [December 5, 2024, 5:54pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/50 "2024-12-05T17:54:44Z")

</div>

The point is that `hash` can change, so dict orderings can change. It can change between restarts and/or it can change between versions. It can also not change. If you don’t find the `hash(+)` example compelling, you can try `hash(Some(1))`.

An open issue suggesting Julia use hash randomization is julia#37166.

---

<div class="post-metadata">

**Author:** ![sgaure](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sgaure/32/14779_2.png) [@sgaure](https://discourse.julialang.org/u/sgaure)\
**Post date:** [December 5, 2024, 10:00pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/51 "2024-12-05T22:00:39Z")

</div>

Isn’t it conceivable that different LLVM versions (15 in 1.10.7, 16 in 1.11.2) can be responsible for subtle differences in optimization of floating point operations?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 5, 2024, 10:19pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/52 "2024-12-05T22:19:08Z")

</div>

We don’t let LLVM change floating point results unless the user explicitly gives permission (e.g. via `@fastmath`/`@simd`/`muladd`).

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [December 6, 2024, 8:10am UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/53 "2024-12-06T08:10:14Z")

</div>

> [@PetrKryslUCSD](#):
>
> Edit: Dictionaries are NOT at fault. Turns out I RANDOMIZE the partitioning myself for nicer graphical output… I just got to make randomization optional, that’s all… Still, something changed between 1.10 and 1.11. Perhaps different random seeds? Perhaps a different default random number generator?
> 
> Edit 2: Perhaps the problem is that I did not set the random number generator seed. Therefore the two Julia versions might have diverged by the time they got to executing my code.

Random number generation is not guaranteed to be identical between different Julia versions; that would effectively stop any improvements from happening in that area. This is documented at [Random Numbers · The Julia Language](https://docs.julialang.org/en/v1/stdlib/Random/) :

> Warning
> 
> Because the precise way in which random numbers are generated is considered an implementation detail, bug fixes and speed improvements may change the stream of numbers that are generated after a version change. Relying on a specific seed or generated stream of numbers during unit testing is thus discouraged - consider testing properties of the methods in question instead.

If you want to have fixed random number sequences in your tests you should use a generator from the [StableRNGs](https://github.com/JuliaRandom/StableRNGs.jl) package and set a fixed seed.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [December 6, 2024, 2:10pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/54 "2024-12-06T14:10:07Z")

</div>

> [@GunnarFarneback](#):
>
> If you want to have fixed random number sequences in your tests you should use a generator from the [StableRNGs](https://github.com/JuliaRandom/StableRNGs.jl) package and set a fixed seed.

Right because, you will get a random seed in Julia for the default RNG when starting, and thus a different sequence.

That said, I doubt this alone is the reason for your changed result, because shouldn’t you then have gotten it on 1.10.5 too, after a few resets? This might be part of the reason explaining 1.11 change.

If you want to keep ~~this~~ [EDIT: rand in at least single-threaded] constant [but NOT `shuffle`, so it might be your problem in 1.11] you can do:

```julia
using Random; Random.seed!(1234);

julia> rand() # now reproducable across 1.10 and 1.11, and down to only 1.7:
0.32597672886359486

```

The currently used RNG, that I proposed adopting (at the time, now also in the current LTS, not the old one), and was adopted in 1.7, could of course be changed again later, so don’t rely on it not changing. But for a quick test might be helpful for you. I’m not actually sure what happens with many threads, then they will get different streams usually, and probably still with my suggestion, so also run with just one thread.

I realized even Libc.malloc, because of e.g. glibc, uses random, and it’s used for huge allocations, to get then random locations for them in memory. Julia takes over for smaller allocations, then doesn’t call malloc for each one, but would still be affected, since Julia needed malloc to initialize memory pools. The location of allocations doesn’t matter in any sense for correctness (usually… unless your code is sensitive to the locations or hashing of them), but yes, it affects hashing of pointers, and Ref then as @mbauman showed. It’s probably very hard to get allocations deterministic (from within Julia), since Julia relies on glibc, that does the randomization. I don’t think C has an API to allow bypassing it, since this is a security feature.

But you could try:

> <https://stackoverflow.com/questions/20003431/disable-the-randomness-in-malloc>

> If the variation is caused by address space layout randomization, then, according to [this page](https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization), you can disable it with:
> 
> ```julia
> echo 0 | sudo tee /proc/sys/kernel/randomize_va_space
> 
> ```
> 
> This should be done only temporarily for debugging purposes.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [December 6, 2024, 2:32pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/55 "2024-12-06T14:32:38Z")

</div>

@Palli, no. None of that is relevant and again there are a number of inaccuracies.

Even with seeding, even if the RNG itself didn’t substantively change, the ways in which Random _uses_ the RNG may change. I’m guessing it may be this change — or if not, it’s something of the same category:

```sh
▶ julia +1.10 -E 'using Random; Random.seed!(0); shuffle(1:4)'
[2, 4, 1, 3]

▶ julia +1.11 -E 'using Random; Random.seed!(0); shuffle(1:4)'
[4, 1, 3, 2]

```

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [December 6, 2024, 3:09pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/56 "2024-12-06T15:09:43Z")

</div>

I tracked down to seemingly this change (for shuffle at least):

> <https://github.com/JuliaLang/julia/commit/2d4d096c599cb92fc2ea2a107693ac1293edcadc>

It’s apparently very hard to get all randomization effects to be the same. All code depending on rand or similar needs to be the same, which it isn’t across versions (nor guaranteed to be).

You can try restoring the old shuffle in 1.11, from 1.10:

```julia
import Random.shuffle!

function shuffle!(r::AbstractRNG, a::AbstractArray) # I only added needed Base.
    Base.require_one_based_indexing(a)
    n = length(a)
    n <= 1 && return a # nextpow below won't work with n == 0
    @assert n <= Int64(2)^52
    mask = nextpow(2, n) - 1
    for i = n:-1:2
        (mask >> 1) == i && (mask >>= 1)
        j = 1 + rand(r, Base.ltm52(i, mask))
        a[i], a[j] = a[j], a[i]
    end
    return a
end

```

I can’t guarantee that’s the only change in 1.11 affecting your code…

[julia/stdlib/Random/src/Random.jl at v1.11.2 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/v1.11.2/stdlib/Random/src/Random.jl) is a stdlib, is it upgradable? I.e. can you use Random from 1.11 in 1.10? And vice versa?

---

<div class="post-metadata">

**Author:** ![Dan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dan/32/42581_2.png) [@Dan](https://discourse.julialang.org/u/Dan)\
**Post date:** [December 6, 2024, 4:05pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/57 "2024-12-06T16:05:52Z")

</div>

Isn’t it easier to detect sensitivity to RNG by just reseeding the working old version with several different seeds and looking at the effect?

My guess the randomization is not the critical difference. It may be exposing a Heisenbug ([Heisenbug - Wikipedia](https://en.wikipedia.org/wiki/Heisenbug)) somewhere else e.g. rounding error picking different branch in some conditional, like 0.999999 not being \>=1. If the bad behaviour is consistently repeatable, then the bug should be defeatable.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [December 7, 2024, 7:17pm UTC](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433/58 "2024-12-07T19:17:36Z")

</div>

Randomness tracked down to the Metis graph partitioning library.

[Previous page](https://discourse.julialang.org/t/julia-1-11-1-gives-different-results-from-1-10-5/123433.md?page=2)