Different dict iteration order between runs when altering unrelated code

New to Julia
1

I’m working on a game simulation. It uses a fixed set of input files to initialize all game state with. To generate random choices I have a single MersenneTwister instance that I allocate at startup, the seed passed to it is specified on the command-line. All calls to rand and shuffle! pass this RNG instance as first argument. So I assume I have all uses of random numbers tied to the seed value provided in order to be able to reproduce and debug failed runs, like in case of a crash or incorrect result.

But I’m trying to figure out whether I’m doing something dumb, or if there is something else going on. The issue I’m facing is that subsequent runs of my program with the same set of input files and same seed value produce the same set of results (as expected), but when I add a println() or @info call for debugging a failed run I suddenly get a different set of results. In the log files I can see that apparently the iteration order of certain dict values has changed (which then leads to a different random choice, different actions being played, etc).

So a first question if is guaranteed in Julia that dict iteration with says keys() and values() will produce the same order between invocations of julia if the underlying dict contains the same values? Of course I assume an unchanged Julia version, same optimization level, etc.

Second question is if calls to print stuff could influence things like dict iteration order? I don’t see how, as I assume the dict code is fully self-contained and does not interact with such calls.

Any other ideas what could be going on? As I mentioned in the beginnen, I might just still be doing something dumb somewhere, but I don’t see it.

Edit: first version was accidentally posted before I was done writing

2

If you need a given iteration order for dictionaries you can use an OrderedDict from GitHub - JuliaCollections/DataStructures.jl: Julia implementation of Data structures.

1 Like
3

Right, that’s always an option. But I’m trying to figure out why I’m seeing what I’m seeing.

5

Okay, I seem to have found at least one source of variation. Some of the values I store in the dicts are tuples containing enums created with @enum. And apparently those do not have a consistent hash value if they’re defined in a module that gets recompiled on-the-fly due to addition of the debug statements I add in the code making up the module.

Example that reproduces this:

$ cat TestEnum.jl 
module TestEnum

export go

@enum E begin
    A
    B
    C
end

function go()
    #println("Hash:")
    println(hash(A))    
end

end

$ cat e.jl 
push!(LOAD_PATH, ".")
using TestEnum

go()

melis@juggle 20:11:~ $ julia e.jl 
17762642078520566858
melis@juggle 20:11:~ $ julia e.jl 
17762642078520566858
melis@juggle 20:11:~ $ julia e.jl 
17762642078520566858
melis@juggle 20:11:~ $ julia e.jl 
17762642078520566858

# After uncommenting the println() in TestEnum.jl:

melis@juggle 20:11:~ $ julia e.jl 
Hash:
5005584565647694511
melis@juggle 20:11:~ $ julia e.jl 
Hash:
5005584565647694511

Interestingly, when not using a module to define the enums the additional println statement does not have any influence:

$ cat e2.jl 
@enum E begin
    A
    B
    C
end

function go()
    #println("Hash:")
    println(hash(A))    
end

go()

melis@juggle 20:13:~ $ julia e2.jl 
10831397044442211192
melis@juggle 20:13:~ $ julia e2.jl 
10831397044442211192
melis@juggle 20:13:~ $ julia e2.jl 
10831397044442211192
melis@juggle 20:13:~ $ julia e2.jl 
10831397044442211192

# Again, after uncommenting the println() statement:

melis@juggle 20:13:~ $ julia e2.jl 
Hash:
10831397044442211192
melis@juggle 20:13:~ $ julia e2.jl 
Hash:
10831397044442211192
melis@juggle 20:13:~ $ julia e2.jl 
Hash:
10831397044442211192

So the enum values changing their hash causes the dict entries to get ordered differently, etc. This is a bit surprising, given that I would expect enums to be merely a symbolic name for an integer. But since they’re implemented with a macro perhaps some more exotic things are going on. And I haven’t looked into macros enough yet to dive into this behaviour at this point.

2 Likes