# Identical code produces different results. sometimes

**URL:** <https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658>\
**Category:** General Usage\
**Tags:** question\
**Created:** [May 11, 2017, 8:56pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658 "2017-05-11T20:56:43Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 11, 2017, 8:56pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/1 "2017-05-11T20:56:43Z")

</div>

I have a very strange bug in my program. The same code run in different sessions produces different results. It is impossible for me to produce a MWE because I cannot reproduce this behaviour in a small example. It only happens in the real application, and that is way too long to share here. I apologise for this, but there is nothing I can do about it and I’m really lost here. Thanks for having a look.

##Test

Here is the core of my test:

```julia
# I have module called bk that contains the code.
function testrun()
	p = bk.Param() # create a parameter type
	m = bk.Model(p); # create a model type
	bk.solve!(m,p); # solve the model
       # get some objects from that solved model
       # those are Array{Float64}
       # not SharedArray or something fancy.
	vown = copy(m.Owner.DW_V)
	vbar = copy(m.Owner.vbar)
	vbar13 = copy(m.Owner13.vbar)
	ccp13 = copy(m.Owner13.CCP)
	vrent = copy(m.Renter.DW_V)
	vown7 = copy(m.Owner7.DW_V)
	vown13 = copy(m.Owner13.DW_V)
	evown = copy(m.Owner.EV)

	# completely erase that model
	m = 0
	gc()

	println("")
	info("run 2")
	println("")

       # create a new model. same parameter value.
	m2 = bk.Model(p);
	bk.solve!(m2,p);
        # see whether the result is the same
	dd = maxabs(vown[:] - m2.Owner.DW_V[:])
	dr = maxabs(vrent[:] - m2.Renter.DW_V[:])
        # success?
	succ = (dd == 0.0) && (dr == 0.0)

	m2 = 0
	gc()
	return succ
end

```

- There is no `rand` anywhere in this program. it is a deterministic solution. a given `p` should imply a unique solution.
- I start a first Julia session and run this test in a loop many times. it succeeds.
- I start a _second_ julia session and the test fails right away: the two models `m` and `m2` are not identical. The error is very large.
- this is very erratic: sometimes the test fails, sometimes it doesnt.
- The memory footprint of each julia session is about 2.5Gb.
- I can suppress this behaviour by commenting out a certain section of my code. I can’t find anything wrong with that section, it is very similar to several other parts of the code. _it seems to work in session number one_.

## Questions

- Is there anything non-deterministic in the way julia generates code across different sessions?
- Could there be some strange numerical error, overflow/underflow for example, that could only occur when my computer is in a certain state? like some part of memory is empty? or some kind of process runs during compilation?
- I just ran this test successfully for a 100 times. i exit julia, run it again and it fails on the first run. I call the test again (in the same session) _and now it runs fine_. How is this possible?
- Could/Should I use valgrind to track this down? how?
- thanks.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 11, 2017, 9:05pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/2 "2017-05-11T21:05:27Z")

</div>

> Is there anything non-deterministic in the way julia generates code across different sessions?

Yes. All of the pointers will be different.

> Could there be some strange numerical error, overflow/underflow for example, that could only occur when my computer is in a certain state? like some part of memory is empty? or some kind of process runs during compilation?

If you didn’t initialize an array, yes.

> Could/Should I use valgrind to track this down? how?

I don’t think valgrind will be helpful (unless you are calling buggy C code). You can track where the result starts to diviate.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 12, 2017, 1:02am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/3 "2017-05-12T01:02:04Z")

</div>

Change every memory allocation to `zeros` to make sure the arrays are zeroed. Do you still have this problem?

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 12, 2017, 6:30am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/4 "2017-05-12T06:30:12Z")

</div>

You mean it’s not enough to set the type that holds the arrays to zero as I do, but rather go inside the type and zero out each array individually? I thought doing what I do destroys the entire object. I’ll try!

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 12, 2017, 7:01am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/5 "2017-05-12T07:01:55Z")

</div>

> [@floswald](#):
>
> set the type that holds the arrays to zero

It is not clear what you mean here — how do you set a type to `0`? What @ChrisRackauckas meant is that you should create arrays that are meant to contain zeros and not initialized otherwise with `zeros`, because constructors like `Array{Float64}(...)` just contain random values.

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 12, 2017, 7:26am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/6 "2017-05-12T07:26:41Z")

</div>

i thought

```julia
type m
     x :: Array
end

mm = m(rand(10))

mm = 0 #erases the array x?

```

that list line is what i meant.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 12, 2017, 8:04am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/7 "2017-05-12T08:04:38Z")

</div>

You are mistaken: it just sets `m` to the integer `0`. Use `m(zeros(10))` in the above example.

Also, if you ask questions here, investing effort into a MWE pays off.

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 12, 2017, 8:42am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/8 "2017-05-12T08:42:36Z")

</div>

thanks for that - I didn’t know that!  
Apologies again for a non MWE - impossible under my current constraints. However, if you looked at my example, what I find is equivalent to

```julia
type myT
    x :: Array
end

t1 = myT(ones(3))
t2 = myT(ones(3))

maxabs(t1.x[:] - t2.x[:]) == 0.0 #is false

```

with the difference that my code assigns the values to array `x`, instead of the constructor, as here. Again, the weird thing is that sometimes this test passes if i repeat this for 100 times, and sometimes it does not. whether I copy `x` to a separate object and erase `t1` before doing the test shouldn’t matter. `t1` and `t2` should always refer to _different objects_?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 12, 2017, 8:56am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/9 "2017-05-12T08:56:27Z")

</div>

> [@floswald](#):
>
> what I find is equivalent to

Sorry, but lacking an MWE, we only have your claim for this equivalence. If your example indeed gave `false`, that would be a serious bug in Julia. An error on your part is much more likely.

> [@floswald](#):
>
> MWE - impossible under my current constraints.

Nope, an MWE is never impossible, you are just unwilling to invest time in making one. Yet you expect others to help you.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [May 12, 2017, 11:55am UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/10 "2017-05-12T11:55:37Z")

</div>

Nine out of ten times this has happened to me, it has been array initialization. Very often you’ll see very small values, but there’s no guarantee. See how far the codes get before diverging. Then strip away the unnecessary parts (what can you remove without the divergence disappearing), and post whatever steps your code goes through up till that point. That’s your mwe.

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [May 12, 2017, 12:04pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/11 "2017-05-12T12:04:11Z")

</div>

> [@floswald](#):
>
> i exit julia, run it again and it fails on the first run.

Initially, any new memory that you allocate will have been zeroed out by the operating system. After the first garbage collection, new memory that you allocate might contain old data.

If you run the exact same code several times, with explicit garbage collection in between iterations, then you are likely to get objects that are initialized to their old values! This may effectively hide bugs where you read data before you have written it.

Example: (This is what happened for me. Your results would be different.)

```julia
               _
   _ _ _(_)_ | A fresh approach to technical computing
  (_) | (_) (_) | Documentation: https://docs.julialang.org
   _ _ _| |_ __ _ | Type "?help" for help.
  | | | | | | |/ _ | |
  | | |_| | | | (_| | | Version 0.6.0-rc1.0 (2017-05-07 00:00 UTC)
 _/ |\ __'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-apple-darwin13.4.0

julia> m = Array{Float64}(10)
10-element Array{Float64,1}:
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0

julia> m=0; gc()

julia> m = Array{Float64}(10)
10-element Array{Float64,1}:
 5.28715e242 
 7.34851e-307
 0.0         
 0.0         
 0.0         
 0.0         
 0.0         
 0.0         
 0.0         
 0.0         

julia> m[1] = 42
42

julia> m = 0; gc()

julia> m = Array{Float64}(10)
10-element Array{Float64,1}:
 42.0         
  7.34851e-307
  0.0         
  0.0         
  0.0         
  0.0         
  0.0         
  0.0         
  0.0         
  0.0         

```

Moral of the story is, make sure your objects are always properly initialized.

(There’s already an issue on this: [https://github.com/JuliaLang/julia/issues/9147](https://github.com/JuliaLang/julia/issues/9147) )

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 12, 2017, 1:22pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/12 "2017-05-12T13:22:38Z")

</div>

this is a great example @Per, thanks!

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [May 12, 2017, 1:23pm UTC](https://discourse.julialang.org/t/identical-code-produces-different-results-sometimes/3658/13 "2017-05-12T13:23:57Z")

</div>

The culprit was indeed: an array initialization with `y=similar(x)`.