ProfileView.@profview seemingly breaks reproducibility of my results

Tooling Profiling
1

I’ve written an algorithm (spanning many, many lines) to solve a certain optimal control problem, which involves both simulation and optimisation. To make sure that different runs provide the same results, I set ‘Random.seed!(42)’ at the beginning of my file. And indeed, executing the active file in REPL multiple times in a row (in VS Code) gives me the same results consistently.

However, now I’m profiling the code with ProfileView.@profview, and a weird thing happens: after running the code once, I get some results, (-0.36165, -0.3616). Running the code again, I get (-0.36163, -0.38507), which keeps happening on any further re-run.

The only thing that I can imagine is that ProfileView.@profview somehow interferes with something that also affects my simulation/optimisation calls, but I have not been able to reproduce this in a MWE.

What is the advised way to proceed here? I’d like to get to the bottom of this, to make sure that there isn’t actually some error in my code. For this, I think the end-goal would be to identify if ProfileView.@profview modifies some global variable or something?

2

To get some idea of the set-up of my code:

#include some stuff
Random.seed!(42)

a, b, c, d, e = GetParameters()
ProfileView.@profview Run_Alg(a,b,c,d)

What is a bit annoying is that the results after running profview are also worse for the particular algorithm I’m implementing, which makes me extra sceptical.

3

My guess is that @profview modifies the global RNG state.
To determine performance statistics without affecting performance it can not do extremely dense samples so it probably relies on statistical sampling. If this also calls rand() somewhere, this will lead to a different RNG state also in your program.

1 Like
4

Thank you for responding! This was what I was thinking as well; perhaps I shouldn’t be too suspicious of this behaviour then.

5

If you really need reproducibility of random numbers, you should probably create your own RNG instance and use that rather than using the global instance. Further, Julia does not guarantee random behavior across versions, so if you need to guarantee the sequence for future versions, you should use something like StableRNGs.jl.

1 Like
6

That said, it probably would be a good idea for ProfileView to use it’s own RNG stream.

3 Likes
7

I don’t need that level of reproducibility, just that if I run the same code twice in a row, that the results should be the same. I concur that ProfileView having its own RNG stream would be very nice.