Simulation reset causes non-reproducible simulations

[lukicdarkoo]

Describe the Bug
After the simulation reset, the behavior of the robots is not exactly the same. There are very tiny deviations, but enough to consider the simulations non-reproducible.

Steps to Reproduce

1. Download reproducibility.zip
2. Start the world (let it run for 2-3 seconds)
3. Reset the simulation (let it run for 2-3 seconds)
4. There should be two files in the controller's directory, one from step 2 and the other one from step 3. Compare them. Only the first row should be the same.

With PROTOs (this may be a better example):

1. Download reproducability_proto.zip
2. cd controllers/position_tracker
3. webots ../../worlds/reproducability_proto.wbt
4. Put it in the fast mode and reset the simulation like 20 times
5. python3 check.py to check for deviations

[ad-daniel]

I can't seem to replicate it, maybe I'm misunderstanding

• if you mean the box behaves differently after each call of node.restartController() in the supervisor then that's normal as only the controller is reset, not the physics. Calling node.resetPhysics() after node.restartController() gets the job done
  

• if instead you mean that manually resetting the world multiple times after letting it run a bit generates different CSVs, then I can't replicate on Ubuntu 20.04 (compiling from the latest master branch). PS: here I use your ZIP as is, without resetting the node physics
  
  can't see much but here's the difference between the later CSVs and the first one
  

[lukicdarkoo]

Try to restart the world manually using:

Then, compare two generated files using simple diff (instead of plotting).

---

P.S. Let me check again.
P.S2. I have just checked on my laptop and indeed it is reproducible. Initially, I checked on a different computer. I need to further investigate this.

[lukicdarkoo]

@ad-daniel I updated the steps and the example, so it is easier to reproduce the "non-reproducibility". Sorry if you spend a lot of time because of my vague description.

[lukicdarkoo]

I would leave this issue open until the second problem is resolved (described above, related to https://github.com/cyberbotics/webots/files/5856675/reproducability_proto.zip).

[lukicdarkoo]

@ad-daniel I assigned this to you as you have already done a lot of work around PROTOs. Maybe it is even fixed with the new improvements. If not and it takes a lot of time to be fixed I believe it is ok to postpone.