test(restore_latency): do not reuse netns

[kalyazin]

Changes

Disable netns reuse for test_restore_latency test.

Reason

This is because with netns reuse, the first test variant spends time on initialising netns, while the subsequent test variants do not. Disable netns reuse to make the performance data consistent.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

• I have read and understand CONTRIBUTING.md.
• I have run tools/devtool checkstyle to verify that the PR passes the
  automated style checks.
• I have described what is done in these changes, why they are needed, and
  how they are solving the problem in a clear and encompassing way.
• [ ] I have updated any relevant documentation (both in code and in the docs)
  in the PR.
• [ ] I have mentioned all user-facing changes in CHANGELOG.md.
• [ ] If a specific issue led to this PR, this PR closes the issue.
• [ ] When making API changes, I have followed the
  Runbook for Firecracker API changes.
• [ ] I have tested all new and changed functionalities in unit tests and/or
  integration tests.
• [ ] I have linked an issue to every new TODO.

---

• This functionality cannot be added in rust-vmm.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.91%. Comparing base (033825d) to head (cb28847).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5287      +/-   ##
==========================================
+ Coverage   82.86%   82.91%   +0.05%     
==========================================
  Files         250      250              
  Lines       26897    26897              
==========================================
+ Hits        22288    22302      +14     
+ Misses       4609     4595      -14     

Flag	Coverage Δ
5.10-c5n.metal	83.35% <ø> (ø)
5.10-m5n.metal	83.34% <ø> (ø)
5.10-m6a.metal	82.56% <ø> (+<0.01%)	⬆️
5.10-m6g.metal	79.17% <ø> (ø)
5.10-m6i.metal	83.34% <ø> (-0.01%)	⬇️
5.10-m7a.metal-48xl	82.55% <ø> (?)
5.10-m7g.metal	79.17% <ø> (ø)
5.10-m7i.metal-24xl	83.30% <ø> (?)
5.10-m7i.metal-48xl	83.31% <ø> (?)
5.10-m8g.metal-24xl	79.17% <ø> (?)
5.10-m8g.metal-48xl	79.17% <ø> (?)
6.1-c5n.metal	83.40% <ø> (ø)
6.1-m5n.metal	83.40% <ø> (+<0.01%)	⬆️
6.1-m6a.metal	82.61% <ø> (+<0.01%)	⬆️
6.1-m6g.metal	79.17% <ø> (ø)
6.1-m6i.metal	83.38% <ø> (-0.01%)	⬇️
6.1-m7a.metal-48xl	82.60% <ø> (?)
6.1-m7g.metal	79.17% <ø> (ø)
6.1-m7i.metal-24xl	83.40% <ø> (?)
6.1-m7i.metal-48xl	83.41% <ø> (?)
6.1-m8g.metal-24xl	79.17% <ø> (?)
6.1-m8g.metal-48xl	79.17% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[roypat]

Mh, do you see any way we can make this work even if someone only runs a subset of these tests? Maybe something as simple as just always dropping the first datapoint when emitting metrics?

[kalyazin]

Mh, do you see any way we can make this work even if someone only runs a subset of these tests? Maybe something as simple as just always dropping the first datapoint when emitting metrics?

Good point, let me see. The problem is not a single datapoint for some reason, but the entire test result.

[kalyazin]

Mh, do you see any way we can make this work even if someone only runs a subset of these tests? Maybe something as simple as just always dropping the first datapoint when emitting metrics?

Good point, let me see. The problem is not a single datapoint for some reason, but the entire test result.

@roypat Playing with it a bit more, I can't really think of a better way. The netnses are cached between "functions" (ie tests). After a test finishes, it returns all its nses to the pool so subsequent tests can reuse them. There is no reuse within a test, so we can't run something every test that would create an ns and omit metrics from such execution. I was thinking of "prefilling" nses at every test, ie create and return them to the pool immediately. But this doesn't help, because even though the nses are preallocated, they aren't used/initialised (the ioctl isn't called), so they aren't any faster than the ones created on-demand.