Re-enabling continuous benchmarking on vuejs/core

[adriencaccia]

Hey, I am Adrien from CodSpeed

Last month, @sxzz added benchmarks to the repo and enabled CodSpeed to track performance in #9638. From then on, a CI job ran on every pull request to track performance.

The benchmark job was then disabled in 8b25982.

I noticed that there were some flaky benchmarks, resulting in performance regressions comments appearing on completely unrelated pull requests. For example here, I discuss the potential origin of the flakiness.

Is it part of the reason why it was disabled?

I would love to help enable them back again and ensure the experience is as smooth as possible 😉

[adriencaccia]

Hey @sxzz, @yyx990803

We just released a feature to ignore a benchmark on CodSpeed. An ignored benchmark is removed is still run, but its measure is removed from the performance report.

This feature should allow you to re-enable CodSpeed, ignore the flaky packages/reactivity/__tests__/reactiveObject.bench.ts::write reactive obj, read 1000 computeds benchmark and start benefitting from CodSpeed again.
We will also be able to collect more data on the specific flaky benchmarks, which will help us find the source of the problem and fix it 🤞

Let me know if you have any questions 😉