From aa368a8e5438837d01f309fbd5f9910242342d65 Mon Sep 17 00:00:00 2001 From: Ryan Levick Date: Tue, 28 Mar 2023 16:46:13 +0200 Subject: [PATCH 1/2] Add triage for 2023-03-28 --- triage/2023-03-28.md | 234 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 234 insertions(+) create mode 100644 triage/2023-03-28.md diff --git a/triage/2023-03-28.md b/triage/2023-03-28.md new file mode 100644 index 000000000..4f1812269 --- /dev/null +++ b/triage/2023-03-28.md @@ -0,0 +1,234 @@ +# 2023-03-28 Triage Log + +A busy week with lots of real performance gains. Most regressions seemed to be due to noise. The biggest highlight was large wins in incremental compilation leading to a lot of (albeit modest gains) of 1% performance in a majority of incremental compilation test scenarios. Other than that most performance gains were smaller and more incremental. One of the biggest performance regressions came in an update too LLVM. However, nearly just as many test cases showed improvements as regressions. + +Triage done by **@rylev**. +Revision range: [ef03fda3..cbc064b3](https://perf.rust-lang.org/?start=ef03fda339923e659d3d3ca3321de887316d2807&end=cbc064b341be231403d181402a786cce7f1c73f1&absolute=false&stat=instructions%3Au) + +**Summary**: + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:---------------:|:-----:| +| Regressions ❌
(primary) | 1.7% | [0.5%, 3.5%] | 24 | +| Regressions ❌
(secondary) | 1.2% | [0.2%, 2.6%] | 18 | +| Improvements ✅
(primary) | -1.5% | [-10.9%, -0.3%] | 168 | +| Improvements ✅
(secondary) | -4.0% | [-65.3%, -0.4%] | 119 | +| All ❌✅ (primary) | -1.1% | [-10.9%, 3.5%] | 192 | + + +3 Regressions, 7 Improvements, 8 Mixed; 5 of them in rollups +46 artifact comparisons made in total + +#### Regressions + +Add RANLIB_x86_64_unknown_illumos env for dist-x86_64-illumos dockerfile [#109163](https://github.com/rust-lang/rust/pull/109163) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=6502613a8199cab4cb68b08c7ddce6f210d8d69c&end=439292bc7913399e406d9bb7e8da0f70c6317c6e&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:----:|:------------:|:-----:| +| Regressions ❌
(primary) | 0.8% | [0.7%, 0.8%] | 4 | +| Regressions ❌
(secondary) | 0.5% | [0.4%, 0.7%] | 9 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | 0.8% | [0.7%, 0.8%] | 4 | +- Noise + + +Make NLL Type Relating Eager [#108861](https://github.com/rust-lang/rust/pull/108861) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=8859fde21fa4e059c71a843cc7ee995c189a97c0&end=1c771fec3329ef1d8c66697daf9517501d0c129a&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.6% | [0.3%, 0.8%] | 7 | +| Regressions ❌
(secondary) | 0.5% | [0.3%, 0.6%] | 10 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -0.8% | [-0.8%, -0.8%] | 1 | +| All ❌✅ (primary) | 0.6% | [0.3%, 0.8%] | 7 | +- A good chunk of the regressions are noise, and the rest are small enough that I don't think it's worth looking too deeply into. + + +Refactor `try_execute_query` [#109100](https://github.com/rust-lang/rust/pull/109100) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e10eab5956f91c26dcc5ae29a19cfcd747047e4d&end=b72e896268bd741a2a2bb10028dca9e6f8d0ae63&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:----:|:------------:|:-----:| +| Regressions ❌
(primary) | 0.7% | [0.7%, 0.8%] | 4 | +| Regressions ❌
(secondary) | 0.5% | [0.3%, 0.7%] | 8 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | 0.7% | [0.7%, 0.8%] | 4 | +- Noise + + +#### Improvements + +Rollup of 10 pull requests [#109442](https://github.com/rust-lang/rust/pull/109442) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ef03fda339923e659d3d3ca3321de887316d2807&end=a01b4cc9f375f1b95fa8195daeea938d3d9c4c34&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.4% | [-0.5%, -0.3%] | 4 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | -0.4% | [-0.5%, -0.3%] | 4 | + + +mv tests/codegen/issue-* tests/codegen/issues/ [#109172](https://github.com/rust-lang/rust/pull/109172) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1c771fec3329ef1d8c66697daf9517501d0c129a&end=cf811810fe8dc92b3ff25e7dd29fb3178c91d409&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.6% | [-0.7%, -0.6%] | 4 | +| Improvements ✅
(secondary) | -0.5% | [-0.6%, -0.4%] | 2 | +| All ❌✅ (primary) | -0.6% | [-0.7%, -0.6%] | 4 | + + +Rollup of 7 pull requests [#109517](https://github.com/rust-lang/rust/pull/109517) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9a6b0c3326f3577f330d8828d3b706c0eea9cf29&end=99c49d95cd7d704d2226859cfdc419c930cde9c6&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.4% | [-0.4%, -0.4%] | 1 | +| Improvements ✅
(secondary) | -0.5% | [-0.7%, -0.4%] | 6 | +| All ❌✅ (primary) | -0.4% | [-0.4%, -0.4%] | 1 | + + +Don't pass `TreatProjections` separately to `fast_reject` [#109202](https://github.com/rust-lang/rust/pull/109202) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1459b3128e288a85fcc4dd1fee7ada2cdcf28794&end=cf073ec2cbf0b90ded7893f429020b81dd28e6c3&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:---------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.4% | [0.4%, 0.4%] | 1 | +| Improvements ✅
(primary) | -0.9% | [-2.3%, -0.2%] | 17 | +| Improvements ✅
(secondary) | -2.6% | [-10.5%, -0.4%] | 24 | +| All ❌✅ (primary) | -0.9% | [-2.3%, -0.2%] | 17 | + + +Add #[inline] to the Into for From impl [#109546](https://github.com/rust-lang/rust/pull/109546) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=d012d2f96e2959267b14812929b85e17a155f653&end=24a69af213f4bf23014dc91903e7a934c88f6b9e&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.8% | [0.8%, 0.8%] | 1 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.8% | [-1.5%, -0.4%] | 15 | +| Improvements ✅
(secondary) | -0.9% | [-1.4%, -0.3%] | 19 | +| All ❌✅ (primary) | -0.7% | [-1.5%, 0.8%] | 16 | + + +Optimize `incremental_verify_ich` [#109371](https://github.com/rust-lang/rust/pull/109371) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9fa6b3c15758e85657d5be051cfa57022a8bbe57&end=e10eab5956f91c26dcc5ae29a19cfcd747047e4d&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.9% | [-1.6%, -0.4%] | 78 | +| Improvements ✅
(secondary) | -1.1% | [-2.2%, -0.5%] | 39 | +| All ❌✅ (primary) | -0.9% | [-1.6%, -0.4%] | 78 | + + +Rollup of 7 pull requests [#109581](https://github.com/rust-lang/rust/pull/109581) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b72e896268bd741a2a2bb10028dca9e6f8d0ae63&end=96bd50dd47b645efa52d1312b9cc32e507d9cd76&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.7% | [-0.7%, -0.7%] | 2 | +| Improvements ✅
(secondary) | -0.6% | [-0.7%, -0.6%] | 5 | +| All ❌✅ (primary) | -0.7% | [-0.7%, -0.7%] | 2 | + + +#### Mixed + +Only clear written-to locals in ConstProp [#109087](https://github.com/rust-lang/rust/pull/109087) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=6dc3999c2699461aa930b8c1e00f99e73dcc0174&end=5fa73a75ce94faf1d1008ec96c043134ef0df427&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:------:|:---------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.5% | [0.5%, 0.5%] | 3 | +| Improvements ✅
(primary) | -2.7% | [-6.1%, -0.8%] | 27 | +| Improvements ✅
(secondary) | -15.8% | [-63.2%, -0.2%] | 17 | +| All ❌✅ (primary) | -2.7% | [-6.1%, -0.8%] | 27 | +- Given the overall positive impact of this PR and the complex relationship it has with some other PRs, I think it's safe to say the perf results are fine here. + + +Rollup of 11 pull requests [#109496](https://github.com/rust-lang/rust/pull/109496) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=439292bc7913399e406d9bb7e8da0f70c6317c6e&end=a266f11990d9544ee408e213e1eec8cc9eb032b7&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.5% | [0.3%, 0.8%] | 4 | +| Improvements ✅
(primary) | -0.7% | [-0.8%, -0.7%] | 4 | +| Improvements ✅
(secondary) | -0.6% | [-0.7%, -0.4%] | 4 | +| All ❌✅ (primary) | -0.7% | [-0.8%, -0.7%] | 4 | +- Unfortunately most of the positives here seem to be correction from noise. The regressions are really small though so I don't think investigation is worth it. + + +Rollup of 9 pull requests [#109503](https://github.com/rust-lang/rust/pull/109503) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=cf811810fe8dc92b3ff25e7dd29fb3178c91d409&end=84dd6dfd9d19176cc3c94bc1448a841e44d57890&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.5% | [0.5%, 0.5%] | 1 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -1.5% | [-1.7%, -1.4%] | 2 | +| Improvements ✅
(secondary) | -2.4% | [-5.3%, -0.3%] | 13 | +| All ❌✅ (primary) | -0.9% | [-1.7%, 0.5%] | 3 | +- The one regression is outweighed by many other improvements. Given this is a rollup which requires extra steps to investigate regressions, I think it's safe to mark this as triaged. + + +Add `CastKind::Transmute` to MIR [#108442](https://github.com/rust-lang/rust/pull/108442) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=df7fd9995f10627f25ccb325f693a11b3395a73c&end=e2163008763c326ec4003e07b8e6eef0c98f6204&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.7% | [0.3%, 1.5%] | 4 | +| Improvements ✅
(primary) | -1.6% | [-1.6%, -1.6%] | 1 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | -1.6% | [-1.6%, -1.6%] | 1 | +- The regressions are small enough that I don't think this is worth investigating. + + +rustdoc: Optimize impl sorting during rendering [#109399](https://github.com/rust-lang/rust/pull/109399) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=8be3c2bda6b683f87b24714ba595e8b04faef54c&end=d012d2f96e2959267b14812929b85e17a155f653&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:------:|:----------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.7% | [0.7%, 0.7%] | 1 | +| Improvements ✅
(primary) | -10.1% | [-10.1%, -10.1%] | 1 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | -10.1% | [-10.1%, -10.1%] | 1 | +- The one regression is noise. + + +Implement Default for some alloc/core iterators [#99929](https://github.com/rust-lang/rust/pull/99929) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=24a69af213f4bf23014dc91903e7a934c88f6b9e&end=9fa6b3c15758e85657d5be051cfa57022a8bbe57&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.7% | [0.7%, 0.7%] | 4 | +| Regressions ❌
(secondary) | 0.5% | [0.3%, 0.6%] | 9 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -0.3% | [-0.4%, -0.3%] | 6 | +| All ❌✅ (primary) | 0.7% | [0.7%, 0.7%] | 4 | +- Given the nature of this PR (adding APIs) and the fact that the primary benchmark impacted is noisy, I think we can triage this. + + +Use `SmallVec` in bitsets [#109458](https://github.com/rust-lang/rust/pull/109458) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=96bd50dd47b645efa52d1312b9cc32e507d9cd76&end=2d429f3064cb67710fe64dee293329089871d92b&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.8% | [0.7%, 1.0%] | 8 | +| Improvements ✅
(primary) | -0.8% | [-1.4%, -0.3%] | 15 | +| Improvements ✅
(secondary) | -0.7% | [-1.4%, -0.3%] | 15 | +| All ❌✅ (primary) | -0.8% | [-1.4%, -0.3%] | 15 | +- Improvements outweigh the regressions and [for some of the regressions, it should be fine to ignore the result as noise](https://github.com/rust-lang/rust/pull/109458#issuecomment-1483148687). + + +Upgrade to LLVM 16, again [#109474](https://github.com/rust-lang/rust/pull/109474) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2d429f3064cb67710fe64dee293329089871d92b&end=0c61c7a978fe9f7b77a1d667c77d2202dadd1c10&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 1.1% | [0.3%, 3.6%] | 64 | +| Regressions ❌
(secondary) | 0.9% | [0.2%, 2.5%] | 23 | +| Improvements ✅
(primary) | -1.0% | [-2.9%, -0.5%] | 49 | +| Improvements ✅
(secondary) | -1.1% | [-4.1%, -0.3%] | 75 | +| All ❌✅ (primary) | 0.2% | [-2.9%, 3.6%] | 113 | +- Given that the perf results are somewhat even (though regressions do win out), I think we sort of have to take this as is. I don't imagine we would revert an LLVM upgrade unless the perf results were really bad. From 860c2bea21f4b1d10070d421584d679db42cddd4 Mon Sep 17 00:00:00 2001 From: Ryan Levick Date: Wed, 5 Apr 2023 12:07:53 +0200 Subject: [PATCH 2/2] Update triage/2023-03-28.md Co-authored-by: Nicholas Nethercote --- triage/2023-03-28.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/triage/2023-03-28.md b/triage/2023-03-28.md index 4f1812269..45d4ae1fc 100644 --- a/triage/2023-03-28.md +++ b/triage/2023-03-28.md @@ -1,6 +1,6 @@ # 2023-03-28 Triage Log -A busy week with lots of real performance gains. Most regressions seemed to be due to noise. The biggest highlight was large wins in incremental compilation leading to a lot of (albeit modest gains) of 1% performance in a majority of incremental compilation test scenarios. Other than that most performance gains were smaller and more incremental. One of the biggest performance regressions came in an update too LLVM. However, nearly just as many test cases showed improvements as regressions. +A busy week with lots of real performance gains. Most regressions seemed to be due to noise. The biggest highlight was large wins in incremental compilation leading to a lot of (albeit modest) gains of 1% performance in a majority of incremental compilation test scenarios. Other than that most performance gains were smaller and more incremental. One of the biggest performance regressions came in an update to LLVM. However, nearly just as many test cases showed improvements as regressions. Triage done by **@rylev**. Revision range: [ef03fda3..cbc064b3](https://perf.rust-lang.org/?start=ef03fda339923e659d3d3ca3321de887316d2807&end=cbc064b341be231403d181402a786cce7f1c73f1&absolute=false&stat=instructions%3Au)