Skip to content

Split integration tests in groups #7733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Oct 30, 2025
Merged

Split integration tests in groups #7733

merged 6 commits into from
Oct 30, 2025
+97 −43

Conversation

@pablomartinezbernardo
Copy link
Contributor

@pablomartinezbernardo pablomartinezbernardo commented Oct 29, 2025 •
edited
Loading

Summary of changes

  • Extract integration test running logic to template
  • Split DockerTest job in two arbitrary groups
  • Categorize tests with a new DockerGroup trait indicating which group they belong to
  • Categorize docker containers using service profiles

Reason for change

The number of containers in a single job was starting to be problematic for startup. This approach splits images in two groups (attempting to keep them similarly weighted) easily extensible to more groups.

@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Oct 29, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (79ms)  : 76, 83
     .   : milestone, 79,
    master - mean (79ms)  : 76, 82
     .   : milestone, 79,

    section Baseline
    This PR (7733) - mean (75ms)  : 72, 78
     .   : milestone, 75,
    master - mean (74ms)  : 71, 78
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (1,096ms)  : 1046, 1147
     .   : milestone, 1096,
    master - mean (1,113ms)  : 1031, 1195
     .   : milestone, 1113,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (117ms)  : 113, 121
     .   : milestone, 117,
    master - mean (119ms)  : 116, 122
     .   : milestone, 119,

    section Baseline
    This PR (7733) - mean (116ms)  : 113, 120
     .   : milestone, 116,
    master - mean (117ms)  : 113, 121
     .   : milestone, 117,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (797ms)  : 761, 832
     .   : milestone, 797,
    master - mean (795ms)  : 773, 817
     .   : milestone, 795,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (105ms)  : 102, 107
     .   : milestone, 105,
    master - mean (105ms)  : 102, 107
     .   : milestone, 105,

    section Baseline
    This PR (7733) - mean (103ms)  : 99, 107
     .   : milestone, 103,
    master - mean (102ms)  : 99, 106
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (738ms)  : 710, 766
     .   : milestone, 738,
    master - mean (748ms)  : 719, 776
     .   : milestone, 748,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (103ms)  : 100, 106
     .   : milestone, 103,

    section Baseline
    This PR (7733) - mean (101ms)  : 97, 106
     .   : milestone, 101,
    master - mean (102ms)  : 99, 105
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (698ms)  : 678, 718
     .   : milestone, 698,
    master - mean (709ms)  : 689, 729
     .   : milestone, 709,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (200ms)  : 194, 206
     .   : milestone, 200,
    master - mean (198ms)  : 193, 202
     .   : milestone, 198,

    section Baseline
    This PR (7733) - mean (195ms)  : 189, 201
     .   : milestone, 195,
    master - mean (195ms)  : 189, 201
     .   : milestone, 195,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (1,169ms)  : 1100, 1237
     .   : milestone, 1169,
    master - mean (1,172ms)  : 1102, 1243
     .   : milestone, 1172,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (279ms)  : 273, 284
     .   : milestone, 279,
    master - mean (281ms)  : 274, 289
     .   : milestone, 281,

    section Baseline
    This PR (7733) - mean (279ms)  : 272, 286
     .   : milestone, 279,
    master - mean (283ms)  : 269, 296
     .   : milestone, 283,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (954ms)  : 909, 998
     .   : milestone, 954,
    master - mean (954ms)  : 905, 1003
     .   : milestone, 954,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (276ms)  : 263, 290
     .   : milestone, 276,
    master - mean (270ms)  : 266, 274
     .   : milestone, 270,

    section Baseline
    This PR (7733) - mean (277ms)  : 269, 284
     .   : milestone, 277,
    master - mean (270ms)  : 263, 276
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (939ms)  : 894, 984
     .   : milestone, 939,
    master - mean (940ms)  : 882, 999
     .   : milestone, 940,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7733) - mean (274ms)  : 264, 284
     .   : milestone, 274,
    master - mean (273ms)  : 263, 282
     .   : milestone, 273,

    section Baseline
    This PR (7733) - mean (270ms)  : 265, 274
     .   : milestone, 270,
    master - mean (270ms)  : 264, 276
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (7733) - mean (857ms)  : 829, 884
     .   : milestone, 857,
    master - mean (861ms)  : 837, 884
     .   : milestone, 861,
Loading

@pr-commenter
Copy link

pr-commenter bot commented Oct 29, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7733 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.178
  • 1 benchmarks are slower, with geometric mean 1.182
  • 4 benchmarks have fewer allocations
  • 6 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.13 KB 6.04 KB -93 B -1.52%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 9.99μs 15.7ns 54.5ns 0 0 0 5.52 KB
master StartStopWithChild netcoreapp3.1 14.1μs 72.2ns 331ns 0 0 0 5.74 KB
master StartStopWithChild net472 22.6μs 127ns 844ns 0.887 0.332 0 6.13 KB
#7733 StartStopWithChild net6.0 10.7μs 57.1ns 343ns 0 0 0 5.52 KB
#7733 StartStopWithChild netcoreapp3.1 13.9μs 70.4ns 307ns 0 0 0 5.71 KB
#7733 StartStopWithChild net472 21.8μs 118ns 679ns 1.02 0.338 0.113 6.04 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 928μs 150ns 581ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 51.6ns 186ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.19ms 78ns 302ns 0 0 0 3.31 KB
#7733 WriteAndFlushEnrichedTraces net6.0 935μs 51.9ns 180ns 0 0 0 2.71 KB
#7733 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 230ns 830ns 0 0 0 2.7 KB
#7733 WriteAndFlushEnrichedTraces net472 1.2ms 1.16μs 4.02μs 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleSimpleBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleSimpleBody net472 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody net472 N/A N/A N/A NaN NaN NaN 0 b
master ObjectExtractorSimpleBody net6.0 320ns 1.63ns 7.47ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 410ns 2.27ns 14.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 295ns 0.0436ns 0.169ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.31μs 33.3ns 176ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.91μs 39.9ns 178ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.64μs 2.9ns 11.2ns 0.598 0 0 3.8 KB
#7733 AllCycleSimpleBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7733 AllCycleSimpleBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7733 AllCycleSimpleBody net472 N/A N/A N/A NaN NaN NaN 0 b
#7733 AllCycleMoreComplexBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
#7733 AllCycleMoreComplexBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
#7733 AllCycleMoreComplexBody net472 N/A N/A N/A NaN NaN NaN 0 b
#7733 ObjectExtractorSimpleBody net6.0 326ns 1.68ns 8.22ns 0 0 0 280 B
#7733 ObjectExtractorSimpleBody netcoreapp3.1 401ns 1.98ns 8.15ns 0 0 0 272 B
#7733 ObjectExtractorSimpleBody net472 317ns 0.218ns 0.845ns 0.0446 0 0 281 B
#7733 ObjectExtractorMoreComplexBody net6.0 6.32μs 34ns 173ns 0 0 0 3.78 KB
#7733 ObjectExtractorMoreComplexBody netcoreapp3.1 7.82μs 37.3ns 144ns 0 0 0 3.69 KB
#7733 ObjectExtractorMoreComplexBody net472 6.7μs 8.06ns 31.2ns 0.574 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 76.8μs 102ns 382ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 97.1μs 215ns 832ns 0 0 0 32.4 KB
master EncodeArgs net472 110μs 18.4ns 66.3ns 5.04 0 0 32.51 KB
master EncodeLegacyArgs net6.0 145μs 33.7ns 126ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 204μs 419ns 1.62μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 263μs 44.8ns 162ns 0 0 0 2.16 KB
#7733 EncodeArgs net6.0 77μs 184ns 687ns 0 0 0 32.4 KB
#7733 EncodeArgs netcoreapp3.1 98.1μs 217ns 842ns 0 0 0 32.4 KB
#7733 EncodeArgs net472 109μs 18.2ns 68.2ns 4.91 0 0 32.51 KB
#7733 EncodeLegacyArgs net6.0 143μs 52.6ns 197ns 0 0 0 2.15 KB
#7733 EncodeLegacyArgs netcoreapp3.1 200μs 497ns 1.86μs 0 0 0 2.14 KB
#7733 EncodeLegacyArgs net472 266μs 61.8ns 214ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 394μs 76.9ns 288ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 416μs 114ns 410ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 429μs 33.7ns 130ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 287μs 56.4ns 211ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 300μs 83.3ns 323ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 309μs 61.6ns 238ns 0 0 0 2.29 KB
#7733 RunWafRealisticBenchmark net6.0 393μs 107ns 414ns 0 0 0 4.56 KB
#7733 RunWafRealisticBenchmark netcoreapp3.1 429μs 343ns 1.24μs 0 0 0 4.48 KB
#7733 RunWafRealisticBenchmark net472 430μs 51.4ns 192ns 0 0 0 4.68 KB
#7733 RunWafRealisticBenchmarkWithAttack net6.0 284μs 54.8ns 205ns 0 0 0 2.24 KB
#7733 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 297μs 329ns 1.27μs 0 0 0 2.22 KB
#7733 RunWafRealisticBenchmarkWithAttack net472 308μs 22ns 82.3ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.3μs 70.3ns 244ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 70.9μs 42.9ns 155ns 0 0 0 17.42 KB
master SendRequest net472 0.00442ns 0.00215ns 0.00831ns 0 0 0 0 b
#7733 SendRequest net6.0 59.9μs 46.2ns 173ns 0 0 0 14.52 KB
#7733 SendRequest netcoreapp3.1 71.9μs 62.9ns 218ns 0 0 0 17.42 KB
#7733 SendRequest net472 0.00707ns 0.00292ns 0.0113ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1 1 B 2 B 1 B 100.00%

Fewer allocations 🎉 in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472 73 B 0 b -73 B -100.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472 47 B 0 b -47 B -100.00%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.9ms 3.62μs 13.5μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.14ms 6.52μs 25.3μs 0 0 0 640 KB
master OriginalCharSlice net472 2.62ms 457ns 1.71μs 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.35ms 135ns 524ns 0 0 0 4 B
master OptimizedCharSlice netcoreapp3.1 1.76ms 292ns 1.09μs 0 0 0 1 B
master OptimizedCharSlice net472 1.99ms 187ns 723ns 0 0 0 73 B
master OptimizedCharSliceWithPool net6.0 880μs 71.8ns 278ns 0 0 0 2 B
master OptimizedCharSliceWithPool netcoreapp3.1 880μs 49.9ns 187ns 0 0 0 0 b
master OptimizedCharSliceWithPool net472 1.15ms 109ns 422ns 0 0 0 47 B
#7733 OriginalCharSlice net6.0 1.94ms 354ns 1.28μs 0 0 0 640.01 KB
#7733 OriginalCharSlice netcoreapp3.1 2.27ms 10.6μs 41.1μs 0 0 0 640 KB
#7733 OriginalCharSlice net472 2.69ms 396ns 1.48μs 100 0 0 641.95 KB
#7733 OptimizedCharSlice net6.0 1.41ms 120ns 449ns 0 0 0 4 B
#7733 OptimizedCharSlice netcoreapp3.1 1.83ms 406ns 1.57μs 0 0 0 2 B
#7733 OptimizedCharSlice net472 1.91ms 159ns 595ns 0 0 0 0 b
#7733 OptimizedCharSliceWithPool net6.0 894μs 34.5ns 129ns 0 0 0 2 B
#7733 OptimizedCharSliceWithPool netcoreapp3.1 849μs 69.2ns 268ns 0 0 0 0 b
#7733 OptimizedCharSliceWithPool net472 1.14ms 121ns 470ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7733

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 1.182 918,370.70 1,085,843.75

More allocations ⚠️ in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.83 KB 42.59 KB 765 B 1.83%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.87 KB 42.32 KB 442 B 1.06%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 778μs 4.12μs 20.6μs 0 0 0 41.83 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 786μs 4.38μs 29μs 0 0 0 41.87 KB
master WriteAndFlushEnrichedTraces net472 918μs 3.54μs 13.7μs 7.81 0 0 55.65 KB
#7733 WriteAndFlushEnrichedTraces net6.0 717μs 3.44μs 14.6μs 0 0 0 42.59 KB
#7733 WriteAndFlushEnrichedTraces netcoreapp3.1 756μs 2.2μs 8.5μs 0 0 0 42.32 KB
#7733 WriteAndFlushEnrichedTraces net472 1.09ms 5.81μs 29μs 5.21 0 0 55.85 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.95μs 2.08ns 7.78ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.6μs 8.57ns 33.2ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.79μs 1.43ns 5.53ns 0.152 0.0139 0 987 B
#7733 ExecuteNonQuery net6.0 1.85μs 6.74ns 25.2ns 0 0 0 1.02 KB
#7733 ExecuteNonQuery netcoreapp3.1 2.7μs 9.67ns 36.2ns 0 0 0 1.02 KB
#7733 ExecuteNonQuery net472 2.85μs 2.51ns 9.73ns 0.156 0.0142 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.67μs 1.05ns 4.06ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.28μs 10.9ns 46.4ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.66μs 0.752ns 2.91ns 0.165 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.85μs 6.34ns 24.6ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.37μs 11.6ns 46.4ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.68μs 3.31ns 12.8ns 0.167 0 0 1.1 KB
#7733 CallElasticsearch net6.0 1.74μs 8.78ns 40.2ns 0 0 0 1.03 KB
#7733 CallElasticsearch netcoreapp3.1 2.28μs 5.29ns 20.5ns 0 0 0 1.03 KB
#7733 CallElasticsearch net472 3.54μs 1.6ns 5.77ns 0.16 0 0 1.04 KB
#7733 CallElasticsearchAsync net6.0 1.83μs 4.05ns 15.7ns 0 0 0 1.01 KB
#7733 CallElasticsearchAsync netcoreapp3.1 2.41μs 11.9ns 49ns 0 0 0 1.08 KB
#7733 CallElasticsearchAsync net472 3.67μs 3.05ns 11.8ns 0.164 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.86μs 8.81ns 34.1ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.49μs 8.45ns 31.6ns 0 0 0 952 B
master ExecuteAsync net472 2.6μs 1.61ns 5.8ns 0.144 0 0 915 B
#7733 ExecuteAsync net6.0 1.91μs 3.95ns 15.3ns 0 0 0 952 B
#7733 ExecuteAsync netcoreapp3.1 2.49μs 10.5ns 40.6ns 0 0 0 952 B
#7733 ExecuteAsync net472 2.67μs 3.21ns 12.4ns 0.134 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.12μs 6.99ns 26.2ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.73μs 26.1ns 101ns 0 0 0 2.9 KB
master SendAsync net472 12.2μs 5.9ns 22.1ns 0.489 0 0 3.18 KB
#7733 SendAsync net6.0 6.76μs 16.8ns 65ns 0 0 0 2.36 KB
#7733 SendAsync netcoreapp3.1 8.86μs 26.5ns 103ns 0 0 0 2.9 KB
#7733 SendAsync net472 11.9μs 9.68ns 34.9ns 0.476 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7733

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 1.189 522,200.00 439,100.00
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 1.166 49,100.00 42,100.00 bimodal

More allocations ⚠️ in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 45.53 KB 2.89 KB 6.77%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 272.82 KB 275.78 KB 2.96 KB 1.08%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.74 KB 44.07 KB 328 B 0.75%

Fewer allocations 🎉 in #7733

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 275.82 KB 249.41 KB -26.42 KB -9.58%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 49.6μs 272ns 1.49μs 0 0 0 43.74 KB
master StringConcatBenchmark netcoreapp3.1 50μs 293ns 2.48μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 57μs 283ns 1.23μs 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 525μs 1.98μs 6.86μs 0 0 0 275.82 KB
master StringConcatAspectBenchmark netcoreapp3.1 512μs 1.97μs 7.11μs 0 0 0 272.82 KB
master StringConcatAspectBenchmark net472 407μs 2.34μs 18.9μs 0 0 0 278.53 KB
#7733 StringConcatBenchmark net6.0 42.2μs 140ns 755ns 0 0 0 44.07 KB
#7733 StringConcatBenchmark netcoreapp3.1 56.1μs 639ns 6.32μs 0 0 0 45.53 KB
#7733 StringConcatBenchmark net472 57.4μs 278ns 1.08μs 0 0 0 57.34 KB
#7733 StringConcatAspectBenchmark net6.0 441μs 2.04μs 9.55μs 0 0 0 249.41 KB
#7733 StringConcatAspectBenchmark netcoreapp3.1 518μs 1.93μs 6.94μs 0 0 0 275.78 KB
#7733 StringConcatAspectBenchmark net472 413μs 2.29μs 13.9μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.75μs 13ns 51.9ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.6μs 15.9ns 61.7ns 0 0 0 1.7 KB
master EnrichedLog net472 3.89μs 4.05ns 15.7ns 0.252 0 0 1.64 KB
#7733 EnrichedLog net6.0 2.62μs 11.9ns 47.7ns 0 0 0 1.7 KB
#7733 EnrichedLog netcoreapp3.1 3.65μs 13.8ns 53.5ns 0 0 0 1.7 KB
#7733 EnrichedLog net472 3.92μs 2.64ns 10.2ns 0.256 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 72.2ns 270ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 129μs 355ns 1.33μs 0 0 0 4.31 KB
master EnrichedLog net472 167μs 36.6ns 142ns 0 0 0 4.52 KB
#7733 EnrichedLog net6.0 123μs 95.6ns 370ns 0 0 0 4.31 KB
#7733 EnrichedLog netcoreapp3.1 129μs 128ns 481ns 0 0 0 4.31 KB
#7733 EnrichedLog net472 167μs 296ns 1.15μs 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.12μs 17.6ns 66ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.94μs 16.2ns 62.6ns 0 0 0 2.26 KB
master EnrichedLog net472 7.52μs 5.38ns 20.8ns 0.302 0 0 2.08 KB
#7733 EnrichedLog net6.0 5.1μs 24.8ns 105ns 0 0 0 2.26 KB
#7733 EnrichedLog netcoreapp3.1 7.2μs 16.3ns 63.1ns 0 0 0 2.26 KB
#7733 EnrichedLog net472 7.56μs 4.91ns 19ns 0.302 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.01μs 6.58ns 24.6ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.72μs 7.03ns 26.3ns 0 0 0 1.2 KB
master SendReceive net472 3.12μs 5.44ns 20.4ns 0.185 0 0 1.2 KB
#7733 SendReceive net6.0 1.94μs 0.411ns 1.59ns 0 0 0 1.2 KB
#7733 SendReceive netcoreapp3.1 2.62μs 12.4ns 51ns 0 0 0 1.2 KB
#7733 SendReceive net472 3.06μs 3.02ns 11.7ns 0.184 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.45μs 2.69ns 9.7ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.67μs 18ns 69.6ns 0 0 0 1.63 KB
master EnrichedLog net472 6.57μs 8.39ns 32.5ns 0.296 0 0 2.03 KB
#7733 EnrichedLog net6.0 4.28μs 14.9ns 57.5ns 0 0 0 1.58 KB
#7733 EnrichedLog netcoreapp3.1 5.71μs 15.8ns 61.1ns 0 0 0 1.63 KB
#7733 EnrichedLog net472 6.64μs 8.7ns 33.7ns 0.3 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 764ns 4ns 19.2ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 986ns 5.19ns 27ns 0 0 0 576 B
master StartFinishSpan net472 959ns 0.419ns 1.62ns 0.0916 0 0 578 B
master StartFinishScope net6.0 916ns 4.43ns 18.8ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.2μs 5.57ns 21.6ns 0 0 0 696 B
master StartFinishScope net472 1.14μs 0.513ns 1.99ns 0.103 0 0 658 B
#7733 StartFinishSpan net6.0 777ns 3.87ns 16ns 0 0 0 576 B
#7733 StartFinishSpan netcoreapp3.1 975ns 4.48ns 17.4ns 0 0 0 576 B
#7733 StartFinishSpan net472 976ns 1.38ns 5.36ns 0.0876 0 0 578 B
#7733 StartFinishScope net6.0 931ns 0.437ns 1.69ns 0 0 0 696 B
#7733 StartFinishScope netcoreapp3.1 1.2μs 5.98ns 26.1ns 0 0 0 696 B
#7733 StartFinishScope net472 1.14μs 0.265ns 1.03ns 0.103 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.06μs 5.16ns 24.7ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.42μs 6.79ns 27.1ns 0 0 0 696 B
master RunOnMethodBegin net472 1.43μs 1.01ns 3.9ns 0.1 0 0 658 B
#7733 RunOnMethodBegin net6.0 1.06μs 5.54ns 27.1ns 0 0 0 696 B
#7733 RunOnMethodBegin netcoreapp3.1 1.45μs 5.05ns 19.6ns 0 0 0 696 B
#7733 RunOnMethodBegin net472 1.46μs 0.923ns 3.58ns 0.102 0 0 658 B

@datadog-official

This comment has been minimized.

Sign in to view
@pablomartinezbernardo pablomartinezbernardo changed the title Test Split integration test containers in two Oct 29, 2025
@pablomartinezbernardo pablomartinezbernardo force-pushed the pmartinez/integration-tests branch from b2831b7 to b478d25 Compare October 30, 2025 09:35
@pablomartinezbernardo pablomartinezbernardo changed the title Split integration test containers in two Split integration tests in groups Oct 30, 2025
@pablomartinezbernardo pablomartinezbernardo marked this pull request as ready for review October 30, 2025 12:10
@pablomartinezbernardo pablomartinezbernardo requested review from a team as code owners October 30, 2025 12:10
andrewlock
andrewlock reviewed Oct 30, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks! I think this is a good stop-gap approach overall, but I don't think we need (or should) extract the step out - instead we can just update the integration_tests_linux_docker_matrix matrix we calculate in Build.VariableGenerations.cs to include a groupNumber variable 🙂

.azure-pipelines/steps/run-docker-integration-tests.yml Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of extracting this into a separate file, we can instead update the integration_tests_linux_docker_matrix test matrix to have a new groupNumber variable. That will mean we can easily keep this all in the main file.

I think keeping it all together here is important, because there's a lot of "find and replace" that often happens to the pipeline when we refactor things, and this will likely get forgotten. By adding the variable to the generation matrix we're not going to be duplicating anything, so I think it's fine 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also results in a cleaner diff, so I'm all for it

bouwkast
bouwkast approved these changes Oct 30, 2025
View reviewed changes
Copy link
Collaborator

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review super closely but it looks right to me

Thank you very much!

@pablomartinezbernardo pablomartinezbernardo marked this pull request as draft October 30, 2025 15:35
@pablomartinezbernardo pablomartinezbernardo marked this pull request as ready for review October 30, 2025 16:52
andrewlock
andrewlock approved these changes Oct 30, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:chef-kiss: thanks!

@pablomartinezbernardo pablomartinezbernardo merged commit 05622b1 into master Oct 30, 2025
152 checks passed
@pablomartinezbernardo pablomartinezbernardo deleted the pmartinez/integration-tests branch October 30, 2025 17:19
@github-actions github-actions bot added this to the vNext-v3 milestone Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes

@andrewlock andrewlock andrewlock approved these changes

@lucaspimentel lucaspimentel Awaiting requested review from lucaspimentel

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

vNext-v3

Development

Successfully merging this pull request may close these issues.

4 participants

@pablomartinezbernardo @bouwkast @andrewlock