Skip to content

ci: stop caching buildx bake #1027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 16, 2025
Merged

ci: stop caching buildx bake #1027

merged 2 commits into from
Apr 16, 2025

Conversation

@morrisonlevi
Copy link
Contributor

What does this PR do?

This stops using the buildx bake cache.

Motivation

This routinely seems to hang, possibly the same as this:
docker/buildx#537

Additional Notes

Caches are hard, some would say one of the hardest problems in computer science.

How to test the change?

CI should run as normal, the buildx bake job should just be a bit slower, but shouldn't try to cache and hopefully 🤞🏻 the slowdown is acceptable.

@pr-commenter
Copy link

pr-commenter bot commented Apr 16, 2025 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2025-04-16 03:52:19

Comparing candidate commit a8e4daa in PR branch levi/buildx with baseline commit daf50ad in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

scenario:benching deserializing traces from msgpack to their internal representation

  • 🟥 execution_time [+3.235ms; +3.632ms] or [+4.369%; +4.905%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.892µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.916µs 3.920µs 3.922µs 0.25% -1.307 9.132 0.08% 0.000µs 1 200
credit_card/is_card_number/ throughput 254958982.127op/s 255611482.744op/s ± 197627.025op/s 255604664.785op/s ± 91572.677op/s 255687451.501op/s 255911692.766op/s 256064477.417op/s 256921630.560op/s 0.52% 1.329 9.270 0.08% 13974.341op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 76.673µs 77.670µs ± 0.477µs 77.647µs ± 0.338µs 77.990µs 78.536µs 78.919µs 79.272µs 2.09% 0.457 0.143 0.61% 0.034µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12614864.946op/s 12875390.679op/s ± 78902.105op/s 12878856.791op/s ± 56256.869op/s 12935081.883op/s 12988184.892op/s 13016141.082op/s 13042357.330op/s 1.27% -0.421 0.080 0.61% 5579.221op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 70.800µs 71.390µs ± 0.294µs 71.340µs ± 0.180µs 71.561µs 71.921µs 72.157µs 72.237µs 1.26% 0.585 0.046 0.41% 0.021µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13843237.791op/s 14007884.089op/s ± 57483.305op/s 14017347.170op/s ± 35381.393op/s 14048833.049op/s 14088953.071op/s 14113952.569op/s 14124275.261op/s 0.76% -0.564 0.017 0.41% 4064.683op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.894µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.917µs 3.920µs 3.924µs 0.29% -0.646 5.338 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254863487.063op/s 255603913.312op/s ± 207201.747op/s 255606141.434op/s ± 82261.747op/s 255668476.096op/s 255968762.084op/s 256049387.071op/s 256784194.225op/s 0.46% 0.662 5.403 0.08% 14651.376op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 67.103µs 67.820µs ± 0.349µs 67.769µs ± 0.228µs 68.041µs 68.553µs 68.675µs 68.704µs 1.38% 0.490 -0.094 0.51% 0.025µs 1 200
credit_card/is_card_number/378282246310005 throughput 14555121.721op/s 14745241.437op/s ± 75620.075op/s 14756073.383op/s ± 49592.174op/s 14794865.095op/s 14852706.392op/s 14894999.166op/s 14902391.350op/s 0.99% -0.465 -0.120 0.51% 5347.147op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.723µs 51.822µs ± 0.040µs 51.819µs ± 0.016µs 51.836µs 51.886µs 51.952µs 52.036µs 0.42% 1.106 4.536 0.08% 0.003µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19217418.907op/s 19296792.368op/s ± 14925.128op/s 19297765.319op/s ± 6051.175op/s 19303485.727op/s 19321206.280op/s 19326608.050op/s 19333706.384op/s 0.19% -1.094 4.484 0.08% 1055.366op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.027µs 6.060µs ± 0.031µs 6.042µs ± 0.010µs 6.074µs 6.122µs 6.151µs 6.190µs 2.45% 1.527 2.679 0.51% 0.002µs 1 200
credit_card/is_card_number/x371413321323331 throughput 161547786.501op/s 165021834.633op/s ± 831433.538op/s 165500360.994op/s ± 280624.032op/s 165667995.420op/s 165756526.298op/s 165849929.693op/s 165925636.135op/s 0.26% -1.492 2.512 0.50% 58791.229op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.898µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.913µs 3.916µs 3.917µs 3.917µs 0.13% -1.034 3.371 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255301271.982op/s 255646006.888op/s ± 173664.728op/s 255636632.729op/s ± 78632.377op/s 255703035.544op/s 255967520.825op/s 256028416.876op/s 256571018.677op/s 0.37% 1.043 3.413 0.07% 12279.951op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 66.006µs 66.513µs ± 0.227µs 66.473µs ± 0.132µs 66.641µs 66.958µs 67.206µs 67.544µs 1.61% 1.066 2.409 0.34% 0.016µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 14805063.881op/s 15034793.432op/s ± 51194.683op/s 15043764.903op/s ± 29892.374op/s 15068159.474op/s 15100791.594op/s 15139438.290op/s 15150092.265op/s 0.71% -1.033 2.296 0.34% 3620.011op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 59.533µs 59.678µs ± 0.069µs 59.676µs ± 0.037µs 59.710µs 59.774µs 59.874µs 60.142µs 0.78% 2.102 11.029 0.12% 0.005µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 16627310.417op/s 16756541.828op/s ± 19289.685op/s 16757241.387op/s ± 10528.514op/s 16768120.034op/s 16780004.189op/s 16790545.147op/s 16797340.981op/s 0.24% -2.073 10.806 0.11% 1363.987op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.913µs ± 0.002µs 3.913µs ± 0.001µs 3.914µs 3.916µs 3.918µs 3.918µs 0.14% -1.957 13.211 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255241207.435op/s 255584797.146op/s ± 162325.987op/s 255590811.511op/s ± 62952.476op/s 255640800.273op/s 255806765.494op/s 255989585.698op/s 256772371.293op/s 0.46% 1.978 13.393 0.06% 11478.181op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 56.201µs 56.394µs ± 0.101µs 56.389µs ± 0.069µs 56.452µs 56.575µs 56.634µs 56.706µs 0.56% 0.392 -0.198 0.18% 0.007µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17634713.359op/s 17732488.075op/s ± 31704.771op/s 17733807.677op/s ± 21649.286op/s 17755807.128op/s 17780591.507op/s 17789889.990op/s 17793315.052op/s 0.34% -0.384 -0.210 0.18% 2241.866op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.739µs 51.826µs ± 0.031µs 51.820µs ± 0.013µs 51.842µs 51.878µs 51.905µs 51.973µs 0.29% 0.663 3.098 0.06% 0.002µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19240920.753op/s 19295525.258op/s ± 11445.907op/s 19297417.041op/s ± 4778.584op/s 19301257.960op/s 19313572.325op/s 19322713.319op/s 19327946.278op/s 0.16% -0.655 3.077 0.06% 809.348op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.025µs 6.053µs ± 0.027µs 6.039µs ± 0.007µs 6.073µs 6.111µs 6.148µs 6.217µs 2.95% 2.154 7.188 0.45% 0.002µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 160836419.437op/s 165204025.554op/s ± 730925.696op/s 165577773.292op/s ± 189110.334op/s 165674938.955op/s 165855191.963op/s 165885949.401op/s 165974903.751op/s 0.24% -2.095 6.725 0.44% 51684.252op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.913µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/ throughput [255584093.539op/s; 255638871.949op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [77.604µs; 77.737µs] or [-0.085%; +0.085%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12864455.606op/s; 12886325.752op/s] or [-0.085%; +0.085%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [71.349µs; 71.430µs] or [-0.057%; +0.057%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13999917.455op/s; 14015850.722op/s] or [-0.057%; +0.057%] None None None
credit_card/is_card_number/37828224631 execution_time [3.912µs; 3.913µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631 throughput [255575197.142op/s; 255632629.481op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/378282246310005 execution_time [67.772µs; 67.869µs] or [-0.071%; +0.071%] None None None
credit_card/is_card_number/378282246310005 throughput [14734761.222op/s; 14755721.652op/s] or [-0.071%; +0.071%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [51.817µs; 51.828µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19294723.889op/s; 19298860.848op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.056µs; 6.064µs] or [-0.070%; +0.070%] None None None
credit_card/is_card_number/x371413321323331 throughput [164906605.941op/s; 165137063.325op/s] or [-0.070%; +0.070%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.911µs; 3.912µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255621938.627op/s; 255670075.150op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [66.482µs; 66.545µs] or [-0.047%; +0.047%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15027698.341op/s; 15041888.523op/s] or [-0.047%; +0.047%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [59.669µs; 59.688µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [16753868.463op/s; 16759215.193op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255562300.326op/s; 255607293.967op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [56.380µs; 56.408µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [17728094.099op/s; 17736882.052op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [51.821µs; 51.830µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19293938.966op/s; 19297111.551op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.049µs; 6.057µs] or [-0.062%; +0.062%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [165102726.282op/s; 165305324.826op/s] or [-0.061%; +0.061%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.065ms 6.079ms ± 0.011ms 6.078ms ± 0.004ms 6.082ms 6.090ms 6.110ms 6.194ms 1.91% 6.339 60.115 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.078ms; 6.081ms] or [-0.025%; +0.025%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.573µs 33.484µs ± 1.135µs 32.794µs ± 0.103µs 34.663µs 35.571µs 35.908µs 35.925µs 9.55% 1.012 -0.778 3.38% 0.080µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.326µs; 33.641µs] or [-0.470%; +0.470%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 76.886ms 77.483ms ± 1.105ms 77.315ms ± 0.149ms 77.476ms 77.868ms 80.361ms 91.126ms 17.86% 9.915 115.643 1.42% 0.078ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [77.330ms; 77.636ms] or [-0.198%; +0.198%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 243.762ns 253.144ns ± 11.906ns 248.587ns ± 3.473ns 254.147ns 283.894ns 286.886ns 291.127ns 17.11% 1.704 1.740 4.69% 0.842ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [251.494ns; 254.794ns] or [-0.652%; +0.652%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.465µs 25.493µs ± 10.551µs 17.885µs ± 0.268µs 34.581µs 44.130µs 55.963µs 80.422µs 349.67% 1.516 3.626 41.29% 0.746µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.030µs; 26.955µs] or [-5.736%; +5.736%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 503.132µs 504.896µs ± 0.629µs 504.897µs ± 0.308µs 505.212µs 505.788µs 506.167µs 509.422µs 0.90% 1.661 12.261 0.12% 0.044µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1963009.485op/s 1980608.018op/s ± 2462.319op/s 1980602.093op/s ± 1209.937op/s 1981805.266op/s 1984703.067op/s 1985138.951op/s 1987549.767op/s 0.35% -1.619 11.921 0.12% 174.112op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 453.768µs 455.139µs ± 1.571µs 454.934µs ± 0.331µs 455.345µs 455.885µs 466.720µs 467.971µs 2.87% 6.801 49.878 0.34% 0.111µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2136884.094op/s 2197154.652op/s ± 7412.675op/s 2198122.980op/s ± 1597.139op/s 2199433.525op/s 2202042.559op/s 2203048.873op/s 2203768.113op/s 0.26% -6.742 49.283 0.34% 524.155op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 177.293µs 177.724µs ± 0.156µs 177.731µs ± 0.081µs 177.813µs 178.010µs 178.083µs 178.127µs 0.22% -0.073 0.262 0.09% 0.011µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5613971.285op/s 5626694.848op/s ± 4946.454op/s 5626481.240op/s ± 2557.295op/s 5628788.152op/s 5635017.743op/s 5638310.369op/s 5640388.372op/s 0.25% 0.079 0.262 0.09% 349.767op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 37.488µs 37.595µs ± 0.037µs 37.593µs ± 0.023µs 37.618µs 37.655µs 37.700µs 37.714µs 0.32% 0.174 0.586 0.10% 0.003µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26515064.775op/s 26599626.026op/s ± 26115.886op/s 26600343.150op/s ± 16545.596op/s 26616217.844op/s 26642946.371op/s 26660412.134op/s 26674871.210op/s 0.28% -0.167 0.580 0.10% 1846.672op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.196µs 48.315µs ± 0.153µs 48.299µs ± 0.031µs 48.330µs 48.384µs 48.487µs 49.775µs 3.06% 8.439 77.012 0.32% 0.011µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20090462.503op/s 20697530.146op/s ± 63770.062op/s 20704350.091op/s ± 13497.936op/s 20717931.094op/s 20733542.241op/s 20739901.567op/s 20748743.449op/s 0.21% -8.364 76.085 0.31% 4509.224op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [504.809µs; 504.983µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1980266.764op/s; 1980949.271op/s] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [454.922µs; 455.357µs] or [-0.048%; +0.048%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2196127.327op/s; 2198181.978op/s] or [-0.047%; +0.047%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [177.703µs; 177.746µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5626009.317op/s; 5627380.379op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.589µs; 37.600µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26596006.616op/s; 26603245.437op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.294µs; 48.337µs] or [-0.044%; +0.044%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20688692.229op/s; 20706368.064op/s] or [-0.043%; +0.043%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.182µs 3.232µs ± 1.438µs 3.023µs ± 0.023µs 3.046µs 3.690µs 14.020µs 14.940µs 394.17% 7.358 55.191 44.37% 0.102µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.033µs; 3.431µs] or [-6.164%; +6.164%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.954µs 5.052µs ± 0.050µs 5.053µs ± 0.037µs 5.079µs 5.156µs 5.158µs 5.163µs 2.18% 0.536 -0.287 0.99% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.045µs; 5.059µs] or [-0.138%; +0.138%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 66.097µs 66.345µs ± 0.417µs 66.293µs ± 0.075µs 66.367µs 66.522µs 67.759µs 70.949µs 7.02% 8.629 83.879 0.63% 0.030µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [66.287µs; 66.403µs] or [-0.087%; +0.087%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 148.887µs 149.777µs ± 0.416µs 149.764µs ± 0.143µs 149.902µs 150.288µs 150.635µs 154.062µs 2.87% 5.518 54.782 0.28% 0.029µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [149.719µs; 149.834µs] or [-0.038%; +0.038%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 208.613µs 209.002µs ± 0.139µs 208.992µs ± 0.091µs 209.095µs 209.218µs 209.255µs 209.384µs 0.19% -0.184 0.043 0.07% 0.010µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4775910.308op/s 4784655.087op/s ± 3173.383op/s 4784867.696op/s ± 2093.578op/s 4786512.883op/s 4790138.958op/s 4792961.690op/s 4793567.033op/s 0.18% 0.188 0.045 0.07% 224.392op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.622µs 18.684µs ± 0.029µs 18.681µs ± 0.020µs 18.703µs 18.722µs 18.784µs 18.788µs 0.57% 0.476 1.372 0.15% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 53224238.679op/s 53522939.833op/s ± 81920.920op/s 53528928.654op/s ± 55828.116op/s 53574156.745op/s 53663560.088op/s 53692505.010op/s 53700731.286op/s 0.32% -0.462 1.336 0.15% 5792.684op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.895µs 10.958µs ± 0.040µs 10.954µs ± 0.020µs 10.975µs 11.005µs 11.066µs 11.333µs 3.46% 4.406 38.122 0.36% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 88235134.416op/s 91256957.241op/s ± 326230.083op/s 91289570.558op/s ± 167816.983op/s 91454717.449op/s 91657099.369op/s 91756753.281op/s 91788462.896op/s 0.55% -4.187 35.424 0.36% 23067.950op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [208.982µs; 209.021µs] or [-0.009%; +0.009%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4784215.286op/s; 4785094.887op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.680µs; 18.688µs] or [-0.021%; +0.021%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [53511586.381op/s; 53534293.284op/s] or [-0.021%; +0.021%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.953µs; 10.964µs] or [-0.050%; +0.050%] None None None
normalization/normalize_name/normalize_name/good throughput [91211744.889op/s; 91302169.593op/s] or [-0.050%; +0.050%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz a8e4daa 1744774906 levi/buildx
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.320µs 2.385µs ± 0.018µs 2.386µs ± 0.007µs 2.395µs 2.404µs 2.407µs 2.421µs 1.43% -2.238 5.521 0.74% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.382µs; 2.387µs] or [-0.103%; +0.103%] None None None

Baseline

Omitted due to size.

@morrisonlevi morrisonlevi marked this pull request as ready for review April 16, 2025 03:36
@morrisonlevi morrisonlevi requested a review from a team as a code owner April 16, 2025 03:36
@codecov-commenter
Copy link

codecov-commenter commented Apr 16, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.51%. Comparing base (daf50ad) to head (a8e4daa).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1027      +/-   ##
==========================================
- Coverage   71.52%   71.51%   -0.02%     
==========================================
  Files         339      339              
  Lines       50751    50751              
==========================================
- Hits        36302    36295       -7     
- Misses      14449    14456       +7
Components Coverage Δ
crashtracker 42.84% <ø> (ø)
crashtracker-ffi 6.30% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.93% <ø> (ø)
data-pipeline-ffi 90.35% <ø> (ø)
ddcommon 78.57% <ø> (ø)
ddcommon-ffi 66.37% <ø> (ø)
ddtelemetry 60.29% <ø> (ø)
ddtelemetry-ffi 21.43% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
ipc 82.41% <ø> (ø)
profiling 77.49% <ø> (ø)
profiling-ffi 62.12% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 41.14% <ø> (ø)
sidecar-ffi 2.05% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 89.86% <ø> (ø)
trace-mini-agent 73.82% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 96.00% <ø> (ø)
trace-protobuf 78.50% <ø> (ø)
trace-utils 92.82% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@morrisonlevi morrisonlevi enabled auto-merge (squash) April 16, 2025 03:43
@r1viollet
Copy link
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 24.21 MB 24.21 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.79 MB 7.79 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 78.03 MB 78.03 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 72.40 MB 72.40 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.81 MB 22.81 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 7.71 MB 7.71 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 17.01 MB 17.01 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 61.83 KB 61.83 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 114.18 MB 114.17 MB -0% (-8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 633.63 MB 633.63 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 5.06 MB 5.06 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 61.83 KB 61.83 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 16.15 MB 16.15 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.83 MB 26.83 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.40 MB 14.40 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 62.78 KB 62.78 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 116.19 MB 116.19 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 625.37 MB 625.37 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.83 MB 3.83 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 62.78 KB 62.78 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.77 MB 16.77 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.74 MB 24.74 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 67.28 MB 67.28 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.27 MB 8.27 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 23.34 MB 23.34 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 68.24 MB 68.24 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.15 MB 8.15 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.99 MB 20.99 MB 0% (0 B) 👌

@morrisonlevi morrisonlevi merged commit 1096293 into main Apr 16, 2025
35 checks passed
@morrisonlevi morrisonlevi deleted the levi/buildx branch April 16, 2025 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ekump ekump ekump approved these changes

Assignees

No one assigned

Labels

ci-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morrisonlevi @codecov-commenter @r1viollet @ekump