Skip to content

chore(ci): fix the instance type for codspeed benches#1811

Merged
jonathanpwang merged 2 commits into
feat/new-executionfrom
ci/codspeed-fix-instance
Jul 2, 2025
Merged

chore(ci): fix the instance type for codspeed benches#1811
jonathanpwang merged 2 commits into
feat/new-executionfrom
ci/codspeed-fix-instance

Conversation

@jonathanpwang

Copy link
Copy Markdown
Contributor

No description provided.

@codspeed-hq

codspeed-hq Bot commented Jul 2, 2025

Copy link
Copy Markdown

CodSpeed WallTime Performance Report

Merging #1811 will degrade performances by 63.77%

Comparing ci/codspeed-fix-instance (9b86249) with feat/new-execution (6dffa9b)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

#### 🎉 Hooray! `codspeed-rust` just leveled up to 3.0.2!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

❌ 20 (👁 20) regressions

Benchmarks breakdown

Benchmark BASE HEAD Change
👁 benchmark_execute[bubblesort] 166.8 ms 402.6 ms -58.57%
👁 benchmark_execute[factorial_iterative_u256] 439.5 ms 1,150.8 ms -61.81%
👁 benchmark_execute[fibonacci_iterative] 176.1 ms 428.1 ms -58.86%
👁 benchmark_execute[fibonacci_recursive] 225.2 ms 597.3 ms -62.29%
👁 benchmark_execute[keccak256] 168.8 ms 438.6 ms -61.52%
👁 benchmark_execute[keccak256_iter] 256.8 ms 679.2 ms -62.19%
👁 benchmark_execute[quicksort] 174.7 ms 452.3 ms -61.37%
👁 benchmark_execute[revm_snailtracer] 141 ms 257 ms -45.14%
👁 benchmark_execute[sha256] 165.8 ms 434.5 ms -61.83%
👁 benchmark_execute[sha256_iter] 258.1 ms 698.5 ms -63.05%
👁 benchmark_execute_metered[bubblesort] 261.5 ms 671.1 ms -61.03%
👁 benchmark_execute_metered[factorial_iterative_u256] 841.7 ms 2,119.4 ms -60.29%
👁 benchmark_execute_metered[fibonacci_iterative] 325.8 ms 739.3 ms -55.93%
👁 benchmark_execute_metered[fibonacci_recursive] 410.2 ms 1,034.7 ms -60.36%
👁 benchmark_execute_metered[keccak256] 277.4 ms 762.5 ms -63.62%
👁 benchmark_execute_metered[keccak256_iter] 476.7 ms 1,242.3 ms -61.63%
👁 benchmark_execute_metered[quicksort] 291.1 ms 760.2 ms -61.7%
👁 benchmark_execute_metered[revm_snailtracer] 145.3 ms 268.1 ms -45.8%
👁 benchmark_execute_metered[sha256] 275.1 ms 759.2 ms -63.77%
👁 benchmark_execute_metered[sha256_iter] 490.5 ms 1,226.4 ms -60%

@jonathanpwang

Copy link
Copy Markdown
Contributor Author

Acknowledging the performance degradation because: codspeed updated to v3.0.2 and changed to a fixed m5a.xlarge instance type (intentionally only 2.5Ghz instead of the faster 3.5 Ghz available on m6a.xlarge).

@jonathanpwang jonathanpwang merged commit 9fafdbb into feat/new-execution Jul 2, 2025
5 checks passed
@jonathanpwang jonathanpwang deleted the ci/codspeed-fix-instance branch July 2, 2025 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant