Skip to content

chore: codspeed benchmarks for pr to feat/aot#2183

Merged
GunaDD merged 11 commits into
feat/aotfrom
feat/aot-codspeed-bench
Oct 29, 2025
Merged

chore: codspeed benchmarks for pr to feat/aot#2183
GunaDD merged 11 commits into
feat/aotfrom
feat/aot-codspeed-bench

Conversation

@GunaDD

@GunaDD GunaDD commented Oct 28, 2025

Copy link
Copy Markdown
Contributor
  • Change workflow to enable codpseed bench with aot features
  • Run the extension tests with aot feature on
  • Aot skips test_load_x0 since it is an untrusted program

@GunaDD GunaDD marked this pull request as draft October 28, 2025 22:21
@GunaDD GunaDD changed the base branch from main to feat/aot October 28, 2025 22:26
@GunaDD GunaDD changed the title chore:(will not merge) pr to check codspeed bench chore: codspeed benchmarks for pr to feat/aot Oct 28, 2025
@codspeed-hq

codspeed-hq Bot commented Oct 28, 2025

Copy link
Copy Markdown

CodSpeed Performance Report

Merging #2183 will degrade performances by 88.78%

Comparing feat/aot-codspeed-bench (4b88c7d) with main (c2e376e)1

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

❌ 18 regressions
⏩ 42 skipped2

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Mode Benchmark BASE HEAD Change
WallTime benchmark_execute[bubblesort] 27.4 ms 233 ms -88.22%
WallTime benchmark_execute[fibonacci_iterative] 31.4 ms 279.5 ms -88.78%
WallTime benchmark_execute[fibonacci_recursive] 45.1 ms 380.8 ms -88.16%
WallTime benchmark_execute[keccak256] 33 ms 251.4 ms -86.88%
WallTime benchmark_execute[pairing] 137.8 ms 225.2 ms -38.8%
WallTime benchmark_execute[quicksort] 32 ms 265.3 ms -87.94%
WallTime benchmark_execute[revm_snailtracer] 16.1 ms 20.3 ms -20.67%
WallTime benchmark_execute[revm_transfer] 43.1 ms 169.2 ms -74.55%
WallTime benchmark_execute[sha256] 31.1 ms 249.9 ms -87.56%
WallTime benchmark_execute_metered[bubblesort] 53 ms 279.6 ms -81.06%
WallTime benchmark_execute_metered[fibonacci_iterative] 72.5 ms 323.4 ms -77.58%
WallTime benchmark_execute_metered[fibonacci_recursive] 101.4 ms 456.1 ms -77.76%
WallTime benchmark_execute_metered[keccak256] 68.9 ms 302.9 ms -77.26%
WallTime benchmark_execute_metered[pairing] 156.1 ms 246.8 ms -36.77%
WallTime benchmark_execute_metered[quicksort] 60.3 ms 314.3 ms -80.82%
WallTime benchmark_execute_metered[revm_snailtracer] 17.1 ms 21.1 ms -19%
WallTime benchmark_execute_metered[revm_transfer] 64.1 ms 201.7 ms -68.19%
WallTime benchmark_execute_metered[sha256] 67.1 ms 302.3 ms -77.79%

Footnotes

  1. No successful run was found on feat/aot (6b4ab05) during the generation of this report, so main (c2e376e) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

  2. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@GunaDD GunaDD marked this pull request as ready for review October 29, 2025 00:35
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
group app.proof_time_ms app.cycles app.cells_used leaf.proof_time_ms leaf.cycles leaf.cells_used
verify_fibair 234 322,610 2,058,654 - - -
fibonacci 988 1,500,209 2,100,402 - - -
regex 2,273 4,137,502 17,695,216 - - -
ecrecover 698 122,859 2,263,820 - - -
pairing 1,429 1,745,742 25,468,210 - - -

Commit: 4b88c7d

Benchmark Workflow

@GunaDD GunaDD merged commit f804542 into feat/aot Oct 29, 2025
44 of 56 checks passed
@GunaDD GunaDD deleted the feat/aot-codspeed-bench branch October 29, 2025 01:47
GunaDD added a commit that referenced this pull request Oct 29, 2025
* Change workflow to enable codpseed bench with aot features
* Run the extension tests with aot feature on
* Aot skips `test_load_x0` since it is an untrusted program

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-18-208.ec2.internal>
Maillew pushed a commit that referenced this pull request Oct 30, 2025
* Change workflow to enable codpseed bench with aot features
* Run the extension tests with aot feature on
* Aot skips `test_load_x0` since it is an untrusted program

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-18-208.ec2.internal>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant