Improve CFG traversal - #160193

nnethercote · 2026-07-30T04:59:11Z

This PR implements a new CFG traversal algorithm in iterate_to_fixpoint that drastically speeds up a pathological case in cranelift-codegen.

r? @cjgillot

nnethercote · 2026-07-30T05:52:27Z

@bors try @rust-timer queue

Improve CFG traversal

rust-bors · 2026-07-30T07:58:20Z

☀️ Try build successful (CI)
Build commit: 0d785d5 (0d785d51b8f9b098d3ec7f1530b5b75acd518c1c)
Base parent: 1a833e1 (1a833e16546c2eb012758ddd499964fd8afee29e)

rust-timer · 2026-07-30T10:02:24Z

Finished benchmarking commit (0d785d5): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.3%	[0.2%, 0.3%]	10
Improvements ✅ (primary)	-6.4%	[-21.7%, -0.3%]	10
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	5
All ❌✅ (primary)	-6.4%	[-21.7%, -0.3%]	10

Max RSS (memory usage)

Results (primary -3.5%, secondary -1.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	0.5%	[0.4%, 0.5%]	3
Improvements ✅ (primary)	-5.4%	[-8.1%, -0.4%]	3
Improvements ✅ (secondary)	-5.5%	[-5.5%, -5.5%]	1
All ❌✅ (primary)	-3.5%	[-8.1%, 2.3%]	4

Cycles

Results (primary -7.0%, secondary -6.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.4%, 1.3%]	5
Regressions ❌ (secondary)	1.3%	[0.5%, 2.2%]	6
Improvements ✅ (primary)	-10.8%	[-35.2%, -0.4%]	10
Improvements ✅ (secondary)	-10.2%	[-17.3%, -0.5%]	13
All ❌✅ (primary)	-7.0%	[-35.2%, 1.3%]	15

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 497.309s -> 488.837s (-1.70%)
Artifact size: 390.20 MiB -> 391.03 MiB (0.21%)

nnethercote · 2026-07-30T11:10:10Z

Great results on cranelift-codegen as expected: -33% walltime in the best case.

Also -12% walltime on tt-muncher somehow even though icounts were unchanged? I think that might just be a random fluctuation.

cjgillot · 2026-07-30T13:48:36Z

Wow great results! mir_borrowck shows a 68% improvement. Do you know which analysis benefits the most from the changes?

`iterate_to_fixpoint` uses a FIFO worklist. It is seeded in a good order (reverse postorder for forward analyses, postorder for backward analyses). But when back edges are present, re-dirtied blocks are pushed to the back of the queue. Successor blocks of these re-dirtied blocks are then analyzed even though we know the input states may change later on. This commit switches to a new min-rank algorithm that always processes the earliest dirty block in dataflow order. On most code this makes little difference, but cranelift-codegen has one enormous function with over 18,000 basic blocks. The old algorithm required 1.5 million calls to `apply_effects_in_block` to reach a fixpoint for `EverInitializedPlaces`. The new algorithm requires 90,000. This results in a 17% reduction in instruction counts for a `Check Full` build. Note: the new algorithm also avoids doing anything for unreachable blocks in backward analyses. (The old algorithm computed states but never used them meaningfully.)

It's no longer used.

nnethercote · 2026-07-30T23:23:45Z

The improvement is entirely from EverInitializedPlaces. I instrumented each call to iterate_to_fixpoint to print out the analysis name, the number of basic blocks, and the number of calls to apply_effects_in_block in the loop. Here are the top 13 case which accounts for 82.7% of all calls to apply_effects_in_block, prior to this PR:

(  1)  1508264 (73.7%, 73.7%): cfg ever_init                     : 18073 1508264
(  2)    78168 ( 3.8%, 77.5%): cfg liveness-lint                 : 18073 78168
(  3)    18073 ( 0.9%, 78.4%): cfg borrows                       : 18073 18073
(  4)    18073 ( 0.9%, 79.3%): cfg maybe_uninit                  : 18073 18073
(  5)    11600 ( 0.6%, 79.9%): cfg borrows                       : 11600 11600
(  6)    11600 ( 0.6%, 80.4%): cfg ever_init                     : 11600 11600
(  7)    11600 ( 0.6%, 81.0%): cfg liveness-lint                 : 11600 11600
(  8)    11600 ( 0.6%, 81.6%): cfg maybe_uninit                  : 11600 11600
(  9)     4777 ( 0.2%, 81.8%): cfg ever_init                     : 4673 4777
( 10)     4750 ( 0.2%, 82.0%): cfg liveness-lint                 : 4673 4750
( 11)     4673 ( 0.2%, 82.3%): cfg borrows                       : 4673 4673
( 12)     4673 ( 0.2%, 82.5%): cfg maybe_uninit                  : 4673 4673
( 13)     4445 ( 0.2%, 82.7%): cfg ever_init                     : 561 4445

After this PR, the 1508264 drops to 89809 and everything else is the same or barely changes. (Actually, the 561 4445 drops to 561 3750 which is a non-trivial improvement.)

I also have a draft PR at #160033 to improve EverInitializedPlaces, but I think this PR will render it unnecessary.

nnethercote · 2026-07-30T23:28:31Z

I have updated the code to address the review comments.

cjgillot · 2026-07-31T02:01:09Z

@bors r+

rust-bors · 2026-07-31T02:01:12Z

📌 Commit ac49776 has been approved by cjgillot

It is now in the queue for this repository.

nnethercote · 2026-07-31T11:44:16Z

-8.5s in the bootstrap results is large enough that it might be a real effect and not just random fluctuations. Especially given that it all mostly from rustc_parse dropping from 19.8s to 11.6s.

rust-bors · 2026-08-01T04:45:43Z

☀️ Test successful - CI
Approved by: cjgillot
Duration: 3h 17m 25s
Pushing cb9d1b0 to main...

github-actions · 2026-08-01T04:48:37Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 6c04025 (parent) -> cb9d1b0 (this PR)

Test differences

Show 6 test diffs

6 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard cb9d1b0640549f1b041aae430dc413ce93f8c204 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

x86_64-gnu-nopt: 1h 47m -> 2h 22m (+32.6%)
x86_64-gnu-llvm-21-3: 1h 23m -> 1h 50m (+31.2%)
dist-x86_64-musl: 1h 46m -> 2h 18m (+30.2%)
dist-arm-linux-musl: 1h 43m -> 1h 16m (-26.2%)
optional-x86_64-gnu-autodiff: 56m 21s -> 41m 48s (-25.8%)
pr-check-1: 26m 57s -> 33m 54s (+25.8%)
dist-x86_64-illumos: 1h 54m -> 1h 27m (-23.5%)
dist-x86_64-msvc: 2h 13m -> 1h 44m (-22.2%)
x86_64-gnu-llvm-22-1: 1h 13m -> 57m 7s (-21.9%)
dist-x86_64-linux-alt: 2h 21m -> 1h 52m (-20.4%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2026-08-01T05:28:27Z

Finished benchmarking commit (cb9d1b0): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.2%, 0.3%]	6
Improvements ✅ (primary)	-6.4%	[-21.7%, -0.3%]	10
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-6.4%	[-21.7%, -0.3%]	10

Max RSS (memory usage)

Results (primary -0.4%, secondary 0.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.1%	[0.5%, 2.5%]	9
Regressions ❌ (secondary)	1.7%	[0.4%, 3.7%]	11
Improvements ✅ (primary)	-7.1%	[-7.5%, -6.8%]	2
Improvements ✅ (secondary)	-2.9%	[-5.0%, -0.7%]	3
All ❌✅ (primary)	-0.4%	[-7.5%, 2.5%]	11

Cycles

Results (primary -5.8%, secondary 0.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.1%	[0.4%, 2.1%]	10
Regressions ❌ (secondary)	3.4%	[0.4%, 17.9%]	7
Improvements ✅ (primary)	-17.2%	[-34.7%, -3.5%]	6
Improvements ✅ (secondary)	-1.5%	[-3.9%, -0.5%]	7
All ❌✅ (primary)	-5.8%	[-34.7%, 2.1%]	16

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 491.569s -> 490.618s (-0.19%)
Artifact size: 390.31 MiB -> 390.50 MiB (0.05%)

panstromek · 2026-08-03T17:16:57Z

perf triage:

include-blob regressions are noise.

@rustbot label: +perf-regression-triaged

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 30, 2026

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 30, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Jul 30, 2026

Auto merge of #160193 - nnethercote:improve-cfg-traversal, r=<try>

0d785d5

Improve CFG traversal

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 30, 2026

rustbot assigned cjgillot Jul 30, 2026

nnethercote marked this pull request as ready for review July 30, 2026 11:10

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 30, 2026

cjgillot reviewed Jul 30, 2026

View reviewed changes

Comment thread compiler/rustc_mir_dataflow/src/framework/mod.rs Outdated

nnethercote added 2 commits July 31, 2026 09:18

Remove rustc_data_structures::work_queue

ac49776

It's no longer used.

nnethercote force-pushed the improve-cfg-traversal branch from df3dea1 to ac49776 Compare July 30, 2026 23:27

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 31, 2026

cjgillot mentioned this pull request Jul 31, 2026

Speed up EverInitializedPlaces #160033

Merged

This comment has been minimized.

Sign in to view

rust-bors Bot added merged-by-bors This PR was explicitly merged by bors. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 1, 2026

rust-bors Bot merged commit cb9d1b0 into rust-lang:main Aug 1, 2026
14 checks passed

rustbot added this to the 1.99.0 milestone Aug 1, 2026

nnethercote deleted the improve-cfg-traversal branch August 2, 2026 09:28

cjgillot mentioned this pull request Aug 2, 2026

Unschedule moved locals #159577

Draft

rustbot added the perf-regression-triaged The performance regression has been triaged. label Aug 3, 2026

Uh oh!

Uh oh!

Conversation

nnethercote commented Jul 30, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nnethercote commented Jul 30, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Jul 30, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Jul 30, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

nnethercote commented Jul 30, 2026

Uh oh!

Uh oh!

cjgillot commented Jul 30, 2026

Uh oh!

nnethercote commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nnethercote commented Jul 30, 2026

Uh oh!

cjgillot commented Jul 31, 2026

Uh oh!

rust-bors Bot commented Jul 31, 2026

Uh oh!

nnethercote commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

rust-bors Bot commented Aug 1, 2026

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026

Test differences

Job duration changes

Uh oh!

rust-timer commented Aug 1, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

panstromek commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nnethercote commented Jul 30, 2026 •

edited by rustbot

Loading

nnethercote commented Jul 30, 2026 •

edited

Loading

nnethercote commented Jul 31, 2026 •

edited

Loading