Add post-merge analysis CI workflow#138013
Merged
bors merged 1 commit intorust-lang:masterfrom Mar 8, 2025
Merged
Conversation
Collaborator
|
rustbot
added
A-testsuite
This comment has been minimized.
This comment has been minimized.
Collaborator
|
☔ The latest upstream changes (presumably #138021) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
c40b798 to
6ec42d7
Compare
Member
Author
|
Rebased. @rustbot ready |
This comment has been minimized.
This comment has been minimized.
Collaborator
|
☔ The latest upstream changes (presumably #138058) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
6ec42d7 to
f04a2cc
Compare
Member
Author
|
Rebased. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
f04a2cc to
d5da6b7
Compare
marcoieni
reviewed
Mar 7, 2025
Member
marcoieni
left a comment
marcoieni
left a comment
There was a problem hiding this comment.
very small details that don't matter, I'll approve this PR as it is now
|
|
||
| for job in &job_db.auto_jobs { | ||
| eprintln!("Downloading metrics of job {}", job.name); | ||
| let metrics_parent = match download_job_metrics(&job.name, parent) { |
Member
There was a problem hiding this comment.
you could simplify this match with something like this:
download_job_metrics(&job.name, parent).inspect_err( eprintln!()).ok()
Comment on lines
+41
to
+43
| r#"Did not find metrics for job `{}` at `{}`: {error:?}. | ||
| Maybe it was newly added?"#, | ||
| job.name, parent |
Member
There was a problem hiding this comment.
Suggested change
| r#"Did not find metrics for job `{}` at `{}`: {error:?}. | |
| Maybe it was newly added?"#, | |
| job.name, parent | |
| r#"Did not find metrics for job `{}` at `{parent}`: {error:?}. | |
| Maybe it was newly added?"#, | |
| job.name |
Member
|
@bors r+ |
Collaborator
bors
added
S-waiting-on-bors
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup of 12 pull requests Successful merges: - rust-lang#137337 (Add verbatim linker to AIXLinker) - rust-lang#137363 (compiler: factor Windows x86-32 ABI impl into its own file) - rust-lang#137537 (Prevent `rmake.rs` from using unstable features, and fix 3 run-make tests that currently do) - rust-lang#137606 (add a "future" edition) - rust-lang#137957 (Remove i586-pc-windows-msvc) - rust-lang#138000 (atomic: clarify that failing conditional RMW operations are not 'writes') - rust-lang#138013 (Add post-merge analysis CI workflow) - rust-lang#138033 (rustdoc: Add attribute-related tests for rustdoc JSON.) - rust-lang#138137 (setTargetTriple now accepts Triple rather than string) - rust-lang#138173 (Delay bug for negative auto trait rather than ICEing) - rust-lang#138184 (Allow anyone to relabel `CI-spurious-*`) - rust-lang#138187 (remove clones) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup merge of rust-lang#138013 - Kobzol:ci-post-merge-analysis, r=marcoieni Add post-merge analysis CI workflow This PR adds a post-merge analysis workflow, which was discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Reporting.20test.20suite.20statistics.20after.20merge). The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations. It can be tested locally e.g. using this: ``` cargo run --release --manifest-path src/ci/citool/Cargo.toml post-merge-report 3cb0272 fd17dea ``` This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large). CC `@jieyouxu` r? `@marcoieni`
klensy
reviewed
Mar 13, 2025
jhpratt
added a commit
to jhpratt/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? `@jieyouxu`
fmease
added a commit
to fmease/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ``@jieyouxu``
fmease
added a commit
to fmease/rust
that referenced
this pull request
Mar 14, 2025
…r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ```@jieyouxu```
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 15, 2025
Rollup merge of rust-lang#138454 - Kobzol:post-merge-workflow-fixes, r=jieyouxu Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in rust-lang#138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ```@jieyouxu```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a post-merge analysis workflow, which was discussed here.
The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations.
It can be tested locally e.g. using this:
This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large).
CC @jieyouxu
r? @marcoieni