The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1304564	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#52938
1304562	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52936
2909929	dotnet-dotnet	Build (centOSStreamContainer)	Log
1304399	dotnet/maui	Controls CollectionView	Log	dotnet/maui#34169
1304402	dotnet/maui	Controls Entry	Log	dotnet/maui#34023
2909878	dotnet-roslyn-analyzers	Build and Test	Log
2909868	dotnet-roslyn-analyzers	Build and Test	Log
2909865	dotnet-roslyn-analyzers	Build and Test	Log
1304312	dotnet/runtime	Send tests to Helix (Unix)	Log
1304235	dotnet/runtime	Publish Build Assets	Log
1304181	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124709
1304197	dotnet/roslyn	Test	Log	dotnet/roslyn#82491
2909626	dotnet-dotnet	Build (centOSStreamContainer)	Log
2909594	dotnet-runtime	Send job to Helix (Windows)	Log
1304093	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123611
1304070	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124374
1304107	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124705
1304120	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53108
2909529	dotnet-runtime	Run performance job script	Log
2909463	dotnet-runtime	Send tests to Helix (Unix)	Log
2909413	dotnet-runtime	Send job to Helix (Windows)	Log
1303906	dotnet/runtime	Send job to Helix (Unix)	Log
1303735	dotnet/runtime	Send tests to Helix (Unix)	Log
2909300	dotnet-razor	🛡 Guardian: PoliCheck	Log
2909262	dotnet-runtime	Send job to Helix (Windows)	Log
1303648	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33977
1303542	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#124560
1303539	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124684
1303633	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53096
1303564	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124575
1303563	dotnet/maui	Controls (API 30) Shell	Log	dotnet/maui#33246
1303482	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124675
1303504	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#123736
1303453	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124669
1303409	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#124250
1303548	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53108
1303468	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124682
1303475	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123920
1303510	dotnet/sdk	🟣 Run FullFramework Tests	Log	dotnet/sdk#53102
1303507	dotnet/performance	Send job to Helix (Windows)	Log	dotnet/performance#5049
2909187	dotnet-performance	Send job to Helix (Windows)	Log
2909002	dotnet-dotnet	Run Tests (ubuntuArmContainer)	Log
1301822	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33977
2909079	dotnet-arcade	Checkout dotnet-arcade@main to s	Log
2909056	dotnet-runtime	Send job to Helix (Windows)	Log
2909057	dotnet-runtime	Send to Helix	Log	#57892
1302996	dotnet/aspnetcore	Run Quarantined Tests	Log
1303051	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53102
1303017	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52695
2908990	dotnet-runtime	Send job to Helix (Windows)	Log
1302861	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34089
1302835	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124589
1302792	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124641
1302739	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124637
2908922	dotnet-dotnet	Run Binary Analysis Scan Optimization (Expand Archives) (1ES PT)	Log
1302764	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34155
1302693	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124656
1302688	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124621
1299257	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33951
1302669	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124603
2908844	dotnet-dotnet	Run Tests (alpineContainer)	Log
1302520	dotnet/maui	Run Integration Tests - Build	Log
2908865	dotnet-dotnet	Build (centOSStreamContainer)	Log
2908876	dotnet-maui	Pack	Log
1302490	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34133
2908847	dotnet-runtime	Send job to Helix (Windows)	Log
1302371	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52937
1302271	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122065
1302318	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#64964
1302324	dotnet/maui	Controls CollectionView	Log
1302311	dotnet/maui	Controls CollectionView	Log
1302104	dotnet/maui	Controls CollectionView	Log	dotnet/maui#34146
2908770	dotnet-performance	Send job to Helix (Windows)	Log
2908760	dotnet-runtime	Send to Helix	Log	#57892
1302095	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34150
1301985	dotnet/maui	Controls Layout	Log	dotnet/maui#33646
2908655	dotnet-runtime	Send job to Helix (Windows)	Log
1302027	dotnet/aspnetcore	Run Quarantined Tests	Log
2908613	dotnet-sdk	Run AoT Tests	Log	#58175
1301938	dotnet/maui	Controls CollectionView	Log	dotnet/maui#34023
1301907	dotnet/runtime	Send tests to Helix (Unix)	Log
2908448	dotnet-runtime	Send job to Helix (Windows)	Log
1301687	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52792
2908358	dotnet-runtime	Send job to Helix (Windows)	Log
2908375	dotnet-performance	Send job to Helix (Unix)	Log
1301526	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124621
1301544	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124537
1301642	dotnet/performance	Send job to Helix (Windows)	Log
1301495	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124514
1301505	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124564
1301574	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123738
2908332	dotnet-sdk	Run TestBuild Tests	Log	#58156
1301549	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124534
1301492	dotnet/runtime	Send to Helix	Log	dotnet/runtime#111439
2908328	dotnet-efcore	Send job to helix	Log	#58152
1301406	dotnet/runtime	Checkout dotnet/runtime@refs/pull/124619/merge to s	Log	dotnet/runtime#124619
2908279	dotnet-sdk	🟣 Build	Log
1301202	dotnet/runtime	Send tests to Helix (Unix)	Log
1301166	dotnet/runtime	Publish	Log	dotnet/runtime#124603
2908250	dotnet-performance	Send job to Helix (Windows)	Log
Displaying 100 of 2702 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
18	313	2702

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez