The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2906490	dotnet-maui	Controls (v18.4) Accessibility,ActionSheet,ActivityIndicator,Animation,AppLinks	Log
1297927	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#53062
1297935	dotnet/sdk	Run Tests in Helix	Log	dotnet/sdk#53012
1297818	dotnet/runtime	Publish Build Assets	Log
1297854	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123780
2906563	dotnet-runtime	Send job to Helix (Windows)	Log
1297804	dotnet/maui	Controls CollectionView	Log
2906578	dotnet-arcade-validation	Run Tests (Windows)	Log
1297890	dotnet/roslyn	Test	Log
2906512	dotnet-runtime	Send to Helix	Log	#57892
1297770	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#119432
2906509	dotnet-runtime	Evaluate eng/Version.Details.xml for dependency changes	Log	#57892
2906408	dotnet-runtime	Generate CORE_ROOT	Log
2906474	dotnet-performance	Send job to Helix (Windows)	Log
2906434	dotnet-runtime	Send job to Helix (Windows)	Log
1297745	dotnet/aspnetcore	Run Quarantined Tests	Log
1297716	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#64964
2906450	dotnet-arcade-validation	🛡 Generate SBoM Manifest	Log
2906438	dotnet-arcade-validation	Build	Log
1297580	dotnet/aspnetcore	Run Quarantined Tests	Log
1297546	dotnet/runtime	Checkout dotnet/runtime@refs/pull/123377/merge to s	Log	dotnet/runtime#123377
2906326	dotnet-dotnet	Build (centOSStreamContainer)	Log
1297471	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#34072
1297561	dotnet/runtime	Checkout dotnet/runtime@refs/pull/124519/merge to s	Log	dotnet/runtime#124519
1297541	dotnet/runtime	Checkout dotnet/runtime@refs/pull/124483/merge to s	Log	dotnet/runtime#124483
1297495	dotnet/xharness	Run tests in Helix	Log
1297387	dotnet/runtime	Checkout dotnet/runtime@refs/pull/124003/merge to s	Log	dotnet/runtime#124003
2906291	dotnet-maui	Controls (API 30) Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log
1297444	dotnet/sdk	Checkout dotnet/sdk@refs/pull/53055/merge to s	Log	dotnet/sdk#53055
1297395	dotnet/maui	DeviceTestsAndroid_CoreCLR (Windows)	Log
1297323	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65415
1297280	dotnet/roslyn	Run Unit Tests	Log	dotnet/roslyn#82429
2906208	dotnet-runtime	Send job to Helix (Windows)	Log
2906178	dotnet-dotnet	Build (centOSStreamContainer)	Log
1297277	dotnet/roslyn	Run BuildValidator	Log	dotnet/roslyn#82428
1297240	dotnet/aspnetcore	Run Quarantined Tests	Log
2906202	dotnet-performance	Send job to Helix (Windows)	Log
1297134	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65304
1296334	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1540
1296965	dotnet/maui	Controls CollectionView	Log
1297008	dotnet/runtime	Send tests to Helix (Unix)	Log
2906033	dotnet-runtime	Send job to Helix (Windows)	Log
1296913	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124485
1296880	dotnet/runtime	Send to Helix	Log
1296891	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#124475
1296885	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124493
1296888	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124492
1296907	dotnet/aspnetcore	Run Quarantined Tests	Log
1296767	dotnet/runtime	Send to Helix	Log	dotnet/runtime#119432
1296842	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53053
2905873	dotnet-runtime	Send job to Helix (Windows)	Log
1296478	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124173
1296602	dotnet/runtime	Send tests to Helix (Unix)	Log
1296565	dotnet/runtime	Zip Build Assets	Log
1296543	dotnet/runtime	Build product	Log	dotnet/runtime#123819
2905710	dotnet-runtime	Send job to Helix (Windows)	Log
2905638	dotnet-runtime	Send to Helix	Log	#57892
1296194	dotnet/runtime	Send tests to Helix (Unix)	Log
1296117	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124173
2905520	dotnet-runtime	Send job to Helix (Windows)	Log
1296112	dotnet/aspnetcore	Run Quarantined Tests	Log
2905519	dotnet-performance	Send job to Helix (Windows)	Log
1296001	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34072
1295992	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#34070
1295935	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#30875
1295930	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65444
2905290	dotnet-dotnet	Run Tests (licenseScanContainer)	Log
2905285	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2905287	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2905292	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2905288	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2905282	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1295695	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#124452
2905259	dotnet-runtime	Send job to Helix (Windows)	Log
1295693	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65437
1295722	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1531
1295751	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7570
1295731	dotnet/xharness	Run tests in Helix	Log	dotnet/xharness#1532
1295705	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7566
1295625	dotnet/runtime	Send tests to Helix (Unix)	Log
1295647	dotnet/aspnetcore	Run Quarantined Tests	Log
1295536	dotnet/aspire	Run helix tests	Log
2905088	dotnet-runtime	Send job to Helix (Windows)	Log
1295505	dotnet/maui	Controls Layout	Log	dotnet/maui#29924
2905055	dotnet-maui	Controls (v18.4) SafeAreaEdges,Shadow	Log
1295444	dotnet/maui	Controls Layout	Log	dotnet/maui#30133
2904957	dotnet-runtime	Send job to Helix (Windows)	Log
2904939	dotnet-runtime	Send tests to Helix (Unix)	Log
1295373	dotnet/aspnetcore	Run Quarantined Tests	Log
2904857	dotnet-dotnet	Build (centOSStreamContainer)	Log
2904817	dotnet-runtime	Send job to Helix (Windows)	Log
2904824	dotnet-performance	Send job to Helix (Windows)	Log
2904764	dotnet-dotnet	Build (centOSStreamContainer)	Log
2904706	dotnet-runtime	Send job to Helix (Windows)	Log
1295158	dotnet/runtime	Send tests to Helix (Unix)	Log
1295163	dotnet/aspnetcore	Run Quarantined Tests	Log
2904536	dotnet-runtime	Send job to Helix (Windows)	Log
2904491	dotnet-runtime	Generate CORE_ROOT	Log
2904404	dotnet-runtime	Send job to Helix (Windows)	Log
1294857	dotnet/machinelearning	Run Helix Tests	Log
Displaying 100 of 2864 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
50	506	2864

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez