[infra][Apple] Fix Apple mobile runtime test reporting#114862
[infra][Apple] Fix Apple mobile runtime test reporting#114862matouskozak merged 14 commits intodotnet:mainfrom
Conversation
|
These tests are disabled in |
Good catch. I suppose the new merged runner might not be respecting these correctly (cc: @kotlarmilos) . I'll disable them directly in the test file using ActiveIssue attribute. Edit: disabling directly in the test file looks to be the only way now #112246 |
|
I'm pretty sure we have a ton of non-working disabled tests in |
|
Azure Pipelines successfully started running 1 pipeline(s). |
I believe you're right. I can confirm that the issue is with TCP tunnel by invoking mlaunch directly with and without TCP tunnel option. With TCP: Without TCP:
I'm a bit confused by the merged runner system, do you know where that change could be made? |
|
Let's test this with XHarness once dotnet/xharness#1416 is merged. In the meantime, try to disable the project build in the merged test runner ( |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull Request Overview
This PR fixes Apple mobile CI test reporting by switching from apple run to apple test to capture exit codes correctly, removes base64 encoding of XML results, disables problematic tests on Apple mobile and non-Windows Mono, and adds a MacCatalyst arm64/x64 pipeline.
- Disable building managed tests on non-Windows Mono
- Annotate and disable specific JIT tests on Apple mobile via
ActiveIssue - Turn off base64 XML encoding in
GeneratedTestRunnerand add MacCatalyst pipeline
Reviewed Changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/tests/managed/Managed.csproj | Disable managed test build for Mono on non-Windows platforms |
| src/tests/JIT/opt/Structs/MemsetMemcpyNullref.* | Add TestLibrary reference and ActiveIssue for Apple mobile |
| src/tests/JIT/Regression/**/GitHub_*.{csproj,cs,il} | Add TestLibrary reference and ActiveIssue in each test |
| src/tests/Directory.Build.targets | Remove AppleMobile test exclusion |
| src/tests/Common/XHarnessRunnerLibrary/RunnerEntryPoint.cs | Disable base64 encoding of XML results |
| src/mono/msbuild/apple/build/AppleBuild.targets | Enable UseMonoRuntime for Mono builds |
| eng/pipelines/extra-platforms/runtime-extra-platforms-maccatalyst.yml | Add MacCatalyst arm64/x64 runtime test pipeline |
|
/ba-g Failure on extra-platforms are unrelated to this PR and are visible on main rolling builds |
4 participants
Runtime test results reporting is broken on Apple mobile CI, see e.g., https://dev.azure.com/dnceng-public/public/_build/results?buildId=1020751&view=results and the log below.
This PR switches to using
apple testinstead ofapple run(introduced in #107118) to correctly detect exit codes. Also removing the base64 encoding forAppleEntryPointXML results dump (introduced in #67393, not sure why?)Additional changes
Directed_1on ios/iossimulators jobs is passing yet the logs contain a SIGSEGV error message.