IOException running NuGet-Migrations during tests in dotnet CLI first run

[akoeplinger]

This is affecting 6.0, 7.0 and 8.0.

In #80510 (comment) we saw a failure because some NuGet migration code failed to run during the dotnet CLI first run experience:

+ dotnet /datadisks/disk1/work/ADFC09B1/p/xunit/xunit.console.dll JIT/HardwareIntrinsics/JIT.HardwareIntrinsics.XUnitWrapper.dll -parallel collections -nocolor -noshadow -xml testResults.xml -trait TestGroup=JIT.HardwareIntrinsics.Arm.ArmBase
Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET 7.0.2)
  Discovering: JIT.HardwareIntrinsics.XUnitWrapper (method display = ClassAndMethod, method display options = None)
  Discovered:  JIT.HardwareIntrinsics.XUnitWrapper (found 6 of 362 test cases)
  Starting:    JIT.HardwareIntrinsics.XUnitWrapper (parallel test collections = on, max threads = 2)
    JIT/HardwareIntrinsics/Arm/ArmBase.Arm64/ArmBase.Arm64_ro/ArmBase.Arm64_ro.sh [FAIL]
      System.IO.IOException: The system cannot open the device or file specified. : 'NuGet-Migrations'
         at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
         at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
         at NuGet.Common.Migrations.MigrationRunner.Run()
         at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure()
         at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(IFirstTimeUseNoticeSentinel firstTimeUseNoticeSentinel, IAspNetCertificateSentinel aspNetCertificateSentinel, IFileSentinel toolPathSentinel, Boolean isDotnetBeingInvokedFromNativeInstaller, DotnetFirstRunConfiguration dotnetFirstRunConfiguration, IEnvironmentProvider environmentProvider, Dictionary`2 performanceMeasurements)
         at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, TimeSpan startupTime, ITelemetry telemetryClient)
         at Microsoft.DotNet.Cli.Program.Main(String[] args)
      
      Return code:      1

We should look into disabling the first run experience via the DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 env var, it doesn't make sense to run that for the tests.

Report

Build	Definition	Test	Pull Request
368683	dotnet/runtime	JIT.Methodical.eE.WorkItemExecution	#90241
362596	dotnet/runtime	JIT.Methodical.eE.WorkItemExecution	#89003
361655	dotnet/runtime	JIT.Math.WorkItemExecution	#89905
361259	dotnet/runtime	JIT.Generics.WorkItemExecution
361075	dotnet/runtime	JIT.Regression.WorkItemExecution	#89867
360768	dotnet/runtime	JIT.Regression.JitBlue.WorkItemExecution	#89815
360148	dotnet/runtime	JIT.jit64.WorkItemExecution	#89814
359522	dotnet/runtime	JIT.Methodical.WorkItemExecution	#89805
356421	dotnet/runtime	JIT.Methodical.a-dA-D.WorkItemExecution
355235	dotnet/runtime	JIT.Methodical.f-iF-I.WorkItemExecution	#89009
343534	dotnet/runtime	JIT.Methodical.WorkItemExecution
342037	dotnet/runtime	JIT.Regression.JitBlue.WorkItemExecution	#89008
341937	dotnet/runtime	JIT.Methodical.a-dA-D.WorkItemExecution	#89003
337982	dotnet/runtime	JIT.HardwareIntrinsics.X86.Avx2.WorkItemExecution	#88809
337742	dotnet/runtime	JIT.Regression.CLR-x86-JIT.V1-M12-M13.WorkItemExecution	#88769

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	1	15

Known Issue Error Message

Fill the error message using known issues guidance.

{
  "ErrorPattern": "The system cannot open the device or file specified. : ('|')NuGet-Migrations('|')",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 6/28/2023 10:04:55 PM UTC

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In #80510 (comment) we saw a failure because some NuGet migration code failed to run during the dotnet CLI first run experience:

+ dotnet /datadisks/disk1/work/ADFC09B1/p/xunit/xunit.console.dll JIT/HardwareIntrinsics/JIT.HardwareIntrinsics.XUnitWrapper.dll -parallel collections -nocolor -noshadow -xml testResults.xml -trait TestGroup=JIT.HardwareIntrinsics.Arm.ArmBase
Microsoft.DotNet.XUnitConsoleRunner v2.5.0 (64-bit .NET 7.0.2)
  Discovering: JIT.HardwareIntrinsics.XUnitWrapper (method display = ClassAndMethod, method display options = None)
  Discovered:  JIT.HardwareIntrinsics.XUnitWrapper (found 6 of 362 test cases)
  Starting:    JIT.HardwareIntrinsics.XUnitWrapper (parallel test collections = on, max threads = 2)
    JIT/HardwareIntrinsics/Arm/ArmBase.Arm64/ArmBase.Arm64_ro/ArmBase.Arm64_ro.sh [FAIL]
      System.IO.IOException: The system cannot open the device or file specified. : 'NuGet-Migrations'
         at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
         at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
         at NuGet.Common.Migrations.MigrationRunner.Run()
         at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure()
         at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(IFirstTimeUseNoticeSentinel firstTimeUseNoticeSentinel, IAspNetCertificateSentinel aspNetCertificateSentinel, IFileSentinel toolPathSentinel, Boolean isDotnetBeingInvokedFromNativeInstaller, DotnetFirstRunConfiguration dotnetFirstRunConfiguration, IEnvironmentProvider environmentProvider, Dictionary`2 performanceMeasurements)
         at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, TimeSpan startupTime, ITelemetry telemetryClient)
         at Microsoft.DotNet.Cli.Program.Main(String[] args)
      
      Return code:      1

We should look into disabling the first run experience via the DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 env var, it doesn't make sense to run that for the tests.

Author:	akoeplinger
Assignees:	-
Labels:	area-Infrastructure
Milestone:	-

[ViktorHofer]

I thought that we were already setting that property at least for libraries tests. Maybe not for runtime tests? cc @jkoritzinsky

[carlossanlop]

Another PR that hit this: #80615

[radical]

I'm hitting this fairly often. For example on a rolling build(log).

[radical]

Another rolling build(log).

[carlossanlop]

Another 7.0 dependency flow PR affected by this: #81812

[carlossanlop]

I thought that we were already setting that property at least for libraries tests. Maybe not for runtime tests? cc @jkoritzinsky

Ping @jkoritzinsky

[lewing]

did we do something in net8 that needs to also be done in net7 here?

[radical]

This is affecting lot of builds.

[trylek]

I don't see any property / variable named DOTNET_SKIP_FIRST_TIME_EXPERIENCE in the dotnet/sdk repo, @ViktorHofer / @akoeplinger - am I looking in the wrong place (can you please point me to the place where it's used?) or are you saying we need to build this as a new feature? In fact I'm somewhat puzzled by the above code, these are apparently CoreCLR tests and they shouldn't be normally executed via dotnet, that must be coming from somewhere else like Crossgen2 execution.