Test failure JIT/jit64/opt/cse/HugeField2/HugeField2.cmd

[v-wenyuxu]

Failed in: runtime-coreclr jitstress-random 20230625.1

Failed tests:

coreclr windows x86 Checked jitstress_random_2 @ Windows.10.Amd64.Open
    - JIT/jit64/opt/cse/HugeField2/HugeField2.cmd

Error message:

 
Return code:      1
Raw output file:      C:hwA2C2093DwAABC097AuploadsoptcseHugeField2output.txt
Raw output:
BEGIN EXECUTION
 "C:hwA2C2093Dpcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  HugeField2.dll 
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at CseTest.Test_Main.TestEntryPoint()
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 6/25/2023 7:44:20 PM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> C:hwA2C2093DwAABC097AeJITjit64jit64_3../opt/cse/HugeField2/HugeField2.cmd
Expected: True
Actual:   False

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor131|0_132(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Failed in: runtime-coreclr jitstress-random 20230625.1

Failed tests:

coreclr windows x86 Checked jitstress_random_2 @ Windows.10.Amd64.Open
    - JIT/jit64/opt/cse/HugeField2/HugeField2.cmd

Error message:

 
Return code:      1
Raw output file:      C:hwA2C2093DwAABC097AuploadsoptcseHugeField2output.txt
Raw output:
BEGIN EXECUTION
 "C:hwA2C2093Dpcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  HugeField2.dll 
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at CseTest.Test_Main.TestEntryPoint()
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 6/25/2023 7:44:20 PM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> C:hwA2C2093DwAABC097AeJITjit64jit64_3../opt/cse/HugeField2/HugeField2.cmd
Expected: True
Actual:   False

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor131|0_132(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

Author:	v-wenyuxu
Assignees:	-
Labels:	arch-x86, os-windows, JitStress, area-CodeGen-coreclr
Milestone:	-

[BruceForstall]

set DOTNET_TieredCompilation=0
set DOTNET_JitStress=5b

I wonder if this is a test that needs RequiresProcessIsolation -- @markples ?

[markples]

HugeField2 already has RequiresProcessIsolation due to GCStressIncompatible. The two DOTNET_ lines that you see are for the overall test run (from SetStressModes_jitstress_random_2.cmd).

[v-wenyuxu]

Failed again in: runtime-coreclr jitstress-random 20230709.1

Failed tests:

coreclr linux arm Checked jitstress_random_1 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - JIT/jit64/opt/cse/HugeField2/HugeField2.cmd

Error message:

 Assert failure(PID 180 [0x000000b4], Thread: 180 [0x00b4]): Assertion failed 'distVal <= CALL_DIST_MAX_POS' in 'CseTest.Test_Main:TestEntryPoint():int' during 'Emit code' (IL size 3230358; hash 0x930ee156; MinOpts)

    File: /__w/1/s/src/coreclr/jit/emitarm.cpp Line: 5528
    Image: /root/helix/work/correlation/corerun

waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
/root/helix/work/workitem/e/JIT/jit64/jit64_3/../opt/cse/HugeField2/HugeField2.sh: line 455:   180 Aborted                 (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/opt/cse/HugeField2/output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true HugeField2.dll ''
[createdump] Gathering state for process 180 corerun
[createdump] Crashing thread 00b4 signal 6 (0006)
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.180.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.180.dmp
[createdump] Written 2613555200 bytes (638075 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 13012ms
Expected: 100
Actual: 134
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 7/9/2023 7:43:55 PM
Processing /home/helixbot/dotnetbuild/dumps/coredump.180.dmp.crashreport.json
Printing stacktrace from '/home/helixbot/dotnetbuild/dumps/coredump.180.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/root/helix/work/workitem/e/JIT/jit64/jit64_3'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 496
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 456
   at System.Diagnostics.Process.Start() in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs:line 1282
   at CoreclrTestLib.CoreclrTestWrapperLib.TryPrintStackTraceFromCrashReport(String crashReportJsonFile, TextWriter outputWriter)
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> /root/helix/work/workitem/e/JIT/jit64/jit64_3/../opt/cse/HugeField2/HugeField2.sh
Expected: True
Actual:   False

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor131|0_132(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

[jakobbotsch]

JitStress=5b here turns on STRESS_SPLIT_TREES_REMOVE_COMMAS, and removing all commas here results in a lot of new locals:

Num locals before SplitTreesRemoveCommas: 19694
Max tree ID before SplitTreesRemoveCommas: 8722530
Num locals after SplitTreesRemoveCommas: 902066
Max tree ID after SplitTreesRemoveCommas: 10936326

This causes CheckLclVarSemanticsHelper to use a ton of memory.

I'll add some limits to these.