System.Drawing.Common.Tests fails during libraries test run

[buyaa-n]

Found while running all tests in libraries, running tests manually from the individual project also fails.

System.Drawing.Tests.BitmapTests.SaveWmfAsPngDoesntChangeImageBoundaries [FAIL]
        Assert.Equal() Failure
                              ↓ (pos 19)
        Expected: [..., 0, 1, 68, 0, 0, ...]
        Actual:   [..., 0, 1, 149, 0, 0, ...]
                              ↑ (pos 19)
        Stack Trace:
          D:\dotnet\runtime\src\libraries\System.Drawing.Common\tests\BitmapTests.cs(821,0): at System.Drawing.Tests.BitmapTests.SaveWmfAsPngDoesntChangeImageBoundaries()
             at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
          D:\dotnet\runtime\src\coreclr\System.Private.CoreLib\src\System\Reflection\RuntimeMethodInfo.CoreCLR.cs(387,0): at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr)

Failing with clean build having no any changes

Tagging subscribers to this area: @dotnet/area-system-drawing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Found while running all tests in libraries, running tests manually from the individual project also fails.

System.Drawing.Tests.BitmapTests.SaveWmfAsPngDoesntChangeImageBoundaries [FAIL]
        Assert.Equal() Failure
                              ↓ (pos 19)
        Expected: [..., 0, 1, 68, 0, 0, ...]
        Actual:   [..., 0, 1, 149, 0, 0, ...]
                              ↑ (pos 19)
        Stack Trace:
          D:\dotnet\runtime\src\libraries\System.Drawing.Common\tests\BitmapTests.cs(821,0): at System.Drawing.Tests.BitmapTests.SaveWmfAsPngDoesntChangeImageBoundaries()
             at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
          D:\dotnet\runtime\src\coreclr\System.Private.CoreLib\src\System\Reflection\RuntimeMethodInfo.CoreCLR.cs(387,0): at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr)

Author:	buyaa-n
Assignees:	-
Labels:	area-System.Drawing
Milestone:	-

[ViktorHofer]

Thanks for filing @buyaa-n. What's your hardware configuration? Also, did only this one test fail or multiple ones?

[buyaa-n]

What's your hardware configuration?

Windows 11, 64 bit, not sure what else needed ...

Also, did only this one test fail or multiple ones?

Two more tests were failing constantly, filed issue for one. The other one looks like depend on my system conf, so did not filed an issue, here is the log if you want to check:

<test name="System.Xml.Tests.XsltcTestBasicFunctionality.Var2(param0: \&quot;nul.xsl\&quot;, param1: \&quot;bft35.txt\&quot;, englishOnly: True)" type="System.Xml.Tests.XsltcTestBasicFunctionality" method="Var2" time="0.4372131" result="Fail">
        <traits>
          <trait name="category" value="XsltcExeRequired" />
        </traits>
        <failure exception-type="OLEDB.Test.ModuleCore.CTestFailedException">
          <message><![CDATA[OLEDB.Test.ModuleCore.CTestFailedException : Output was not as expected.\r\nExpected: `fatal error : Source file 'nul.xsl' could not be found. ---> Win32 device names are not allowed.\r\n`\r\nActual: `error : Source file 'nul.xsl' could not be found.\r\n`\r\nResult: 0\r\n]]></message>
          <stack-trace><![CDATA[   at System.Xml.Tests.XsltcTestCaseBase.CompareOutput(Stream expectedStream, Stream actualStream, Int32 count) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestCaseBase.cs:line 96
   at System.Xml.Tests.XsltcTestCaseBase.VerifyTest(String cmdLine, String asmName, Boolean asmCreated, String typeName, String pdbName, Boolean pdbCreated, String baselineFile, Boolean runAssemblyVerification, Boolean loadFromFile) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestCaseBase.cs:line 173
   at System.Xml.Tests.XsltcTestCaseBase.VerifyTest(String cmdLine, String asmName, Boolean asmCreated, String typeName, String pdbName, Boolean pdbCreated, String baselineFile, Boolean loadFromFile) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestCaseBase.cs:line 132
   at System.Xml.Tests.XsltcTestCaseBase.VerifyTest(String cmdLine, String asmName, Boolean asmCreated, String typeName, String baselineFile, Boolean loadFromFile) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestCaseBase.cs:line 127
   at System.Xml.Tests.XsltcTestCaseBase.VerifyTest(String cmdLine, String baselineFile, Boolean loadFromFile) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestCaseBase.cs:line 122
   at System.Xml.Tests.XsltcTestBasicFunctionality.Var2(Object param0, Object param1, Boolean englishOnly) in D:\dotnet\runtime\src\libraries\System.Private.Xml\tests\Xslt\XsltCompiler\CommonScenarios\XsltcTestBasicFunctionality.cs:line 42
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr) in D:\dotnet\runtime\src\coreclr\System.Private.CoreLib\src\System\Reflection\RuntimeMethodInfo.CoreCLR.cs:line 383]]></stack-trace>
        </failure>
      </test>

[janvorli]

I have just hit the same issue on my local devbox, only the wrong value was a bit different:

        Assert.Equal() Failure
                              ↓ (pos 19)
        Expected: [..., 0, 1, 68, 0, 0, ...]
        Actual:   [..., 0, 1, 229, 0, 0, ...]
                              ↑ (pos 19)

[MichalStrehovsky]

Ran into this too on Windows 10.0.19044.1826 x64.

[jkotas]

Test disabled in dotnet/runtime#72416