AV in F# compiler when run on .NET 5 runtime

[nguerrera]

I don't have a minimal repro yet, this is happening in core-sdk end-to-end tests that are newly merging to the master branch.

Managed stack below and there's a dump at "\\mlangfs1\public\nicholg\fsharp_dotnet5\fsharp_dotnet5.dmp"

I'm filing this on coreclr since this doesn't happen when running on .NET Core 3.x with the same F# compiler.

If the dump is insufficient, I can work on a repro.

cc @jkotas @KevinRansom

Microsoft (R) F# Compiler version 10.7.0.0 for F# 4.7
    Copyright (c) Microsoft Corporation. All Rights Reserved.
    Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
       at FSharp.Compiler.TypeChecker+MakeAndPublishSimpleVals@1892.Invoke(System.Tuple`2<NameResolutionEnv,range>, System.Tuple`8<pos,Item,Item,Microsoft.FSharp.Collections.FSharpList`1<System.Tuple`2<Typar,TType>>,ItemOccurence,DisplayEnv,NameResolutionEnv,System.Tuple`3<AccessorDomain,range,Boolean>>)
       at Microsoft.FSharp.Collections.SeqModule.Fold[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,System.__Canon>>, System.__Canon, System.Collections.Generic.IEnumerable`1<System.__Canon>)
       at FSharp.Compiler.TypeChecker.MakeAndPublishSimpleVals$cont@1870(cenv, TcEnv, Microsoft.FSharp.Collections.FSharpMap`2<System.String,PrelimValScheme1>, Microsoft.FSharp.Core.Unit)
       at FSharp.Compiler.TypeChecker.TcIteratedLambdas(cenv, Boolean, TcEnv, TType, Microsoft.FSharp.Collections.FSharpSet`1<System.String>, SyntacticUnscopedTyparEnv, SynExpr)
       at FSharp.Compiler.TypeChecker.TcExprUndelayed(cenv, TType, TcEnv, SyntacticUnscopedTyparEnv, SynExpr)
       at FSharp.Compiler.TypeChecker.TcExpr(cenv, TType, TcEnv, SyntacticUnscopedTyparEnv, SynExpr)
       at FSharp.Compiler.TypeChecker.TcNormalizedBinding(DeclKind, cenv, TcEnv, SyntacticUnscopedTyparEnv, TType, Microsoft.FSharp.Core.FSharpOption`1<Val>, SafeInitData, Microsoft.FSharp.Collections.FSharpList`1<Typar>, ExplicitTyparInfo, NormalizedBinding)
       at FSharp.Compiler.TypeChecker.TcNonRecursiveBinding(DeclKind, cenv, TcEnv, SyntacticUnscopedTyparEnv, TType, SynBinding)
       at FSharp.Compiler.TypeChecker+TcLetBinding@11058.Invoke(SyntacticUnscopedTyparEnv, SynBinding)
       at Microsoft.FSharp.Primitives.Basics.List.mapFold[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,System.Tuple`2<System.__Canon,System.__Canon>>>, System.__Canon, Microsoft.FSharp.Collections.FSharpList`1<System.__Canon>)
       at FSharp.Compiler.TypeChecker.TcLetBinding(cenv, Boolean, TcEnv, ContainerInfo, DeclKind, SyntacticUnscopedTyparEnv, Microsoft.FSharp.Collections.FSharpList`1<SynBinding>, range, range)
       at FSharp.Compiler.TypeChecker.TcLetBindings(cenv, TcEnv, ContainerInfo, DeclKind, SyntacticUnscopedTyparEnv, Microsoft.FSharp.Collections.FSharpList`1<SynBinding>, range, range)
       at FSharp.Compiler.TypeChecker+TcModuleOrNamespaceElementNonMutRec@17050-1.Invoke(Microsoft.FSharp.Core.Unit)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+catch@902-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+catch@902-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+catch@902-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+catch@902-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule+bind@893-1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Invoke(CompilationThreadToken)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule.forceWhile[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](CompilationThreadToken, Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit,Boolean>, Eventually`1<System.__Canon>)
       at FSharp.Compiler.AbstractIL.Internal.Library+EventuallyModule.force[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](CompilationThreadToken, Eventually`1<System.__Canon>)
       at FSharp.Compiler.CompileOps.TypeCheckOneInput(CompilationThreadToken, Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit,Boolean>, TcConfig, TcImports, TcGlobals, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Collections.FSharpList`1<Ident>>, TcState, ParsedInput)
       at FSharp.Compiler.CompileOps+TypeCheckClosedInputSet@5589.Invoke(TcState, ParsedInput)
       at Microsoft.FSharp.Primitives.Basics.List.mapFoldToFreshConsTail[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Collections.FSharpList`1<System.__Canon>, FSharpFunc`3<System.__Canon,System.__Canon,System.Tuple`2<System.__Canon,System.__Canon>>, System.__Canon, Microsoft.FSharp.Collections.FSharpList`1<System.__Canon>)
       at Microsoft.FSharp.Primitives.Basics.List.mapFold[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,System.Tuple`2<System.__Canon,System.__Canon>>>, System.__Canon, Microsoft.FSharp.Collections.FSharpList`1<System.__Canon>)
       at Microsoft.FSharp.Collections.ListModule.MapFold[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,System.Tuple`2<System.__Canon,System.__Canon>>>, System.__Canon, Microsoft.FSharp.Collections.FSharpList`1<System.__Canon>)
       at FSharp.Compiler.CompileOps.TypeCheckClosedInputSet(CompilationThreadToken, Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit,Boolean>, TcConfig, TcImports, TcGlobals, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Collections.FSharpList`1<Ident>>, TcState, Microsoft.FSharp.Collections.FSharpList`1<ParsedInput>)
       at FSharp.Compiler.Driver.TypeCheck(CompilationThreadToken, TcConfig, TcImports, TcGlobals, ErrorLogger, System.String, FSharp.Compiler.NiceNameGenerator, TcEnv, Microsoft.FSharp.Collections.FSharpList`1<ParsedInput>, Exiter)
       at FSharp.Compiler.Driver.main0[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.__Canon, System.String[], Resolver, Boolean, ReduceMemoryFlag, CopyFSharpCoreFlag, Exiter, ErrorLoggerProvider, DisposablesTracker)
       at FSharp.Compiler.Driver.typecheckAndCompile(CompilationThreadToken, System.String[], Resolver, Boolean, ReduceMemoryFlag, CopyFSharpCoreFlag, Exiter, ErrorLoggerProvider, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Core.FSharpFunc`2<TcImports,Microsoft.FSharp.Core.Unit>>, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Core.FSharpFunc`2<System.Tuple`3<TcGlobals,System.String,ILModuleDef>,Microsoft.FSharp.Core.Unit>>)
       at FSharp.Compiler.Driver.mainCompile(CompilationThreadToken, System.String[], Resolver, Boolean, ReduceMemoryFlag, CopyFSharpCoreFlag, Exiter, ErrorLoggerProvider, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Core.FSharpFunc`2<TcImports,Microsoft.FSharp.Core.Unit>>, Microsoft.FSharp.Core.FSharpOption`1<Microsoft.FSharp.Core.FSharpFunc`2<System.Tuple`3<TcGlobals,System.String,ILModuleDef>,Microsoft.FSharp.Core.Unit>>)
       at FSharp.Compiler.CommandLineMain+Driver.main(System.String[])
       at FSharp.Compiler.CommandLineMain.main(System.String[])

[jkotas]

Repro (it does not need to be a minimal repro) would help a lot. Is there a way to run the end-to-end tests locally?

[nguerrera]

"\\mlangfs1\public\nicholg\fsharp_dotnet5\repro.bat"

This will invoke the compilation of hello world directly using a private sdk that has this problem

[jkotas]

The GC heap is corrupted. !VerifyHeap detects number of inconsistencies when the crash happens. Most likely a GC hole.

[jakobbotsch]

What bits are in .NET 5.0? Maybe this could be dotnet/coreclr#26818 (comment) (read comment above for some context).

[jkotas]

It repros with current master

[jakobbotsch]

Then it is probably the same issue.

[jkotas]

Yes, it is a problem with slow tailcalls. I got time travel trace of the crash and found what looks like a problem that matches dotnet/coreclr#26818 (comment) . Also, the crash no longer reproduces if slow tailcalls are disabled in current master.

cc @jashook @BruceForstall