Provide a generic API to read from and write to a pointer

[terrajobst]

C# doesn't support pointer operations on generic types. We can, however, express them in IL. A prototype could like this:

public static class Unsafe
{
      [MethodImpl(MethodImplOptions.AggressiveInlining)]
      public static T Read<T>(void * p)
      {
          // Can't be expressed in C#, thus in IL:
          ldarg.0
          ldobj !!T
          ret
      }

      [MethodImpl(MethodImplOptions.AggressiveInlining)]
      public static void Write<T>(void * p, T value)
      {
          // Can't be expressed in C#, thus in IL:
          ldarg.0
          ldarg.1
          stobj !!T
          ret
      }
}

This would also solve issues like #16026 without any additional work.

[nietras]

I assume the discussion should continue here. Since I can't wait to be able to use this I created an implementation of this and published it as a NuGet package called DotNetCross.Memory.Unsafe this contains just the static class Unsafe and the proposed functions. This package has been marked alpha so be sure to include prereleases if you want to check this out. I haven't had time to add unit tests and tried doing some benchmark tests but BenchmarkDotNet does not work out of the box for this.

I have constrained the type to be a value type, as compared to @jkotas code and the proposal above, so the definition is:

public static class Unsafe
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static unsafe T Read<T>(void* p) where T : struct

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static unsafe void Write<T>(void* p, T value)  where T : struct
    }

I assume this shouldn't work for reference types. I also have not changed the order of parameters for the Write method, although I would prefer it to be Write<T>(T value, void* p), as it seems this was chosen to match existing behaviour and the IL, so it might be more efficient this way (maybe?).

The implementation in IL, which I did by hand from a normal class library, can be found at https://github.com/DotNetCross/Memory.Unsafe/blob/master/il/DotNetCross.Memory.Unsafe.il which contains the following definition of the Unsafe class:

.class public abstract auto ansi sealed beforefieldinit DotNetCross.Memory.Unsafe
       extends [System.Runtime]System.Object
{
  .method public hidebysig static !!T  Read<valuetype .ctor ([System.Runtime]System.ValueType) T>(void* p) cil managed aggressiveinlining
  {
    // Can't be expressed in C#, thus in IL:
    .maxstack 1
    ldarg.0
    ldobj !!T
    ret
  } // end of method Unsafe::Read

  .method public hidebysig static void  Write<valuetype .ctor ([System.Runtime]System.ValueType) T>(void* p, !!T 'value') cil managed aggressiveinlining
  {
    // Can't be expressed in C#, thus in IL:
    .maxstack 2
    ldarg.0
    ldarg.1
    stobj !!T
    ret  
    } // end of method Unsafe::Write
}

With regard to alignment, I agree with @CarolEidt that unaligned read/write does only have a small extra cost for Intel Sandy Bridge or later. However, it is still an extra cost and also means higher cache usage if I remember correctly (I am not an assembly/micro-architecture expert). More importantly, what about other architectures such as ARM? Some ARM architectures don't even have unaligned read/wrire.

Additionally, what about dynamically allocated memory on the stack. Does this also assume aligned access? That is, a contrived exampled could be:

 public static unsafe void StackAllocAlignmentDoNotDoThis()
{
      var ptr = stackalloc byte[128];
      ptr += 3;
      var value = Unsafe.Read<double>(ptr);    
}

Now, you definitely shouldn't do this, but just trying to understand how alignment will work. Perhaps this should be discussed separately, as I have nothing against the proposed API. I would just prefer an API which also had explicit AlignedRead/AlignedWrite, UnalignedRead/UnalignedWrite, with fallback to whatever makes sense for a given platform. There still should be Read/Write which have sensible defaults as @CarolEidt have mentioned are:

• Heap pointers (e.g. unmanaged or fixed managed: unaligned access.
• Stack pointers: aligned access.

[jkotas]

I assume this shouldn't work for reference types

Constraining T to reference type does not prevent one from using it for reference types. One just needs to go through extra hassle to wrap the reference type in struct:

struct Wrapper<T> { public T Value; };
Unsafe.Read<Wrapper<String>>(p).Value

There are valid situations for Unsafe.Read on reference type (not with Vector<T>). It is interesting question whether the Unsafe class should have artificial restrictions to make it "safer" ... the artificial restrictions in current C# unsafe constructs is why we are having this discussion.

how alignment will work

The abstraction level that this is coded against is IL. In IL, all access is assumed to be sufficiently aligned by default. The unaligned access has to be explicitly requested by unaligned prefix for IL instruction. So the simple natural way to implement UnalignedRead would be:

    ldarg.0
    unaligned. 1 
    ldobj !!T
    ret

It is JIT problem to figure out the most efficient way to translate this to machine instructions on given platform in given context. E.g. on platforms without direct instructions for unaligned access, the JIT has to expand this into series of byte loads unless it can prove that the pointer is aligned.

[nietras]

@jkotas of course, I wasn't trying to imply this would make it "safe" as such, just that this could indicate the expected use case, guess I wasn't imaginative enough :). Could you elaborate on the valid situations for using Unsafe.Read for reference types? And how this would work?

[jkotas]

situations for using Unsafe.Read

E.g. Low level code that does reflection:
https://github.com/dotnet/corert/blob/11d3cd78d4b656d3053651cb7cd13f1291b69000/src/System.Private.CoreLib/src/Internal/Runtime/Augments/RuntimeAugments.cs#L269 .

[nietras]

I have updated DotNetCross.Memory.Unsafe to no longer have the generic value type constraint. I have also added SizeOf<T> and AddressOf<T>.

@jkotas not sure I understand how the code you point to would benefit from Unsafe.Read for a reference type?

            fixed (IntPtr* pObj = &obj.m_pEEType)
            {
                IntPtr pData = (IntPtr)pObj;
                IntPtr pField = pData + fieldOffset;
                return LoadReferenceTypeField(pField);
            }

However, I do understand that it is beneficial to be able to cast a pointer (i.e. address) to be seen as a reference to a reference type. That makes sense. Despite all the issues that might follow with the GC etc.

The abstraction level that this is coded against is IL. In IL, all access is assumed to be sufficiently aligned by default. The unaligned access has to be explicitly requested by unaligned prefix for IL instruction.

From this I would assume that all Unsafe.Read/Write should be translated to aligned. However, according to @CarolEidt:

the JIT currently only assumes alignment for SIMD types on the stack. For reads or writes to non-stack locations it uses unaligned writes. It is my understanding that these incur
0-1 cycle of additional overhead for modern processors if the memory location is actually aligned.

Which means that no SIMD non-stack locations will be accessed unaligned. Which would be terrible if it had to do byte loads as mentioned:

It is JIT problem to figure out the most efficient way to translate this to machine instructions on given platform in given context. E.g. on platforms without direct instructions for unaligned access, the JIT has to expand this into series of byte loads unless it can prove that the pointer is aligned.

And this is exactly my concern. That we have no way of forcing through IL the JIT to do aligned accessing, but as you say we are expressing in IL that this should be aligned, the JIT, however, does what it thinks best. So I am not saying we should change the API, I'm just trying to express what we would ideally want. Full control ;)

Finally, there is one scenario that isn't covered. Casting a pointer to a value type pointer i.e. without loading the value type to the stack/register. This could be an issue for big value types e.g.

public struct BigValueType
{
    double M00;
    double M01;
    double M02;
    double M10;
    double M11;
    double M12;
    double M20;
    double M21;
    double M22;
}

Really, we do not want to load this completely if we only wanted to change a part of it e.g.

public void AddToM02(void* ptr, double value)
{
    BigValueType* big = Unsafe.Cast<BigValueType>(ptr);
    big->M02 += value;
}

Something we would like to do in a generic way. So would it even be possible to cast a pointer too? Like this:

   T* Cast<T>(void* p)

Pretty sure C# won't accept this or am I wrong? We would to have the unmanaged constraint for this right?

[mikedn]

From this I would assume that all Unsafe.Read/Write should be translated to aligned. However, according to @CarolEidt:

The thing is that IL knows nothing about SIMD. As far as IL is concerned Vector<float> is a struct that needs to be 4 byte aligned.

I'm just trying to express what we would ideally want. Full control ;)

It would be better to investigate if aligned SIMD access is truly needed. AFAIK there's no perf penalty if you use unaligned loads/stores (movups & co.) if the address is in fact properly aligned.

Casting a pointer to a value type pointer i.e. without loading the value type to the stack/register. This could be an issue for big value types e.g.

Fixing this requires compiler support. Either ref returns or generic pointers.

We would to have the unmanaged constraint for this right?

I don't think that's needed in this case. The input pointer is already unmanaged.

[jkotas]

not sure I understand how the code you point to would benefit from Unsafe.Read

LoadReferenceTypeField can be replaced with a clean Unsafe.Read<Object>(pField) instead of abusing internal Interlock overload that takes address as IntPtr.

we have no way of forcing through IL the JIT to do aligned accessing

This sounds like as optimizer hints, like what C/C++ compilers have. A potential way to express this in IL would be via void Unsafe.Assume(bool condition) that works very similar. So you would write:

    Unsafe.Assume((p & 0x1F) == 0);
    Unsafe.Write<Vector<double>>(p, v);

I would track the optimizer hints separately because of it is a non-trivial feature on its own, and it is unlikely to help the Vector<T> case much.

[nietras]

LoadReferenceTypeField can be replaced with a clean Unsafe.Read<Object>(pField) instead of abusing internal Interlock overload that takes address as IntPtr.

Yeah ok I thought the Interlocked was there for a reason. So the problem was in LoadReferenceTypeField where the Interlocked.Compare was. That makes sense then.

It would be better to investigate if aligned SIMD access is truly needed. AFAIK there's no perf penalty if you use unaligned loads/stores (movups & co.) if the address is in fact properly aligned.

It would definitely be much better to test this instead of my perhaps unfounded theoretical concerns. Although it is not unheard of that unaligned reads/writes can have significant consequences, see http://www.agner.org/optimize/blog/read.php?i=285 where:

Store-to-load forwarding is generally good, but in some unfortunate cases of an unaligned 256-bit read after a smaller write, there is an unusually large delay of more than 200 clock cycles.

But that is probably an outlier. Agner has more info at http://www.agner.org/optimize/#manuals

[mikedn]

But that is probably an outlier. Agner has more info at http://www.agner.org/optimize/#manuals

Incidentally Agner's manual is one source for my "AFAIK":

13.5 Accessing unaligned data and partial vectors
...
On contemporary processors, there is no penalty for using the unaligned instruction MOVDQU rather than the aligned MOVDQA if the data are in fact aligned. Therefore, it is convenient to use MOVDQU if you are not sure whether the data are aligned or not.

unaligned 256-bit read after a smaller write

That's a bit more complicated and not solely caused by unaligned data. And again, there's a distinction between using unaligned load instructions and unaligned loads.

[nietras]

@jkotas so Unsafe.Assume is a proposed feature? Interesting. I agree, though, that this should be discussed separately.

@mikedn I agree and for x64 this discussion is probably mute, but given that .NET Core is targetting cross platform, we should at least consider consequences for ARM etc. However, this should probably be discussed separately too, I guess.

Fixing this requires compiler support. Either ref returns or generic pointers.
I don't think that's needed in this case. The input pointer is already unmanaged.

Not sure I understand this, isn't it the output pointer type that is the problem for T* Cast<T>(void* p)? For C# to accept this would they not want to make sure T is an unmanaged value type or? Does T* make sense for a reference type? Or is that why you say something like ref T Cast<T>(void* p)?

By the way I have noticed many other projects that have the Unsafe.Read/Write and other unsafe methods e.g. https://github.com/sharpdx/SharpDX/blob/master/Source/SharpDX/Interop.cs So I am sure a lot of people will find value in these.

With regards to API I am happy with its elegant simplicity, so perhaps to move forward placement of API is a more pertinent question. We would prefer it to be as central as possible, as this would also make it useable for most projects without extra dependencies.

[mikedn]

we should at least consider consequences for ARM etc. However, this should probably be discussed separately too, I guess.

Yep, that's a long story since the SIMD support for ARM is simply non-existent at the moment. And before discussing about Unsafe.Read/Write there's a lot of other things to settle and that includes how SIMD vectors are loaded from normal arrays.

Not sure I understand this, isn't it the output pointer type that is the problem for T* Cast(void* p) ?

The devil is in the details. What does T* mean when T is a reference type? It can be a pointer to an object of type T (which may sound weird but it is technically doable provided that the object is pinned) or it can be a pointer to a reference of type T. I think that the later is what we need in this case.

Either way it works fine as long as the location pointed by the pointer is in stack or in a pinned GC heap object or in native memory. And that's expected since the input to these functions is a void*.

I don't think we'll see T* any time soon in C#. ref returns are more likely and ref T Cast<T>(void* p) makes perfect sense.

[jkotas]

@jkotas so Unsafe.Assume is a proposed feature?

I have created one.

[nietras]

@mikedn I though ARM NEON was pretty wide spread in high end smart phones and server ARM.

I have been playing around a bit with Unsafe.Read/Write and for the work we do we think it is essential to add SizeOf to the Unsafe class e.g.:

  [MethodImpl(MethodImplOptions.AggressiveInlining)]
  public static unsafe int SizeOf<T>()

Or in IL:

  .method public hidebysig static int32  SizeOf<T>() cil managed aggressiveinlining
  {
    sizeof !!T
    ret
  }

Not sure how this differs from System.Runtime.InteropServices.Marshal.SizeOf<T>() though, haven't had time to test it, but the Marshal.SizeOf<T> does not appear to be as efficient.

Not sure if SizeOfshould be added as a new issue or it can be handled here too.

[nietras]

A completely different issue I have with Unsafe.Read/Write for Vector<T> types, or probably with Vector<T> as such, is this only allows for reading the full or largest SIMD register i.e. for AVX(2) a register is 256-bit or 32 bytes. For image processing this means we can handle 32 bytes in one SIMD op.

However if we have say 31 bytes, AVX2 is as such not possible since we have less than 32 bytes, therefore, we often revert to using 128-bit (e.g. SSE2) registers instead, for the remaining, so at least 16 of the 32 bytes can be handled fast. As far as I can tell there is no way to use SIMD registers less than the largest available for the given architecture one is running on. I.e. if 256-bit registers are available, there is no way to access/use 128-bit registers, and with Unsafe.Read/Write there is no way to be able to do...

Unless one would have specific types for vectors with static sizes e.g. Vector256bit<T> (pretty bad naming), but I hope you understand. Coming from the world of C++ intrinsics where one can choose the exact instructions to use and one can leverage the fact that a AVX2 CPU also supports SSE2, it seems less than ideal that there is no way of doing the same in C# (and for our case in unmanaged memory). I presume this is actually an issue with how Vector<T> works. If so, I should file an issue for Vector<T>.