Optimize SimpleRWLock

[sdmaclea]

Keep hot/cold data on separate cache lines
Fix errors in exponential backoff
Make m_WriterWaiting Volatile and only use MemoryBarriers on write
Simplify code

[sdmaclea]

@kouvel @vancem PTAL . I just realized this was not the same lock you were optimizing. The ideas may still overlap....

[sdmaclea]

@jkotas @janvorli This was a patch I drafted when this lock was being hit hard by the plaintext test on arm64. At the time it made a significant improvement, but another patch made it unnecessary for plaintext optimization. PTAL it may still be useful in contended cases

[sdmaclea]

It looks like this is failing lots of tests. If you want the optimization, I'll clean it up and make the tests pass.

[sdmaclea]

The failure look to be related to changes in object size. There are some constants which will need to be updated because The size of SimpleRWLock has changed or perhaps the offset of some of its member variables.

[jkotas]

If this change is not improving anything meaningful, I would not bother with it.

For general purpose unmanaged locks like this one , I think we should be rather going towards using the OS provided ones (pthread_rwlock_t on Unix or SRWLock on Windows).

[sdmaclea]

@jkotas Thanks

Closing till this appears in a profile.