[API Proposal]: Improve RateLimiter metrics #71804
Description
Background and motivation
One of the original potential use cases cited for RateLimiter.GetAvailablePermits() was for diagnostics. While it's a good starting point, it would make more sense to include more information to include in diagnostics. So, taking inspiration from MemoryCache we are proposing changing GetAvailablePermits to a more useful API: GetStatistics.
API Proposal
namespace System.Threading.RateLimiting;
public abstract class RateLimiter : IAsyncDisposable, IDisposable
{
- public abstract int GetAvailablePermits();
+ public abstract RateLimiterStatistics? GetStatistics();
}
public abstract class PartitionedRateLimiter<TResource> : IAsyncDisposable, IDisposable
{
- public abstract int GetAvailablePermits(TResource resourceID);
+ public abstract RateLimiterStatistics? GetStatistics(TResource resourceID);
}
+ public class RateLimiterStatistics
+ {
+ public RateLimiterStatistics();
+ public long CurrentAvailablePermits { get; init; }
+ public long CurrentQueuedCount { get; init; }
+ public long TotalFailedLeases { get; init; }
+ public long TotalSuccessfulLeases { get; init; }
+ }API Usage
Meter meter = new Meter("RateLimiter", "1.0.0");
RateLimiter limiter = new ConcurrencyLimiter(
new ConcurrencyLimiterOptions(100, QueueProcessingOrder.OldestFirst, 20));
meter.CreateObservableGauge<long>("available-permits", GetAvailablePermits);
IEnumerable<Measurement<long>> GetAvailablePermits()
{
return new Measurement<long>[]
{
new Measurement<long>(limiter.GetStatistics()!.CurrentAvailablePermits,
new KeyValuePair<string, object?>("Limiter", "Concurrent")),
};
}Alternative Designs
An alternative name is GetCurrentStatistics which is what MemoryCache uses.
Another property that might be useful to include is long TotalAcquiredPermits { get; init; }
There has been discussion around allowing rate limiter specific state to be returned, either via this API or another API. This would be similar to HttpClientHandler.Properties
Two ways to allow state without introducing additional APIs to what is already proposed is to use the existing metadata on
RateLimitLease and call Acquire with 0 permits to always get back a lease without acquiring any permits. Or we keep the new RateLimiterStatistics class that's being proposed as non-sealed and allow implementations to provide their own statistics type that contains custom state.
Risks
If someone was wanting to query GetAvailablePermits before every call to Acquire or WaitAsync they would now need to call a more expensive method (allocates every call) to get the same behavior. I don't believe this would be a common pattern, if someone wants to acquire permits quickly without potential waiting, they can call Acquire which will already check for availability and be more accurate than calling GetAvailablePermits then Acquire if there are enough permits.
