Adding distinct activity for distributed tracing to YARP

[samsp-msft]

Adding distinct tracing objects to YARP.

These can then be picked up via monitoring middleware such as Open Telemetry.

using Azure.Monitor.OpenTelemetry.AspNetCore;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddReverseProxy().LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var otel = builder.Services.AddOpenTelemetry();
otel.UseAzureMonitor(o =>
{
    o.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
});
otel.WithTracing(t =>
{
    t.AddSource("Yarp.*");
});

var app = builder.Build();

app.MapReverseProxy();

app.Run();

The activity source is only active if something is listening to it, such as t.AddSource("Yarp.*");.

[samsp-msft]

Result when seen in App Insights

[MihaZupan]

If you add .AddHttpClientInstrumentation() to the builder, you should see another dependency node at about the same time you see Forward now (assuming things aren't broken). That means you don't gain new timing information from the new activity.

Re: Tags like routeId, is there prior art in adding such information to existing Activities?
E.g. to the root one ASP.NET will create, or to the one HttpClient creates internally?

Things like that should be trivial to do now that ActivityListener is a thing

ActivitySource.AddActivityListener(new ActivityListener()
{
    ShouldListenTo = source => source.Name == "System.Net.Http",
    ActivityStarted = activity =>
    {
        string routeId = "..."; // Get that somehow (IHttpContextAccessor?)
        activity.AddTag("YarpRoute", routeId);
    }
});

[samsp-msft]

If you add .AddHttpClientInstrumentation() to the builder, you should see another dependency node at about the same time you see Forward now (assuming things aren't broken). That means you don't gain new timing information from the new activity.

HttpClientInstrumentation is added as part of the AzureMonitor helper.

Adding to the existing activity verses adding another is an interesting question. You don't even need to get to the listener, you can probably just do Activity.Current.SetTag(...)

[Tratcher]

Great, just needs some cleaup.

[MihaZupan]

This will now allocate an extra async state machine even if the request isn't being sampled.
Can you please change the logic to something along the lines of what I shared here: #2098 (comment)?

[samsp-msft]

This will now allocate an extra async state machine even if the request isn't being sampled. Can you please change the logic to something along the lines of what I shared here: #2098 (comment)?

It only adds it if the activity is created, which depends on the listener. If the activity is null, nothing gets set.

[MihaZupan]

The issue is that the method is now async, which means you get an extra state machine allocation even if you don't have the activity.

[MihaZupan]

If you want you can keep it as-is and I can clean it up in a follow up PR