LLM Observability Experiments supports the entire lifecycle of building LLM applications and agents. It helps you understand how changes to prompts, models, providers, or system architecture affect performance. With this feature, you can:

• Create and version datasets
• Run and manage experiments
• Compare results to evaluate impact

Cookbooks

For in-depth examples of what you can do with LLM Experiments, see Datadog’s provided Jupyter notebooks.

Further reading

Additional helpful documentation, links, and articles:

Set up and use LLM Observability ExperimentsDOCUMENTATION Create and monitor LLM experiments with DatadogBLOG Datadog LLM Observability natively supports OpenTelemetry GenAI Semantic ConventionsBLOG Driving AI ROI: How Datadog connects cost, performance, and infrastructure so you can scale responsiblyBLOG How we cut our NLQ agent debugging time from hours to minutes with LLM ObservabilityBLOG