Agent Trace Solution

An Agent Trace data pipeline captures the complete execution history of an AI agent and turns it into data that teams can query and reuse.

This includes model inputs and outputs, tool calls, retrieval steps, state changes, token usage, intermediate results, errors, and final outcomes. Unlike a basic logging pipeline, it must support long-running and branching executions, evolving JSON structures, incremental transformation, and downstream workflows such as Evals, replay, attribution, and training.

Databend Cloud provides the storage, processing, scheduling, and analytical layers required to operate this pipeline on one S3-backed data foundation.

Self-hosted Langfuse requires teams to operate PostgreSQL, ClickHouse, Redis or Valkey, blob storage, and application containers. At high volume and concurrency, scaling and maintaining these components can increase operational complexity and infrastructure costs. Langfuse Cloud removes this burden, but data access and retention vary by plan, with additional usage billed in units.

Databend Cloud provides one managed pipeline for large-scale Agent Trace data. It stores heterogeneous payloads as native VARIANT data in a customer-controlled lakehouse, where teams control retention and define JSON extraction and analytical models in SQL. It is built for TB-scale workloads with less infrastructure to operate.

Yes. Databend Cloud stores raw JSON payloads in the native VARIANT data type, including nested objects and arrays.

Teams do not need to define every possible field before ingestion. They can preserve the original payload, extract frequently queried fields with SQL, and update their transformation logic as agents, models, tools, and Trace formats evolve.

This separates reliable data ingestion from downstream modeling, preventing every upstream JSON change from becoming an immediate pipeline migration.

A Stream tracks changes that have not yet been consumed from the raw Trace table. A Task runs SQL on a schedule or when new rows are available.

Together, Stream and Task can extract JSON fields, normalize event names, enrich spans, update detail tables, and generate aggregations using only newly arrived data. Teams avoid repeatedly scanning and transforming the complete historical dataset.

The original raw records remain available for auditing, replay, and reprocessing when analytical requirements change.

Not for the core Trace transformation pipeline.

Databend Cloud can sit behind an existing ingestion layer such as Kafka or object storage, then handle raw data retention, incremental SQL transformation, task scheduling, analytical modeling, and querying in one managed service.

Teams can continue using external systems where their architecture requires them, but they do not need to deploy separate engines for every stage between raw Trace ingestion and analytical data production.