Roman Eisner opened merge request !231530: Draft: Session Analytics at GitLab.org / GitLab

Roman Eisner
2026-04-15T18:44:36Z

What does this MR do and why?

Check the session-analytics page for your project:

http://gdk.test:3000/gitlab-duo/test/-/automate/session-analytics

References

Valuable additions:

Token usage — usage_metadata on every AIMessage has input_tokens, output_tokens, cache_read, and cache_creation tokens. These are summed across all nodes per workflow and could be aggregated per definition — useful for cost analysis.

Application-level tool errors — ToolMessage.status is always "success" (transport-level), but the content frequently contains soft errors: "Action error: File not found", {"error": "API request failed..."}, "Tool runtime exception". These are invisible without scanning the content. Worth counting separately from the HTTP status.

LangGraph workflow status — channel_values.status holds the LangGraph-level completion state ("Completed", "Not Started", etc.), which is distinct from workflow.status_name. In the error checkpoint the model status may say one thing while channel_values.status says "Not Started" — could be a useful signal.

Less actionable:

channel_values.context has metadata (goal, project_id, session_url) but that's per-workflow context, not aggregate analytics

invalid_tool_calls was empty in both checkpoints

The two highest-value additions would be total token usage per definition (input/output/cache) and application-level tool error counts — they're both invisible in the current stats. Want me to add either or both?

Screenshots or screen recordings

Before | After
|

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.