Add AI Request Logging experiment with observability dashboard #149
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds a comprehensive logging system for AI requests that provides: - Observability: Track all AI operations with request/response previews - Cost tracking: Automatic cost estimation based on provider/model pricing - Admin UI: Full-featured log viewer with filtering, search, and stats - REST API: Endpoints for querying logs programmatically Architecture: - AI_Request_Log_Manager: Facade coordinating all logging operations - AI_Request_Log_Schema: Database table creation and management - AI_Request_Log_Repository: CRUD with caching and cursor pagination - AI_Request_Cost_Calculator: Provider pricing and cost estimation - Logging_HTTP_Client: Decorator for automatic request capture Features: - Configurable retention period and max rows - Daily cleanup cron job - FULLTEXT search with LIKE fallback - Filter by type, provider, status, operation, date range - Summary statistics by period
Replace the PSR-18 client discovery approach with a cleaner architecture that uses the AiClient SDK's public setHttpTransporter() API: - Add Logging_Http_Transporter: decorator implementing HttpTransporterInterface - Add Logging_Integration: initializes logging by wrapping the SDK transporter - Remove Logging_HTTP_Client and Logging_Discovery_Strategy (reflection-based) - Update bootstrap to use new Logging_Integration class - Update documentation with new architecture details This approach is more production-ready as it: - Uses the SDK's public API rather than reflection hacks - Wraps at the correct abstraction level (transporter, not HTTP client) - Is resilient to SDK internal changes - Follows the decorator pattern cleanly
Extract data parsing logic from Logging_Http_Transporter into a dedicated Log_Data_Extractor class with filter hooks for extensibility: - ai_request_log_providers: customize provider detection patterns - ai_request_log_context: filter context data before storage - ai_request_log_tokens: custom token extraction for new providers - ai_request_log_kind: customize request kind detection Architecture improvements: - Separation of concerns: transport vs extraction logic - Transporter reduced from 574 to 130 lines - Media metadata stored instead of full base64 data (prevents DB bloat) - All extraction methods are public for testability Supports 14 providers out of the box: OpenAI, Anthropic, Google, Fal, Cloudflare, Groq, Grok, HuggingFace, DeepSeek, Ollama, OpenRouter, Azure, Cohere, and Mistral.
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive AI Request Logging experiment that provides observability for AI operations through real-time logging, a React-powered admin dashboard, REST API endpoints, and configurable retention settings. The implementation uses the decorator pattern to wrap the SDK's HTTP transporter and supports 14 AI providers out of the box.
Key changes:
- Logging infrastructure with database schema, repository pattern, and cost estimation
- React-based admin dashboard with DataViews for log viewing and filtering
- REST API endpoints for programmatic access to logs and settings
- Automatic cleanup based on retention period and row limits
Reviewed changes
Copilot reviewed 46 out of 47 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| webpack.config.js | Adds webpack entry point for AI request logs admin bundle |
| src/admin/hooks/usePersistedView.ts | Custom hook wrapping @wordpress/views for persisting DataViews state |
| src/admin/components/provider-icons.tsx | Provider icon component mapping for displaying AI provider logos |
| src/admin/components/icons/* | SVG icon components for various AI providers |
| src/admin/ai-request-logs/index.tsx | Main React app component with state management and API integration |
| src/admin/ai-request-logs/components/* | UI components for logs table, summary cards, settings panel, and modal |
| src/admin/ai-request-logs/style.scss | Comprehensive styles for the admin dashboard |
| includes/helpers.php | Helper functions for experiment checking, icon loading, and timeout configuration |
| includes/bootstrap.php | Bootstrap integration for logging initialization |
| includes/Logging/* | Core logging infrastructure (Manager, Schema, Repository, HTTP Transporter) |
| includes/Experiment_Loader.php | Registers the AI Request Logging experiment |
| tests/Integration/Includes/Logging/* | Integration tests for log manager functionality |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7 tasks
|
This is great. Building something like this was at the top of my to-do list for January, before I start developing more AI abilities for our staff. |
jeffpaul
reviewed
Dec 20, 2025
Member
There was a problem hiding this comment.
Can we use the colored variant like https://github.com/WordPress/ai/blob/develop/.wordpress-org/icon.svg instead?
Member
|
@Jameswlepage can you add tests here like was done in https://github.com/WordPress/ai/pull/147/changes#diff-0cf348eda48cae411ad9d44a2e5b449603a971f2313b0a849893ff4cf8280c46? |
Member
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Adds a comprehensive AI Request Logging experiment that provides observability for all AI operations:
Why?
AI operations are opaque—users have no visibility into what requests are being made, how much they cost, or whether they're succeeding. This experiment provides the observability surface needed for debugging, cost tracking, and usage analysis.
How?
Architecture uses the decorator pattern to wrap the SDK's HTTP transporter:
Logging_Integrationwraps the transporter onwp_loaded/admin_initusing the publicsetHttpTransporter()APILogging_Http_Transporterdecorates requests, capturing timing and delegating toLog_Data_ExtractorLog_Data_Extractorparses request/response payloads with 4 filter hooks for extensibility:ai_request_log_providers- customize provider detectionai_request_log_context- filter context dataai_request_log_tokens- custom token extractionai_request_log_kind- request type detectionAI_Request_Log_Managercoordinates schema, repository, and cost calculationAI_Request_Cost_Calculatorestimates costs based on provider pricing tablesSupports 14 providers out of the box: OpenAI, Anthropic, Google, Fal, Cloudflare, Groq, Grok, HuggingFace, DeepSeek, Ollama, OpenRouter, Azure, Cohere, and Mistral.
Testing Instructions