[Feature]: AI Request Logging#437

TylerB24890 · 2026-04-16T14:19:15Z

Original PR: #149

What?

Adds a comprehensive AI Request Logging experiment that provides observability for all AI operations:

Real-time logging of every AI request (provider, model, tokens, duration)
React-powered admin dashboard under Tools → AI Request Logs
REST API endpoints for programmatic access
Configurable retention period and automatic cleanup

Closes #345.

Why?

AI operations are opaque—users have no visibility into what requests are being made, how much they cost, or whether they're succeeding. This experiment provides the observability surface needed for debugging, cost tracking, and usage analysis.

How?

Architecture uses the decorator pattern to wrap the SDK's HTTP transporter:

Logging_Integration wraps the transporter on wp_loaded/admin_init using the public setHttpTransporter() API
Logging_Http_Transporter decorates requests, capturing timing and delegating to Log_Data_Extractor
Log_Data_Extractor parses request/response payloads with 4 filter hooks for extensibility:
- ai_request_log_providers - customize provider detection
- ai_request_log_context - filter context data
- ai_request_log_tokens - custom token extraction
- ai_request_log_kind - request type detection
AI_Request_Log_Manager coordinates schema, repository, and cost calculation
~~AI_Request_Cost_Calculator estimates costs based on provider pricing tables~~

Use of AI Tools

AI assistance: Yes
Tool(s): Claude
Model(s): Opus 4.7
Used for:
- Thorough review and analysis of original PR codebase & changes
- Assistance in updating unit tests
- Assistance in tracking down duplicate logic/unnecessary function wrappers.

Testing Instructions

Enable the AI Request Logging experiment in Settings → AI
Trigger an AI-powered feature (e.g., Title Generation, Excerpt Generation)
Navigate to the logs via Tools → AI Request Logs
Verify the log table populates with request details (provider, model, tokens, duration)
Test filtering by provider, status, and date range
Verify "Purge logs" clears all entries
Disable the experiment and confirm no new logs appear

- Replace hand-rolled table, filters, search, and pagination with @wordpress/dataviews for the Request Logs screen - Bundle @wordpress/dataviews and @wordpress/ui (not registered in WP 7.0-RC1) via custom DependencyExtractionWebpackPlugin config - Copy DataViews CSS into build/ and enqueue from PHP with fallback to wp-dataviews style handle when available - Filter unregistered script dependencies in Asset_Loader to prevent wp_enqueue_script notices - Expand AI_Request_Cost_Calculator from 3 providers / 25 models to 8 providers / 50+ models (adds DeepSeek, Mistral, Cohere, Groq, Grok, Ollama; adds o-series reasoning models, Claude 4.5/4.6 IDs) - Add Azure → OpenAI provider alias for cost lookup - Return $0.00 for local providers (Ollama) instead of null - Fix provider tooltip: click-to-toggle instead of hover so links are reachable - Compact operation cell to max 2 rows (code + kind inline) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Fix PHPCS array arrow alignment in cost calculator (phpcbf auto-fix) - Fix PHPStan: cast filemtime() to string for wp_enqueue_style version - Fix ESLint a11y: add role, tabIndex, onKeyDown to provider cell - Fix ESLint prettier: auto-formatted multiline expressions - Fix ESLint import: suppress no-extraneous-dependencies for webpack build-tool requires (dependency-extraction-webpack-plugin, copy-webpack-plugin) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…encies instead of React.

codecov · 2026-04-16T14:27:47Z

Codecov Report

❌ Patch coverage is 72.96443% with 342 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.16%. Comparing base (841b3da) to head (4f9b9f5).

Files with missing lines	Patch %	Lines
includes/Logging/Logging_Http_Transporter.php	19.17%	118 Missing ⚠️
includes/Logging/AI_Request_Log_Repository.php	81.62%	70 Missing ⚠️
includes/Logging/AI_Request_Log_Page.php	18.30%	58 Missing ⚠️
includes/Logging/Log_Data_Extractor.php	85.65%	36 Missing ⚠️
includes/Logging/AI_Request_Log_Manager.php	69.01%	22 Missing ⚠️
includes/Logging/AI_Request_Log_Schema.php	76.34%	22 Missing ⚠️
includes/Logging/Logging_Integration.php	36.00%	16 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #437      +/-   ##
=============================================
+ Coverage      72.24%   73.16%   +0.92%     
- Complexity      1162     1526     +364     
=============================================
  Files             68       77       +9     
  Lines           5609     6867    +1258     
=============================================
+ Hits            4052     5024     +972     
- Misses          1557     1843     +286

Flag	Coverage Δ
unit	`73.16% <72.96%> (+0.92%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

dkotter

Overall this looks decent to me but have a couple bigger questions:

I'd love to not have to manually support a bunch of providers. If possible, I'd suggest we remove any hardcoded provider references and rely on what information we can get from the AI Client
I'd suggest we drop cost tracking / estimation from this as it gets overly complicated. My opinion is that should be implemented at the provider level

dkotter · 2026-04-20T20:49:19Z

+		// Copy DataViews CSS into build/ so the plugin can enqueue it
+		// on WP versions where wp-dataviews is not a registered style.


If all the code changes in this file are only to support older WP versions, I think we can just remove all of this as we only support 7.0+ anyway

It seems WP 7.0 doesn't actually register the wp-dataviews stylesheet handle. I tried removing this but the UI ended up breaking.

Is there a different handle for the wp-dataviews stylesheet I should be targeting? Left this in place for now until we figure this out.

…return type definitions.

github-actions · 2026-05-11T14:05:20Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: Jameswlepage <isotropic@git.wordpress.org>
Co-authored-by: TylerB24890 <tyb@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: justlevine <justlevine@git.wordpress.org>
Co-authored-by: gohelkunjan <gohelkunjan@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

dkotter · 2026-05-11T14:08:27Z

This is looking good to me. I think final thing needed here is some discussion around the introduction of a new database table in this PR. I don't have any concerns with that but others may.

If we decide we're fine with that, I think final step is to ensure we are okay with the structure of the table and the data we are storing as it's harder to change that once this goes live. cc / @jeffpaul

dkotter · 2026-05-14T16:32:56Z

+		$table_name      = $this->get_table_name();
+		$charset_collate = $wpdb->get_charset_collate();
+
+		$sql = "CREATE TABLE {$table_name} (


I asked for feedback on this new table and schema on our last AI call and one question was raised around the logging of cached tokens. We log input and output tokens here but not cached tokens.

Looking into it, I think this will require changes both in the PHP AI Client and within each AI Provider plugin, as right now the AI Client has a TokenUsage class that doesn't consider cached tokens.

And looking at the OpenAI Provider plugin as an example, it removes all usage information after sending it to the TokenUsage class so you can't even manually pull that information yourself.

Hmmm, that's a good flag. I didn't really even think about cached tokens as a separate metric here. Are cached tokens returned as a value in that usage key? I think supplying them as another parameter to the GenerativeAiResult object would be the best path forward, as that already stores tokenUsage.

Given the GenerativeAiResult object doesn't currently do anything for cached tokens is that something we could add support for in like a phase 2 of this experiment or should we aim to get that squared away now?

I didn't dive too far into it but I think upstream changes are needed to fully support this, both in the PHP AI Client and in each individual provider plugin. As such, I wouldn't put that as a blocker here

Yeah I reviewed a bit and that was my understanding as well 👍

jeffpaul · 2026-05-19T15:34:30Z

Using WordPress 7.0-RC4, OAI provider Version 1.0.3, and the latest ZIP here I'm not seeing any data appearing in the AI Request Logs screen.

jeffpaul · 2026-05-19T18:07:33Z

Tested locally again, still nothing coming through. Tested on Playground with WordPress 7.0-RC4 running, OAI Version 1.0.3, and things worked as expected, so maybe just something off with my local.

jeffpaul · 2026-05-19T18:08:36Z

May end up replacing this with the updated WPORG icon asset from #546

Ah, yep, good catch. I can update that now that #546 has been merged

Actually it doesn't look like that icon was even being used, it was already defaulting to the same icon we use everywhere else. I've gone ahead and removed that extra icon file

Added - Request Logging experiment that provides observability for all AI operations Co-authored-by: Jameswlepage <isotropic@git.wordpress.org> Co-authored-by: TylerB24890 <tyb@git.wordpress.org> Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org> Co-authored-by: dkotter <dkotter@git.wordpress.org> Co-authored-by: justlevine <justlevine@git.wordpress.org> Co-authored-by: gohelkunjan <gohelkunjan@git.wordpress.org>

justlevine · 2026-05-25T20:30:43Z

 				- tests/
 				- vendor/
+				- includes/Logging/Logging_Integration.php
+				- includes/Logging/Logging_Http_Transporter.php


I know I'm late here, but I'm not seeing it in the previous comments.
@TylerB24890 @dkotter Why aren't we analyzing these files?

I believe the issue here is these files reference code coming from the AI Client and that code isn't stubbed yet, so PHPStan throws errors. And the errors thrown were internal to PHPStan so they weren't errors we could suppress individually.

I couldn't find the discussion in a quick search but the idea was we could either stub the AI Client ourselves or ignore those files until the upstream WordPress Core stubs are updated. We decided on the latter but open to revisiting that decision

Definitely no need to revisit here, we should be able to update to the latest WordPress stubs as soon as wp-cli-stubs cuts a new release. If it's not out this week though, I'll open a PR to add in a # Reason why we're ignoring these

Jameswlepage and others added 16 commits March 25, 2026 16:52

feat: refresh AI request logging for current develop

f93095e

fix: guard provider metadata without AI client runtime

299d817

fix: align request logs localized settings

56f7774

fix: render request logs header icon

f566267

refactor: simplify request logs admin UI

69f4db3

fix: stabilize CI checks

666459e

fix: refine request logs filtering

073052f

Merge latest develop & resolve conflicts.

2916bc2

Update DataView with correct type definitions

744e479

Fix DataViews fields and layout switching

0ff6282

Fix table filters and query logic. Use @WordPress packages for depden…

e53aa8f

…encies instead of React.

Fix linting errors

98e1853

Remove default operations filter. Remove unused functions.

fce11d9

Fix static analysis stubs

45a83f4

Add Python-2.0 license to dependency review action

f588fc8

TylerB24890 changed the title ~~Feature/ai request logging~~ [Feature]: AI Request Logging Apr 16, 2026

TylerB24890 added 6 commits April 16, 2026 11:02

Add Cost Calculator Tests.

4568a0d

Add Log Controller test for API integration

3ce093b

Add unit tests for repository log

379f94c

Add log extractor tests

ee862bb

Merge latest develop and resolve conflicts

82d513f

Merge branch 'develop' into feature/ai-request-logging

fc57e92

jeffpaul mentioned this pull request Apr 16, 2026

Add AI Request Logging experiment with observability dashboard #149

Closed

jeffpaul assigned TylerB24890 Apr 16, 2026

jeffpaul added this to the 0.8.0 milestone Apr 16, 2026

Merge latest develop and resolve conflicts

63b1ac9

dkotter requested changes Apr 20, 2026

View reviewed changes

TylerB24890 added 3 commits May 7, 2026 09:13

Add basic E2E tests for request logging

c13f6e0

Add E2E tests for period selector and search filtering

9fa8e26

Merge branch 'develop' into feature/ai-request-logging

cb1010f

TylerB24890 requested a review from dkotter May 7, 2026 15:18

TylerB24890 added 2 commits May 7, 2026 12:00

Merge branch 'develop' into feature/ai-request-logging

3564f6b

Exclude problematic files from static analysis until 7.0 launch. Fix …

9d2f32c

…return type definitions.

dkotter marked this pull request as ready for review May 11, 2026 14:05

dkotter mentioned this pull request May 12, 2026

Connector Approval experiment to limit what plugins can access what connectors #467

Merged

jeffpaul requested a review from justlevine May 12, 2026 15:29

Merge branch 'develop' into feature/ai-request-logging

a630959

dkotter reviewed May 14, 2026

View reviewed changes

mateuswetah mentioned this pull request May 18, 2026

Unexpected Universal Open AI Connector API response: Missing the "choices" key rtCamp/universal-openai-connector#39

Closed

jeffpaul mentioned this pull request May 18, 2026

Release version 1.0.0 #567

Closed

42 tasks

jeffpaul added 2 commits May 18, 2026 22:11

Merge branch 'develop' into feature/ai-request-logging

c9c64a7

Merge branch 'develop' into feature/ai-request-logging

11fe02f

Merge branch 'develop' into feature/ai-request-logging

6199d37

jeffpaul previously approved these changes May 19, 2026

View reviewed changes

jeffpaul reviewed May 19, 2026

View reviewed changes

dkotter added 2 commits May 19, 2026 12:32

Merge branch 'develop' into feature/ai-request-logging

69dcc16

Remove icon file that wasn't used

4f9b9f5

dkotter dismissed jeffpaul’s stale review via 4f9b9f5 May 19, 2026 18:36

dkotter approved these changes May 19, 2026

View reviewed changes

dkotter merged commit 01f6393 into WordPress:develop May 19, 2026
21 of 35 checks passed

mateuswetah mentioned this pull request May 19, 2026

Adapt logging strategy to use WordPress AI logging screen tainacan/tainacan-ai#4

Closed

justlevine reviewed May 25, 2026

View reviewed changes

		// Copy DataViews CSS into build/ so the plugin can enqueue it
		// on WP versions where wp-dataviews is not a registered style.

Conversation

TylerB24890 commented Apr 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What?

Why?

How?

Use of AI Tools

Testing Instructions

Uh oh!

codecov Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dkotter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkotter commented May 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffpaul commented May 19, 2026

Uh oh!

jeffpaul commented May 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

TylerB24890 commented Apr 16, 2026 •

edited by github-actions Bot

Loading

codecov Bot commented Apr 16, 2026 •

edited

Loading

github-actions Bot commented May 11, 2026 •

edited

Loading