Add the Image Generation Experiment base

[dkotter]

What?

Partially closes #13

Adds the base Experiment and Ability code for Image Generation and Import. There will be a followup PR that adds in the UI to trigger image generation and import.

Why?

We want an Experiment that allows you to generate images, whether that is in-content images, featured images or an image used elsewhere and then import those images into your media library. To keep PRs as small as possible, this introduces the base framework for image generation and import, both the Image Generation Experiment and the Image Generation and Image Import Abilities.

How?

• Adds a new Image Generation Experiment class and registers this as one of our default Experiments
• Adds a new Image Generation Ability that allows you to pass in a prompt and it will then generate a base64 encoded image using the WP AI Client
• Adds a new Image Import Ability that allows you to pass in a base64 encoded image and it then imports that into your media library, along with any optional information like title, filename and description
  • Note: could extend this to support image URLs as well but mostly built this to handle importing images we generate, which will be base64 only
• Adds in new tests to cover these changes

Testing Instructions

Since there isn't a UI for this yet, will need to make direct API requests to see this working.

Make an authenticated GET request to the main abilities endpoint and ensure the Image Generation and Image Import abilities shows there

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****'

Make an authenticated GET request to the Image Generation endpoint and ensure information about the ability shows

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-generation' \
--header 'Authorization: Basic ****'

Make an authenticated GET request to the Image Import endpoint and ensure information about the ability shows

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-import' \
--header 'Authorization: Basic ****'

Make an authenticated POST request to the Image Generation run endpoint without passing a prompt and ensure an error shows

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
    }
}'

Make an authenticated POST request to the Image Generation run endpoint and pass in a prompt and ensure base64 encoded data is returned

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "prompt": "Sunrise over the mountains reflecting in a mountain lake"
    }
}'

Make an authenticated POST request to the Image Import run endpoint without passing data and ensure an error shows

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-import/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
    }
}'

Make an authenticated POST request to the Image Import run endpoint and pass in valid base64 data (and any optional data) and ensure the image data is returned and the image is in your media library

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/image-import/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
    }
}'

Test using WordPress Playground

The changes in this pull request can be previewed and tested using this WordPress Playground instance:

Click here to test this pull request.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @prabinjha, @kurtrank.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: prabinjha, kurtrank.

Co-authored-by: dkotter <[email protected]>
Co-authored-by: JasonTheAdams <[email protected]>
Co-authored-by: felixarntz <[email protected]>
Co-authored-by: karmatosed <[email protected]>
Co-authored-by: jeffpaul <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 78.37838% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.89%. Comparing base (f5b1892) to head (7dbbc44).
⚠️ Report is 35 commits behind head on develop.

Files with missing lines	Patch %	Lines
includes/Abilities/Image/Generate_Image.php	52.17%	22 Missing ⚠️
includes/Abilities/Image/Import_Base64_Image.php	88.69%	19 Missing ⚠️
.../Experiments/Image_Generation/Image_Generation.php	31.81%	15 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #134      +/-   ##
=============================================
+ Coverage      38.83%   46.89%   +8.05%     
- Complexity       177      208      +31     
=============================================
  Files             16       19       +3     
  Lines           1012     1271     +259     
=============================================
+ Hits             393      596     +203     
- Misses           619      675      +56     

Flag	Coverage Δ
unit	46.89% <78.37%> (+8.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[JasonTheAdams]

Good work, @dkotter! Couple small suggestions. 😄

[JasonTheAdams]

Let's call this Import_Base64_Image for clarity.

[dkotter]

Renamed: 6536248

[JasonTheAdams]

Good work, @dkotter!

[felixarntz]

@dkotter This looks excellent! A few small notes, but after that it'll be good to merge.

[felixarntz]

@dkotter Great! Just one somewhat unrelated note, but this is good to merge 👍

[felixarntz]

Just a note: In WordPress/performance, WordPress/php-ai-client, and a few other projects we established the convention of @since n.e.x.t. Obviously it's an arbitrary decision, but I wonder whether we should use that here too.

Most importantly, we should probably choose a convention and document it, so that it's easy for a pre-release step to replace all those with the actual version. Maybe we can even build linting for it at some point.

This is mostly a thing we'll want to figure out soon, feel free to change it here or leave as is if you prefer.

[dkotter]

I'm fine if we update to using this, no strong opinion on my end but noting this is currently documented as the approach (which is why I used that here) and there is a step in our release process to update these