Add the Featured Image Generation UI

[dkotter]

What?

Partially closes #13. This only handles generating featured images, I'm assuming we'll want image generation in other places as well (though could be tracked in new Issues if we want).

Adds in the UI to trigger Featured Image Generation

Why?

This builds on top of the work done in #134, which added Abilities that could be used to generate an image and/or import an image. This PR adds in the actual UI that triggers those Abilities in the Featured Image section of the edit post screen.

How?

• Modifies the Image Import Ability to allow passing in additional meta data that will be stored with the image. This allows us to easily track which images are AI generated based on a specific meta key
• Adds a new utility Ability that can be used to generate generic LLM-friendly prompts. This is then used to generate a prompt, based on the post context, to generate a featured image
• Loads in the necessary admin scripts and admin script data
• Adds in all the needed javascript to wire things up, including:
  • a Generate featured image button that shows above the standard Set featured image button
  • When that button is clicked, first make a request to the get-post-details Ability. Then pass those details to the new generate-prompt Ability, along with our base purpose. Take the prompt this generates and pass it to our existing image-generation Ability. Finally take the base64-encoded image it returns and pass it to our existing image-import Ability to import the image into the Media Library
• Set this image as the Featured Image and change the text of the generation button from Generate featured image to Generate new featured image
• Add a label below the Featured Image (if set and if AI generated) that makes it clear the image was AI generated

Testing Instructions

1. Ensure you have valid AI credentials in place for OpenAI or Google under Settings > AI Credentials
2. Ensure you turn on the Image Generation Experiment under Settings > AI Experiments
3. Edit a piece of content that doesn't have a Featured Image yet
4. Ensure you see the Generate featured image button
5. Click this button (noting this will make an API request and will cost money)
6. Ensure a proper loading state is shown and that eventually (this can take around 90s) an image is rendered in the normal featured image section
7. Ensure this image imported properly into the Media Library and has a proper title and description
8. Ensure the AI label shows below the image in the featured image section

There is also one new Ability introduced in this PR that can be used to generate prompts. This will be tested as part of the above testing instructions but you can also test this Ability directly by making API requests, like the following:

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/generate-prompt/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "purpose": "Given the following paragraph of text, create a prompt that can be passed to an image generation LLM to generate an image that will be associated with the text",
        "context": "Content: Welcome to WordPress. This is your first post. Edit or delete it, then start writing!"
    }
}'

Screenshots or screencast

Test using WordPress Playground

The changes in this pull request can be previewed and tested using this WordPress Playground instance:

Click here to test this pull request.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @prabinjha, @kurtrank.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: prabinjha, kurtrank.

Co-authored-by: dkotter <[email protected]>
Co-authored-by: JasonTheAdams <[email protected]>
Co-authored-by: jeffpaul <[email protected]>
Co-authored-by: karmatosed <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 68.59504% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.45%. Comparing base (6478545) to head (6f77dd3).
⚠️ Report is 7 commits behind head on develop.

Files with missing lines	Patch %	Lines
.../Experiments/Image_Generation/Image_Generation.php	36.66%	19 Missing ⚠️
includes/Abilities/Utilities/Prompts.php	75.00%	14 Missing ⚠️
includes/Abilities/Image/Generate_Image.php	0.00%	3 Missing ⚠️
includes/bootstrap.php	0.00%	2 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff               @@
##             develop     #146       +/-   ##
==============================================
+ Coverage      46.89%   59.45%   +12.56%     
- Complexity       208      219       +11     
==============================================
  Files             19       20        +1     
  Lines           1271     1391      +120     
==============================================
+ Hits             596      827      +231     
+ Misses           675      564      -111     

Flag	Coverage Δ
unit	59.45% <68.59%> (+12.56%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[JasonTheAdams]

One straightforward suggestion, but I'm also humming pretty hard at the flow of providing an AI prompt writing Ability in order to run the subsequent prompt. What problem is this extra roundtrip solving that we couldn't accomplish by compiling a prompt ourselves based on the post data? It feels like we could make due without this.

The only place I could see an Ability like that being genuinely useful is if a user had an input to do something like describe a picture, and we added a "Improve prompt" button. But even that would require more specific instructions based on the context. This feels too generic.

[JasonTheAdams]

Fortunately, there's a much better way of doing this on a per prompt basis! The builder has a using_request_options() method. So for this you'd add:

$request_options = new WordPress\AiClient\Providers\Http\DTO\RequestOptions();
$request_options->setTimeout(90);

$prompt->using_request_options($request_options);

[dkotter]

Ah, that's good to know, thanks for pointing that out. Updated in 460f529

[jeffpaul]

@dkotter perhaps a docs page for this experiment like done in https://github.com/WordPress/ai/pull/155/changes#diff-f70665388390657902e73741dea03efcdadc4bd0ceb4432959f8903ae015c042?

[dkotter]

One straightforward suggestion, but I'm also humming pretty hard at the flow of providing an AI prompt writing Ability in order to run the subsequent prompt. What problem is this extra roundtrip solving that we couldn't accomplish by compiling a prompt ourselves based on the post data? It feels like we could make due without this.

The only place I could see an Ability like that being genuinely useful is if a user had an input to do something like describe a picture, and we added a "Improve prompt" button. But even that would require more specific instructions based on the context. This feels too generic.

@JasonTheAdams I struggled coming up with an approach to turn an entire article into a useful image generation prompt (useful being key there), thus relying on AI to help us out here. I initially tried sending the entire article with a prompt along the lines of Generate a featured image for the following article but that didn't work great and leads to huge prompts. Open to suggestions though on other alternatives to solve this but given the wide-range of content someone may be using, I couldn't think of a better way to do this.

I did go back and forth on if this should be an Ability (the "generate a prompt from this context" piece) so happy to change that to be more specific to this use case instead of a globally exposed Ability if that helps at all.

[dkotter]

@dkotter perhaps a docs page for this experiment like done in https://github.com/WordPress/ai/pull/155/changes#diff-f70665388390657902e73741dea03efcdadc4bd0ceb4432959f8903ae015c042?

@jeffpaul I know James has mentioned these PRs are more WIP at the moment so wondering if that's the best example to copy from? I know our default Experiment did add a README file within the Experiments directory but we haven't followed that approach so far, though I can do that if desired. I just think we need to decide on what our standard documentation structure should be so we can ensure it matches across everything.