Add the Title Generation execute functionality #67

dkotter · 2025-11-03T21:46:00Z

What?

Partially closes #10

Builds off of the Title Generation feature base work done in #61 to wire up the actual execution of title generation. Note the actual UI still isn't in this PR, there will be a close followup that adds that.

Why?

In PR #61, we introduced the Title Generation Ability and Feature classes but the actual generation of titles wasn't completed there. This PR finishes that piece, though note there still isn't any UI to trigger this functionality, it all needs to be done via direct API requests.

How?

Pulls in the latest changes from the trunk branch of the WP AI Client repo to take advantage of the new settings screen and helper classes introduced there. This can be changed to pull from an actual release once that is out
Initializes the WP AI Client
Adds in a new API_Request class that can be used to make requests to an AI Provider. This ultimately utilizes the WP AI Client. Debated whether having this extra abstraction was worth it so can change this if desired. But it felt like the right approach knowing we'll have multiple other AI integrations coming that will all have slightly different requirements (i.e. arguments) they'll need to send
Adds a generate_titles method to the Title Generation Ability and uses that in the execute callback
Adds some helper functions to normalize content and get additional context we can send to the LLM
Adds functionality where we can store system instructions in their own PHP file that the Ability can then pull in. This could be hardcoded in the Ability class but felt cleaner to keep these in their own files for easier management

Testing Instructions

There is no UI yet but can be tested by making direct API requests to the Title Generation Ability run endpoint.

First you'll need to login to WordPress and go to Settings > AI Credentials and add in at least one API key.

Then make an authenticated POST request to the Title Generation run endpoint and ensure a title is generated based on the content you provide

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/title-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "content": "this is some content!",
    }
}'

Make an authenticated POST request to the Title Generation run endpoint, passing in a valid post ID, and ensure a title is generated

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/title-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "post_id": 1,
        "candidates": 1
    }
}'

Make an authenticated POST request to the Title Generation run endpoint, passing in a valid post ID and number and ensure multiple titles are generated

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/title-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "post_id": 1,
        "candidates": 2
    }
}'

Make an authenticated POST request to the Title Generation run endpoint and don't pass any content or post ID and ensure an error is returned

curl --location 'https://example.com/wp-json/wp-abilities/v1/abilities/ai/title-generation/run' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
    "input": {
        "candidates": 2
    }
}'

Test using WordPress Playground

The changes in this pull request can be previewed and tested using this WordPress Playground instance:

Click here to test this pull request.

…ategory for each ability

…when it is run

…he AI Client to actually make the requests

… easily set credentials. Initialize the settings screen

… this into our request method

codecov · 2025-11-03T21:48:41Z

Codecov Report

❌ Patch coverage is 63.24786% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.94%. Comparing base (2567e18) to head (7e9c935).
⚠️ Report is 78 commits behind head on trunk.

Files with missing lines	Patch %	Lines
...es/Abilities/Title_Generation/Title_Generation.php	21.73%	36 Missing ⚠️
includes/Abstracts/Abstract_Ability.php	71.42%	6 Missing ⚠️
includes/bootstrap.php	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##              trunk      #67       +/-   ##
=============================================
+ Coverage     48.48%   60.94%   +12.45%     
- Complexity       45       81       +36     
=============================================
  Files             6       10        +4     
  Lines           198      425      +227     
=============================================
+ Hits             96      259      +163     
- Misses          102      166       +64

Flag	Coverage Δ
unit	`60.94% <63.24%> (+12.45%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…end as well

…text

…ge. Load this file as our prompt if it exists

…ert that to a string when sending it in a request

…using newer models that may not be fully supported

…s been merged

…ation-execute

…assing the feature into the ability. Move the get_post_context method into a helper function so other functionality can take advantage of this. Move the system instructions into the ability

jeffpaul · 2025-11-13T13:59:41Z

@dkotter with #61 merged in, looks like you're good to continue here with resolving the merge conflicts and getting this ready for review/testing

…pt builder

includes/Abilities/Title_Generation/system-instruction.php

…ts those instructions to account for this

…nt now. Remove line changes in test files that weren't intended

github-actions · 2025-11-13T20:42:48Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @matysanchez, @prabinjha, @Meenakshi-bose.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: matysanchez, prabinjha, Meenakshi-bose.

Co-authored-by: dkotter <[email protected]>
Co-authored-by: JasonTheAdams <[email protected]>
Co-authored-by: felixarntz <[email protected]>
Co-authored-by: jeffpaul <[email protected]>
Co-authored-by: karmatosed <[email protected]>
Co-authored-by: justlevine <[email protected]>
Co-authored-by: swissspidy <[email protected]>
Co-authored-by: Ref34t <[email protected]>
Co-authored-by: t-hamano <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

JasonTheAdams

Excited to see the first usage of the WP AI Client, @dkotter!

I didn't fully review this, but I got confused enough by the architectural direction to pause here for some questions. I feel like this is over abstracted, and if it's not I'm really curious about the needs if not.

includes/Abilities/Title_Generation/system-instruction.php

includes/Abilities/Title_Generation.php

includes/Abilities/Title_Generation/Title_Generation.php

JasonTheAdams · 2025-11-13T21:04:14Z

includes/API_Request.php

+	/**
+	 * Check if the AI SDK Client is available.
+	 *
+	 * @since 0.1.0
+	 *
+	 * @return bool True if the client is available, false otherwise.
+	 */
+	protected function is_client_available(): bool {
+		return class_exists( AI_Client::class );
+	}


Why wouldn't this be available?

JasonTheAdams · 2025-11-13T21:12:50Z

includes/API_Request.php

I'm really confused by this class. Why do we need this? Why not just do AI_Client::prompt() in the Title_Generation Ability and do everything we need there? Maybe I'm missing something, but the whole abstraction of this class seems unnecessary to me.

I think the biggest thing it's doing is applying the model and provider, if specified. If that's all it's doing, then I'd make a function (or class) somewhere that just creates the Prompt_Builder, applies the Provider/Model preferences, and then returns the Prompt_Builder instance. That really leans into the point of the builder concept, where you can hand over a partially configured builder intsance.

So I debated on this so happy to simplify. Few thoughts here:

Was trying to reduce the amount of code we would need to duplicate as we add functionality in the future that needs to make requests. Meaning for generating titles, beyond just passing in a prompt, we set the system instructions, we set the temperature value, we set the candidate count value and we set the default model preferences. I could easily move that out of this class and into the Title Generation Ability but we would then have that exact (or similar) code duplicated each time.

This class also standardizes how we process the response, ensuring we return an error if we get an empty array back and sanitizing the responses before returning. Again, all things that could happen in the Ability but would require us to duplicate that code each time.

That said, I'm fine with the approach of a helper function or two that will build the Prompt_Builder and return it and then each Ability/Feature/Experiment/Whatever that wants to make requests can take over from there. Let me make those changes and see if that feels like a better approach

I see where you're coming from. My main motivation here is that we want to think of these Abilities as examples, and I want devs to see how easy it is to construct a prompt and get the results in the ability.

If we need to normalize the response for our abilities in some way, then I'd like to do that after using the builder. That will also help us see if there's some sort of sanitization/normalization that we should introduce in the WP AI Client in some way.

Just to make sure you're aware, there's also a Prompt_Builder_With_WP_Error that can be used if you want to avoid exceptions.

So I've simplified this a bit in ca74ff1, getting rid of the API_Request class and adding a helper function that builds our prompt builder instead.

If that still feels too abstracted (which I've gone back and forth on and can't decide), we can remove all of that and instead, just do this directly in the generate_titles method:

$prompt_builder = AI_Client::prompt_with_wp_error( '"""' . $context . '"""' ); $prompt_builder = $prompt_builder->using_system_instruction( $this->get_system_instruction() ); $prompt_builder = $prompt_builder->using_temperature( 0.7 ); $prompt_builder = $prompt_builder->using_candidate_count( (int) $candidates ); $prompt_builder = $prompt_builder->using_model_preference( ...get_preferred_models() );

That's obviously cleaner and a lot less code but does mean we would have very similar (if not exact) code for each experiment we add. But as you've said, would be great if people can look at the code here and easily understand how things work, even if that means we end up with some code duplication

Getting there! A couple notes:

If you specify a model you don't need to specify a provider, as a model implies a provider

You probably don't need to specify the preferred models array. The Prompt Builder will simply select a model that fits the prompt. Preferring a model is for when you have something like Nano Banana setup and you want to prefer that for image generation.

Also, chaining can be cleaned up a lot (without re-assigning the variable repeatedly):

$prompt_builder = AI_Client::prompt_with_wp_error( '"""' . $context . '"""' ); ->using_system_instruction( $this->get_system_instruction() ); ->using_temperature( 0.7 ); ->using_candidate_count( (int) $candidates ); ->using_model_preference( ...get_preferred_models() );

I don't think the helper is really needed, as I'm not really seeing any code duplication that's necessary.

Okay, more simplification done in 94dff32

You probably don't need to specify the preferred models array

The reason I added this is because I ran into this issue when testing: WordPress/php-ai-client#117. Basically things worked great one day and the next day things started to fail. In debugging, found out it was because the Prompt Builder started using the gpt-4o-mini-search-preview model which doesn't work properly for text generation.

So I can remove this but feels safer to have some defaults to avoid issues where things might break from one day to the next if new models are released or the models API results change.

Ohhh, that's good for us to know. I'm going to ping @felixarntz on this one and get his thoughts. That sounds like a potential bug for that model's capabilities in PHP AI Client.

Definitely let us know if you run into stuff like that. You're the first one bringing this all together, so there are quite likely bugs like this. 😆

I replied in WordPress/php-ai-client#117 (comment):

Concrete problem: Our OpenAI model capability and option assignments are probably inaccurate / out of date.

Underlying problem: We should not return the alphabetically first eligible model, but rather a flagship model of the provider that makes sense to prefer by default.

I'd say for now let's put a preference list here, and then remove it once we've fixed the upstream bug.

…ions. Use this new helper function to get a prompt builder and then use that to make our requests

…uilder into the ability execution

JasonTheAdams

Finished going through it! Left a few minor suggestions, but it's really close!

includes/Abilities/Title_Generation/Title_Generation.php

JasonTheAdams · 2025-11-14T00:12:34Z

includes/helpers.php

+	/**
+	 * TODO: Might be interesting to add simple Abilities for the following,
+	 * just as a way to demonstrate a different approach to registering Abilities,
+	 * how to call Abilities via PHP and how multiple Abilities can be used together.
+	 *
+	 * Example: Get post content Ability; get post author Ability; get post terms Ability.
+	 */


Agreed! I like the idea of composing abilities!

I'll open another PR after this merges that adds these in and we can continue conversation on that

includes/Abilities/Title_Generation/Title_Generation.php

JasonTheAdams · 2025-11-14T00:18:13Z

includes/Abstracts/Abstract_Ability.php

+	 * Automatic detection order:
+	 * 1. `system-instruction.php`
+	 * 2. `prompt.php`


I'm curious, why have more than one option at this time? Why not just stick with one to stick to a single convention?

Good catch. Originally had this named as prompt and then renamed to match what the Prompt Builder expects and left support for both. But as this is a net-new thing, no need to support both right now. Removed in 8619a72

JasonTheAdams · 2025-11-14T00:20:03Z

includes/Abstracts/Abstract_Ability.php

+			}
+		}
+
+		return '';


Do we want silent failure here? Maybe yes, just double-checking. 🙂

That was the thought, yes, as the Prompt Builder handles that fine. Could change this to return an error that we output, though that would require us to not have the same chaining we have right now on the Prompt Builder as we'd need to check if an error was returned before calling using_system_instruction

includes/helpers.php

… call

JasonTheAdams

Great work, @dkotter! LGTM! 🎉

JasonTheAdams · 2025-11-14T17:45:03Z

includes/Abilities/Title_Generation/Title_Generation.php

+		return AI_Client::prompt_with_wp_error( '"""' . $context . '"""' )
+			->using_system_instruction( $this->get_system_instruction() )
+			->using_temperature( 0.7 )
+			->using_candidate_count( (int) $candidates )
+			->using_model_preference( ...get_preferred_models() )
+			->generate_texts();


One note I'll make here that I don't think we need to do anything about at this time, is the use of the Prompt_Builder::is_supporteed_for_text_generation() method. This is intended to make it clear if there is not a model that supports this prompt's needs, allowing us to do something in that condition.

I want to add a WP_Ability::is_available() method in the future for checking ahead of time if an Ability will contextually work. I think this check would fit well there.

dkotter added 9 commits October 31, 2025 15:29

Set the category from the parent class as we'll likely use the same c…

ce7002c

…ategory for each ability

Add a generate titles method to our feature. Use this in our ability …

76da245

…when it is run

Add an API Request class that can be used to make API requests. Use t…

e86c1b4

…he AI Client to actually make the requests

Install guzzle so things work

64062f3

Switch to the settings branch of the WP AI Client repo so we can more…

6afdbf6

… easily set credentials. Initialize the settings screen

Add the ability to set system instructions at the feature level. Pass…

c08f292

… this into our request method

Add a prompt builder method to our API Request class

ec8c3b8

Add unit tests to cover changes

ff16051

Fix lint errors

e8fa887

dkotter added this to the 0.1.0 milestone Nov 3, 2025

dkotter self-assigned this Nov 3, 2025

dkotter added 18 commits November 3, 2025 14:59

Fix PHPStan errors

d9b7965

More PHPStan fixes

9c6f479

Fix typo

44fbe59

Update our system instructions

9930535

Set our default title suggestions to 3

0176418

Instead of just sending the content, build up additional context to s…

797439f

…end as well

Add helper method to normalize content, basically stripping unwanted …

6d0127a

…text

Move our prompt to a separate markdown file to make it easier to mana…

6a192ee

…ge. Load this file as our prompt if it exists

Fix tests

55d5ec2

Fix PHPCS error

50248ab

Ensure we properly handle thrown exceptions when making the request

b32ae5c

Set our context as an array instead of a string for ease of use. Conv…

09da805

…ert that to a string when sending it in a request

Set our preferred models for each provider to ensure we don't end up …

280b5ac

…using newer models that may not be fully supported

Fix tests

5394a70

Fix PHPStan error

10a0d95

Switch back to trunk for the WP AI Client now that the settings PR ha…

70677b9

…s been merged

Merge branch 'feature/title-generation-base' into feature/title-gener…

ee00d69

…ation-execute

Move the generate_titles method into the ability as we're no longer p…

5bd5bed

…assing the feature into the ability. Move the get_post_context method into a helper function so other functionality can take advantage of this. Move the system instructions into the ability

dkotter added 2 commits November 12, 2025 14:38

Update tests with new changes

cdc6b73

Pull in latest WP AI Client code and start using more of their classes

256970f

Pull in latest WP AI Client code and fully switch to using their prom…

8355a89

…pt builder

JasonTheAdams reviewed Nov 13, 2025

View reviewed changes

includes/Abilities/Title_Generation/system-instruction.php Show resolved Hide resolved

dkotter added 2 commits November 13, 2025 13:31

Store our system instructions in PHP files. Update our helper that ge…

85503eb

…ts those instructions to account for this

Remove guzzle as a dependency as that's coming in from the WP AI Clie…

7ff9130

…nt now. Remove line changes in test files that weren't intended

dkotter marked this pull request as ready for review November 13, 2025 20:42

Merge branch 'trunk' into feature/title-generation-execute

b7a2e71

dkotter requested a review from JasonTheAdams November 13, 2025 20:44

Remove unneeded comment. Add TODO comment

4b7b78d

JasonTheAdams requested changes Nov 13, 2025

View reviewed changes

dkotter added 2 commits November 13, 2025 14:44

Use Nowdoc formatting in our instructions

eb226ba

Rename n to candidates

556d6db

JasonTheAdams mentioned this pull request Nov 13, 2025

Refactor: changes Features to Experiments #76

Merged

dkotter added 4 commits November 13, 2025 15:46

Remove the API_Request class and move a few methods into helper funct…

ca74ff1

…ions. Use this new helper function to get a prompt builder and then use that to make our requests

Update tests

009ef44

Remove prompt builder helper methods. Directly integrate the prompt b…

94dff32

…uilder into the ability execution

Remove unneeded tests

13aea7a

JasonTheAdams requested changes Nov 14, 2025

View reviewed changes

dkotter added 5 commits November 14, 2025 10:10

Add class constant to control the default candidate count

560dc31

Simplify getting our prompt builder and getting results into a single…

919b660

… call

Only check for a system-instruction.php file for now

8619a72

Rename file

145cc15

Fix lint issue

7e9c935

JasonTheAdams approved these changes Nov 14, 2025

View reviewed changes

dkotter merged commit a9cb31a into WordPress:trunk Nov 14, 2025
33 checks passed

dkotter deleted the feature/title-generation-execute branch November 14, 2025 18:28

dkotter mentioned this pull request Nov 14, 2025

Add the Title Generation UI #83

Draft

Add the Title Generation execute functionality #67

Add the Title Generation execute functionality #67

Uh oh!

Conversation

dkotter commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What?

Why?

How?

Testing Instructions

Test using WordPress Playground

Uh oh!

codecov bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jeffpaul commented Nov 13, 2025

Uh oh!

Uh oh!

github-actions bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unlinked Accounts

Uh oh!

JasonTheAdams left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JasonTheAdams left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JasonTheAdams left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

dkotter commented Nov 3, 2025 •

edited

Loading

codecov bot commented Nov 3, 2025 •

edited

Loading

github-actions bot commented Nov 13, 2025 •

edited

Loading