[Workflows] support passing attachments to run_agent step

[KDKHD]

Summary

Summarize your PR. If it involves visual changes include a screenshot or gif.

Add support for passing attachments to the run agent step in workflows.

Example workflow with alert attachment

name: New workflow
enabled: false
description: This is a new workflow
triggers:
  - type: alert

steps:
  - name: ai_agent_step
    type: ai.agent
    connector-id: "<connector_id>"
    with:
      message: "Summarise these alert"
      attachments:
        - type: security.alert
          data:
            alert: "{{event.alerts | json:2}}"
            attachmentLabel: "Alert attachment."

PR also moves attachment validation down into ExecutionService so that attachments passed via workflows are validated.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

• See some risk examples
• ...

[pgayvallet]

Feature makes sense, thanks for adding that.

There's only one thing we need to address, about attachment validation - added a specific comment below.

[pgayvallet]

We're going to have a problem with the way attachment validation currently work.

Atm, validation is done in the converse route handler directly (which is pretty bad - agreed). This was "fine" because before this PR, the converse API was the only entry point for attachments, so for their validation.

kibana/x-pack/platform/plugins/shared/agent_builder/server/routes/chat.ts

Lines 172 to 189 in f2c007b

	const validateAttachments = async ({
	attachments,
	attachmentsService,
	}: {
	attachments: AttachmentInput[];
	attachmentsService: AttachmentServiceStart;
	}) => {
	const results: AttachmentInput[] = [];
	for (const attachment of attachments) {
	const validation = await attachmentsService.validate(attachment);
	if (validation.valid) {
	results.push(validation.attachment);
	} else {
	throw createBadRequestError(`Attachment validation failed: ${validation.error}`);
	}
	}
	return results;
	};

So passing attachments directly to chatService.converse as done here bypass this validation.

But now we need to find a way to address that, either by:

1. moving the validation which is currently living in the converse route handler a few layers down, either in chatService or even lower in the chain, so that we hit the attachment validation when calling an agent via that workflow step (probably the most robust way, even if more work - there might be a few type issues between Attachment vs AttachmentInput)
2. extract / re-use the logic we use from the route handler in the step definition - move that validateAttachments living in the handler file elsewhere, and then re-use it from the step definition (I would be fine with that too)
3. re-use the "attachment state manager" we use for the attachment routes (x-pack/platform/plugins/shared/agent_builder/server/routes/attachments.ts). It doesn't have an API for validation atm, but it has an internal validateAttachmentData function. we could add a new public "validateAttachment" which would reproduce the logic of what we do in validateAttachments` in the converse route handler today, and then use this in the step definition and in the route (seems like a good compromise)

Fine with either, you can pick.

[KDKHD]

Moved into ExecutionService since ChatService has been removed.

[KDKHD]

@elasticmachine merge upstream

[elasticmachine]

merge conflict between base and head

[KDKHD]

Moved attachment validation down into the execution service.

[KDKHD]

@elasticmachine merge upstream

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 140dfe3

Failed CI Steps

• Jest Tests #9

Test Failures

• [job] [logs] Jest Tests #9 / Header QueryTabHeader should render the immutable timeline call out providers
• [job] [logs] Jest Tests #9 / Header QueryTabHeader should render the immutable timeline call out with correct message

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
agentBuilder	86.8KB	87.0KB	+162.0B

Unknown metric groups

References to deprecated APIs

id	before	after	diff
agentBuilder	38	41	+3

Unreferenced deprecated APIs

id	before	after	diff
agentBuilder	38	41	+3

History

• 💔 Build #399092 failed b575537
• 💚 Build #397183 succeeded 59fbf9e
• 💔 Build #396924 failed 7b48574
• 💔 Build #396713 failed e2a0c83
• 💔 Build #396694 failed 7deed9f
• 💛 Build #389792 was flaky 21d7bd1

[shahargl]

lgtm