[Agent Builder] Fix truncated inline attachments

[viduni94]

Closes #249908

Summary

Problem

When attachment content exceeds 10,000 characters in inline mode, the content is truncated with the message [content truncated, use attachment_read for full content].

However, the system prompt mentions, "Since the content is shown inline, you don't need to read it - just reference it directly."
This creates conflicting instructions - the truncation message specifies the LLM to use the attachment_read tool, but the system prompt specifies to reference it directly. As a result, the LLM never calls the attachment_read for truncated attachments and only a portion of the data is passed to the LLM.

Solution

Track whether any attachment content was truncated and provide different system prompts based on that.

• When content is not truncated: "Since the content is shown inline, you don't need to read it - just reference it directly."
• When content is truncated: "If you see '[content truncated, use attachment_read for full content]', you MUST call attachment_read(id) to get the complete content before analyzing or referencing that attachment."

This PR also updates the order of the content shown in the AI Insight attachment to show the summary first and then the context data. Therefore, even if the data is truncated, the summary will always be shown.

Checklist

• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

[elasticmachine]

Pinging @elastic/obs-ai-team (Team:obs-ai)

[sorenlouv]

@viduni94 perhaps another bug with bounded tools: in this trace the system prompt contains this

The error ID is provided - use the get_error_details tool to fetch the full error information

But no get_error_details tool is available to call.

[sorenlouv]

Also, as mentioned on Slack, if the attachment content is truncated, we should make sure to put the important content first. For the AI insights we should not do JSON.stringify({context, summary}) . Instead we should output them like

Summary:
${JSON.stringify(summary)}

Context:
${JSON.stringify(context)}

This way we ensure, even if the attachment is truncated, at least we show the summary.

[viduni94]

@viduni94 perhaps another bug with bounded tools: in this trace the system prompt contains this

The error ID is provided - use the get_error_details tool to fetch the full error information

But no get_error_details tool is available to call.

@sorenlouv
Not sure I understood what you meant by "But no get_error_details tool is available to call"

We do have that bounded tool

[ppisljar]

LGTM, code review only

[viduni94]

@viduni94 perhaps another bug with bounded tools: in this trace the system prompt contains this

The error ID is provided - use the get_error_details tool to fetch the full error information

But no get_error_details tool is available to call.

@sorenlouv Not sure I understood what you meant by "But no get_error_details tool is available to call"

We do have that bounded tool

The tool is getting called as expected.

[sorenlouv]

@viduni94 perhaps another bug with bounded tools: in this trace the system prompt contains this

The error ID is provided - use the get_error_details tool to fetch the full error information

But no get_error_details tool is available to call.

@sorenlouv Not sure I understood what you meant by "But no get_error_details tool is available to call"

We do have that bounded tool

Do you see the tool in the trace? Maybe I'm just overlooking it https://oblt-apps.elastic.dev/phoenix-ai/projects/UHJvamVjdDoz/traces/6c4c42999fe209c838145cc315a90bac?selected=&selectedSpanNodeId=U3BhbjozMjkzOTQ%3D

[viduni94]

Do you see the tool in the trace? Maybe I'm just overlooking it https://oblt-apps.elastic.dev/phoenix-ai/projects/UHJvamVjdDoz/traces/6c4c42999fe209c838145cc315a90bac?selected=&selectedSpanNodeId=U3BhbjozMjkzOTQ%3D

I see what you mean.

When we start the conversation from AI Insights, since we have the data inline, this tool is not called.
But if we open the flyout from the global nav, this tool should be called using the errorId in the attachment to fetch the full error document. The comment I posted here was when I was testing on the 9.3 branch.
Re-tested on 9.3 and the bounded tool is working - See trace

On main however, I don't see this tool being called as you mentioned 🤔
Here's another trace on main where I ask the same question as the 9.3 trace.

Asking on the agent-builder channel

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 4855049

Failed CI Steps

• FTR Configs #16
• FTR Configs #117

Test Failures

• [job] [logs] FTR Configs #117 / Cloud Security Posture POST /internal/cloud_security_posture/graph Happy flows Enrich graph with entity metadata "before all" hook for "should contain entity data when asset inventory is enabled"
• [job] [logs] FTR Configs #16 / Options list control Allow expensive queries setting is off Can search options list for available options - does not find partial match

Metrics [docs]

✅ unchanged

History

• 💔 Build #384319 failed 599d92e

cc @viduni94

[sorenlouv]

i'm old enough to remember when key order in objects were not guaranteed. These days it's probably not a problem in modern browsers - but it still feels fragile.

That being said, it would be cleaner, and less noisy for the LLM if we did not have to stringify { summary context, } but avoid the outer object and send this string to the LLM:

Summary:
${summary}

Comtext:
${context}

I believe that would yield better results. If that's impossible, or very hard, so be it. But would be better imo.

[viduni94]

We can send the data as a string but we'll have to remove the custom attachment label and use the default one (Summary). Is that okay?
(Or I have to send the content and attachment label as a string and parse the string to extract it)

[sorenlouv]

Let's merge this PR as-is and then discuss this separately