Skip to content

[1Chat] Add support for structured tool call results#229562

Merged
joemcelroy merged 13 commits intoelastic:mainfrom
sorenlouv:artificats-support
Aug 1, 2025
Merged

[1Chat] Add support for structured tool call results#229562
joemcelroy merged 13 commits intoelastic:mainfrom
sorenlouv:artificats-support

Conversation

@sorenlouv
Copy link
Member

@sorenlouv sorenlouv commented Jul 28, 2025

Closes https://github.com/elastic/search-team/issues/10552

This PR implements structured tool results (outlined in Content Artifacts RFC). This will form the foundation for citations, visualizations, and query (re-)execution.

Breaking Change: Modified tool handler contract from { result: T } to { results: ToolResult[] }

4 tool result types are defined:

@sorenlouv sorenlouv force-pushed the artificats-support branch from 184c6cd to 4c240ea Compare July 29, 2025 07:41
@sorenlouv sorenlouv force-pushed the artificats-support branch from 55c7ed5 to 4f7480d Compare July 29, 2025 10:06
@sorenlouv sorenlouv marked this pull request as ready for review July 29, 2025 11:20
@sorenlouv sorenlouv requested a review from a team as a code owner July 29, 2025 11:20
@sorenlouv sorenlouv added backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes labels Jul 29, 2025
@joemcelroy
Copy link
Member

still need to go through the code changes but found a couple of points whilst testing this:

suggestion: errors should have a "error" structured type rather than using the "other"

{
    "type": "tool_call",
    "tool_call_id": "call_HvVM1uSBbv3jj9hXbwhNW2jT",
    "tool_id": ".execute_esql",
    "params": {
    "query": "FROM financial_accounts | SORT BY total_portfolio_value DESC | LIMIT 1"
    },
    "results": [
        {
            "type": "other",
            "data": {
            "reason": """parsing_exception
    Root causes:
        parsing_exception: line 1:32: extraneous input 'BY' expecting {QUOTED_STRING, INTEGER_LITERAL, DECIMAL_LITERAL, 'false', 'not', 'null', '?', 'true', '+', '-', '??', NAMED_OR_POSITIONAL_PARAM, NAMED_OR_POSITIONAL_DOUBLE_PARAMS, '[', '(', UNQUOTED_IDENTIFIER, QUOTED_IDENTIFIER}"""
            }
        }
    ]
}

question: noticed the index_mapping_response was using the "other" type but seems to have another type below. Is this intentional?

{
    "type": "tool_call",
    "tool_call_id": "call_5w5JIn2trsWEaIeNUrF0XOKJ",
    "tool_id": ".get_index_mapping",
    "params": {
      "indices": [
        "financial_accounts"
      ]
    },
    "results": [
      {
        "type": "other",
        "data": {
          "type": "index_mappings_response",
          "mappings": {
            "financial_accounts": {
              "mappings": {
                "properties": {
                  "account_holder_name": {
                    "type": "text"
                  },
                  "account_id": {
                    "type": "keyword"
                  },
                  "account_type": {
                    "type": "keyword"
                  },
                  "contact_preference": {
                    "type": "keyword"
                  },
                  "first_name": {
                    "type": "text"
                  },
                  "last_name": {
                    "type": "text"
                  },
                  "last_updated": {
                    "type": "date"
                  },
                  "risk_profile": {
                    "type": "keyword"
                  },
                  "state": {
                    "type": "keyword"
                  },
                  "total_portfolio_value": {
                    "type": "double"
                  },
                  "zip_code": {
                    "type": "keyword"
                  }
                }
              }
            }
          },
          "indices": [
            "financial_accounts"
          ]
        }
      }
    ]
  },

@sorenlouv
Copy link
Member Author

sorenlouv commented Jul 31, 2025

question: noticed the index_mapping_response was using the "other" type but seems to have another type below. Is this intentional?

I was toying with the idea of providing a subtype to "other" to distinguish between different "other" artifacts. It's probably a bad idea for now so will remove it.

@joemcelroy
Copy link
Member

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/onechat-genai-utils 75 74 -1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
onechat 592.3KB 592.4KB +38.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/onechat-common 0 1 +1
Unknown metric groups

API count

id before after diff
@kbn/onechat-genai-utils 79 78 -1

History

@joemcelroy joemcelroy merged commit ab0f4f3 into elastic:main Aug 1, 2025
13 checks passed
@sorenlouv sorenlouv deleted the artificats-support branch August 1, 2025 22:39
szaffarano pushed a commit to szaffarano/kibana that referenced this pull request Aug 5, 2025
Closes elastic/search-team#10552

This PR implements structured tool results (outlined in [Content
Artifacts
RFC](https://docs.google.com/document/d/10ZlcUZhrnxpWc6NDgv1YIXIUxc6AtNVhTpto_yQl9PU/edit?tab=t.jlbbbsa981p6)).
This will form the foundation for citations, visualizations, and query
(re-)execution.

Breaking Change: Modified tool handler contract from `{ result: T }` to
`{ results: ToolResult[] }`

4 tool result types are defined:
- `ResourceResult`
- `TabularDataResult`
- `QueryResult`
- `OtherResult`

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
delanni pushed a commit to delanni/kibana that referenced this pull request Aug 5, 2025
Closes elastic/search-team#10552

This PR implements structured tool results (outlined in [Content
Artifacts
RFC](https://docs.google.com/document/d/10ZlcUZhrnxpWc6NDgv1YIXIUxc6AtNVhTpto_yQl9PU/edit?tab=t.jlbbbsa981p6)).
This will form the foundation for citations, visualizations, and query
(re-)execution.

Breaking Change: Modified tool handler contract from `{ result: T }` to
`{ results: ToolResult[] }`

4 tool result types are defined:
- `ResourceResult`
- `TabularDataResult`
- `QueryResult`
- `OtherResult`

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@wildemat wildemat mentioned this pull request Aug 7, 2025
10 tasks
NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request Aug 18, 2025
Closes elastic/search-team#10552

This PR implements structured tool results (outlined in [Content
Artifacts
RFC](https://docs.google.com/document/d/10ZlcUZhrnxpWc6NDgv1YIXIUxc6AtNVhTpto_yQl9PU/edit?tab=t.jlbbbsa981p6)).
This will form the foundation for citations, visualizations, and query
(re-)execution.

Breaking Change: Modified tool handler contract from `{ result: T }` to
`{ results: ToolResult[] }`

4 tool result types are defined:
- `ResourceResult`
- `TabularDataResult`
- `QueryResult`
- `OtherResult`

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@jedrazb jedrazb mentioned this pull request Aug 19, 2025
10 tasks
sorenlouv added a commit that referenced this pull request Aug 20, 2025
…32258)

Regression introduced in #229562

Tool result messages must be of type `MessageContentComplex` or
`DataContentBlock`. This PR converts the result to match the type
`PlainTextContentBlock`.
qn895 pushed a commit to qn895/kibana that referenced this pull request Aug 26, 2025
…astic#232258)

Regression introduced in elastic#229562

Tool result messages must be of type `MessageContentComplex` or
`DataContentBlock`. This PR converts the result to match the type
`PlainTextContentBlock`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants