Add JSON_EXTRACT ES|QL scalar function

[quackaplop]

Summary

Adds a new ES|QL scalar function JSON_EXTRACT(string, path) → keyword that extracts values from JSON strings using a subset of JSONPath (RFC 9535).

Tech Preview — available from 9.4.0, gated behind FN_JSON_EXTRACT capability. Wired into the string functions documentation page.

Parameters

Parameter	Type	Description
string	keyword, text, _source	JSON input or _source metadata field
path	keyword, text	JSONPath expression (e.g., name, user.address.city, orders[1].item)

Supported path syntax

Paths can use dot notation (user.address.city), bracket notation (['user']['address']['city']), or a mix of both (user['address'].city). For simple keys they are interchangeable — a.b and a['b'] produce the same result.

Bracket notation is required for keys containing dots or special characters (['user.name']), empty string keys (['']), and array indices (items[0]). Dots in dot notation are always path separators per RFC 9535 — a JSON key literally named "user.name" must use ['user.name'].

The JSONPath $ root selector is supported for compatibility but is always optional — $.name and name are equivalent, and $[0] and [0] are equivalent. Optional whitespace is allowed inside brackets ([ 0 ] is equivalent to [0]). Path matching is case-sensitive per the JSON specification.

Multi-value JSONPath features (wildcards, recursive descent, filters, slicing, negative indices) are not currently supported.

Return value

The extracted value is returned as a keyword string: string values without surrounding quotes, numbers and booleans as their string representation, and objects or arrays as JSON strings. Returns null if either parameter is null or if the extracted JSON value is null.

Returns null and emits a warning if the input is not valid JSON, the input is empty, the path is malformed, the path does not exist, the array index is out of bounds, or the path attempts to traverse through a non-object/non-array value.

Key implementation details

• Streaming parser: Uses XContentParser — skips fields not on the path without materializing them
• Multi-encoding _source: Detects encoding via XContentFactory.xContentType() — works with JSON, SMILE, CBOR, and YAML
• Structure serialization: Uses XContentBuilder.copyCurrentStructure() for objects/arrays. Future zero-copy byte slicing tracked in Expose byte offsets on XContentParser for zero-copy sub-structure extraction #142873
• Constant path optimization: Parses path once, reuses across all rows via specialized evaluator
• Duplicate keys: Returns first match (streaming parser semantics)
• Error handling: Throws IllegalArgumentException directly for all error conditions (empty input, invalid JSON, missing path, out-of-bounds index). The evaluator catches these and emits warnings.

Documentation

• Function listed in string functions page with {applies_to} version tags (preview 9.4.0)
• @FunctionInfo uses description (short summary) + detailedDescription (full details) per docs team guidance
• Kibana definition and docs auto-generated from annotations

Test coverage

• Unit tests (JsonExtractTests.java) — ~70 inline tests + parameterized suppliers + 4 randomized XContent encoding tests
• Path parsing tests (JsonPathTests.java) — dot/bracket notation, quoted keys, escapes, error positions, randomized round-trips
• CSV spec tests (json_extract.csv-spec) — bracket notation (simple, nested, dot-in-key, mixed), top-level arrays (with and without $), top-level scalars, deep mixed nesting, duplicate keys, null-in-array, FROM-index tests with OTel-style json_logs dataset, FROM employees CONCAT round-trip test
• REST integration tests (JsonExtractSourceIT.java in qa/server/single-node) — JSON_EXTRACT(_source, ...) against real indexed documents across all four XContent encodings (JSON, SMILE, CBOR, YAML) in both SYNC and ASYNC modes
• Serialization tests (JsonExtractSerializationTests.java)
• Error tests (JsonExtractErrorTests.java)

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/functions-operators/string-functions/json_extract.md
• docs/reference/query-languages/esql/kibana/docs/functions/json_extract.md

[github-actions]

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

• Check out the cumulative docs guidelines
• Reach out in the #docs Slack channel

[quackaplop]

This PR continues the work from #141507, moved to a clean branch with correct authorship.

[quackaplop]

buildkite test this

[quackaplop]

buildkite test this