Skip to content

Fix CSV-escaped quotes in generated docs examples#143449

Merged
quackaplop merged 2 commits intoelastic:mainfrom
quackaplop:feature/json-extract
Mar 3, 2026
Merged

Fix CSV-escaped quotes in generated docs examples#143449
quackaplop merged 2 commits intoelastic:mainfrom
quackaplop:feature/json-extract

Conversation

@quackaplop
Copy link
Copy Markdown
Contributor

@quackaplop quackaplop commented Mar 3, 2026

Summary

  • DocsV3Support.renderTableLine() reads CSV spec result blocks as raw text but didn't unescape RFC 4180 quoting — quoted fields with doubled quotes ("") were rendered literally in Markdown tables
  • Added CSV unescaping: strip outer " delimiters and replace "" with " so JSON strings render correctly in published docs
  • Example: {"severity":"ERROR"} instead of {""severity"":""ERROR""}

Fixes these published doc pages

  • JSON_EXTRACT — 5 example result tables show CSV-escaped JSON instead of proper JSON
  • TO_TDIGEST — 1 example result table shows CSV-escaped JSON instead of proper JSON

Affected docs

  • json_extract.md — 5 result cells with JSON values now render correctly
  • to_tdigest.md — 1 result cell with JSON value now renders correctly

Test plan

  • Precommit passes (spotless, checkstyle, forbidden APIs)
  • JsonExtractTests — doc generation + function logic
  • ToTDigestTests — doc generation + function logic
  • CsvTests with *json* filter — CSV spec end-to-end
  • Verified to_boolean.md NOT affected (empty string "" is preserved because the cell doesn't start/end with " as delimiters)

@quackaplop quackaplop requested a review from a team as a code owner March 3, 2026 09:55
@elasticsearchmachine elasticsearchmachine added v9.4.0 needs:triage Requires assignment of a team area label labels Mar 3, 2026
@quackaplop quackaplop added >bug :Analytics/ES|QL AKA ESQL Team:ES|QL and removed needs:triage Requires assignment of a team area label labels Mar 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 3, 2026

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 3, 2026
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Copy Markdown
Member

@leemthompo leemthompo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs output LGTM, but I'm not competent to validate the java changes :)

@quackaplop quackaplop enabled auto-merge (squash) March 3, 2026 10:48
@quackaplop quackaplop disabled auto-merge March 3, 2026 10:52
DocsV3Support.renderTableLine() now unescapes RFC 4180 CSV quoting
(strips outer quote delimiters and replaces "" with ") so that JSON
strings in function example tables render correctly — e.g.
{"key":"value"} instead of {""key"":""value""}.

Affects json_extract and to_tdigest doc examples.
…ests

The previous approach stripped outer quotes from ALL quoted values, breaking
simple quoted values like "POINT(...)" and "foo". Now only cells with actual
RFC 4180 doubled-quote escaping ("") are unescaped, leaving simple quoted
values unchanged.

Added tests: testRenderingExampleResultCsvJsonUnescaping verifies JSON
unescaping works, testRenderingExampleResultSimpleQuotesPreserved verifies
simple quoted values are not modified.

Also adds changelog YAML for the PR.
@quackaplop quackaplop force-pushed the feature/json-extract branch from 9a5bdc6 to 677f99c Compare March 3, 2026 11:20
@quackaplop quackaplop enabled auto-merge (squash) March 3, 2026 11:46
@quackaplop quackaplop merged commit 6dce4ba into elastic:main Mar 3, 2026
35 checks passed
szybia added a commit to szybia/elasticsearch that referenced this pull request Mar 3, 2026
…cations

* upstream/main: (56 commits)
  Mute org.elasticsearch.compute.lucene.read.ValueSourceReaderTypeConversionTests testLoadAll elastic#143471
  [DOCS] Fix ES|QL function and commands lists versioning metadata (elastic#143402)
  Fix MMROperatorTests (elastic#143453)
  Fix CSV-escaped quotes in generated docs examples (elastic#143449)
  Fix SQL client parsing of array header values (elastic#143408)
  ESQL: Add extended distribution tests and fault injection for external sources (elastic#143420)
  ESQL: Fix datasource test failures on Windows and FIPS (elastic#143417)
  Add circuit breaker for query construction to prevent OOM from automaton-based queries (elastic#142150)
  Cleanup SpecIT logging configuration (elastic#143365)
  ESQL: Prune unused regex extract nodes in optimizer (elastic#140982)
  Ensure supported locale outside of Entitlements check (elastic#143405)
  feat(es|ql): add dense_vector support in coalesce (elastic#142974)
  [Test] Unmute SnapshotStressTestsIT (elastic#143359)
  Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:lookup-join.LookupJoinWithCoalesceFilterOnRight} elastic#143443
  Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:lookup-join.MvJoinKeyOnTheLookupIndex} elastic#143442
  ESQL: Fix CCS exchange sink cleanup (elastic#143325)
  Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:lookup-join.MvJoinKeyOnTheLookupIndexAfterStats} elastic#143434
  Mute org.elasticsearch.xpack.esql.CsvIT test {csv-spec:lookup-join.MvJoinKeyFromRow} elastic#143432
  Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT test {csv-spec:k8s-timeseries.Datenanos_derivative_compared_to_rate} elastic#143431
  Mute org.elasticsearch.multiproject.test.CoreWithMultipleProjectsClientYamlTestSuiteIT test {yaml=search.retrievers/result-diversification/10_mmr_result_diversification_retriever/Test MMR result diversification single index float type} elastic#143430
  ...
tballison pushed a commit to tballison/elasticsearch that referenced this pull request Mar 3, 2026
* Fix CSV-escaped quotes rendering in generated docs examples

DocsV3Support.renderTableLine() now unescapes RFC 4180 CSV quoting
(strips outer quote delimiters and replaces "" with ") so that JSON
strings in function example tables render correctly — e.g.
{"key":"value"} instead of {""key"":""value""}.

Affects json_extract and to_tdigest doc examples.

* Refine CSV unescaping to only unescape RFC 4180 doubled quotes; add tests

The previous approach stripped outer quotes from ALL quoted values, breaking
simple quoted values like "POINT(...)" and "foo". Now only cells with actual
RFC 4180 doubled-quote escaping ("") are unescaped, leaving simple quoted
values unchanged.

Added tests: testRenderingExampleResultCsvJsonUnescaping verifies JSON
unescaping works, testRenderingExampleResultSimpleQuotesPreserved verifies
simple quoted values are not modified.

Also adds changelog YAML for the PR.
GalLalouche pushed a commit to GalLalouche/elasticsearch that referenced this pull request Mar 3, 2026
* Fix CSV-escaped quotes rendering in generated docs examples

DocsV3Support.renderTableLine() now unescapes RFC 4180 CSV quoting
(strips outer quote delimiters and replaces "" with ") so that JSON
strings in function example tables render correctly — e.g.
{"key":"value"} instead of {""key"":""value""}.

Affects json_extract and to_tdigest doc examples.

* Refine CSV unescaping to only unescape RFC 4180 doubled quotes; add tests

The previous approach stripped outer quotes from ALL quoted values, breaking
simple quoted values like "POINT(...)" and "foo". Now only cells with actual
RFC 4180 doubled-quote escaping ("") are unescaped, leaving simple quoted
values unchanged.

Added tests: testRenderingExampleResultCsvJsonUnescaping verifies JSON
unescaping works, testRenderingExampleResultSimpleQuotesPreserved verifies
simple quoted values are not modified.

Also adds changelog YAML for the PR.
shmuelhanoch pushed a commit to shmuelhanoch/elasticsearch that referenced this pull request Mar 4, 2026
* Fix CSV-escaped quotes rendering in generated docs examples

DocsV3Support.renderTableLine() now unescapes RFC 4180 CSV quoting
(strips outer quote delimiters and replaces "" with ") so that JSON
strings in function example tables render correctly — e.g.
{"key":"value"} instead of {""key"":""value""}.

Affects json_extract and to_tdigest doc examples.

* Refine CSV unescaping to only unescape RFC 4180 doubled quotes; add tests

The previous approach stripped outer quotes from ALL quoted values, breaking
simple quoted values like "POINT(...)" and "foo". Now only cells with actual
RFC 4180 doubled-quote escaping ("") are unescaped, leaving simple quoted
values unchanged.

Added tests: testRenderingExampleResultCsvJsonUnescaping verifies JSON
unescaping works, testRenderingExampleResultSimpleQuotesPreserved verifies
simple quoted values are not modified.

Also adds changelog YAML for the PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:ES|QL v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants