[Lens as code] preserve all chart style properties during API format round-trip

[ghudgins]

Summary

Closes #253049

When lens.apiFormat is enabled, visualization style properties are silently dropped during the Lens state <-> API format round-trip conversion on save/load. This causes style changes made in the UI to revert to defaults after saving and reloading a dashboard.

Root cause: The API format schemas for multiple chart types did not include fields for certain visualization state properties. During the enrichConfigurationWithVisualizationProperties (Lens -> API) step they were not written, and during buildVisualizationState (API -> Lens) they were not read back, so they defaulted.

Properties fixed

Chart Type	Properties	User-configurable
Metric	primaryPosition, titleWeight, secondaryAlign	Yes
XY	reference line decoration_position (renamed from icon_position)	Yes
Partition (pie/treemap/mosaic/waffle)	legendPosition	Yes
Datatable	showRowNumbers	Yes
Heatmap	shouldTruncate, strokeWidth, strokeColor, percentageMode, showTooltip, highlightInHover	Partial
Gauge	respectRanges, commonLabel	No
Shared coloring	palette continuity	Yes

Fix pattern (same for all)

1. Schema — add the missing field to the API schema definition
2. Transform out (Lens -> API) — write the property during serialization
3. Transform in (API -> Lens) — read the property during deserialization

Files changed

Area	Files	Change
Metric schema	schema/charts/metric.ts	Added position, title_weight, secondary alignments
Metric transforms	transforms/charts/metric.ts	Write/read new props; fix secondaryAlign to read from secondary metric
XY schema	schema/charts/xy.ts	Added decoration_position (auto/left/right) to reference line
XY transforms	transforms/charts/xy/api_layers.ts, state_layers.ts	Write/read decoration_position
Partition schemas	schema/charts/pie.ts, treemap.ts, mosaic.ts, waffle.ts	Added position, values to legend (with maxSize: 2)
Partition transforms	transforms/charts/partition.ts	Write/read legendPosition
Datatable schema	schema/charts/datatable.ts	Added show_row_numbers
Datatable transforms	to_api/appearance.ts, to_state/appearance.ts	Write/read via buildShowRowNumbers helper
Heatmap schema	schema/charts/heatmap.ts	Added 6 new fields (removed aria_label)
Heatmap transforms	transforms/charts/heatmap/to_api.ts, from_api.ts	Write/read all new fields
Gauge schema	schema/charts/gauge.ts	Added respect_ranges, common_label
Gauge transforms	transforms/charts/gauge.ts	Write/read all new fields
Shared coloring schema	schema/color.ts	Added continuity to color-by-value
Shared coloring transforms	transforms/coloring.ts	Write/read continuity
Test mocks	8 mock files across metric, xy, partition, datatable, heatmap, gauge	Added non-default values to exercise round-trip coverage

Review feedback addressed

• Renamed icon_position to decoration_position and limited values to auto|left|right (removed above/below)
• Removed aria_label from heatmap API (inferred, not user-facing)
• Removed legendStats from treemap mock (not exposed in treemap UI)
• Extracted buildShowRowNumbers helper to follow existing datatable pattern
• Added maxSize: 2 to legend values arrays in partition schemas

Test plan

• With feature_flags.overrides: { lens.apiFormat: true } in kibana.yml:
  • Create metric chart, change primary position/title weight/secondary alignment, save & reload — verify preserved
  • Create XY chart with reference line, change decoration position, save & reload — verify preserved
  • Create pie/treemap chart, change legend position, save & reload — verify preserved
  • Create datatable, toggle row numbers, save & reload — verify preserved
  • Create heatmap, toggle legend truncation, save & reload — verify preserved
• Verify the same workflows work without the feature flag (no regression)
• yarn test:jest src/platform/packages/shared/kbn-lens-embeddable-utils/config_builder/tests/ — 273/273 pass
• yarn test:jest src/platform/packages/shared/kbn-lens-embeddable-utils/config_builder/schema/ — 660/660 pass
• yarn test:type_check --project src/platform/packages/shared/kbn-lens-embeddable-utils/tsconfig.json — passes

[ghudgins]

/ci

[walterra]

To fix the code scanning warnings you need to add checks like here to the schemas: #253411

[ghudgins]

/ci

[walterra]

It looks like the settings to have "angled" axis labels in the heatmap doesn't work (it works for regular bar charts).

[ghudgins]

config_builder correctly round-trips the angled orientation value. there is a separate "angled axis labels don't work in heatmap" is a rendering issue in the heatmap visualization expression itself — it receives the -45 rotation value but doesn't render it at an angle. That's a separate bug from this PR's scope #211214 (which is about values being lost during the API format round-trip).

[ghudgins]

/ci

[elasticmachine]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

[ghudgins]

/ci

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: bb41bd5

Failed CI Steps

• Jest Tests #1
• Jest Tests #4

Test Failures

• [job] [logs] Jest Tests #4 / Endpoint exceptions flyout When valid form state should save endpoint exception with correct os_types based on alert data
• [job] [logs] Jest Tests #4 / Endpoint exceptions flyout When wildcard warning is active pressing confirm should show confirm modal instead of saving exception

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
agentBuilder	569.3KB	571.0KB	+1.6KB
dashboard	819.9KB	821.5KB	+1.6KB
dashboardAgent	173.3KB	174.9KB	+1.6KB
discover	1.6MB	1.6MB	+1.6KB
infra	1.2MB	1.2MB	+1.6KB
lens	2.0MB	2.0MB	+4.0KB
observability	2.0MB	2.0MB	+1.6KB
observabilityAIAssistantApp	287.3KB	289.0KB	+1.6KB
unifiedDocViewer	398.4KB	400.0KB	+1.6KB
total			+17.1KB

History

• 💔 Build #397833 failed 4baaa42
• 💚 Build #397345 succeeded 3d748d3
• 💔 Build #396507 failed 1ab5e00

[markov00]

Thank you Graham for your attempt, but as proved by multiple review comments, allucinations, context missing and no overall schema design process in the generation creates a lot more work on our side as reviewer then for some engineer to pick this task from the beginning and work on that.
I get your willingness to supporting us, but this create way more problems than the the benefits.
I believe is way more useful if, instead of asking LLM to fix these things, you ask LLM to help you idenfity properly every missing parameters, providing a good and validated list of what is missing, from where, what are the parameters available etc. This will help way more then a generated code. The important aspect is the manual Validation: as seen in the review comments, some parameters are just made up, so a manual validation on the interface is greatly appreciated and valuated more then an unsupervised LLM trying its own findings.

We also follows some guidelines on how to correctly reply to PR reviews that creates a structured but simplified process, that in this PR is not followed (if you want I can share the docs on slack as it is not public)