Add shape fill-by-column controls and outline defaults

AGENTS.md

@@ -16,8 +16,15 @@ joining work on this repository.
 
 ## Working norms
 
+- Use the Node.js and pnpm versions pinned in `package.json` under `volta`.
+  If `node` or `pnpm` is missing or resolves to a different version, prefer
+  Volta-managed commands (for example `$(volta which pnpm)` or `~/.volta/bin/pnpm`) rather
+  than falling back to the Codex app bundled Node or the system Node.
 - Prefer behavioral tests over cache-key unit tests. If a change is
   performance-related, the test should observe runtime side effects (e.g.
   fetch counts), not internal cache hits.
+- Treat layers as independent views of spatial elements: it must be valid for
+  multiple layer configs to represent the same underlying element with different
+  visual properties, filters, or table-driven encodings.
 - Worktrees share `.git` but not working state. Documents intended to outlive
   the current branch must land on `main`.

docs/docs/core/elements.mdx

@@ -113,6 +113,12 @@ labels.ndim;
 labels.getTransformation('global');
 ```
 
+Current implementation note:
+labels currently expose picked feature identity from raster values (for example
+segment/object ids) at render-time. A dedicated `LabelsElement` feature-id
+loading API, parallel to `ShapesElement.loadFeatureIds()`, is planned but not
+yet part of the public surface.
+
 ## ShapesElement
 
 Shapes represent vector geometries like polygons and circles.
@@ -185,6 +191,55 @@ table.attrs.region;         // Element(s) this table annotates
 table.attrs.region_key;     // Column name linking to region
 ```
 
+### Experimental extension idea: equivalent region encodings
+
+The SpatialData table contract currently maps each row to a single target
+region+instance pair via `region_key` + `instance_key`. For some workflows, we
+may want to assert that one row can be interpreted across multiple equivalent
+elements (for example both `labels/cell_labels` and `shapes/cell_shapes`), then choose
+whichever runtime representation is more appropriate for a task.
+
+This is not part of the SpatialData spec today. In this repo, a possible
+experimental approach is to keep the canonical mapping unchanged and add
+optional sidecar metadata under `table.uns.spatialdata_attrs`, for example:
+
+```ts
+table.uns.spatialdata_attrs.experimental_equivalent_mappings = {
+  canonical: {
+    region_key: 'region',
+    instance_key: 'instance_id',
+  },
+  aliases: {
+    labels: {
+      region: 'labels/cell_labels',
+      // Optional: when labels use a different obs column than canonical.
+      region_key: 'label_region_key',
+      instance_key: 'label_instance_id',
+    },
+    shapes: {
+      region: 'shapes/cell_shapes',
+      // Omit keys when canonical columns already apply.
+    },
+  },
+};
+```
+
+Guidelines for experimentation:
+
+- Treat this metadata as optional and non-authoritative.
+- Preserve normal `region_key`/`instance_key` behavior when it is absent.
+- Prefer column-level equivalence (`*_region_key`, `*_instance_key`) over
+  per-id dictionaries.
+- Keep one canonical identity per row to avoid ambiguous write/update paths.
+- Do not assume other SpatialData tools will read this field.
+
+Alternative worth discussing upstream:
+for common cases where an alias always targets one fixed region, it may be
+simpler to store only `region: '<fixed-region-name>'` for that alias (without a
+`region_key` column reference). This is more divergent from today's
+`region`/`region_key`/`instance_key` table-keys contract, but could reduce
+friction for simple one-region equivalence workflows.
+
 Implementation note:
 `loadObsIndex()` and `loadObsColumns()` are used by the feature-association
 helpers and currently stay on the direct zarr/parquet loader path rather than

docs/docs/core/internals.mdx

@@ -80,6 +80,32 @@ Planned evolution:
 - keep the current convenience helpers focused on association / tooltip use
 - continue feeding capability gaps back upstream to `anndata.js`
 
+### Feature Identity Convergence (labels and shapes)
+
+Current state:
+
+- shapes expose stable feature ids through `ShapesElement.loadFeatureIds()`
+  (plus render data carrying `featureIds` and row-alignment metadata);
+- labels currently expose feature identity from picked raster values during
+  interactive rendering/tooltips.
+
+Near-term unification direction:
+
+1. Add a labels-side feature-id API in `@spatialdata/core` (for example
+   non-zero unique ids at the selected scale).
+2. Route both labels and shapes through the same association helper shape:
+   `featureId -> table row index`.
+3. Keep render-time picking as one producer of feature ids, not the only one.
+
+This keeps the table contract (`region`, `region_key`, `instance_key`) intact
+while making labels/shapes more symmetric in higher-level APIs.
+
+Experimental metadata idea (non-spec):
+we may prototype optional table metadata that asserts equivalence classes across
+elements (for example one canonical row identity corresponding to both
+`labels/<key>/<id>` and `shapes/<key>/<id>`). This should remain explicitly
+experimental and advisory until there is upstream spec support.
+
 ## Element Factory (Internal)
 
 Elements are created internally when loading stores. These functions are not intended for application use:

docs/docs/vis/feature-table-associations.mdx

@@ -0,0 +1,135 @@
+---
+sidebar_position: 6
+---
+
+# Feature table associations and annotation columns
+
+This note records the intended foundation for linking SpatialData features to
+table rows, tooltip values, and visual encodings. The current `SpatialCanvas`
+demo supports table-driven shape fill colour and aggregated feature tooltips,
+but some of that logic still lives too close to the demo UI. Before publishing
+the first stable visualization API, these responsibilities should be pulled
+into reusable core/layer utilities.
+
+## Reference semantics
+
+Use Python `spatialdata` as the source of truth for association semantics.
+
+- Tables annotate regions through the `region`, `region_key`, `instance_key`
+  triplet. `region_key` identifies which spatial element a row annotates, and
+  `instance_key` identifies which instance within that element the row
+  annotates. See the upstream
+  [table annotations tutorial](https://spatialdata.scverse.org/en/stable/tutorials/notebooks/notebooks/examples/tables.html)
+  and
+  [SpatialData design document](https://github.com/scverse/spatialdata/blob/main/docs/design_doc.md).
+- Shapes are GeoDataFrames. Their semantic feature ids are the GeoDataFrame
+  index values, and extra GeoDataFrame columns are valid shape annotations.
+  The table `instance_key` should match those shape index values; row order is
+  not the association contract.
+- Labels instances are raster label values, excluding background. Table
+  `instance_key` values annotate those label values.
+- Points are not regions in the same sense as shapes/labels. They can carry
+  annotations directly and may also participate in future feature-key based
+  workflows, but they should not be forced into the region-table model by
+  default.
+
+In TypeScript terms, `featureId` should mean the canonical SpatialData
+instance id. `featureIndex` should mean render/order position only. A
+zero-based `featureIndex` can match a table row only when the upstream element
+index is actually a zero-based range and has been exposed as the feature id.
+
+## Desired API shape
+
+`@spatialdata/core` should expose a single canonical alignment helper for
+regions:
+
+```ts
+type FeatureTableAlignment = {
+  rowIndexByFeatureIndex: Int32Array;
+  rowIndexByFeatureId?: Map<string, number>;
+  resolveRowIndex(feature: {
+    featureId: string;
+    featureIndex: number;
+    rowIndex?: number;
+  }): number | undefined;
+};
+```
+
+The exact type may change, but the principle should not: tooltip resolution,
+click/hover events, table-driven fill colour, filtering, and downstream
+applications should all use one shared resolver. Avoid adding one-off helpers
+such as `resolveShapeFillColorRowIndex` in `SpatialCanvas`.
+
+The resolver should:
+
+- Load association metadata from `region`, `region_key`, and `instance_key`.
+- Filter rows by the target region/element.
+- Match table rows to canonical feature ids, not to render order.
+- Preserve unmatched features explicitly, rather than silently inventing a row.
+- Treat positional fallback as a compatibility path only when the element ids
+  are known to be positional ids.
+
+## Package boundaries
+
+`@spatialdata/core` should own semantic association:
+
+- Reading table keys and annotation metadata.
+- Loading element feature ids / instances.
+- Building `FeatureTableAlignment`.
+- Exposing shape/label/point annotation columns in a consistent way.
+
+`@spatialdata/layers` should own reusable deck/layer helpers:
+
+- Shape/label feature state runtimes.
+- Pick datum interpretation and logical layer id normalization for composite
+  layers where needed.
+- Optional column-to-colour encoders when they are renderer-agnostic and useful
+  to downstream apps.
+
+`@spatialdata/vis` and `SpatialCanvas` should remain UI glue:
+
+- Choosing which column to use.
+- Displaying property panels.
+- Loading the requested columns through core helpers.
+- Passing resolved feature state into deck layers.
+
+The demo UI can exercise a feature before the public API is final, but new
+semantic rules should not be invented in `SpatialCanvas`.
+
+## Annotation column roadmap
+
+Column choices in the demo should eventually include more than associated table
+obs columns:
+
+- Associated table obs columns, excluding `instance_key` and `region_key`.
+- Extra annotation columns stored directly on shape elements.
+- Future entries corresponding to `vars` in `X` / `layers` for expression-like
+  matrices.
+
+Those sources should be surfaced through a common annotation-column discovery
+API so downstream apps do not need to know whether a value came from AnnData
+obs, a GeoDataFrame column, or a matrix-backed feature.
+
+## Current branch status
+
+The current `SpatialCanvas` behaviour is acceptable as demo functionality:
+
+- Shape fill colour can be driven by a chosen table column.
+- Tooltips can aggregate multiple visible layers under the cursor.
+- Multiple layer configs may represent the same element with different visual
+  properties.
+
+However, these are not yet the desired foundations for a first stable
+library-facing API. Before publishing, revisit the implementation with this
+checklist:
+
+1. Move feature-to-row association into a core helper with tests against the
+   SpatialData table semantics above.
+2. Replace local row-index precedence rules in tooltip, fill-colour, and pick
+   event paths with that helper.
+3. Move reusable colour encoders and feature-state helpers out of the demo UI
+   when downstream apps need them.
+4. Keep `SpatialCanvas` as the consumer of these utilities, not the owner of
+   the semantics.
+5. Add fixture coverage for non-matching row order, missing rows, mixed-region
+   tables, labels values, and shape annotation columns.

docs/docs/vis/layer-prop-flow.mdx

@@ -54,6 +54,11 @@ refetches. The fix is upstream stability, not downstream caching.
 - `loader`: same object reference until the underlying element changes.
 - `selections`: same array reference until the selected values change. Memoize
   with `useMemo` keyed on a stable signature of the selection values.
+- Layer caches for derived visual state must be keyed by layer identity when
+  the value can differ between two layers that point at the same element. A
+  single shapes element may appear in multiple layer configs with different
+  opacity, filtering, table-driven colour encodings, or other visual
+  properties.
 - Cosmetic props (`colors`, `contrastLimits`, `channelsVisible`, `opacity`,
   per-channel arrays, `modelMatrix`): identity may churn freely. Deck diffs
   them efficiently and updates uniforms without disturbing tile loading.
@@ -184,6 +189,99 @@ Use this checklist when changing images, labels, shapes, or future layer types.
    should not call `getWorldBoundsForLayer()` unless they are executing a user
    command.
 
+## Shapes feature state and deck.gl performance
+
+Shapes layers can carry hundreds of thousands to millions of features. Most
+deck.gl update cost on large layers is **accessor re-execution** (see
+[deck.gl performance — optimize accessors](https://deck.gl/docs/developer-guide/performance#optimize-accessors)),
+not React reconciliation. Treat shapes rendering with the same discipline as
+tile layers: **minimize how often per-feature work runs**, and prefer uniform
+props (`opacity`, `radiusScale`, etc.) for cosmetic changes.
+
+### What is structural vs cosmetic for shapes
+
+| Change | Category | Expected work |
+|---|---|---|
+| `opacity`, `fillColor` / `strokeColor` defaults, stroke width clamps | Cosmetic | Deck uniform / attribute invalidation only; no geometry or feature-state rebuild |
+| `hiddenFeatureIds`, `fadedFeatureIds`, per-feature colour maps, table-driven fill column | Structural | Rebuild `shapePrebuiltData` and/or `ShapeFeatureStateRuntime` |
+| Geometry load, transform, element key | Structural | Reload / re-decode geometry |
+
+Cosmetic edits must **not** re-run `Record`→`Map` conversion, re-filter the
+feature list, or change `updateTriggers` keys that point at fresh objects every
+frame.
+
+### Producer contract (`useLayerData`)
+
+- **`shapePrebuiltData`** (keyed by layer id): built when geometry loads and when
+  `hiddenFeatureIds` changes — not on opacity tweaks.
+- **`stableShapeFeatureStateRef`** (keyed by layer id): holds a
+  `ShapeFeatureStateRuntime` (Maps/Sets) rebuilt only when the feature-state
+  signature changes (hidden/faded ids, opacity multiplier, manual colour record
+  identity, table fill-colour signature). Pass this into
+  `renderShapesLayer({ featureStateRuntime })` so `getLayers()` does not allocate
+  on every render.
+- **Merged feature-state objects must not be allocated each frame** when only
+  cosmetic layer props change. Table-driven fill colours are merged inside
+  `getStableShapeFeatureStateRuntime` only when the signature changes.
+
+### Layer contract (`@spatialdata/layers`)
+
+- **`buildShapeFeatureStateRuntime`**: converts serializable `featureState`
+  records to Maps/Sets once. Accepts an existing runtime and returns it
+  unchanged (`isShapeFeatureStateRuntime`).
+- **`normalizeShapeFeatureState`**: thin wrapper; use at layer boundaries when the
+  caller may still pass records (tests, `SpatialLayer`).
+- **`updateTriggers` for `getFillColor` / `getLineColor`**: list the specific
+  Maps/Sets/scalars that affect colour, not the whole `featureState` object.
+- Prefer **constant accessors** and layer `opacity` over per-feature callbacks
+  when the visual change is uniform.
+
+### Anti-patterns (shapes)
+
+| Anti-pattern | Why it's wrong |
+|---|---|
+| `new Map(Object.entries(fillColorByFeatureId))` on every `getLayers()` | O(n) allocation and GC pressure on large feature sets |
+| Spreading `featureState` each render for table fill colours | Defeats WeakMap identity caches; triggers full accessor rebuilds |
+| Putting `featureState` wholesale in `updateTriggers` | Any new object identity invalidates all colour attributes |
+| Per-feature accessors for layer-wide opacity | Use deck `opacity` on the layer instead |
+
+### Direction: binary attributes (not implemented)
+
+deck.gl's fastest path is **precomputed attribute buffers** (optionally built in
+a worker/WASM) passed via `data.attributes`, bypassing accessor calls entirely.
+Our shapes path still uses `PolygonLayer` / `ScatterplotLayer` with per-feature
+accessors for fill/stroke lookup by `featureId`. That is acceptable for current
+scale targets but is not the long-term ceiling.
+
+Future work (document-only for now):
+
+1. Columnar fill/stroke colours aligned with `featureIds` / `rowIndexByFeatureIndex`
+2. Push colours into typed arrays when encodings change, not when opacity changes
+3. Optional worker/WASM attribute generation for multi-million feature sets
+
+Until then, keep hot paths allocation-free across cosmetic renders and rebuild
+feature-state runtimes only when filtering or encodings actually change.
+
+## Shape/table annotation controls
+
+Shape UI controls that expose table-backed values should not be limited to
+associated table `obs` columns forever. They should also be able to use extra
+annotation columns carried by the shapes element itself. Future work should
+extend the same concept to entries corresponding to `var` values in `X` /
+`layers`, once the core table/annotation API exposes those data sources
+cleanly.
+
+When adding these controls, keep the visual encoding layer-specific: choosing
+one fill-colour column for a shapes layer must not affect another layer that
+renders the same shapes element.
+
+Fill-colour encodings should normally control both polygon fill and outline
+colour. Polygon outlines should use the shared shape stroke defaults from
+`@spatialdata/layers`: common-coordinate line width, `lineWidthMinPixels: 0`,
+and a small max-pixel clamp. That keeps outlines from dominating when many
+shapes become tiny on screen, while still allowing headless callers to override
+stroke width, units, and min/max pixel clamps per layer.
+
 ## See also
 
 - [`packages/vis/src/SpatialCanvas/useLayerData.ts`](https://github.com/Taylor-CCB-Group/SpatialData.js/blob/main/packages/vis/src/SpatialCanvas/useLayerData.ts)

docs/docs/vis/overview.mdx

@@ -15,6 +15,8 @@ This document describes how **`@spatialdata/vis`** fits into the wider visualiza
 Dependencies include **`@hms-dbmi/viv`**, **deck.gl**, **`@spatialdata/avivatorish`**, **`@spatialdata/layers`**, and **`@spatialdata/core`** / **`@spatialdata/react`**.
 
 See [SpatialCanvas + images — status and roadmap](./spatial-canvas-status).
+For the current boundary between demo UI behaviour and reusable feature/table
+semantics, see [Feature table associations and annotation columns](./feature-table-associations).
 
 ## `@spatialdata/layers`
 

docs/docs/vis/spatial-canvas-status.mdx

@@ -49,6 +49,10 @@ sidebar_position: 1
 
 - **Histogram** (optional) and richer channel UX (MDV parity).
 - **Fold or replace `ImageView`** once SpatialCanvas covers the single-image case well.
+- **Feature/table foundations:** keep current shape colour and aggregate
+  tooltip behaviour as demo functionality, then move canonical feature-to-row
+  association and annotation column discovery into reusable core/layer APIs.
+  See [Feature table associations and annotation columns](./feature-table-associations).
 
 ## Medium-term roadmap