Document the feature-table association split and core/layers responsibilities

.github/workflows/test.yml

@@ -72,19 +72,19 @@ jobs:
           # Wait for server to be ready (up to ~30s)
           # Check both root and a fixture path to ensure server is fully ready
           for i in {1..30}; do
-            if curl -sSf http://localhost:8080/ >/dev/null && \
-               curl -sSf http://localhost:8080/v0.5.0/blobs.zarr/zmetadata >/dev/null; then
+            if curl -sSf http://localhost:38473/ >/dev/null && \
+               curl -sSf http://localhost:38473/v0.5.0/blobs.zarr/zmetadata >/dev/null; then
               echo "Test server is up and serving fixtures"
               break
             fi
-            echo "Waiting for test server on http://localhost:8080/ ..."
+            echo "Waiting for test server on http://localhost:38473/ ..."
             sleep 1
           done
 
           # Small additional delay to ensure server is fully ready
           sleep 2
 
-          # Run integration tests (will hit http://localhost:8080/…)
+          # Run integration tests (will hit http://localhost:38473/…)
           pnpm test:integration
 
           # Clean up server

.vscode/settings.json

@@ -8,5 +8,8 @@
     "editor.defaultFormatter": "ms-python.black-formatter",
     "editor.formatOnSave": true
   },
-  "python.analysis.extraPaths": ["${workspaceFolder}/python"]
+  "python.analysis.extraPaths": [
+    "${workspaceFolder}/python"
+  ],
+  "typescript.tsdk": "node_modules/typescript/lib"
 }

README.md

@@ -195,14 +195,16 @@ pnpm test:fixtures:generate:0.7.2
 The main Node integration tests now load fixtures directly from a `FileSystemStore`. This server is still useful for HTTP smoke tests and browser-oriented local development with `FetchStore`:
 
 ```bash
-# Start the test fixture server (runs on http://localhost:8080)
+# Start the test fixture server (default http://localhost:38473)
 pnpm test:server
 ```
 
+Override the port with `SPATIALDATA_FIXTURE_PORT` if needed.
+
 Once running, fixtures are accessible at:
-- `http://localhost:8080/test-fixtures/v0.5.0/blobs.zarr`
-- `http://localhost:8080/test-fixtures/v0.6.1/blobs.zarr`
-- `http://localhost:8080/test-fixtures/v0.7.2/blobs.zarr`
+- `http://localhost:38473/test-fixtures/v0.5.0/blobs.zarr`
+- `http://localhost:38473/test-fixtures/v0.6.1/blobs.zarr`
+- `http://localhost:38473/test-fixtures/v0.7.2/blobs.zarr`
 
 The server provides directory listings and serves all zarr metadata files with appropriate CORS headers.
 

docs/docs/core/overview.mdx

@@ -29,7 +29,7 @@ pnpm add @spatialdata/core
 ## Quick Start
 
 ```ts
-import { readZarr } from '@spatialdata/core';
+import { readZarr, loadFeatureRowIndexByFeatureIndex } from '@spatialdata/core';
 
 // Load a SpatialData store from a URL
 const sdata = await readZarr('https://example.com/my-spatialdata.zarr');
@@ -46,8 +46,23 @@ const points = sdata.points;    // Record<string, PointsElement>
 
 // Get coordinate systems
 const coordinateSystems = sdata.coordinateSystems; // ['global', ...]
+
+// Table-backed shape association (see also @spatialdata/layers encoders)
+const shapesEl = sdata.shapes.cell_shapes;
+const renderData = await shapesEl.loadRenderData();
+const rowIndexByFeatureIndex = await loadFeatureRowIndexByFeatureIndex({
+  spatialData: sdata,
+  kind: 'shapes',
+  key: 'cell_shapes',
+  featureIds: renderData.featureIds,
+});
 ```
 
+Tables load through **`anndata.js`** on **`zarrita`** stores. Association helpers
+today use targeted obs-column loaders; the roadmap is a unified AnnData.js /
+zarrita **`DataLoader`** surface for richer `obs` / `var` / `X` / `obsm` access
+(see [MDV integration — tables](../vis/mdv-integration#tables-anndatajs-and-zarrita-dataloader)).
+
 ## Design Philosophy
 
 The library follows several key principles:
@@ -75,6 +90,13 @@ The main entry points for application code are:
 | Element classes | `ImageElement`, `ShapesElement`, `LabelsElement`, `PointsElement`, `TableElement` |
 | `Result` utilities | `Ok`, `Err`, `isOk`, `isErr`, `unwrap`, `unwrapOr` |
 | `getTransformMatrix()` | Convenience function for getting Matrix4 transforms |
+| Table association helpers | `loadAssociatedTableFeatureRows`, `loadFeatureRowIndexByFeatureIndex`, `createFeatureTableAlignment` |
+
+The table association helpers follow Python `spatialdata` semantics: regions
+are matched through `region`, `region_key`, and `instance_key`, with feature ids
+coming from SpatialElement instances such as `GeoDataFrame.index` for shapes.
+Visual encoders in `@spatialdata/layers` consume the row alignment produced
+here rather than reimplementing association rules.
 
 For internal architecture details, module organization, and APIs intended for contributors or advanced tooling, see [Internals & Architecture](./internals).
 
@@ -101,4 +123,3 @@ These have coordinate transformations and can be rendered in shared coordinate s
 - [Transformations](./transformations) - Understanding coordinate systems and transforms
 - [Error Handling](./error-handling) - Using the Result type pattern
 - [Internals](./internals) - Architecture and internal APIs (for contributors)
-

docs/docs/intro.mdx

@@ -12,31 +12,162 @@ The approach taken to the design is to follow the structure of the original libr
 
 There are several packages that aim to facilitate the use and development of this functionality:
 
-## `@spatialdata/core`
+## Package map
 
-This is a vanilla-JS library that provides a reasonably lightweight, clean and efficient way of loading data from SpatialData stores, reflecting the functionality of the Python library.
+| Package | Role | React? | npm status |
+|---------|------|--------|------------|
+| `@spatialdata/core` | Load SpatialData Zarr stores; elements, transforms, table association | No | Not published yet |
+| `@spatialdata/react` | Thin hooks around core (`useSpatialData`, provider) | Yes | Not published yet |
+| `@spatialdata/layers` | deck.gl layers, shape/point renderers, versioned layer props | No | Not published yet |
+| `@spatialdata/avivatorish` | Viv loaders, channel stats, image Zustand stores | No | Not published yet |
+| `@spatialdata/vis` | `SpatialCanvas` UI + headless `SpatialCanvasViewer` | Yes | Not published yet; MDV-targeted prerelease in progress |
 
-See the [Core Package documentation](./core/overview) for detailed API documentation, including:
-- [Element Classes](./core/elements) - Working with images, shapes, labels, points, and tables
-- [Transformations](./core/transformations) - Coordinate systems and spatial transforms
-- [Error Handling](./core/error-handling) - The Result type pattern
-- [Internals](./core/internals) - Architecture and internal APIs (for contributors)
+Nothing is on npm yet. Clone the monorepo or `pnpm pack` workspace packages for local integration (see [Headless viewer guide](./vis/headless-viewer)).
 
+## `@spatialdata/core`
 
-## `@spatialdata/react`
+Vanilla-JS library for loading SpatialData Zarr stores. Mirrors the Python
+`spatialdata` API where practical: elements, coordinate systems, lazy data
+loads, and table–feature association (`region` / `region_key` / `instance_key`).
+
+Built on **`zarrita`** for Zarr I/O and **`anndata.js`** for AnnData-backed
+tables. Table obs-column helpers used today (`loadObsColumns`, association
+resolvers) sit on a direct loader path; the roadmap is a coherent
+**`TableElement` → AnnData.js / zarrita `DataLoader`** surface for `obs`,
+`var`, `X`, `obsm`, and related AnnData views without every integrator calling
+`getAnnDataJS()` directly.
+
+See [Core Package documentation](./core/overview):
+- [Element Classes](./core/elements) — images, shapes, labels, points, tables
+- [Transformations](./core/transformations) — coordinate systems and spatial transforms
+- [Error Handling](./core/error-handling) — the `Result` type pattern
+- [Internals](./core/internals) — architecture (for contributors)
+
+### Core-only example
+
+```ts
+import {
+  readZarr,
+  createFeatureTableAlignment,
+  loadFeatureRowIndexByFeatureIndex,
+} from '@spatialdata/core';
+
+const sdata = await readZarr('https://example.com/data.zarr');
+
+// Load shape geometry + stable feature ids (async per element)
+const shapesEl = sdata.shapes.cell_shapes;
+const renderData = await shapesEl.loadRenderData();
+
+// Join shapes to an associated AnnData table row index
+const rowIndexByFeatureIndex = await loadFeatureRowIndexByFeatureIndex({
+  spatialData: sdata,
+  kind: 'shapes',
+  key: 'cell_shapes',
+  featureIds: renderData.featureIds,
+});
+const alignment = createFeatureTableAlignment({ rowIndexByFeatureIndex });
+
+// alignment.resolveRowIndex({ featureId, featureIndex }) → table row or undefined
+```
+
+## `@spatialdata/layers`
+
+**deck.gl–native, React-free** rendering package. This is the layer between
+`core` data and any viewer (custom `DeckGL`, MDV, Vitessce, or `@spatialdata/vis`).
+
+What it owns today:
+
+- **`createShapesDeckLayer`** / **`buildShapesPrebuiltData`** — polygon, circle, and GeoArrow-table shape paths with feature-state styling (hide, fade, per-id colours)
+- **`buildShapeFillColorByFeatureId`** — table column → per-feature colours (consumes core row alignment; does not resolve associations itself)
+- **`SpatialLayer`** + **`spatialLayerPropsSchema`** / **`migrateSpatialLayerProps`** — versioned, JSON-serializable composite-layer contract (image/points sublayers still maturing)
+- Pick/tooltip helpers keyed by stable **`featureId`**
+
+`@spatialdata/vis` calls into `@spatialdata/layers` for shapes; MDV can drive
+the same contracts by updating layer config / feature state without importing vis
+internals.
+
+See [Layers package overview](./layers/overview).
+
+### React-agnostic deck example
+
+```ts
+import { createShapesDeckLayer, buildShapesPrebuiltData } from '@spatialdata/layers';
+import { Deck } from '@deck.gl/core';
+
+// renderData from core (see above)
+const prebuilt = buildShapesPrebuiltData(renderData);
+const shapesLayer = createShapesDeckLayer(
+  renderData,
+  {
+    kind: 'shapes',
+    elementKey: 'cell_shapes',
+    visible: true,
+    defaultFillColor: [70, 130, 180, 200],
+    featureState: { hiddenFeatureIds: ['42'] },
+  },
+  { id: 'my-shapes', prebuilt }
+);
+
+const deck = new Deck({
+  canvas: document.getElementById('deck-canvas'),
+  initialViewState: { longitude: 0, latitude: 0, zoom: 1 },
+  layers: shapesLayer ? [shapesLayer] : [],
+});
+```
 
-React hooks for SpatialData. This package should have a minimal set of dependencies, such that it should be easy to integrate into other React applications. It is mostly focused on providing react-idiomatic ways of accessing the data itself, with appropriate abstractions around the core vanilla API, managing the `async` nature of fetching data etc.
+## `@spatialdata/react`
 
+Minimal React hooks for SpatialData: `SpatialDataProvider` + `useSpatialData`
+wrap `readZarr` and expose `{ spatialData, loading, error }`. No deck/Viv
+dependencies — use with `@spatialdata/vis` or your own renderer.
 
 ## `@spatialdata/vis`
 
-High-level React-components and deck.gl layers for visualising SpatialData. This package is explicitly less stable and more experimental than the related `core` and `react` packages - the dependency on the `viv`/`deck.gl`/`luma.gl` stack means that installing it into an app (particularly one that already uses anything from this ecosystem) entails a certain amount of care around potential conflicts or breaking changes between versions.
+React components and the SpatialCanvas stack (Viv images + deck vectors).
+**2D only today** — orthographic pan/zoom via Viv `DetailView`; no 3D orbit or volume rendering.
+
+**Two entry modes:**
+
+1. **Batteries included** — `SpatialCanvas` with coordinate-system picker, layer list, properties panels, tooltips.
+2. **Headless / controlled** — `SpatialCanvasViewer` or `useSpatialCanvasRenderer`: same render path, **your** state for `layers`, `layerOrder`, `viewState`, plus optional `deckLayers` for app overlays.
+
+The vis package is more experimental than `core`/`layers` because of the
+Viv/deck/luma dependency stack. The near-term goal is an MDV-consumable
+prerelease with headless embedding first; API stability guarantees come after
+that smoke test.
+
+- [Headless viewer guide](./vis/headless-viewer) — practical guide for controlled / MDV-style embedding
+- [SpatialCanvas status and roadmap](./vis/spatial-canvas-status) — what works, known gaps
+- [Visualization overview](./vis/overview) — package topology (`vis`, `layers`, `avivatorish`)
+- [MDV integration](./vis/mdv-integration) — phased MDV/Vitessce alignment
+- [MDV release checklist](./vis/mdv-release-checklist) — branch shipping criteria
+
+### Headless vis example
+
+```tsx
+import { SpatialCanvasViewer, type LayerConfig } from '@spatialdata/vis';
+
+<SpatialCanvasViewer
+  spatialData={spatialData}
+  coordinateSystem="global"
+  layers={layers}
+  layerOrder={['image', 'shapes']}
+  viewState={viewState}
+  onViewStateChange={setViewState}
+  deckLayers={myMdvOverlays}
+  renderTooltip={false}
+  onShapeHover={handleShapeHover}
+/>
+```
+
+### Full UI example
 
-It is hoped that it will be robust and of a high quality, but especially at the early stage of development, the goal is not to have a lean bundle size, guarantees of API stability between releases, etc.
+```tsx
+import { SpatialCanvas } from '@spatialdata/vis';
 
-It is used to provide working examples for displaying in the `docs` site, as well as a sample app primarily for prototyping while developing features.
+<SpatialCanvas spatialData={spatialData} />
+```
 
-- [SpatialCanvas + images — status and roadmap](./vis/spatial-canvas-status) — what works today, known gaps, and planned work
-- [Visualization package overview](./vis/overview) — packages topology and direction (`vis`, `layers`, `avivatorish`)
-- [Layers package overview](./layers/overview) — `SpatialLayer`, versioned `SpatialLayerProps`, migrations
-- [MDV integration notes](./vis/mdv-integration) — how this repo aligns with MDV over time
+`SpatialCanvas` manages its own zustand store (layer visibility, order, channel
+UI). Use it for demos and exploration; MDV and other hosts should prefer
+`SpatialCanvasViewer` with externally owned state.

docs/docs/layers/overview.mdx

@@ -31,6 +31,9 @@ See also the [visualization overview](../vis/overview): deck-only integrators ca
 - shapes rendering is owned primarily at the `@spatialdata/layers` tier rather than `@spatialdata/vis`
 - the current shipping path is mixed-mode: polygon fallback for JS/WKB payloads and a shared columnar backend branch for `geoarrow-table` runtime data
 - row alignment, picking, and tooltip resolution are now driven by stable feature identity plus shared row-index metadata
+- `buildShapeFillColorByFeatureId()` turns a table column plus already-resolved
+  `rowIndexByFeatureIndex` into per-feature colour maps; it deliberately does
+  not resolve SpatialData table associations itself
 - the public shapes config stays representation-agnostic so a stronger future `deck.gl-geoarrow` backend can slot in without changing saved props
 
 For shapes, the important contract is stable feature identity plus table-join-driven styling/filtering. `@spatialdata/core` loads render-oriented shape data, `@spatialdata/layers` turns that into deck layers, and `@spatialdata/vis` consumes the shared behavior for viewer use.
@@ -51,3 +54,44 @@ load-bearing:
 The migration test is the same as shapes: MDV or Vitessce should be able to
 drive hide/fade/color/radius state without knowing whether the renderer used JS
 arrays, deck binary attributes, or GeoArrow batches underneath.
+
+## React-agnostic usage
+
+`@spatialdata/layers` has no React dependency. A minimal deck-only path:
+
+```ts
+import {
+  buildShapesPrebuiltData,
+  createShapesDeckLayer,
+  buildShapeFillColorByFeatureId,
+} from '@spatialdata/layers';
+import {
+  createFeatureTableAlignment,
+  loadFeatureRowIndexByFeatureIndex,
+} from '@spatialdata/core';
+
+// After loading renderData + table column from core:
+const fillColorByFeatureId = buildShapeFillColorByFeatureId({
+  featureIds: renderData.featureIds,
+  rowIndexByFeatureIndex: alignment.rowIndexByFeatureIndex,
+  column: obsColumn,
+  mode: 'categorical',
+  alpha: 200,
+});
+
+const layer = createShapesDeckLayer(
+  renderData,
+  {
+    kind: 'shapes',
+    elementKey: 'cell_shapes',
+    visible: true,
+    featureState: { fillColorByFeatureId },
+  },
+  { id: 'shapes', prebuilt: buildShapesPrebuiltData(renderData) }
+);
+```
+
+For Viv images and async SpatialData layer orchestration without the
+`SpatialCanvas` UI, use `@spatialdata/vis` headless APIs
+([Headless viewer guide](../vis/headless-viewer)) or compose
+`createShapesDeckLayer` output with your own `Deck` / `DeckGL` stack.