reinitialize (#823)

* document layouts (as "scale-aware transforms")

* document binWidth

* document the initialize option after 42ac4f0

* sort hex bins by radius (descending)
group by z
inline hexbin
binWidth is the distance between two centers

(rebased on mbostock/reinitialize)

* dodge

rebased on mbostock/reinitialize

* compose intializers

* use composeInitialize to make dodge composable

* add new channels as you compose initializers

* darker transform, to demonstrate composition with dodgeY

(added as an example, but we could promote it to a transform)

* a more generic "remap"

* jiggle layout
(using the same remap intializer as in the darkerDodge plot)

Author: Fil

README.md

@@ -1037,6 +1037,14 @@ Plot.dotY(cars.map(d => d["economy (mpg)"]))
 
 Equivalent to [Plot.dot](#plotdotdata-options) except that if the **y** option is not specified, it defaults to the identity function and assumes that *data* = [*y₀*, *y₁*, *y₂*, …].
 
+### Hexgrid
+
+The hexgrid mark can be used to support marks using the [hexbin](#hexbin) layout.
+
+#### Plot.hexgrid([*options*])
+
+The *binWidth* option specifies the distance between the centers of neighboring hexagons, in pixels (defaults to 20). The *clip* option defaults to true, clipping the mark to the frame’s dimensions.
+
 ### Image
 
 [<img src="./img/image.png" width="320" height="198" alt="a scatterplot of Presidential portraits">](https://observablehq.com/@observablehq/plot-image)
@@ -1524,10 +1532,10 @@ The following aggregation methods are supported:
 * *pXX* - the percentile value, where XX is a number in [00,99]
 * *deviation* - the standard deviation
 * *variance* - the variance per [Welford’s algorithm](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm)
-* *x* - the middle the bin’s *x*-extent (when binning on *x*)
+* *x* - the middle of the bin’s *x*-extent (when binning on *x*)
 * *x1* - the lower bound of the bin’s *x*-extent (when binning on *x*)
 * *x2* - the upper bound of the bin’s *x*-extent (when binning on *x*)
-* *y* - the middle the bin’s *y*-extent (when binning on *y*)
+* *y* - the middle of the bin’s *y*-extent (when binning on *y*)
 * *y1* - the lower bound of the bin’s *y*-extent (when binning on *y*)
 * *y2* - the upper bound of the bin’s *y*-extent (when binning on *y*)
 * a function to be passed the array of values for each bin and the extent of the bin
@@ -2145,6 +2153,69 @@ This helper for constructing derived columns returns a [*column*, *setColumn*] a
 
 Plot.column is typically used by options transforms to define new channels; the associated columns are populated (derived) when the **transform** option function is invoked.
 
+## Scale-aware transforms
+
+Some transforms need to operate in representation space (such as pixels and colors, *i.e.* after scales have been applied) rather than data space. Such a transform might, for example, modify the marks’ positions in screen space to avoid occlusion. These scale-aware transforms are applied *after* the initial setting of the scales, and can modify the channels or derive new channels—which can in turn be passed to scales.
+
+### Dodge
+
+The dodge transform can be applied to any mark that consumes *x* or *y*, such as the Dot, Image, Text and Vector marks.
+#### Plot.dodgeY([*layoutOptions*, ]*options*)
+
+```js
+Plot.dodgeY({x: "date"})
+```
+
+If the marks are arranged along the *x* axis, the dodgeY transform piles them vertically, keeping their *x* position unchanged, and creating a *y* position that avoids overlapping.
+
+#### Plot.dodgeX([*layoutOptions*, ]*options*)
+
+```js
+Plot.dodgeX({y: "value"})
+```
+
+Equivalent to Plot.dodgeY, but the piling is horizontal, keeping the marks’ *y* position unchanged, and creating an *x* position that avoids overlapping.
+The dodge transforms accept the following options:
+* **padding** — a number of pixels added to the radius of the mark to estimate its size
+* **anchor** - the frame anchor: one of *middle*, *right*, and *left* (default) for dodgeX, and one of *middle*, *top*, and *bottom* (default) for dodgeY. With the *middle* anchor the piles will grow from the center in both directions; with the other anchors, the piles will grow from the specified anchor towards the opposite direction.
+
+### Hexbin
+
+The hexbin transform can be applied to any mark that consumes *x* and *y*, such as the Dot, Image, Text and Vector marks. It aggregates the values into hexagonal bins of the given *radius* (in pixel space), and computes new values *x* and *y* as the centers of each bin. It can also return new channels by applying a reducer to each bin, such as the number of elements in the bin.
+
+#### Plot.hexbin(*outputs*, *options*)
+
+[Source](./src/transforms/hexbin.js) · [Examples](https://observablehq.com/@observablehq/plot-hexbin) · Aggregates the given inputs into hexagonal bins, and creates output channels with the reduced data. The options must specify the *x* and *y* channels, and can optionally indicate the *binWidth* in pixels (defaults to 20), defined as the distance between the centers of two neighboring hexagons. If any of **z**, **fill**, or **stroke** is a channel, the first of these channels will be used to subdivide bins. The *outputs* options are similar to Plot.bin’s outputs; each output channel receives as input, for each hexagon, the subset of the data which has been matched to its center. The outputs object specifies the aggregation method for each output channel.
+
+The following aggregation methods are supported:
+
+* *first* - the first value, in input order
+* *last* - the last value, in input order
+* *count* - the number of elements (frequency)
+* *distinct* - the number of distinct values
+* *sum* - the sum of values
+* *proportion* - the sum proportional to the overall total (weighted frequency)
+* *proportion-facet* - the sum proportional to the facet total
+* *min* - the minimum value
+* *min-index* - the zero-based index of the minimum value
+* *max* - the maximum value
+* *max-index* - the zero-based index of the maximum value
+* *mean* - the mean value (average)
+* *median* - the median value
+* *deviation* - the standard deviation
+* *variance* - the variance per [Welford’s algorithm](https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm)
+* *mode* - the value with the most occurrences
+* a function to be passed the array of values for each bin and the extent of the bin
+* an object with a *reduce* method
+
+When the hexbin transform has an *r* output, the bins are returned in decreasing size order.
+
+See also the [hexgrid](#hexgrid) mark.
+
+### Custom scale-aware transforms
+
+When its *options* have an *initialize* property, the initialize function is called after the scales have been computed. It receives as inputs the (possibly transformed) data array, the index of elements of this array that belong to each facet, the input channels (as a key: array object), the scales, and the dimensions, with the mark as this. It must return the data, index, and the channels that need to be scaled in a second pass.
+
 ## Curves
 
 A curve defines how to turn a discrete representation of a line as a sequence of points [[*x₀*, *y₀*], [*x₁*, *y₁*], [*x₂*, *y₂*], …] into a continuous path; *i.e.*, how to interpolate between points. Curves are used by the [line](#line), [area](#area), and [link](#link) mark, and are implemented by [d3-shape](https://github.com/d3/d3-shape/blob/master/README.md#curves).

package.json

@@ -35,6 +35,7 @@
   },
   "sideEffects": false,
   "devDependencies": {
+    "@rollup/plugin-commonjs": "^21.0.1",
     "@rollup/plugin-json": "4",
     "@rollup/plugin-node-resolve": "13",
     "canvas": "2",
@@ -50,7 +51,7 @@
   },
   "dependencies": {
     "d3": "^7.3.0",
-    "d3-hexbin": "^0.2.2",
+    "interval-tree-1d": "1",
     "isoformat": "0.2"
   },
   "engines": {

rollup.config.js

@@ -1,5 +1,6 @@
 import fs from "fs";
 import {terser} from "rollup-plugin-terser";
+import commonjs from "@rollup/plugin-commonjs";
 import json from "@rollup/plugin-json";
 import node from "@rollup/plugin-node-resolve";
 import * as meta from "./package.json";
@@ -25,6 +26,7 @@ const config = {
     banner: `// ${meta.name} v${meta.version} Copyright ${copyrights.join(", ")}`
   },
   plugins: [
+    commonjs(),
     json(),
     node()
   ]

src/index.js

@@ -19,6 +19,7 @@ export {Vector, vector, vectorX, vectorY} from "./marks/vector.js";
 export {valueof, column} from "./options.js";
 export {filter, reverse, sort, shuffle, basic as transform} from "./transforms/basic.js";
 export {bin, binX, binY} from "./transforms/bin.js";
+export {dodgeX, dodgeY} from "./transforms/dodge.js";
 export {group, groupX, groupY, groupZ} from "./transforms/group.js";
 export {hexbin} from "./transforms/hexbin.js";
 export {normalize, normalizeX, normalizeY} from "./transforms/normalize.js";

src/marks/hexgrid.js

@@ -17,15 +17,15 @@ export function hexgrid(options) {
 }
 
 export class Hexgrid extends Mark {
-  constructor({radius = 10, clip = true, ...options} = {}) {
+  constructor({binWidth = 20, clip = true, ...options} = {}) {
     super(undefined, undefined, {clip, ...options}, defaults);
-    this.radius = number(radius);
+    this.binWidth = number(binWidth);
   }
   render(index, scales, channels, dimensions) {
-    const {dx, dy, radius: rx} = this;
+    const {dx, dy, binWidth} = this;
     const {marginTop, marginRight, marginBottom, marginLeft, width, height} = dimensions;
     const x0 = marginLeft - ox, x1 = width - marginRight - ox, y0 = marginTop - oy, y1 = height - marginBottom - oy;
-    const ry = rx * sqrt4_3, hy = ry / 2, wx = rx * 2, wy = ry * 1.5;
+    const rx = binWidth / 2, ry = rx * sqrt4_3, hy = ry / 2, wx = rx * 2, wy = ry * 1.5;
     const path = `m0,${-ry}l${rx},${hy}v${ry}l${-rx},${hy}`;
     const i0 = Math.floor(x0 / wx), i1 = Math.ceil(x1 / wx);
     const j0 = Math.floor((y0 + hy) / wy), j1 = Math.ceil((y1 - hy) / wy) + 1;

src/plot.js

@@ -90,11 +90,16 @@ export function plot(options = {}) {
   autoScaleLabels(channelsByScale, scaleDescriptors, axes, dimensions, options);
   autoAxisTicks(scaleDescriptors, axes);
 
+  const {fx, fy} = scales;
+  const fyMargins = fy && {marginTop: 0, marginBottom: 0, height: fy.bandwidth()};
+  const fxMargins = fx && {marginRight: 0, marginLeft: 0, width: fx.bandwidth()};
+  const subdimensions = {...dimensions, ...fxMargins, ...fyMargins};
+
   // Reinitialize; for deriving channels dependent on other channels.
   const newByScale = new Set();
   for (const [mark, state] of stateByMark) {
     if (mark.reinitialize != null) {
-      const {facets, channels} = mark.reinitialize(state.data, state.facets, state.channels, scales);
+      const {facets, channels} = mark.reinitialize(state.data, state.facets, state.channels, scales, subdimensions);
       if (facets !== undefined) state.facets = facets;
       if (channels !== undefined) {
         inferChannelScale(channels, mark);
@@ -148,7 +153,6 @@ export function plot(options = {}) {
     .node();
 
   // When faceting, render axes for fx and fy instead of x and y.
-  const {fx, fy} = scales;
   const axisY = axes[facets !== undefined && fy ? "fy" : "y"];
   const axisX = axes[facets !== undefined && fx ? "fx" : "x"];
   if (axisY) svg.appendChild(axisY.render(null, scales, dimensions));
@@ -158,9 +162,6 @@ export function plot(options = {}) {
   if (facets !== undefined) {
     const fyDomain = fy && fy.domain();
     const fxDomain = fx && fx.domain();
-    const fyMargins = fy && {marginTop: 0, marginBottom: 0, height: fy.bandwidth()};
-    const fxMargins = fx && {marginRight: 0, marginLeft: 0, width: fx.bandwidth()};
-    const subdimensions = {...dimensions, ...fxMargins, ...fyMargins};
     const indexByFacet = facetMap(facetChannels);
     facets.forEach(([key], i) => indexByFacet.set(key, i));
     const selection = select(svg);

src/symbols.js

@@ -1,7 +1,8 @@
 import {symbolAsterisk, symbolDiamond2, symbolPlus, symbolSquare2, symbolTriangle2, symbolX as symbolTimes} from "d3";
 import {symbolCircle, symbolCross, symbolDiamond, symbolSquare, symbolStar, symbolTriangle, symbolWye} from "d3";
 
-export const sqrt4_3 = 2 / Math.sqrt(3);
+export const sqrt3 = Math.sqrt(3);
+export const sqrt4_3 = 2 / sqrt3;
 
 const symbolHexagon = {
   draw(context, size) {

src/transforms/basic.js

@@ -9,13 +9,15 @@ export function basic({
   sort: s1,
   reverse: r1,
   transform: t1,
+  initialize: i1,
   ...options
 } = {}, t2) {
   if (t1 === undefined) { // explicit transform overrides filter, sort, and reverse
     if (f1 != null) t1 = filterTransform(f1);
     if (s1 != null && !isOptions(s1)) t1 = composeTransform(t1, sortTransform(s1));
     if (r1) t1 = composeTransform(t1, reverseTransform);
   }
+  if (t2 != null && i1 != null) throw new Error("Data transforms must appear before any channel transform");
   return {
     ...options,
     ...isOptions(s1) && {sort: s1},
@@ -32,6 +34,20 @@ function composeTransform(t1, t2) {
   };
 }
 
+export function composeInitialize({initialize: i1, ...options} = {}, i2) {
+  return i1 == null
+  ? {...options, initialize: i2}
+  : {
+    ...options,
+    initialize(data, facets, channels, scales, dimensions) {
+      let newChannels;
+      ({data, facets, channels: newChannels} = i1.call(this, data, facets, channels, scales, dimensions));
+      Object.assign(channels, newChannels);
+      return i2.call(this, data, facets, channels, scales, dimensions);
+    }
+  };
+}
+
 export function filter(value, options) {
   return basic(options, filterTransform(value));
 }

src/transforms/dodge.js

@@ -0,0 +1,101 @@
+import {max} from "d3";
+import IntervalTree from "interval-tree-1d";
+import {finite, positive} from "../defined.js";
+import {composeInitialize} from "./basic.js";
+
+const anchorXLeft = ({marginLeft}) => [1, marginLeft];
+const anchorXRight = ({width, marginRight}) => [-1, width - marginRight];
+const anchorXMiddle = ({width, marginLeft, marginRight}) => [0, (marginLeft + width - marginRight) / 2];
+const anchorYTop = ({marginTop}) => [1, marginTop];
+const anchorYBottom = ({height, marginBottom}) => [-1, height - marginBottom];
+const anchorYMiddle = ({height, marginTop, marginBottom}) => [0, (marginTop + height - marginBottom) / 2];
+
+function maybeAnchor(anchor) {
+  return typeof anchor === "string" ? {anchor} : anchor;
+}
+
+export function dodgeX(dodgeOptions = {}, options = {}) {
+  if (arguments.length === 1) [options, dodgeOptions] = [dodgeOptions, options];
+  let {anchor = "left", padding = 1} = maybeAnchor(dodgeOptions);
+  switch (`${anchor}`.toLowerCase()) {
+    case "left": anchor = anchorXLeft; break;
+    case "right": anchor = anchorXRight; break;
+    case "middle": anchor = anchorXMiddle; break;
+    default: throw new Error(`unknown dodge anchor: ${anchor}`);
+  }
+  return dodge("x", "y", anchor, +padding, options);
+}
+
+export function dodgeY(dodgeOptions = {}, options = {}) {
+  if (arguments.length === 1) [options, dodgeOptions] = [dodgeOptions, options];
+  let {anchor = "bottom", padding = 1} = maybeAnchor(dodgeOptions);
+  switch (`${anchor}`.toLowerCase()) {
+    case "top": anchor = anchorYTop; break;
+    case "bottom": anchor = anchorYBottom; break;
+    case "middle": anchor = anchorYMiddle; break;
+    default: throw new Error(`unknown dodge anchor: ${anchor}`);
+  }
+  return dodge("y", "x", anchor, +padding, options);
+}
+
+function dodge(y, x, anchor, padding, options) {
+  return composeInitialize(options, function(data, facets, {[x]: X, r: R}, {[x]: xscale, r: rscale}, dimensions) {
+    if (!X) throw new Error(`missing channel ${x}`);
+    X = X.value.map(xscale);
+    const r = R ? undefined : this.r !== undefined ? this.r : options.r !== undefined ? +options.r : 3;
+    if (R) R = R.value.map(rscale);
+    if (X == null) throw new Error(`missing channel: ${x}`);
+    let [ky, ty] = anchor(dimensions);
+    const compare = ky ? compareAscending : compareSymmetric;
+    if (ky) ty += ky * ((R ? max(facets.flat(), i => R[i]) : r) + padding); else ky = 1;
+    const Y = new Float64Array(X.length);
+    const radius = R ? i => R[i] : () => r;
+    for (let I of facets) {
+      const tree = IntervalTree();
+      I = I.filter(R
+        ? i => finite(X[i]) && positive(R[i])
+        : i => finite(X[i]));
+      for (const i of I) {
+        const intervals = [];
+        const l = X[i] - radius(i);
+        const h = X[i] + radius(i);
+
+        // For any previously placed circles that may overlap this circle, compute
+        // the y-positions that place this circle tangent to these other circles.
+        // https://observablehq.com/@mbostock/circle-offset-along-line
+        tree.queryInterval(l - padding, h + padding, ([,, j]) => {
+          const yj = Y[j];
+          const dx = X[i] - X[j];
+          const dr = padding + (R ? R[i] + R[j] : 2 * r);
+          const dy = Math.sqrt(dr * dr - dx * dx);
+          intervals.push([yj - dy, yj + dy]);
+        });
+
+        // Find the best y-value where this circle can fit.
+        for (let y of intervals.flat().sort(compare)) {
+          if (intervals.every(([lo, hi]) => y <= lo || y >= hi)) {
+            Y[i] = y;
+            break;
+          }
+        }
+
+        // Insert the placed circle into the interval tree.
+        tree.insert([l, h, i]);
+      }
+      for (const i of I) Y[i] = Y[i] * ky + ty;
+    }
+    return {data, facets, channels: {
+      [x]: {value: X},
+      [y]: {value: Y},
+      ...R && {r: {value: R}}
+    }};
+  });
+}
+
+function compareSymmetric(a, b) {
+  return Math.abs(a) - Math.abs(b);
+}
+
+function compareAscending(a, b) {
+  return (a < 0) - (b < 0) || (a - b);
+}