Upgrading from 0.6.1 to 0.6.2 seems to have broken (or changed in unexpected way) date handling

[nobarany]

Upgrading from 0.6.1 to 0.6.2 seems to have broken something related to date handling. The chart bellow that has the 2023 data is the original one, and after the upgrade all the 2023 data is not shown.
---update---
replaced the screenshots with this notebook:

[mbostock]

Can you share your data and code to reproduce this example? Otherwise there’s not really anything to go on and we can’t investigate. Thank you.

[mbostock]

This is probably related to changes in the bin transform #1225, but I can’t speculate further without knowing either the data or the code.

[nobarany]

Thanks for looking into it! Yep, should be related to those changes, here's the simplified of the code

  chart = Plot.plot({
    y: {
      grid: true,
      zero: true,
    },
    x: {
      type: 'utc',
    },
    marks: [
      Plot.rectY(
        rows,
        Plot.binX(
          { y: 'sum' },
          {
            x: xProp,
            y: yProp,
            thresholds: [4, 3, 2, 1, 0].map(q =>
              startOfQuarter(currentQuarter() - q)
            ),
            fill: (d: { special: boolean }) =>
              d.special ? '#ccc' : 'blue',
            inset: 1,
          }
        )
      ),
    ],
  });

[mbostock]

Can you share the data please too?

[mbostock]

And the rest of the code? A link to a working notebook would be best. Thanks.

[nobarany]

Ok, here's the notebook. This helped me fix my chart. The data after the last threshold is not shown. So adding another threshold is a workaround. (the fix is commented out in the notebook)

[mbostock]

Here is a minimal reproduction:

Plot.plot({
  y: { grid: true },
  marks: [
    Plot.rectY(
      [
        [3, 1],
        [4, 1],
        [5, 2],
        [6, 3],
        [7, 4],
        [8, 5],
        [9, 4],
        [10, 3],
        [11, 2],
        [12, 1],
        [13, 1],
        [14, 2],
        [15, 3]
      ],
      Plot.binX(
        { y: "sum" },
        { x: ([x]) => x, y: ([, y]) => y, thresholds: [1, 4, 7, 10, 13] }
      )
    )
  ]
})

In 0.6.1 this renders as:

In 0.6.2 and later this renders as:

[mbostock]

In the old behavior, the first bin starts at the lowest x value in the data (3) rather than the lowest threshold (1). And likewise the last bin ends at the highest x value in the data (15) rather than the highest threshold (13); an extra bin covers the range beyond the last threshold (13–15).

Whereas in the new behavior, n thresholds specify n - 1 bins. Values below the lowest threshold or above or equal to the highest threshold are effectively discarded.

I think the new behavior is what we want. However the README describes the opposite behavior, where n thresholds specify n + 1 bins. We should fix this.

A problem with the old behavior is that the extreme bins can have an irregular width. And it’s not possible to “nice” these extreme bins because we shouldn’t assume that when arbitrary thresholds are provided that these thresholds are uniformly spaced; for example when using a UTC scale, months have different lengths. In the worst case, the extreme bins can have zero width be invisible. There’s a history of these fiddly issues with binning:

• Fix zero-width last bin. d3/d3-array#177
• In d3.histogram, the rightmost bin by default thresholds is too wide d3/d3-array#65

Glad you were able to fix this. And thank you for the report!

[nobarany]

Thank you for the explanation, and help. In practice either behavior is perfectly fine, but in terms of ease of reasoning, seems to me that one threshold/point should be able to break a line into two.

[mbostock]

FWIW The idiomatic recommendation for your case is thresholds: d3.utcMonth.every(3), which is much simpler than calculating the thresholds by hand. And since #1205 we could add thresholds: "quarter" to make this even simpler.