Skip to content

fix(control-plane): coalesce dates for timelines - #3516

Open
mdbenito wants to merge 1 commit into
vectorize-io:mainfrom
mdbenito:control-plane/coalesce-times-for-entities
Open

fix(control-plane): coalesce dates for timelines#3516
mdbenito wants to merge 1 commit into
vectorize-io:mainfrom
mdbenito:control-plane/coalesce-times-for-entities

Conversation

@mdbenito

@mdbenito mdbenito commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

The entity timeline in the control plane plotted each memory only by occurred_start, a content date the LLM sets on a small fraction of memories, so most entities showed "No Timeline Data" despite every memory carrying mention and ingest timestamps.

This PR makes it so that timelines plot each row by an effective date computed in the UI as occurred_startmentioned_atoccurred_endcreated_atevent_date, replicating the backend's COALESCE order for a unit's effective time and extending it with system fields that are always populated. Only rows with no usable date at all are excluded. Filtering, sorting, grouping, and labels all use this date.

Additionally:

  • the date partition and granularity grouping are extracted into pure helpers (with unit tests)\
  • the empty-state message was reworded across locales
  • the graph endpoint's non-ISO YYYY-MM-DD HH:MM date is parsed explicitly as UTC since it falls outside the ECMAScript date grammar.

Note: The backend actually shows two different date policies (this PR follows the 2nd one):

  1. "Effective time": COALESCE(occurred_start, mentioned_at, occurred_end), used by temporal retrieval (retrieval.py::_coalesce_date) and recency reranking (reranking.py:156, but inlined instead of calling the helper).
  2. Display/any-time: the extended chain including created_at/event_date, used by get_memories_timeseries and now by the timeline view.

This PR adds more behaviour by handling the graph endpoint's non-ISO date format, which no backend helper currently does, but it does mean the display chain now exists in three places.

Two follow-ups worth considering:

  1. Move _coalesce_date to a shared module and have reranking.py use it, eliminating the inline duplicate.
  2. Compute effective_date server-side on /list and /graph endpoints

The timeline only plotted rows with occurred_start, a content date the
LLM sets on a fraction of memories, so most entity timelines rendered
"No Timeline Data". Coalesce the unit's effective time like the backend
does (COALESCE(occurred_start, mentioned_at, occurred_end)), extended
with created_at and the legacy event_date, so every row with any usable
date appears; reword the empty-state message across locales.

Extract date partition and granularity grouping into pure helpers with
unit tests. The graph endpoint's space-separated "YYYY-MM-DD HH:MM"
date is parsed explicitly as UTC because it falls outside ECMA-262's
date grammar.
@nicoloboschi

Copy link
Copy Markdown
Collaborator

Thanks for digging into this — the diagnosis is right (occurred_start alone drops most rows), but I don't think coalescing is the right fix.

Collapsing five fields into one hidden chain conflates two genuinely different questions: when did this happen (occurred_*, content dates) and when did we hear about it (mentioned_at / ingest). After this change a timeline is a silent mix of both, and there's nothing in the UI telling the user which date any given row was plotted by. A memory dated by its real content date looks identical to one dated by ingest time — so the timeline reads as "when things happened" while mostly showing "when we ingested them". That's worse than an honest empty state, because it's wrong in a way the user can't see.

What I'd rather have is an explicit date-basis selector on the timeline — occurred vs mentioned (and ingest, if useful) — so the user picks the axis and knows what they're looking at. Both views are legitimate and answer different questions; the user should choose, not the code.

We already have this pattern: getMemoriesTimeseries(bankId, period, timeField) takes "created_at" | "mentioned_at" | "occurred_start" and the stats charts expose it. The timeline should follow the same shape rather than inventing a hidden precedence order.

Concretely:

  • Add a selector next to the granularity control, defaulting to mentioned_at (that's the one that's actually populated, so it fixes the "No Timeline Data" complaint immediately).
  • Plot strictly by the selected field. Rows lacking it go to the existing "N without dates" counter — which then becomes meaningful again: on the "occurred" view it honestly tells you how few memories carry extracted content dates, which is real, useful signal that coalescing hides.
  • Keep the empty-state copy tied to the selected basis ("No memories have occurred dates — try Mentioned").

Most of this PR survives that change: extracting the partition/grouping logic into pure tested helpers in effective-date.ts is a clear improvement and I'd keep it as-is. It's effectiveDate() itself that would become dateFor(row, basis).

Two smaller things, in case some form of this lands anyway:

  1. created_at is a dead link in the chain. The only caller is entities-view.tsxclient.listMemories()/api/list, and that item type has no created_at (src/lib/api.ts:1022-1040). So the effective chain is occurred_start → mentioned_at → occurred_end → date, and both mentioned_at and event_date are nullable (event_date lost its NOT NULL in migration aa2b3c4d5e6f). The "always-populated system fields" rationale in the doc comment doesn't hold as written — what actually saves it is writes.py:80 setting event_date = occurred_start ?? mentioned_at.
  2. The SPACE_FORMAT graph-date branch is currently unreachable — nothing feeds the graph endpoint's table_rows into TimelineView. Fine to keep if /graph is next, but worth noting it's untested against a real caller.

And I agree with your follow-up #2 in the description: computing this server-side is the better long-term home. If /list returned the basis fields explicitly, the client would just pick one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants