Skip to content

Phase 3.3 + 3.4 — Period rollup sensors + named comparator drill-in#80

Merged
Artic0din merged 20 commits into
devfrom
phase-3.3-period-rollups
May 18, 2026
Merged

Phase 3.3 + 3.4 — Period rollup sensors + named comparator drill-in#80
Artic0din merged 20 commits into
devfrom
phase-3.3-period-rollups

Conversation

@Artic0din
Copy link
Copy Markdown
Owner

@Artic0din Artic0din commented May 18, 2026

Summary

Combined Phase 3.3 + 3.4 (originally PRs #74 and #75, auto-closed when their stacked bases were deleted on the #73 merge). Phase 3.3 adds 15 period rollup sensors, Phase 3.4 adds named comparator drill-in (5 more sensors).

#73 (Phase 3.2) already merged into dev.

Changes

  • Phase 3.3: cdr/rollup.py + PeriodRollupSensor base + 3 subclasses + 15 sensors
  • Phase 3.4: OptionsFlow named_comparator step + CdrPlanProvider registration + 5 named rollup sensors
  • CR rounds 1-4 fixes applied across both phases

Test plan

🤖 Generated with Claude Code

Summary by Sourcery

Implement multi-plan history backfill with rollup sensors and a named comparator drill-in for PriceHawk, replacing the Amber-specific backfill and wiring new config, coordinator, and dashboard-facing entities.

New Features:

  • Add a universal HA-history backfill pipeline that replays grid-power history through the current CDR plan and ranked alternatives, exposing status via a BackfillStatus sensor and a dedicated backfill service entry point.
  • Introduce period rollup sensors that aggregate current-plan cost, best-alternative cost, savings, and an optional named comparator across today, week, month, 3-month, and year windows.
  • Add a named comparator configuration step and provider that lets users pin a specific CDR plan from the ranked alternatives for continuous comparison and rollup display.

Enhancements:

  • Refactor backfill logic into pure history replay helpers and a thin HA adapter, removing Amber-API-specific backfill paths in favour of evaluator-based multi-plan replay.
  • Extend the coordinator with helpers and state to orchestrate backfill runs, construct plan sets for replay, and persist additional provider (named comparator) state across restarts.
  • Update services, strings, translations, and planning documentation to describe the new backfill, rollup, and named comparator behaviour and surface them cleanly in Home Assistant.

Tests:

  • Add extensive unit tests for history replay and rollup helpers, including slot generation, window filtering, alternative selection, and savings calculation.
  • Add tests for the rewritten backfill adapter, HA recorder mocking, and coordinator helpers for building plan sets and named comparator providers.
  • Add tests covering the new OptionsFlow named comparator step and sensor-facing property behaviour for backfill status and rollup sensors.
  • Extend test fixtures and conftest HA module mocks to support recorder imports and the new coordinator and sensor behaviours.

Summary of Changes

Key Additions

  • Phase 3.3 — Period rollup sensors: Added 15 rolling-window cost sensors that report current-plan cost, best-alternative cost, and savings across five windows (today, week, month, 3-month, year), backed by daily_cost_history

    • New pure-logic cdr/rollup.py module with window filtering, summation, best-alternative selection (with deterministic tie-breaking), and savings computation
    • New PeriodRollupSensor base class and three concrete subclasses (CurrentCostRollupSensor, BestAlternativeRollupSensor, SavingsRollupSensor) for rolling-window monetary aggregation
    • last_reset set only for today window (midnight reset); rolling windows leave it unset
    • Sparse-data handling returns None/unknown until sufficient history accrues
  • Phase 3.4 — Named comparator drill-in: Added support for user-pinned "named comparator" plan with conditional sensor registration

    • New OptionsFlow step (named_comparator) with dropdown menu to select and pin a ranked alternative plan
    • New NAMED_COMPARATOR_CLEAR_SENTINEL constant and plan_named_comparator_step() helper to manage plan selection logic
    • Coordinator helper build_named_comparator_provider() to construct a CdrPlanProvider from persisted plan data
    • Coordinator now creates and manages a stable "named" provider registered under fixed key
    • Five conditional NamedComparatorRollupSensor entities (one per window) registered only when a named provider exists
    • State persistence: named comparator provider stored/restored across restarts
  • Test coverage: Extensive unit tests for rollup logic, config flow named-comparator step, coordinator helpers, and sensor behavior (808 tests passing)

Configuration & Constants

  • Added CONF_NAMED_COMPARATOR_PLAN_ID and CONF_NAMED_COMPARATOR_PLAN constants for persisting plan selection and full plan body
  • Updated strings.json with new Options menu entry and sensor entity names for named comparator costs
  • Updated translations/en.json with UI step text, abort messages, and sensor label translations

Breaking Changes

None identified. The PR maintains backward compatibility (legacy Amber seeds retained where needed).

Files Changed

File Added Removed Total
CHANGELOG.md 105 0 105
custom_components/pricehawk/cdr/rollup.py 206 0 206
custom_components/pricehawk/config_flow.py 160 0 160
custom_components/pricehawk/const.py 18 0 18
custom_components/pricehawk/coordinator.py 73 0 73
custom_components/pricehawk/sensor.py 264 1 263
custom_components/pricehawk/strings.json 76 0 76
custom_components/pricehawk/translations/en.json 76 0 76
tests/test_config_flow_phase_3.py 243 1 242
tests/test_coordinator_helpers.py 73 0 73
tests/test_review_improvements.py 130 0 130
tests/test_rollup.py 302 0 302
Total 1,746 2 1,744

Review Change Stack

Artic0din and others added 18 commits May 17, 2026 21:37
11-commit execution plan covering universal HA-history backfill (3.2),
period rollup sensors (3.3), named comparator drill-in (3.4), and
dashboard rewrite (3.5). Locks architectural decisions up front
(daily_cost_history as single source of truth, named comparator as
just another CdrPlanProvider, rollup as computed-on-demand) so the
executing model doesn't re-derive them at each commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds ``cdr/history_replay.py`` with three pure functions that establish
the public API surface Phase 3.2 commit 2 wires the coordinator to.
No HA imports — the module is unit-testable outside the integration
runtime, matching the ``cdr/ranking_job.py`` pattern.

Public API:
  - ``states_to_half_hour_slots`` — converts raw (ts, power_w, unit)
    tuples to evaluator-shaped slot dicts aligned to 30-min boundaries.
    Handles kW→W unit conversion, gap-protection clamping (6 min cap
    matches ``cdr/streaming.py``), and string-vs-float power values
    (HA's recorder serialises some sensor states as strings).
  - ``replay_day_through_plan`` — wraps ``evaluate()`` with the
    standard exception-swallow pattern (mirrors ``deep_rank``).
    Returns None on evaluator exception OR zero slot count.
  - ``fan_out_replay`` — generator yielding (date_str, {plan_key: aud})
    per day. Streaming output keeps peak RAM at ~one day × N plans
    instead of all-days × N plans of full CostBreakdown objects.

25 tests covering boundary alignment, unit conversion, gap protection,
sign handling, plan-failure isolation, date-order preservation, and
opt-in entry_options pass-through. Pattern follows
``tests/test_coordinator_ranking.py``: stdlib only, no pytest-asyncio.

Foundation for Phase 3.2 commit 2 (backfill.py rewrite) and Phase 3.3
(rollup sensors read the daily_cost_history rows this module helps
populate).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… adapter

Rewrites ``backfill.py`` from a 369-LOC Amber-API-coupled module into
a thin coordination layer over the pure-logic ``cdr.history_replay``
fan-out. New public API:

    backfill_daily_cost_history(hass, grid_sensor_entity, plans,
        *, days_back=30, entry_options=None, existing_history=None)

Internals pull recorder history day-by-day (NOT one big query — a
30-day single ``state_changes_during_period`` on a 1Hz grid sensor
returns 100K+ State objects), convert to evaluator slots, fan out
across N plans via the streaming generator, and merge per-date rows
into the coordinator's ``daily_cost_history``. Final list is capped
at 180 entries (matches live coordinator slice).

Day-by-day queries are deliberate and NOT parallelised. HA's
recorder uses a single executor pool so concurrent queries serialise
anyway and just bloat task count. Per-query memory is bounded; the
SQLite index on ``last_changed`` means 30 small queries are not
meaningfully slower than 1 big one. This is commented inline so CR
doesn't suggest parallelisation.

Legacy ``fetch_amber_price_history`` retained — still used by
``coordinator._replay_amber_today_from_api`` to seed the Amber
accumulator on a fresh install. The Phase 3.2 backfill itself no
longer fetches Amber prices: Amber's role narrowed to a *truth
overlay* written once daily by the live coordinator rollover.

Test rewrite: 14 legacy Amber-API tests deleted (those exercised
``backfill_from_history``, ``_build_amber_price_index``,
``_find_amber_rate``, ``_parse_history_states`` — all removed).
14 new tests cover ``_local_date_string`` (AEST-safe formatting),
``_states_to_tuples`` (State + dict shapes), ``_merge_into_history``
(insert/merge/cap), and ``backfill_daily_cost_history`` end-to-end
with the recorder mocked at the import boundary.

``tests/conftest.py`` extended with ``homeassistant.components``,
``homeassistant.components.recorder``, and
``homeassistant.components.recorder.history`` mocks so the lazy
recorder import inside the backfill resolves under the test harness.

``__init__.py``'s ``handle_backfill`` service updated to call the
new API with the current CDR plan; commit 4 will shrink it further
to a one-line delegate through the coordinator (once the wrapper +
status sensor land).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ickoff

Wires the new universal HA-history backfill into the coordinator and
schedules its first run automatically after the initial ranking job
completes.

Coordinator additions:
  - ``build_backfill_plan_set`` — module-level pure helper (mirrors
    the ``cdr.ranking_job`` pattern) composing the {plan_key:
    plan_body} dict from current plan + top-K ranked alternatives.
    Lives outside the class so it's unit-testable without HA's app
    context (the coordinator's ``DataUpdateCoordinator[T]`` base
    gets mocked away by ``tests/conftest.py``).
  - ``_build_backfill_plan_set`` — thin instance wrapper.
  - ``async_run_backfill`` — coordinator-side run wrapper modelled on
    ``async_run_ranking_job``. Status-tracked via ``_backfill_status``
    state machine (``idle | running | complete | failed``) plus
    ``_backfill_last_run_at``, ``_backfill_days_loaded``,
    ``_backfill_plans_replayed``, ``_backfill_error`` attributes
    that the Phase 3.2 commit 4 status sensor will surface.
  - Reuses ``_ranking_lock`` to serialise against the ranking job —
    both mutate ``_daily_cost_history``. REVISIT: split if contention
    observed in prod (cost of being wrong is brief serialisation of
    two rare operations).
  - Local import of ``backfill_daily_cost_history`` inside
    ``async_run_backfill`` (``# noqa: PLC0415``) so the HA recorder
    isn't loaded at module-import time. Matches the existing pattern
    at ``_replay_amber_today_from_api``.

``__init__.py`` kickoff:
  - After ``async_run_ranking_job`` task is scheduled, schedule a
    second task that AWAITS the ranking lock (so the first ranking
    run finishes and the alternatives list is populated) THEN runs
    ``async_run_backfill(days_back=30)``. Without the wait, the
    first backfill would replay history through only the current
    plan, missing the ``alt_*`` columns.

Tests: 7 new in ``test_coordinator_helpers.py`` covering the pure
helper — current-plan composition, alt_* prefix keying, plan-cache
fallback to alt body, malformed-input skipping, empty/non-dict
``cdr_plan`` graceful return. Pattern matches the existing
``_extract_peak_rate_c_inc_gst`` test block in the same file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ne-liner

Final Phase 3.2 commit. Adds the status sensor users will read from
their dashboards / automations and collapses the
``pricehawk.backfill_history`` service handler to a one-line delegate
through ``coordinator.async_run_backfill``.

New entity:
  - ``sensor.pricehawk_backfill_status`` — state machine read-through.
    State: ``idle | running | complete | failed``. Attributes:
    ``last_run`` (ISO timestamp), ``days_loaded``, ``plans_replayed``,
    ``error``. All sourced from the coordinator's status attributes
    written by ``async_run_backfill``.

Service handler:
  - ``handle_backfill`` shrunk from a 60-LOC inline pipeline to
    defensive ``days`` coercion + a single
    ``await coordinator.async_run_backfill(days_back=...)`` call.
    Status tracking, recorder pulls, plan composition, and
    persistence all happen inside the coordinator method now;
    failures surface on the sensor rather than getting lost to log
    lines.

Service description: updated ``services.yaml`` to reflect the new
flow (replay-through-CDR-plan, no Amber API, status sensor pointer).
Unused ``CONF_GRID_POWER_SENSOR`` import removed from
``__init__.py`` (the coordinator now owns the lookup).

Tests: 4 new BackfillStatusSensor smoke tests in
``test_review_improvements.py`` exercising the property-read contract
(state defaults to idle, running propagates, datetime → ISO, error
attribute surfaces on failed runs). The sensor class itself can't be
imported under the conftest mock tree (CoordinatorEntity +
SensorEntity multiple inheritance from MagicMocks triggers a
metaclass conflict), so the tests mirror the EXACT property bodies
inline. Integration test on Ryan's HA will catch any drift.

CHANGELOG.md updated under ``[Unreleased]`` documenting the full
Phase 3.2 surface area: new module, rewritten backfill, status
sensor, service signature change, recorder mocks in conftest.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`target = by_date.get(date_str)` infers as `dict[str, Any] | None`
but the subsequent `target[plan_key] = aud` assignment narrows
incorrectly without an explicit annotation. Annotate the union so
pyright accepts both branches (None → fresh dict, dict → mutate).

No behaviour change; pytest 760/760 still passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mock

Two docstring corrections flagged by Sourcery — no behaviour changes.

1. `build_backfill_plan_set` previously claimed it returned `{}` when
   the current plan was missing. The implementation actually returns
   alternatives even without a current plan (alts-only backfill is
   intentional so rollup sensors can still surface comparative data).
   Updated the docstring to describe the real contract: callers must
   treat the absence of the current-plan entry as a "no-signal"
   condition for the active plan at that time.

2. `_patch_recorder` claimed a 3-tuple return
   `(get_instance_mock, history_call_mock, dt_util_now_mock)` but the
   helper actually returns a 2-tuple `(get_instance, history_mock)`
   (the dt_util mock was removed earlier when the backfill stopped
   depending on it). Fixed the docstring and added a precise
   `tuple[MagicMock, MagicMock]` return annotation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CodeRabbit round-2 fixes for PR #73:

- coordinator.async_run_backfill: ``_backfill_days_loaded`` now reports
  the delta (new days added by THIS run), not the total merged-history
  length. Capture ``prev_len`` before reassignment, compute
  ``new_days = max(0, len(result) - prev_len)``, clamp negatives to
  zero. Return value and log line updated to match. Docstring spells
  out the delta semantics so future callers don't misread the API.
- tests/test_review_improvements.py::TestBackfillStatusSensor: add
  ``-> None`` return type hints to the four public test methods so
  mypy strict-mode is satisfied. Underscored helpers (_coord,
  _native_value, _attrs) left as-is.
- .planning/PHASE-3.2-to-3.5-PLAN.md: section 1.5 was documenting
  ``error_message`` but the implementation uses ``error`` (matches
  sensor.py:571 + tests). Sync the plan to the code. Also defang the
  literal insecure-WebSocket scheme in section 8.4 (``ws-//`` with a
  parenthetical note) so the dashboard-protocol-safety recipe and
  generic CI security scans don't flag the plan doc itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CodeRabbit round-3:
- Replace literal ws://localhost:* token in PHASE-3.2-to-3.5 plan doc
  with descriptive prose so secret/security scanners stop tripping.
- Short-circuit async_run_backfill on _backfill_status BEFORE acquiring
  _ranking_lock so a concurrent caller does not block. Keep the
  re-check inside the lock to close the read/acquire race.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tadata reset

CodeRabbit round-4 fixes on PR #73:

- Plan doc: replace literal absolute `/Users/.../pricehawk/` path with
  `<REPO_ROOT>` placeholder so the planning markdown isn't tied to one
  user's machine layout.
- Plan doc: fix arithmetic typo in commit total. 4+3+2+3 = 12, not 11.
- Coordinator `async_run_backfill`: reset stale success metadata
  (`_backfill_days_loaded`, `_backfill_plans_replayed`) in the failure
  path so the status sensor doesn't surface misleading counts from a
  prior successful run after a failure. Set `_backfill_last_run_at` to
  the time of THIS (failed) run, matching success-path semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…der lookup

Round-5 CR doc nits on the planning artifact:
- Add `text`/`bash` language tags to 3 fenced code blocks (dependency
  graph, dashboard ASCII layout, pre-push checks) for MD040 compliance.
- Fix test description: kW→W is multiply-by-1000, not "doubles".
- Plan snippet's named-provider check should read `coordinator._providers`
  (the instance dict) not `coordinator.data["providers"]`. Matches the
  shipped code in Phase 3.4's sensor.py registration.

Plan doc only; no shipped code changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `cdr/rollup.py` with four pure functions over `daily_cost_history`:
`filter_window` (rolling window selection), `sum_window` (per-key sum
with sparse-row tolerance), `best_alternative_for_window` (lexicographic
tie-break for determinism across ticks), and `savings` (current minus
best, sign-preserving).

No HA imports. Floats throughout. Returns `(None, 0)` rather than
`(0.0, 0)` for missing-data states so the sensor displays `unknown`
rather than misleading `$0.00`. 27 stdlib-only tests covering empty
history, malformed dates, sparse alt presence, string-coerced numerics,
explicit-zero days, ties, and prefix-based alt key scanning.

Wires the source of truth that Phase 3.3 commit 2 will bind 15 rollup
sensors to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tions

Add ``PeriodRollupSensor`` base + three subclasses
(``CurrentCostRollupSensor``, ``BestAlternativeRollupSensor``,
``SavingsRollupSensor``) reading from ``daily_cost_history`` via
``cdr.rollup``. Three kinds × five windows = 15 new entities
(``sensor.pricehawk_{current,best_alt,savings}_cost_{today,week,month,3month,year}``).

Per plan §3.1 "Surprise risk — last_reset": only the ``today`` window
sets ``last_reset`` to midnight. Rolling week/month/3month/year leave
it unset — HA's TOTAL state-class tolerates this for monotonic-with-
occasional-corrections series, and setting an artificial midnight
reset would falsely re-attribute the prior day's value as today's spend.

Per plan §8.6: no ``RollupStrategy`` interface; the three kinds are
dispatched by inline ``if self._ROLLUP_KIND`` in the base class.
Floats throughout (no Decimal). Lazy ``from .cdr.rollup`` import
inside ``native_value`` (``# noqa: PLC0415`` annotated) to keep the
import-time footprint identical to pre-3.3.

3 sensor smoke tests added to ``tests/test_review_improvements.py``,
following the same property-body mirror pattern used by Phase 3.2's
``BackfillStatusSensor`` tests (CoordinatorEntity + SensorEntity
metaclass conflict prevents direct construction under the conftest
mock tree).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the ``entity.sensor.*`` block to both ``strings.json`` and
``translations/en.json`` for all 15 Phase 3.3 rollup sensors (3 kinds
× 5 windows). Names + descriptions disambiguate the new
``savings_cost_today`` rollup from the existing real-time
``saving_today`` sensor (different math, both valid).

CHANGELOG entry under [Unreleased] documents:
- The new pure-logic module + 15 sensors.
- ``last_reset`` semantics (today-only midnight reset).
- The distinction from the legacy ``saving_today`` sensor.
- The "sparse data → ``None`` rather than ``$0.00``" contract.

No new tests in this commit — the rollup logic is covered by 27 tests
in ``test_rollup.py`` (commit 1/3) and the sensor dispatch by 3 smoke
tests in ``test_review_improvements.py`` (commit 2/3). Translations
are static JSON.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… deref

Two CodeRabbit findings — both add defensive guards without changing
the happy-path behaviour.

1. `cdr.rollup.filter_window` iterates `history` and calls `.get("date")`
   on each row unconditionally. The typed signature says
   `list[dict[str, Any]]`, but restored state from `.storage` or 3rd-
   party callers can slip in scalars (a corrupted entry, a future
   schema change). A non-dict row raises AttributeError and crashes
   every sensor read that hits this codepath. Skip non-dict rows up
   front; the rest of the parse/window logic is unchanged.

2. `PeriodRollupSensor.native_value` reads
   `self.coordinator._current_plan_provider.id` for the "current" and
   "savings" branches without guarding. The coordinator's __init__
   raises ConfigEntryNotReady when `cdr_plan` is missing, so the
   attribute *should* always exist — but restart races, partial
   restore, or mocked coordinators in tests can briefly land here
   without it. Added an upfront guard that returns None (sensor
   shows `unknown`) instead of raising AttributeError. The "best_alt"
   branch is unchanged — it doesn't deref the provider.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rd tests

CodeRabbit round-2 fixes for PR #74:

- strings.json + translations/en.json: removed the ``description`` key
  from all 15 rollup ``entity.sensor`` entries. HA's entity schema
  accepts ``name`` only — ``description`` was silently ignored and
  triggered translation-schema warnings on startup. The descriptive
  copy lives in PLAN.md and CHANGELOG.md where it belongs.
- tests/test_review_improvements.py::TestPeriodRollupSensorSmoke: the
  Phase 3.3 defensive guard added in sensor.py:657-660 (returns
  ``None`` when ``_current_plan_provider`` is missing or has no
  ``id``) had no test coverage. Mirror the guard in the
  ``_native_value`` helper and add two tests:
    - test_current_rollup_returns_none_when_provider_missing
    - test_savings_rollup_returns_none_when_provider_missing
  Both exercise the ``today`` and ``week`` windows so we cover the
  guard regardless of which window the user lands on first after a
  restart race or partial restore.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CodeRabbit round-3:
- best_alternative_for_window: skip rows where the alt key is the
  bare prefix (``"alt_"`` with no plan id suffix) so a malformed
  history row cannot be ranked as the cheapest plan.
- Add test_sum_window_handles_negative_values to cover FIT credits
  and refund rows summing alongside positive costs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(coordinator): Phase 3.4 commit 1/2 — named comparator wiring

Adds an OptionsFlow step that pins ONE CDR plan from the current ranked
alternatives as a "named comparator", and wires the coordinator to
construct a second CdrPlanProvider for that plan under the literal
`"named"` key in `_providers`. The named provider then participates in
the existing 30s tick loop (no new tick path) and contributes a
`"named"` column to `daily_cost_history` at daily rollover, which the
Phase 3.4 commit 2 rollup sensors will read.

- `const.py`: add CONF_NAMED_COMPARATOR_PLAN_ID + CONF_NAMED_COMPARATOR_PLAN.
  We persist the FULL PlanDetailV2 body (not the summarised form) because
  the evaluator needs tariffPeriod data the summary omits.

- `config_flow.py`: new `named_comparator` menu entry + step. The step
  reads ranked_alternatives + the per-day _ranking_plan_cache directly
  from the coordinator. Aborts with `no_ranked_alternatives` if either
  is empty (covers the post-install + post-midnight-cache-reset edge
  cases per plan §4.2 #1 + #3). Decision tree extracted to
  `plan_named_comparator_step()` so it's unit-testable without HA's
  app context (the OptionsFlow class itself becomes a MagicMock under
  the conftest mock tree).

- `coordinator.py`: new `build_named_comparator_provider()` module-level
  helper (same testability rationale as build_backfill_plan_set). Called
  from both __init__ AND rebuild_engine so a fresh pin lands on the
  next OptionsFlowWithReload cycle without an HA restart.

- `strings.json` + `translations/en.json`: new step title/description +
  menu label + two abort reasons. Keep distinct from the existing
  `comparators` step (which toggles live-API providers, not CDR pins).

- 22 new tests: 10 in test_config_flow_phase_3 (full decision-tree
  coverage including the full-body persistence guard, dedupe, default
  fallback when prior pin evicted from cache, plan_not_in_cache
  branch), 4 in test_coordinator_helpers (lifecycle of the named
  provider helper across all option shapes).

Lock interaction with ranking lock: none. The named comparator joins
the existing tick loop unchanged. The OptionsFlow step is a READ from
_ranking_plan_cache, which is only written under the ranking lock —
reading without the lock is safe because (a) the worst case is a brief
torn read that resolves to the abort path, and (b) the lock is held
for the duration of the ranking pipeline run, not just dict mutation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sensor): Phase 3.4 commit 2/2 — NamedComparatorRollupSensor × 5

Surfaces the Phase 3.4 commit 1/2 wiring as user-visible HA sensors:

- `sensor.pricehawk_named_cost_today`
- `sensor.pricehawk_named_cost_week`
- `sensor.pricehawk_named_cost_month`
- `sensor.pricehawk_named_cost_3month`
- `sensor.pricehawk_named_cost_year`

Subclasses the Phase 3.3 `PeriodRollupSensor` base; overrides
`native_value` + `extra_state_attributes` to read the `"named"` key
from `daily_cost_history` rather than extend the base's
`_ROLLUP_KIND` dispatch enum (which Phase 3.3 just shipped — localise
the new behaviour here instead of rewriting the base contract for
one extra kind).

Registration is CONDITIONAL on `"named" in coordinator._providers`,
so users who haven't pinned a plan don't see 5 permanently-
unavailable entities clutter their HA UI. Reads `_providers` directly
(not `data["providers"]`) so the registration check fires on first
setup before the coordinator has populated its data dict.

CHANGELOG.md gains a Phase 3.4 block under [Unreleased] documenting
the OptionsFlow step, the new sensors, lock/ranking interaction (none),
and the persistence-through-rank-churn behaviour (the pin survives the
plan dropping out of cheap-rank top-K because it lives in options,
not in derived ranking state).

strings.json + translations/en.json gain entity blocks for the 5 new
sensors. Descriptions call out the tick-by-tick cadence difference vs
the other ranked alternatives (which only refresh at daily rollover)
so users know what they bought by pinning.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(named): persist provider across restart + skip unique_id collision

Two CodeRabbit findings on the Phase 3.4 named comparator.

1. The named provider added to `self._providers["named"]` was
   constructed in `__init__` and `_apply_options_update` but never
   persisted or restored. Every other CdrPlanProvider in the
   coordinator (current plan, amber, flow_power, localvolts) round-
   trips through `_async_persist_state` / `async_restore_state`, so
   on HA restart their per-day accumulators (import_cost_today, kwh)
   survive — but `named` would reset to zero while the others kept
   state, so today's rollup deltas would lie until midnight rollover.
   Added named.to_dict() to the persist block and a from_dict() call
   in restore, using `today = dt_util.now().date()` (already in scope
   above) to satisfy the AEGIS rule that from_dict MUST receive an
   explicit HA-timezone date (no `date.today()` fallback).

2. `GenericProviderCostSensor(provider_id="named")` produces
   unique_id `{entry}_named_cost_today`. `NamedComparatorRollupSensor`
   for the "today" window produces the same unique_id (since its
   _ROLLUP_KIND is "named" and the base class composes
   `{kind}_cost_{window}`). HA's entity registry drops the second
   registration silently, breaking whichever sensor the dashboard
   depends on first. Added a `provider_id == "named"` skip in the
   providers loop in `async_setup_entry` — the named comparator is
   exposed via its dedicated 5-window rollup family instead. Also
   skips the matching GenericProviderRateSensor pair so we don't
   register orphan rate sensors for the "named" key. (Only one such
   loop exists; the Phase 3.3 rollup loop and Phase 3.4 named-rollup
   loop don't have the same collision since their _ROLLUP_KIND
   differs.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented May 18, 2026

Reviewer's Guide

Implements Phase 3.2 (universal HA-history backfill via a new history_replay engine and coordinator-managed backfill), Phase 3.3 (period rollup sensors backed by a pure rollup module), and Phase 3.4 (named comparator drill‑in with OptionsFlow, provider wiring, and rollup sensors), plus associated tests, constants, status sensor, and changelog/plan docs.

File-Level Changes

Change Details Files
Replace Amber-specific backfill with a generic multi-plan HA-recorder backfill pipeline orchestrated by the coordinator, including a new history_replay module and backfill status tracking.
  • Introduce cdr/history_replay.py to convert recorder state tuples into 30-minute evaluator slots, group them by local date, replay per-day slots through multiple CDR plans, and stream per-day cost results via a generator.
  • Rewrite custom_components/pricehawk/backfill.py into a thin HA adapter that queries recorder day-by-day, converts State objects into tuples, builds per-day slot maps, calls fan_out_replay across the current plan set, and merges new rows into daily_cost_history with a 180-day cap.
  • Add coordinator.async_run_backfill, backfill status fields (_backfill_status, _backfill_last_run_at, _backfill_days_loaded, _backfill_plans_replayed, _backfill_error), and a helper to build the backfill plan set from current plan, ranked alternatives, and plan cache.
  • Auto-trigger an initial backfill after the first ranking job via init.py, and refactor the pricehawk.backfill_history service into a thin wrapper around async_run_backfill with input validation for days.
  • Extend tests (test_backfill.py, test_history_replay.py, coordinator helper tests, conftest recorder mocks) to cover the new pipeline, including recorder mocking, history merging, caps, and evaluator-failure handling.
custom_components/pricehawk/cdr/history_replay.py
custom_components/pricehawk/backfill.py
custom_components/pricehawk/coordinator.py
custom_components/pricehawk/__init__.py
tests/test_history_replay.py
tests/test_backfill.py
tests/test_coordinator_helpers.py
tests/conftest.py
custom_components/pricehawk/services.yaml
Add a BackfillStatusSensor and wire it to coordinator backfill state to expose backfill progress and errors to HA.
  • Define BackfillStatusSensor in sensor.py, reading coordinator._backfill_status as state and exposing last_run, days_loaded, plans_replayed, and error attributes.
  • Register BackfillStatusSensor in async_setup_entry so it is created alongside existing PriceHawk sensors.
  • Add smoke tests in test_review_improvements.py that mirror the sensor property logic against a MagicMock coordinator to pin the expected attribute contract.
custom_components/pricehawk/sensor.py
tests/test_review_improvements.py
Introduce a rollup engine and 15 rolling-window cost rollup sensors that summarise current plan cost, best alternative cost, and savings over several time windows.
  • Create cdr/rollup.py with WINDOW_DAYS, filter_window, sum_window, best_alternative_for_window, and savings helpers operating purely on daily_cost_history entries.
  • Add PeriodRollupSensor base and concrete CurrentCostRollupSensor, BestAlternativeRollupSensor, and SavingsRollupSensor classes that use cdr.rollup to compute values for today, week, month, 3month, and year windows, including last_reset semantics for today-only.
  • Register 15 rollup sensors (3 kinds × 5 windows) in async_setup_entry and ensure they rely solely on daily_cost_history and the current plan provider id.
  • Add pure rollup tests in tests/test_rollup.py plus sensor-level smoke tests in test_review_improvements.py that exercise the dispatch logic against mocked coordinator data.
custom_components/pricehawk/cdr/rollup.py
custom_components/pricehawk/sensor.py
tests/test_rollup.py
tests/test_review_improvements.py
custom_components/pricehawk/strings.json
custom_components/pricehawk/translations/en.json
Implement a named comparator feature that lets users pin a specific CDR plan via OptionsFlow, exposes it as a dedicated provider, and adds named rollup sensors.
  • Introduce CONF_NAMED_COMPARATOR_PLAN_ID and CONF_NAMED_COMPARATOR_PLAN constants and a pure helper build_named_comparator_provider to construct a CdrPlanProvider from options.
  • Extend the coordinator to build and register a named comparator provider under the fixed key "named" on init and on rebuild_engine, persist/restore its accumulator state, and include it in provider persistence.
  • Add a named_comparator OptionsFlow step planned by plan_named_comparator_step, which validates ranked alternatives plus plan cache, renders a dropdown including a clear-pin sentinel, persists full PlanDetailV2 bodies from the ranking plan cache, and handles abort cases when no alternatives or plan cache are available.
  • Add NamedComparatorRollupSensor as a PeriodRollupSensor subclass that sums the "named" column across the same 5 windows, register 5 such sensors only when a named provider is present, and avoid creating generic provider sensors for the "named" key to prevent unique_id collisions.
  • Add extensive tests in test_config_flow_phase_3.py for the named-comparator decision helper (option list composition, clear-pin behavior, cache-miss aborts, deduplication) and in test_coordinator_helpers.py for build_named_comparator_provider, plus coordinator restore/persist adjustments for the named provider.
custom_components/pricehawk/const.py
custom_components/pricehawk/config_flow.py
custom_components/pricehawk/coordinator.py
custom_components/pricehawk/sensor.py
tests/test_config_flow_phase_3.py
tests/test_coordinator_helpers.py
custom_components/pricehawk/strings.json
custom_components/pricehawk/translations/en.json
Update tests and documentation scaffolding for the new phases, including a detailed implementation plan and changelog entries.
  • Add .planning/PHASE-3.2-to-3.5-PLAN.md with a detailed phase-by-phase roadmap, architectural decisions, file-level breakdowns, and anticipated review issues.
  • Extend CHANGELOG.md with an Unreleased section describing Phase 3.2 (universal backfill), Phase 3.3 (period rollups), and Phase 3.4 (named comparator), including user-facing behavior and technical notes.
  • Adjust tests in test_review_improvements.py and test_config_flow_phase_3.py to cover new helpers and sensor contracts, and update conftest recorder mocks to support backfill imports.
  • Lightly tweak services.yaml description for backfill_history to reflect the new multi-plan CDR replay instead of the old Amber-API-based behavior.
.planning/PHASE-3.2-to-3.5-PLAN.md
CHANGELOG.md
tests/test_review_improvements.py
tests/test_config_flow_phase_3.py
tests/conftest.py
custom_components/pricehawk/services.yaml

Possibly linked issues

  • #unknown: PR adds the requested smoke tests verifying PeriodRollupSensor.native_value returns None when current-plan provider metadata is missing.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 18, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1ee67b1f-3c2d-4b5f-9a55-57bc8c2eae5f

📥 Commits

Reviewing files that changed from the base of the PR and between 2f19886 and ddb0cc4.

📒 Files selected for processing (12)
  • CHANGELOG.md
  • custom_components/pricehawk/cdr/rollup.py
  • custom_components/pricehawk/config_flow.py
  • custom_components/pricehawk/const.py
  • custom_components/pricehawk/coordinator.py
  • custom_components/pricehawk/sensor.py
  • custom_components/pricehawk/strings.json
  • custom_components/pricehawk/translations/en.json
  • tests/test_config_flow_phase_3.py
  • tests/test_coordinator_helpers.py
  • tests/test_review_improvements.py
  • tests/test_rollup.py

Walkthrough

This PR implements Phases 3.3 and 3.4 of the PriceHawk integration. Phase 3.3 introduces pure-logic rolling-window aggregation over daily cost history and a new period rollup sensor framework. Phase 3.4 adds a user-facing named-comparator drill-in UI to pin and monitor an alternative CDR plan, with full provider lifecycle support and comprehensive tests.

Changes

Phases 3.3 and 3.4: Rollup Sensors and Named-Comparator Drill-In

Layer / File(s) Summary
Pure-logic rolling-window aggregation
custom_components/pricehawk/cdr/rollup.py, tests/test_rollup.py
New cdr/rollup.py module defines rolling-window semantics (today/week/month/3month/year), filters daily cost history by date, sums plan costs with day-count tracking, selects the lowest-cost alternative with lexicographic tie-breaking, and computes savings deltas. Comprehensive pure-logic test suite validates filtering, numeric summation, alternative selection, and edge cases.
Period rollup sensor framework and registration
custom_components/pricehawk/sensor.py, tests/test_review_improvements.py
New PeriodRollupSensor base class dispatches rolling-window aggregation by _ROLLUP_KIND, provides window/day attributes, and sets last_reset only for today window. Concrete subclasses compute current-plan, best-alternative, and savings rollups. Named-comparator subclass aggregates the pinned plan separately. Registration skips "named" provider in generic loop; conditionally registers 15 standard rollups and 5 named-comparator rollups. Smoke tests validate aggregation logic and None returns.
Named comparator provider and lifecycle
custom_components/pricehawk/const.py, custom_components/pricehawk/coordinator.py, tests/test_coordinator_helpers.py
New constants CONF_NAMED_COMPARATOR_PLAN_ID and CONF_NAMED_COMPARATOR_PLAN in const.py. Coordinator build_named_comparator_provider() helper constructs a provider from options; __init__, async_restore_state(), async_persist_state(), and rebuild_engine() manage the named provider lifecycle. Provider is registered under fixed "named" key in _providers. Tests validate provider construction from valid/invalid options.
Named comparator options flow UI
custom_components/pricehawk/config_flow.py, tests/test_config_flow_phase_3.py
New NAMED_COMPARATOR_CLEAR_SENTINEL constant. Pure helper plan_named_comparator_step() validates ranked alternatives and plan cache, builds deduped dropdown from cached plans, and returns abort/form/create_entry. Async step async_step_named_comparator() fetches coordinator state, delegates to helper, and renders form or aborts. Integrated into OptionsFlow menu. Comprehensive tests cover abort conditions, dropdown rendering, selection validation, clear-pin behavior, full plan body persistence, and deduplication.
User-facing strings and translations
custom_components/pricehawk/strings.json, custom_components/pricehawk/translations/en.json
New menu item "named_comparator" with step schema. Abort messages for missing ranked alternatives and cache misses. Entity name translations for 20 rollup sensor variants (current/best-alt/savings/named across five windows).
Documentation
CHANGELOG.md
Phase 3.3 documents 15 rolling-window rollup sensors from pure-logic module, rollup base and subclasses, registration strategy, tests, and last_reset/sparse-data semantics. Phase 3.4 documents named-comparator drill-in, config constants, provider helper, "named" provider key, five named-comparator rollups, translations, and state persistence notes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • Artic0din/ha-pricehawk#75: Phase 3.3 rollup/sensor refactor that introduces the period rollup framework this PR extends and builds named comparator rollup sensors on.
  • Artic0din/ha-pricehawk#70: Adds coordinator.ranked_alternatives and _ranking_plan_cache data structures that the named-comparator options flow consumes for alternative dropdown rendering.
  • Artic0din/ha-pricehawk#68: Introduces daily ranking job that populates ranked alternatives and plan cache state that the named-comparator UI depends on.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch phase-3.3-period-rollups
  • 🛠️ scrub-secrets
  • 🛠️ no-hardcoded-rates
  • 🛠️ amber-api-limits
  • 🛠️ dashboard-protocol-safety

Comment @coderabbitai help to get the list of available commands and usage tips.

Artic0din and others added 2 commits May 18, 2026 12:39
#73 (Phase 3.2) was squash-merged to dev, collapsing phase-3.2's commit
history. phase-3.3 was branched off phase-3.2 pre-squash, so its history
duplicates dev's content. Took 'ours' for CHANGELOG.md, sensor.py, and
test_review_improvements.py — all are phase-3.3 supersets of the
phase-3.2 content now on dev.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Artic0din Artic0din merged commit d301dda into dev May 18, 2026
3 checks passed
@Artic0din Artic0din deleted the phase-3.3-period-rollups branch May 18, 2026 02:39
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @Artic0din, you have reached your weekly rate limit of 1500000 diff characters.

Please try again later or upgrade to continue using Sourcery

Artic0din added a commit that referenced this pull request May 18, 2026
#73 (Phase 3.2) and #80 (Phase 3.3 + 3.4) were squash-merged to dev.
phase-3.5 was branched off the pre-squash stack so its merge produces
duplicate-content conflicts only in CHANGELOG.md. Taking 'ours' —
phase-3.5's CHANGELOG is a superset (includes Phase 3.5 dashboard
rewrite entry on top of 3.2/3.3/3.4 content).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant