v1.5.0 — CDR-native engine + 8-retailer parsers + universal wizard

.coderabbit.yaml

@@ -18,9 +18,38 @@ reviews:
   sequence_diagrams: true
   changed_files_summary: true
 
+  slop_detection:
+    enabled: true
+    label: "slop"
+
   finishing_touches:
     docstrings:
       enabled: true
+    unit_tests:
+      enabled: true
+    custom:
+      - name: "scrub-secrets"
+        instructions: |
+          Audit changed files for hardcoded secrets: Amber API key, HA long-
+          lived tokens, any user-specific bearer tokens or JWTs. Replace
+          with HA config entry storage. Never commit a real value.
+      - name: "no-hardcoded-rates"
+        instructions: |
+          GloBird and other retailer tariff rates are user-specific. Never
+          hardcode rate values as defaults in source. Read from config flow
+          or user-supplied CSV. Flag any literal c/kWh number that looks
+          like a tariff rate.
+      - name: "amber-api-limits"
+        instructions: |
+          Calls to api.amber.com.au must respect: max 90 days history, max
+          7 days per request, max 50 req/5min per account. Flag loops that
+          could exceed this or missing backoff/retry.
+      - name: "dashboard-protocol-safety"
+        instructions: |
+          custom_components/pricehawk/www/dashboard.html MUST use
+          location.protocol for WebSocket URL detection. Never hardcode
+          ws://. Token must come from URL params or postMessage, never
+          hardcoded.
 
   auto_review:
     enabled: true

.gitignore

@@ -1,6 +1,10 @@
 __pycache__/
 *.py[cod]
 *.egg-info/
+.venv/
+venv/
+.codex/
+graphify-out/
 .vbw-planning/
 .claude/
 .agents/
@@ -18,3 +22,4 @@ PROGRESS.md
 /brand/
 logos/
 docs/superpowers/
+.startup.md

.planning/PHASE-3-ROADMAP.md

@@ -0,0 +1,128 @@
+# Phase 3 — Multi-Plan Pivot Roadmap
+
+Locked 2026-05-15 after Phase 2.12.1 ship + product-direction reset.
+
+## Why we pivoted
+
+Phase 2 architected PriceHawk as **one current retailer + one user-chosen comparison plan**, gated on "current retailer must have a live consumer API". User correction: that's the wrong shape.
+
+The actual product: **PriceHawk evaluates every CDR plan eligible for the user's geography (state + postcode + distributor) against their real meter data, ranks them, and surfaces the best alternatives.** API providers (Amber, Flow Power, LocalVolts) are optional truth-source overlays for users who happen to have them — they're not a gate.
+
+Phase 2 incentive parsers (the wedge feature — free-text math nobody else parses) are kept verbatim. Phase 2 orchestration (coordinator wiring, sensor schema, wizard) gets rewritten.
+
+## No migration
+
+Existing entries from Phase 2 are NOT migrated. User must remove + re-add. Justification: migration paths are bug surfaces; this is pre-1.0, expected disruption.
+
+## Phase order
+
+Sequence chosen to land foundation first, then layer features and polish.
+
+### Phase 3.0 — Unify under one evaluator (foundation)
+
+Every cost number flows through `evaluator.evaluate()`. API providers become optional truth-source overlays.
+
+**Files touched:**
+- `coordinator.py` — rip 4-provider dispatch; introduce `_current_plan_provider` (CdrPlanProvider) + optional `_truth_overlay`
+- `config_flow.py` — wizard: state → distributor → retailer → plan → [optional API connect] → done
+- `const.py` — drop `CONF_CURRENT_PROVIDER` enum semantics; keep PROVIDER_* only for truth-overlay identification
+- `providers/{amber,flow_power,localvolts}.py` — repurpose as truth-source overlays (override computed cost when connected)
+- `sensor.py` — drop per-provider sensor classes; introduce CurrentCostSensor, BestAlternativeSensor (placeholder), WinnerExplanation
+- `__init__.py` — entry setup flow + clean async_migrate_entry returning False
+- New: `cdr/ranking.py` skeleton (3.1 fills it)
+
+**Commits:** 5-8 small, each independently testable. Expected ~500 LOC delta.
+
+**Risk:** breaks ~50 existing tests (per-provider sensor tests, single-comparator coord tests). Replace as we go.
+
+### Phase 3.1 — Multi-plan ranking engine
+
+Daily job: filter CDR registry by user geography → cheap-heuristic top-K → deep-evaluate top-K → persist ranked list.
+
+**Files:**
+- `cdr/ranking.py` — eligibility + cheap-rank + deep-rank
+- `cdr/registry.py` — extend `eligible_plans_for(state, postcode, distributor)` query
+- `coordinator.py` — `async_track_time_change` at 00:30 local → ranking job
+- `__init__.py` — `pricehawk.rank_alternatives` service
+
+**Commits:** 4-6. ~400 LOC.
+
+**Heuristic:** rank by `peak_rate * 0.7 + daily_supply * 0.3` (no incentives, no FIT). Top-K=20 default, user-configurable.
+
+### Phase 3.2 — Universal HA-history backfill
+
+At wizard completion, replay HA grid-sensor history through current + top-K plans → populate `daily_cost_history` for full available lookback (HA recorder default: 10 days; longer if user has `purge_keep_days` raised).
+
+**Files:**
+- Rewrite `backfill.py` — generic replay-through-evaluator
+- New: `cdr/history_replay.py` — multi-plan wrapper
+- `__init__.py` — kick off backfill post-setup; surface `sensor.pricehawk_backfill_status`
+
+**Commits:** 3-4. ~300 LOC.
+
+**UX note:** if HA recorder retention is default 10 days, dashboard's "year" rollup will be sparse until 365 days of live data accrues. Surface this in setup.
+
+### Phase 3.3 — Period rollup sensors
+
+Day / week / month / 3-month / 12-month sensors for current + best-alt + savings.
+
+**Files:**
+- `sensor.py` — new `PeriodRollupSensor` class
+- New: `cdr/rollup.py` — rolling-window aggregate math
+
+**Sensor names:**
+- `sensor.pricehawk_current_cost_{today, week, month, 3month, year}`
+- `sensor.pricehawk_best_alternative_cost_{today, week, month, 3month, year}`
+- `sensor.pricehawk_savings_{today, week, month, 3month, year}`
+
+**Commits:** 3-4. ~250 LOC.
+
+### Phase 3.4 — Optional named comparator drill-in
+
+User pins ONE specific CDR plan as primary comparator; gets tick-by-tick computation (vs daily for auto-ranked alternatives).
+
+**Files:**
+- `config_flow.py` OptionsFlow — "named_comparator" step, skippable
+- `coordinator.py` — extends current_evaluator pattern with parallel `_named_comparator` evaluator running every tick
+- `sensor.py` — `named_comparator_cost_{...}` sensors
+
+**Commits:** 2-3. ~150 LOC.
+
+### Phase 3.5 — Dashboard rewrite
+
+HA Lovelace cards: current cost + ranked top-N alternatives + drill-in card.
+
+**Files:**
+- `www/dashboard.html` — rewrite
+- `dashboard_config.py` — entity exposure
+- `assets/DESIGN.claude.md` — design spec update
+
+**Commits:** 2-3. UI-only.
+
+## Cadence
+
+- 3.0 lands first (foundation; everything else depends on it)
+- 3.1 + 3.2 can develop in parallel after 3.0
+- 3.3 / 3.4 / 3.5 are independent polish layers, ship in any order
+
+## Totals
+
+| | |
+|---|---|
+| Phases | 6 |
+| Commits | 19-28 |
+| LOC delta | ~1,600 net |
+| Test count | 600 → ~750-800 |
+| Wall-clock | 2-3 weeks focused |
+
+## Held by user (not part of Phase 3)
+
+- "Dynamic wholesale pricing (Amber-style) for CDR plans" — would require CDR plans to publish half-hourly variable rates, which they don't. Defer until AER pushes a CDR amendment, or until we add a wholesale-overlay feature.
+
+## v1.5.1+ (post-Phase-3)
+
+Per TODOS.md:
+- TODO-5 demand charges (~10% AU plans currently silently wrong)
+- TODO-7 Flow Power Happy Hour FiT parser
+- TODO-8 plan-change diff notifications
+- TODO-9 plan-override YAML

AGENTS.md

@@ -0,0 +1,85 @@
+# PriceHawk — Energy Compare HACS Integration
+
+**Stack:** Python, Home Assistant custom integration (HACS)
+
+Compare real energy costs between [Amber Electric](https://www.amber.com.au) (wholesale spot pricing) and [GloBird Energy](https://www.globirdenergy.com.au) (time-of-use tariffs) using actual Home Assistant consumption data.
+
+## Project Context
+
+- **Target:** Home Assistant custom integration distributed via HACS
+- **Amber side:** Connects to Amber's public API — straightforward
+- **GloBird side:** No API — users manually configure their tariff rates, time periods, and incentives via a config flow
+- **Users:** Australian solar/battery households comparing energy providers
+
+## GloBird Plan Complexity
+
+Three sample plans in project root (PDFs). Key variations the config flow must handle:
+- **Flat vs TOU** import rates
+- **Stepped pricing** (first X kWh at one rate, remainder at another)
+- **Multiple time windows per period** (e.g., Shoulder = 9pm-12am + 12am-10am + 2pm-4pm)
+- **Separate import and export TOU schedules**
+- **Optional incentives:** ZEROHERO ($1/day credit), Super Export (15c/kWh), Critical Peak, free power windows
+- **Daily supply charge** varies per plan
+
+## Integration Structure
+
+```
+custom_components/energy_compare/
+├── __init__.py
+├── manifest.json
+├── config_flow.py       # Amber API key + GloBird tariff builder
+├── sensor.py            # Cost calculation sensors
+├── const.py
+├── strings.json
+└── translations/
+    └── en.json
+```
+
+## Code Conventions
+
+- Follow Home Assistant integration development guidelines
+- Use `async`/`await` for all I/O operations
+- Config flow must validate Amber API key on entry
+- All sensor calculations use HA's energy sensors as source data
+- Support HACS installation via custom repository
+
+## AEGIS-Derived Rules
+
+_Generated from AEGIS diagnostic audit (2026-04-16). Review invalidation conditions before removing._
+
+### Secrets
+
+- NEVER hardcode tokens, API keys, or credentials in any file — use HA config entry storage
+- NEVER commit files containing JWTs or Bearer tokens — run `gitleaks detect` before every push
+- The `energy-dashboard.html` at repo root is DELETED — do not recreate
+
+### Dashboard
+
+- The canonical dashboard is `custom_components/pricehawk/www/dashboard.html` — there is no repo-root copy
+- Dashboard entity IDs MUST use the `pricehawk_` prefix matching sensor.py
+- Dashboard MUST use `location.protocol` for WebSocket URL detection, never hardcode ws://
+- Dashboard MUST read token from URL params or postMessage, never hardcode
+
+### CI/CD
+
+- NEVER interpolate `${{ }}` directly in `run:` blocks — use `env:` intermediate variables
+- NEVER use `permissions: write-all` — specify minimum required permissions per job
+
+### Testing
+
+- Config flow changes require corresponding test updates in test_config_flow.py
+- Tariff rate calculation changes require edge case tests (negative rates, midnight boundaries, empty windows)
+
+### State Persistence
+
+- State restore MUST validate storage version before loading
+- `from_dict()` methods MUST receive an explicit HA-timezone date — no `date.today()` fallback
+
+## graphify
+
+This project has a graphify knowledge graph at graphify-out/.
+
+Rules:
+- Before answering architecture or codebase questions, read graphify-out/GRAPH_REPORT.md for god nodes and community structure
+- If graphify-out/wiki/index.md exists, navigate it instead of reading raw files
+- After modifying code files in this session, run `graphify update .` to keep the graph current (AST-only, no API cost)

CHANGELOG.md

@@ -4,6 +4,127 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [1.5.0-beta.1] - 2026-05-16
+
+CDR-native release. Replaces the manual GloBird-specific tariff wizard with a
+universal Consumer Data Right (CDR) flow that works for any AU retailer
+published on the AER. Sensor cost math is now driven by structured CDR
+PlanDetailV2 data rather than user-entered rates.
+
+### Added
+
+- **Universal CDR wizard.** New 4-step flow: state → distributor → retailer
+  (from the AER registry) → CDR plan. Replaces the bespoke GloBird-only
+  rate-entry form.
+- **117 retailers via EME refdata2** registry (Phase 3.1 prep). Wizard
+  sources retailer endpoints from `api.energymadeeasy.gov.au/refdata2` with
+  the baked-in EME snapshot as the offline fallback.
+- `RetailerEndpoint.cdr_brand` field carries the CDR-PlanDetail `brand`
+  discriminator. Disambiguates the 14 brands that share a base URI
+  (Energy Locals hosts ARCLINE / RAA / Cooperative / Indigo / Sonnen /
+  iO; OVO hosts MYOB + CTM; Radian hosts iO; Future X hosts Sunswitch).
+- `fetch_plan_list` / `fetch_plan_detail` accept optional `brand=`
+  parameter and append `?brand=<cdrBrand>` so shared-base-URI plans are
+  correctly disambiguated.
+- Baked-in EME refdata2 snapshot at
+  `custom_components/pricehawk/cdr/data/eme_refdata.json`.
+- **8 retailer incentive parsers.** GloBird (ZEROHERO + Super Export + 3-for-Free),
+  AGL (Solar Savers bonus FIT + Three for Free), Origin (tiered FIT), Alinta
+  (stepped FiT), EnergyAustralia (Solar Max + PowerResponse VPP), Engie (free
+  windows), OVO (free windows + EV off-peak + interest-on-balance), Red Energy
+  (weekend-only free window).
+- **Shared incentive helpers.** `tiered_fit.py` (multi-tier FIT for Sumo / Red
+  / Origin patterns), `bonus_fit.py` (Super Export + Peak FIT overlap-aware),
+  `free_window.py` (free-import-window engine across 315 published plans),
+  `ev_offpeak.py` (midnight-6am EV rate override), `ovo_interest.py` (3% on
+  credit balances), `vpp_rebate.py` (per-battery monthly credit).
+- **Opt-in fields.** OVO interest balance + VPP batteries enrolled fed through
+  the parser dispatcher so other-user-on-OVO/ENGIE/EA gets correct credits.
+- **Streaming CDR evaluator.** Per-30-min slot pricing with full structural
+  tariff support (TOU, stepped, controlled load) + per-retailer incentive
+  application. Daily / period accumulators persist across HA restarts with
+  storage version validation.
+- **CDR HTTP client** (`cdr/cdr_client.py`) — paginated plan list + detail
+  fetching with retry/backoff + 5xx + 429 handling.
+- **Phase 3.0 evaluator unification.** Single coordinator path for any CDR
+  plan; `CdrPlanProvider` replaces the GloBird-specific provider class.
+
+### Changed
+
+- **Manual tariff entry removed.** Phase 3.0f deleted the 4-step manual
+  GloBird wizard (plan picker / rates / export / incentives) and the 4
+  matching options-flow steps. Users must use a CDR plan. The Skip-CDR
+  sentinel that previously routed to manual entry is gone.
+- **`cdr_plan` is required** for setup. Coordinator raises
+  `ConfigEntryNotReady` when missing — prevents broken half-configured
+  entries.
+- **Daily wins map** generalised from `{amber, globird}` to
+  `{<any-provider-id>}`.
+- **Storage version** validated on restore; loads from unknown schema
+  versions are skipped.
+- **Sensor labels** read provider display name from coordinator instead of
+  hardcoded "GloBird Energy".
+
+### Fixed
+
+- **Dashboard token leak in logs** — `dashboard_url` no longer logs the
+  raw JWT; appears as `&token=<REDACTED>`.
+- **Multi-day under-credit** in `vpp_rebate.apply_rule` and
+  `ovo_interest.apply_rule` — daily credits now scale by distinct days
+  in the slot window instead of being subtracted once.
+- **VPP regex** no longer matches `/month per kWh` plans (those need
+  `critical_peak.py`, deferred).
+- **Plan list deduplication** — `fetch_plan_list` now dedups by
+  `planId` so republish-boundary repeats don't double-count.
+- **404 mapping** — list endpoint 404s raise `CdrAPIError` (bad URL),
+  not `CdrPlanNotFound` (reserved for stale planId on detail).
+- **`saving_month_aud` pollution** when Amber not configured —
+  accumulation skipped entirely instead of computing fake savings vs.
+  $0.
+- **`_last_update` restore** in `CdrStreamingEngine` — only restored
+  when stored state belongs to today; previously synthetic deltas on
+  the first tick of a new day over-counted energy/cost.
+- **Unguarded `float()` on `dailySupplyCharge`** in `CdrPlanProvider` —
+  malformed CDR values now default to $0/day supply rather than
+  crashing provider setup.
+- **`batteries_enrolled` parser crash** — uses `safe_int` defensive
+  helper so garbage option values no-op the VPP credit instead of
+  aborting the whole parser dispatch.
+- **PERIOD-cap over-credit** in `tiered_fit` — cap no longer multiplied
+  by # days in slots (proper billing-period proration deferred; under-credit
+  preferred over the 30× over-credit it replaces).
+
+### Removed
+
+- Manual GloBird tariff wizard + options-flow steps (4 + 4 step methods).
+- `async_step_cdr_override` JSON override path (was never wired into the
+  install flow). The override step, its strings, and `CONF_CDR_OVERRIDE_JSON`
+  are gone.
+- Skip-CDR sentinel and "enter rates manually" copy from the retailer + plan
+  pickers (with manual entry deleted, the affordance dead-ended on itself).
+- `cdr/data/cdr_endpoints.json` (legacy jxeeno snapshot) — superseded by
+  the EME baked-in copy.
+
+### Breaking Changes
+
+- Setup requires a CDR plan. Existing config entries created against
+  1.4.x with manual-only tariffs need to re-run the wizard.
+
+## [1.4.0-beta.2] - 2026-05-02
+
+### Fixed
+
+- **Dashboard cache stuck across upgrades** — iframe URL now appends an epoch
+  suffix to the version cache-buster, so every HA restart / integration reload
+  yields a unique URL. HA serves `/local/` static files with `max-age=2678400`
+  (31 days), which previously caused browsers and the HA companion app to pin a
+  stale `dashboard.html` for weeks even after a HACS upgrade.
+- **Sensor unique_id collision warnings** — removed legacy import/export entries
+  from `RATE_SENSORS`. These duplicated the generic per-provider rate sensors
+  registered in the providers loop, producing four `Platform pricehawk does not
+  generate unique IDs` errors at every startup. Functionally a no-op (the
+  generic sensors won the race), but the log spam is gone.
+
 ## [1.4.0-beta.1] - 2026-05-02
 
 ### Added