docs: log m2 automation implementation plan

pmoves/docs/NEXT_STEPS.md

@@ -1,5 +1,5 @@
 # PMOVES v5 • NEXT_STEPS
-_Last updated: 2025-09-27_
+_Last updated: 2025-09-30_
 
 ## Immediate
 
@@ -9,29 +9,30 @@ _Last updated: 2025-09-27_
 - [ ] Activate the n8n approval poller and echo publisher workflows once secrets are loaded; document the activation + first successful run.
 - [ ] Confirm Jellyfin credentials (API key and optional user id) allow library enumeration; note any dependency gaps that require new guardrails.
 - [ ] Validate that enriched publisher metadata propagates into Agent Zero and Discord events; schedule a backfill for legacy records if fields are missing.
+- [ ] Record step-by-step evidence in `SESSION_IMPLEMENTATION_PLAN.md` while executing the operational reminders list.
 
 ### 2. Jellyfin Publisher Reliability
 - [x] Add a scheduled refresh or webhook trigger so Jellyfin libraries update after publisher runs; include cron/webhook settings in `services/publisher/README.md`.
 - [ ] Expand error/reporting hooks so failures surface with actionable messages (Jellyfin HTTP errors, dependency mismatches, asset gaps).
 - [ ] Backfill historic Jellyfin entries with enriched metadata and confirm downstream consumers (Agent Zero, Discord) render the new fields.
 
 ### 3. Graph & Retrieval Enhancements (Kickoff M3)
-- [ ] Seed Neo4j with the brand alias dictionary (DARKXSIDE, POWERFULMOVES, plus pending community submissions) and record Cypher script locations.
+- [ ] Seed Neo4j with the brand alias dictionary (DARKXSIDE, POWERFULMOVES, plus pending community submissions) and record Cypher script locations (draft plan in `SESSION_IMPLEMENTATION_PLAN.md`).
 - [ ] Outline relation-extraction passes from captions/notes to candidate graph edges; define success metrics and owner in the project tracker.
-- [ ] Prepare reranker parameter sweep plan (datasets, toggles, artifact storage) for integration into CI.
+- [ ] Prepare reranker parameter sweep plan (datasets, toggles, artifact storage) for integration into CI, aligning with the prep checklist captured in `SESSION_IMPLEMENTATION_PLAN.md`.
 
 ### 4. PMOVES.YT High-Priority Lane
 - [ ] Design and document the resilient download module (resume, retries, rate limiting, playlist/channel ingestion, bounded worker pool).
 - [ ] Specify multipart upload + checksum verification approach for MinIO, including lifecycle/retention tag configuration.
 - [ ] Enumerate metadata enrichment requirements (duration, channel, tags, provenance) and map them to Supabase schema updates.
-- [ ] Draft the faster-whisper GPU migration plan (language auto-detect, diarization flags, partial transcript updates).
+- [ ] Draft the faster-whisper GPU migration plan (language auto-detect, diarization flags, partial transcript updates) and confirm smoke expectations defined in `SESSION_IMPLEMENTATION_PLAN.md`.
 - [ ] Document Gemma integration paths: Ollama (`gemma2:9b-instruct`) and HF Transformers (`google/gemma-2-9b-it`), including feature toggles and embedding backstops.
 - [ ] Define API hardening, observability, and security tasks (validation, OpenAPI, health/readiness probes, metrics, signed URL enforcement, optional content filters).
 
 ### 5. Platform Operations & Tooling
 - [x] Publish Windows/WSL smoke scripts (`scripts/smoke.ps1`) with instructions in `pmoves/docs/LOCAL_DEV.md`.
 - [ ] Draft Supabase RLS hardening checklist covering non-dev environments and dependency audits.
-- [ ] Plan optional CLIP + Qwen2-Audio integrations, including toggles, GPU/Jetson expectations, and smoke tests.
+- [ ] Plan optional CLIP + Qwen2-Audio integrations, including toggles, GPU/Jetson expectations, and smoke tests (initial research threads logged in `SESSION_IMPLEMENTATION_PLAN.md`).
 - [ ] Outline the presign notebook walkthrough deliverable once automation stabilizes.
 
 ## n8n Flow Operations
@@ -87,12 +88,12 @@ _Last updated: 2025-09-27_
 - [ ] (Optional) Infrastructure-as-code starter kit for hybrid GPU + Jetson deployments.
 
 ## Next Session Focus
-- [ ] media-video: insert `detections`/`segments` into Supabase and emit `analysis.entities.v1`.
+- [ ] media-video: insert `detections`/`segments` into Supabase and emit `analysis.entities.v1` — reference activation notes in `SESSION_IMPLEMENTATION_PLAN.md`.
 - [ ] media-audio: insert `emotions` into Supabase and emit `analysis.audio.v1`.
-- [ ] ffmpeg-whisper: switch to `faster-whisper` with GPU auto-detect (Jetson/desktop).
+- [ ] ffmpeg-whisper: switch to `faster-whisper` with GPU auto-detect (Jetson/desktop); confirm GPU smoke path documented in `SESSION_IMPLEMENTATION_PLAN.md`.
 - [ ] CLIP embeddings on keyframes (optional; desktop on by default, Jetson off).
 - [ ] n8n flows: end-to-end ingest → transcribe → extract → index → notify.
-- [ ] Jellyfin refresh hook + Discord rich embeds (cover art, duration, link).
+- [ ] Jellyfin refresh hook + Discord rich embeds (cover art, duration, link) with validation evidence logged in `SESSION_IMPLEMENTATION_PLAN.md`.
 - [ ] Supabase RLS hardening pass (non-dev).
 - [ ] Qwen2-Audio provider (desktop-only toggle) for advanced audio QA/summarization.
 - [ ] PMOVES.YT: wire Gemma summaries (Ollama by default), add `/yt/summarize` and `/yt/chapters` endpoints; add smoke target `make yt-smoke URL=...`.

pmoves/docs/ROADMAP.md

@@ -1,5 +1,5 @@
 # PMOVES v5 • ROADMAP
-_Last updated: 2025-09-24_
+_Last updated: 2025-09-30_
 
 ## Vision
 A production-ready, self-hostable orchestration mesh for creative + agent workloads across GPU boxes and Jetsons: **hybrid Hi‑RAG**, **Supabase Studio**, **n8n orchestration**, **Jellyfin publishing**, and **graph-aware retrieval**.
@@ -21,18 +21,18 @@ A production-ready, self-hostable orchestration mesh for creative + agent worklo
 | 🚧 | Publisher (Jellyfin) | `services/publisher/publisher.py` consumes approval events and refreshes Jellyfin; optional dependency guards and envelope fallback landed, but richer metadata handling and error reporting are still pending. |
 | ✅ | PDF/MinIO ingestion | `services/pdf-ingest/app.py` pulls PDFs from MinIO, extracts text, forwards chunks, and emits ingest events. |
 | ⏳ | n8n flows (Discord/webhooks) | `n8n/flows/*.json` only define placeholder workflows; Supabase pollers and Discord actions must be configured. |
-| 🚧 | Jellyfin library refresh hook + Discord rich cards | Jellyfin refresh occurs in the publisher, and `services/publisher-discord` formats embeds, but published-event wiring and asset deep links remain. |
+| 🚧 | Jellyfin library refresh hook + Discord rich cards | Jellyfin refresh occurs in the publisher, and `services/publisher-discord` formats embeds, but published-event wiring and asset deep links remain. Automation activation plan logged in `pmoves/docs/SESSION_IMPLEMENTATION_PLAN.md`. |
 
 **Outstanding to close M2:**
 - publisher metadata/envelope polish — namespace-aware filenames, dependency guards, and fallback envelopes merged; monitor adoption and backfill historic assets if needed
-- add published-event Discord embeds via `content.published.v1`
-- build the Supabase→Discord automation inside the n8n exports
-- execute the Supabase → Agent Zero → Discord activation checklist (`pmoves/docs/SUPABASE_DISCORD_AUTOMATION.md`) and log the validation timestamp
+- add published-event Discord embeds via `content.published.v1`; execution plan staged in `SESSION_IMPLEMENTATION_PLAN.md`
+- build the Supabase→Discord automation inside the n8n exports and track discrete workflow validation steps in the implementation log
+- execute the Supabase → Agent Zero → Discord activation checklist (`pmoves/docs/SUPABASE_DISCORD_AUTOMATION.md`) and log the validation timestamp (see operational reminders captured in the implementation plan)
 
 ### M3 — Retrieval Quality & Graph Enrichment
-- Entity linking dictionaries (DARKXSIDE / POWERFULMOVES aliases)
+- Entity linking dictionaries (DARKXSIDE / POWERFULMOVES aliases) — alias sourcing tasks assigned in `SESSION_IMPLEMENTATION_PLAN.md`
 - Relation extraction (Entity —[REL]→ Entity) from captions/notes
-- Reranker parameter sweeps + CI artifacts (toggle implemented)
+- Reranker parameter sweeps + CI artifacts (toggle implemented) — prep checklist drafted in `SESSION_IMPLEMENTATION_PLAN.md`
 - Cross-namespace routing & intent-based type boosters
 
 ### M4 — Formats & Scale

pmoves/docs/SESSION_IMPLEMENTATION_PLAN.md

@@ -0,0 +1,63 @@
+# M2 Automation Kickoff & Roadmap Prep Plan
+_Last updated: 2025-09-30_
+
+This working session establishes the concrete implementation tasks needed to close Milestone M2 while warming up broader roadmap threads for Milestones M3–M5. It consolidates the operational reminders from the sprint brief and ties each step to the canonical checklists in `pmoves/docs/SUPABASE_DISCORD_AUTOMATION.md`, `pmoves/docs/NEXT_STEPS.md`, and `pmoves/docs/ROADMAP.md`.
+
+## 1. Automation Loop Activation (Supabase → Agent Zero → Discord)
+
+| Step | Owner | Dependencies | Evidence to Capture |
+| --- | --- | --- | --- |
+| Populate `.env` Discord credentials and confirm ping | Automation engineer | Discord webhook provisioned, `.env` write access | Screenshot or log excerpt of successful manual webhook ping |
+| Import & credential n8n flows | Automation engineer | n8n instance live, Supabase + Agent Zero endpoints reachable | n8n workflow screenshots showing credential bindings |
+| Run approval poller dry run | Automation engineer | Supabase row seeded with `status='approved'` | n8n execution log + Agent Zero event log snippet |
+| Run echo publisher workflow | Automation engineer | Previous step complete, Discord ping verified | Discord embed screenshot + Supabase row showing `meta.publish_event_sent_at` |
+| Capture activation log | Automation engineer | All steps above complete | Append validation timestamps + links in `pmoves/docs/SUPABASE_DISCORD_AUTOMATION.md` |
+
+**Operational reminders applied**
+- Use the n8n import and manual verification checklist verbatim during activation.
+- Keep a running evidence trail (timestamps, log excerpts, screenshots) and archive it alongside the automation playbook once the loop completes.
+
+## 2. Jellyfin Publisher Reliability Enhancements
+
+1. Expand error and reporting hooks to surface HTTP failures, missing dependencies, and asset lookup issues in publisher logs.
+2. Schedule Jellyfin metadata backfill job after automation loop validation; capture duration estimates and data volume in the run log.
+3. Verify refreshed metadata renders in Discord embeds and Agent Zero payloads (tie back to automation evidence above).
+
+## 3. Broader Roadmap Prep (M3–M5)
+
+### 3.1 Graph & Retrieval (Milestone M3)
+- **Alias seeding**: Draft Cypher scripts for DARKXSIDE/POWERFULMOVES alias ingestion; source lists from community submissions and Jellyfin metadata exports.
+- **Relation extraction passes**: Outline caption/note parsing strategy and scoring metrics; identify candidate tooling (spaCy, OpenAI function-calling fallback).
+- **Reranker parameter sweeps**: Enumerate datasets (agent transcripts, Jellyfin descriptions) and define CI artifact storage structure; align toggle flags with `HI_RAG_RERANKER` docs.
+
+### 3.2 Formats & Scale (Milestone M4)
+- Document prerequisites for DOCX/PPTX ingestion (LibreOffice container image, conversion queue) and assign research spikes.
+- Specify GPU passthrough and Tailscale policy updates required for Proxmox deployment bundles.
+
+### 3.3 Studio & Ops (Milestone M5)
+- Capture Studio approval UI requirements (Supabase quick-view, moderation controls) to inform backlog grooming.
+- List CI/CD gating targets: retrieval-eval coverage thresholds, artifact retention policy, backup cadence.
+
+## 4. PMOVES.YT Lane: Immediate Prep Items
+
+- Resilient downloader design outline: segmented downloads, retry/backoff strategy, and concurrency guard rails for playlists/channels.
+- Multipart upload plan: presign workflow for MinIO, checksum verification, retention tag defaults.
+- Metadata enrichment matrix: map duration/channel/tag ingestion into Supabase schema fields; note migrations needed.
+- Faster-whisper GPU migration: record Jetson vs desktop flag differences and expected `make` smoke commands.
+- Gemma integration primer: baseline Ollama config (`gemma2:9b-instruct`), HF transformer fallback, and embedding compatibility notes.
+- API/observability/security hardening backlog: request validation, OpenAPI doc generation, metrics to expose, signed URL enforcement, and optional content filters.
+
+## 5. Next Session Setup Checklist
+
+- [ ] Review evidence captured during automation loop dry runs and update `SUPABASE_DISCORD_AUTOMATION.md` with timestamps.
+- [ ] Queue Jellyfin metadata backfill job and document parameters (collections targeted, run duration).
+- [ ] Draft Neo4j alias seeding scripts and store them under `pmoves/neo4j/` (placeholder if scripts not yet committed).
+- [ ] Prepare GPU smoke expectations for faster-whisper path and add them to `pmoves/docs/SMOKETESTS.md` if gaps remain.
+- [ ] Confirm Supabase RLS hardening research tasks have owners and deadlines noted in the project tracker.
+
+---
+
+**References**
+- `pmoves/docs/SUPABASE_DISCORD_AUTOMATION.md`
+- `pmoves/docs/NEXT_STEPS.md`
+- `pmoves/docs/ROADMAP.md`