Skip to content

feat(publisher): enrich published events with metadata - #49

Merged
POWERFULMOVES merged 1 commit into
mainfrom
codex/implement-namespace-aware-output-paths-and-logging
Sep 20, 2025
Merged

POWERFULMOVES merged 1 commit into
mainfrom
codex/implement-namespace-aware-output-paths-and-logging

Conversation

@POWERFULMOVES

@POWERFULMOVES POWERFULMOVES commented Sep 20, 2025 •

Copy link
Copy Markdown
Owner

Summary

  • derive namespace-aware media paths, structured logging, and retry instrumentation in the publisher worker
  • merge description/tags/meta and propagate Jellyfin lookup results when emitting content.published.v1
  • document the richer contract and cover slug/metadata helpers with unit tests

Testing

  • pytest pmoves/services/publisher/tests/test_publisher.py

https://chatgpt.com/codex/tasks/task_b_68ce3ff69dac8324a4977213b854d343

Summary by CodeRabbit

  • New Features

    • Enhanced Content Publisher: stages approved media to structured paths and emits richer publish events including source description, tags, and merged metadata.
    • Optionally attaches a public URL and media library item ID after a successful library refresh.
    • Configurable public base URL for HTTP paths.
  • Improvements

    • More reliable downloads with automatic retries and backoff.
  • Documentation

    • Added smoke tests section for the Content Publisher, including setup and configuration notes.
  • Tests

    • Added unit tests for slug/path generation and metadata merging in publish payloads.

@coderabbitai

coderabbitai Bot commented Sep 20, 2025 •

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

Walkthrough

An async publisher service was introduced and documented. It listens for content.publish.approved.v1, downloads artifacts from MinIO, stages them under MEDIA_LIBRARY_PATH, optionally derives a public URL, triggers Jellyfin refresh and lookup, merges metadata, and publishes content.published.v1. Tests cover slugging, path derivation, and payload/metadata merging.

Changes

Cohort / File(s) Summary
Docs: Smoke Tests
pmoves/docs/SMOKETESTS.md
Added Content Publisher smoke test section detailing event subscription, media staging path via MEDIA_LIBRARY_PATH, enriched content.published.v1 payload (source description, tags, merged meta), optional public_url/jellyfin_item_id, and MEDIA_LIBRARY_PUBLIC_BASE_URL usage.
Publisher Service (Async rewrite)
pmoves/services/publisher/publisher.py
Rewrote to async service with structured logging, metrics (PublisherMetrics), retrying downloads (DownloadError), Jellyfin refresh and item lookup, public URL derivation, slug/path utilities, metadata merge, and richer published payload. Subscribes to content.publish.approved.v1 and emits content.published.v1.
Unit Tests
pmoves/services/publisher/tests/*
Added tests validating slugify, derive_output_path, and build_published_payload metadata merging and fields (artifact_uri, published_path, namespace, public_url, jellyfin_item_id).

Sequence Diagram(s)

sequenceDiagram
    participant A as Producer
    participant N as NATS
    participant P as Publisher Service
    participant M as MinIO
    participant F as Filesystem (MEDIA_LIBRARY_PATH)
    participant J as Jellyfin

    A->>N: Publish content.publish.approved.v1
    N->>P: Deliver approved payload
    P->>P: Validate payload, derive slug/output path
    P->>M: Download artifact (with retries)
    M-->>P: Artifact stream/bytes
    P->>F: Write to staged path
    alt Jellyfin configured
        P->>J: Trigger library refresh
        J-->>P: Refresh ack
        P->>J: Lookup item by title
        J-->>P: Item ID, (optional) URL components
        P->>P: Derive public_url (if base configured)
    else No Jellyfin
        P->>P: Optionally derive public_url from path
    end
    P->>P: Merge metadata, build published payload
    P->>N: Publish content.published.v1 (includes public_url/jellyfin_item_id if available)
    Note over P,N: Logs + metrics for downloads/refreshes/outcomes
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

A hop, a bop, I queue and glide,
From NATS I snag the streaming tide—
I stage the bytes, then ping Jellyfin,
“Refreshed?” it nods; I publish with a grin.
Slugs and tags in tidy rows,
My little paws make payloads glow. 🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Free

📥 Commits

Reviewing files that changed from the base of the PR and between 21f2491 and c1974f8.

📒 Files selected for processing (3)
  • pmoves/docs/SMOKETESTS.md (1 hunks)
  • pmoves/services/publisher/publisher.py (1 hunks)
  • pmoves/services/publisher/tests/test_publisher.py (1 hunks)

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@POWERFULMOVES
POWERFULMOVES merged commit eda35e0 into main Sep 20, 2025
0 of 3 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Comment on lines +241 to +244
async def handle(msg):
env = json.loads(msg.data.decode())
payload = env.get("payload", {})
artifact_uri = payload.get("artifact_uri")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Guard publisher callback against unexpected exceptions

The new handle callback no longer wraps its work in a top-level try/except block. If the incoming message is malformed (e.g. json.loads fails) or any unexpected error occurs later when building the envelope, the exception will now propagate out of the NATS subscription task and can crash the subscriber or leave the queue unprocessed. The previous version caught all exceptions and simply logged them. Consider restoring a defensive error boundary around the entire handler so a single bad message cannot tear down the worker.

Useful? React with 👍 / 👎.

POWERFULMOVES added a commit that referenced this pull request Jan 18, 2026
…ware-output-paths-and-logging

feat(publisher): enrich published events with metadata
@POWERFULMOVES
POWERFULMOVES deleted the codex/implement-namespace-aware-output-paths-and-logging branch March 7, 2026 21:42
POWERFULMOVES added a commit that referenced this pull request May 25, 2026
* chore(.gitmodules): correct stale branch records for 6 submodules tracking main

Audit (post-#1531) revealed 6 submodules where the gitlink in parent's
HEAD points to a SHA on the submodule's `main` branch, but `.gitmodules`
still records `branch = PMOVES.AI-Edition-Hardened`. The recorded branch
is no longer the active development line — running
`git submodule update --remote` against these entries would *regress*
the gitlink to a stale PMOVES.AI-Edition-Hardened tip rather than
advance to current HEAD.

Corrected entries (branch: PMOVES.AI-Edition-Hardened -> main):
- PMOVES-E2B-Danger-Room
- PMOVES-Danger-infra
- PMOVES-Pipecat
- PMOVES-MAI-UI
- PMOVES-Headscale
- Pmoves-cipher

Per-submodule verification (each at origin/main parity, 0/0):
- PMOVES-E2B-Danger-Room  HEAD=7a38b33b  origin/main=7a38b33b
- PMOVES-Danger-infra     HEAD=44d80c70  origin/main=44d80c70
- PMOVES-Pipecat          HEAD=a74aa0cc  origin/main=a74aa0cc
- PMOVES-MAI-UI           HEAD=182bcf4d  origin/main=182bcf4d
- PMOVES-Headscale        HEAD=d050dbe9  origin/main=d050dbe9
- Pmoves-cipher           HEAD=c4f8348f  origin/main=c4f8348f

No gitlink SHAs change in this PR — purely a metadata correction so
future `submodule update --remote` and fleet-audit automation
target the right branch.

Out of scope (require individual investigation, deferred):
- PMOVES-BoTZ           (actual=main but flagged in plan Lane B for branch reconciliation)
- PMOVES-BotZ-gateway   (actual=dependabot/pip/... — wrong-branch-tracking)
- PMOVES-transcribe-and-fetch (actual=dependabot/... — covered by PR #1546)
- PMOVES-n8n            (actual=codex/n8n-authoritative-runtime — codex agent scope)
- pmoves-e2b-mcp-server (HEAD not in any remote branch — needs investigation)
- pmoves/integrations/archon (actual=POWERFULMOVES-patch-1 — unusual)

The top-of-file comment "All submodules track PMOVES.AI-Edition-Hardened"
is now stale (it was already stale for ClawZ and skills/*); a follow-up
policy decision is needed on whether to standardize on main or restore
the PMOVES.AI-Edition-Hardened policy. Left unchanged here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs+ops: CLAUDE.md fleet audit + daily/weekly automation pair

Bundles the audit and the automations that close the gaps it surfaced.

Audit (`pmoves/docs/audit/CLAUDE_MD_AUDIT_2026-05-20.md`):
- 26 CLAUDE.md / .claude/CLAUDE.md files scored across root + top-level
  submodules + pmoves/services + worktree overrides
- Per-file rubric scores, cross-file drift list, P0/P1/P2 priority queue
- One verified false positive: Cipher port drift was already fixed
- Submodule-level fixes already filed as separate PRs (ToKenism #49,
  DoX #170, ClawZ #2, tensorzero #4, Headscale #2, BoTZ #139)

Submodule-count fix (`.claude/CLAUDE.md`):
- Catalog said 20 submodules; `git submodule status` shows 50

Daily digest pair (paired with cloud routine
`trig_01QMXAdArAv8EusKmJPMrcq9`):
- `daily_pmoves_digest.ps1` -- worktree dirty/ahead/behind + light PR
  snapshot, posts to Discord webhook
- `register_daily_digest_task.ps1` -- one-time Windows Task Scheduler
  registration that bakes DISCORD_WEBHOOK_URL into a per-task wrapper

Fleet sweep pair:
- `fleet_stale_node_sweep.ps1` -- weekly tailnet sweep via local
  `tailscale status --json` (no cloud Tailscale connector available),
  flags peers offline > N days, posts to Discord
- `register_fleet_sweep_task.ps1` -- weekly Monday Task Scheduler
  registration

Why local rather than cloud routines: the cloud routine layer can't
reach the tailnet (no Tailscale MCP connector available) and can't see
local worktree state. The daily-digest pair is cloud-for-PRs +
local-for-worktrees; the fleet sweep is local-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tasks): keep webhook wrappers outside repo

* fix(scripts): harden webhook wrappers — ACL + whitespace + gitmodules doc

- Add Set-Acl lock-down after Set-Content in both register_*_task.ps1
  wrappers (wrapper contains plaintext DISCORD_WEBHOOK_URL, restrict to
  current user only)
- Tighten DISCORD_WEBHOOK_URL blank-check in daily_pmoves_digest.ps1
  to also reject whitespace-only values
- Update .gitmodules branch strategy header to reflect mixed strategy:
  most modules on PMOVES.AI-Edition-Hardened, E2B/infra + skills/* on main

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Shaela Bello <slbello@uncg.edu>
POWERFULMOVES added a commit that referenced this pull request May 31, 2026
…1659)

Fleet-wide hardened-branch audit (38 submodules tracking PMOVES.AI-Edition-Hardened).
Promotes 8 gitlinks where hardened already contains the default branch (zero drift,
pure pin advancement, same operation as #1656):

  Pmoves-hyperdimensions, Pmoves-AgentGym-RL, PMOVES-surf, PMOVES-Ultimate-TTS-Studio,
  PMOVES-Tailscale, PMOVES-Neo4j, PMOVES-autoresearch, PMOVES-ToKenism-Multi (incl #49).

Audit doc classifies all 38: 11 clean, 8 gate-OK-stale (this PR), 17 DRIFTED (hardened
missing default-branch commits, incl. BoTZ #72 JWT auth-gate + BotZ-gateway #4 log-sanitize).
Drifted tranche deferred to per-repo research-agent merge-safety verification.

Refs: 5090 deep-bumblebee security handoff.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant