fix(yt): JS runtime for YouTube extraction + cookie-volume wiring gaps - #2358
Conversation
|
Warning Review limit reached
Next review available in: 22 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
chit-aware
Docker Hardening ValidationHardening Validation ReportValidated: Mon Aug 3 15:28:29 UTC 2026Services CheckedPMOVES.AI Docker Hardening Validation[INFO] Checking: pmoves/docker-compose.hardened.yml [INFO] Validating: hi-rag-gateway-v2 [INFO] Validating: extract-worker [INFO] Validating: langextract [INFO] Validating: presign [INFO] Validating: render-webhook [INFO] Validating: retrieval-eval [INFO] Validating: pdf-ingest [INFO] Validating: jellyfin-bridge [INFO] Validating: invidious-companion-proxy [INFO] Validating: ffmpeg-whisper [INFO] Validating: media-video [INFO] Validating: media-audio [INFO] Validating: hi-rag-gateway-v2-gpu [INFO] Validating: hi-rag-gateway-gpu [INFO] Validating: deepresearch [INFO] Validating: supaserch [INFO] Validating: publisher-discord [INFO] Validating: mesh-agent [INFO] Validating: nats-echo-req [INFO] Validating: nats-echo-res [INFO] Validating: comfy-watcher [INFO] Validating: grayjay-plugin-host [INFO] Validating: agent-zero [INFO] Validating: p7-room-orchestrator [INFO] Validating: archon [INFO] Validating: channel-monitor [INFO] Validating: pmoves-yt [INFO] Validating: notebook-sync [INFO] Validating: supabase_service_role_key [INFO] Validating: supabase_jwt_secret [INFO] Validating: p7_control_token ====================================== |
🔏 CHIT-aware change — control-body routingThis PR touches a CHIT-aware service (ports 8086/8087/8103/8106/8113/9224 surface). Before a Control-Body ACK:
Advisory routing only — the blocking contract check is CHIT Contract. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e3d9c3576
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # deno is yt-dlp's default-enabled runtime. Pin via DENO_VERSION (e.g. "v2.6.4") | ||
| # for reproducible builds; "latest" tracks upstream. | ||
| ARG DENO_VERSION=latest | ||
| RUN set -eux; if [ "$DENO_VERSION" = "latest" ]; then url="https://github.com/denoland/deno/releases/latest/download/deno-x86_64-unknown-linux-gnu.zip"; else url="https://github.com/denoland/deno/releases/download/${DENO_VERSION}/deno-x86_64-unknown-linux-gnu.zip"; fi; curl -fsSL "$url" -o /tmp/deno.zip; python -c "import zipfile,os; zipfile.ZipFile('/tmp/deno.zip').extractall('/usr/local/bin'); os.chmod('/usr/local/bin/deno',0o755)"; rm /tmp/deno.zip; deno --version |
There was a problem hiding this comment.
When the existing pmoves-yt workflows build linux/arm64—for example .github/workflows/yt-dlp-bump.yml:59 and .github/workflows/integrations-ghcr.matrix.json:94—this step still downloads the deno-x86_64-unknown-linux-gnu.zip asset and immediately executes it. The arm64 build therefore exits with an executable-format error, preventing the multi-architecture manifest from being built or published; select the archive using BuildKit's target architecture (such as TARGETARCH) and use Deno's aarch64 asset for arm64.
AGENTS.md reference: AGENTS.md:L112-L115
Useful? React with 👍 / 👎.
YouTube deprecated JS-less extraction; the February pmoves-yt image (yt-dlp 2026.02.04, no JS runtime) failed format extraction even with valid cookies. Fixes, each verified live on 5090: - Dockerfile: install deno (yt-dlp's default-enabled runtime), pinnable via DENO_VERSION build arg; unpinned yt-dlp picks up 2026.07.04 on rebuild. Extraction proven: title+format resolve with cookies. - up-yt used $(DC) whose STACK_FILES omits the yt-cookies overlay, so pmoves-yt always started WITHOUT the harvested-cookie volume. Now uses COOKIES_DC (+ yt-cookies profile). - The JuiceFS media mount was inside the cookies overlay with a B850-specific rshared bind default (hard-fails Docker Desktop Windows and volume-backed-JuiceFS nodes). Split into opt-in docker-compose.yt-media.yml, included by up-yt only when JUICEFS_HOST_MOUNT is set. - yt-cookie-writer wrote root:root 0660 files that pmoves-yt (uid 65532) could not read; the old chmod comment intended sharing but never chowned. New _share_with_consumer() chowns cookie/PO/refresh-token files to the consumer uid (env-overridable). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4e3d9c3 to
b6d533a
Compare
Docker Hardening ValidationHardening Validation ReportValidated: Mon Aug 3 18:50:04 UTC 2026Services CheckedPMOVES.AI Docker Hardening Validation[INFO] Checking: pmoves/docker-compose.hardened.yml [INFO] Validating: hi-rag-gateway-v2 [INFO] Validating: extract-worker [INFO] Validating: langextract [INFO] Validating: presign [INFO] Validating: render-webhook [INFO] Validating: retrieval-eval [INFO] Validating: pdf-ingest [INFO] Validating: jellyfin-bridge [INFO] Validating: invidious-companion-proxy [INFO] Validating: ffmpeg-whisper [INFO] Validating: media-video [INFO] Validating: media-audio [INFO] Validating: hi-rag-gateway-v2-gpu [INFO] Validating: hi-rag-gateway-gpu [INFO] Validating: deepresearch [INFO] Validating: supaserch [INFO] Validating: publisher-discord [INFO] Validating: mesh-agent [INFO] Validating: nats-echo-req [INFO] Validating: nats-echo-res [INFO] Validating: comfy-watcher [INFO] Validating: grayjay-plugin-host [INFO] Validating: agent-zero [INFO] Validating: p7-room-orchestrator [INFO] Validating: archon [INFO] Validating: channel-monitor [INFO] Validating: pmoves-yt [INFO] Validating: notebook-sync [INFO] Validating: supabase_service_role_key [INFO] Validating: supabase_jwt_secret [INFO] Validating: p7_control_token ====================================== |
Docker Hardening ValidationHardening Validation ReportValidated: Mon Aug 3 18:52:54 UTC 2026Services CheckedPMOVES.AI Docker Hardening Validation[INFO] Checking: pmoves/docker-compose.hardened.yml [INFO] Validating: hi-rag-gateway-v2 [INFO] Validating: extract-worker [INFO] Validating: langextract [INFO] Validating: presign [INFO] Validating: render-webhook [INFO] Validating: retrieval-eval [INFO] Validating: pdf-ingest [INFO] Validating: jellyfin-bridge [INFO] Validating: invidious-companion-proxy [INFO] Validating: ffmpeg-whisper [INFO] Validating: media-video [INFO] Validating: media-audio [INFO] Validating: hi-rag-gateway-v2-gpu [INFO] Validating: hi-rag-gateway-gpu [INFO] Validating: deepresearch [INFO] Validating: supaserch [INFO] Validating: publisher-discord [INFO] Validating: mesh-agent [INFO] Validating: nats-echo-req [INFO] Validating: nats-echo-res [INFO] Validating: comfy-watcher [INFO] Validating: grayjay-plugin-host [INFO] Validating: agent-zero [INFO] Validating: p7-room-orchestrator [INFO] Validating: archon [INFO] Validating: channel-monitor [INFO] Validating: pmoves-yt [INFO] Validating: notebook-sync [INFO] Validating: supabase_service_role_key [INFO] Validating: supabase_jwt_secret [INFO] Validating: p7_control_token ====================================== |
Docker Hardening ValidationHardening Validation ReportValidated: Mon Aug 3 18:55:46 UTC 2026Services CheckedPMOVES.AI Docker Hardening Validation[INFO] Checking: pmoves/docker-compose.hardened.yml [INFO] Validating: hi-rag-gateway-v2 [INFO] Validating: extract-worker [INFO] Validating: langextract [INFO] Validating: presign [INFO] Validating: render-webhook [INFO] Validating: retrieval-eval [INFO] Validating: pdf-ingest [INFO] Validating: jellyfin-bridge [INFO] Validating: invidious-companion-proxy [INFO] Validating: ffmpeg-whisper [INFO] Validating: media-video [INFO] Validating: media-audio [INFO] Validating: hi-rag-gateway-v2-gpu [INFO] Validating: hi-rag-gateway-gpu [INFO] Validating: deepresearch [INFO] Validating: supaserch [INFO] Validating: publisher-discord [INFO] Validating: mesh-agent [INFO] Validating: nats-echo-req [INFO] Validating: nats-echo-res [INFO] Validating: comfy-watcher [INFO] Validating: grayjay-plugin-host [INFO] Validating: agent-zero [INFO] Validating: p7-room-orchestrator [INFO] Validating: archon [INFO] Validating: channel-monitor [INFO] Validating: pmoves-yt [INFO] Validating: notebook-sync [INFO] Validating: supabase_service_role_key [INFO] Validating: supabase_jwt_secret [INFO] Validating: p7_control_token ====================================== |
…oding (#2400) The provider hardcoded a positional 101-slot param array ('upstream reduced from 121 to 101') while the live studio reports 121 — every layout drift null-crashed synthesis ('Gradio error: null'). Same disease as the #2358 postscript: nothing connected the assumed shape to the real one. Now the provider consumes the studio's own schema (/gradio_api/info, cached per instance): every slot takes the studio's parameter_default, and overrides are resolved BY NAME (text_input/tts_engine/audio_format + per-engine ENGINE_NAME_OVERRIDES for kokoro/kitten, names verified against the live schema). A renamed core param fails loudly instead of silently; unknown-to-us engines run on the engine author's defaults. vibevoice rejection and unknown-engine errors preserved. Verified on 5090 through the full gateway loop: POST /v1/voice/synthesize/audio (ultimate_tts) -> 200, 262KB wav -> POST /v1/voice/recognize -> exact text back. Operator doctrine applied: 'thats what archon and BoTZ for mcp discover' — the API publishes its shape; consume it. Co-authored-by: Mavis <Mavis@pmoves.local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…release Three entries appended at the bottom of pmoves/docs/AGENTS/AGNOTE4482PHI.t1.md: 1. PR #2415 (RELEASE, 2026-08-05T20:08Z) — env.local stragglers across 3 hand-maintained compose overlays. The entry was lost in the rebase/reset cycle that left main at `0ae6a99fd8`. Re-adding so the AGNOTE chain reflects the actual merge history. 2. PR #2416 (RELEASE, 2026-08-05T20:39Z) — validate-composes + validate-tac ratchets. Same: lost in the rebase cycle, re-added. 3. PR #2418 (CLAIM, 2026-08-05T22:15Z) — this PR's own validate-dockerfile-paths ratchet. Third ratchet from the #2358 meta-callout, same shape as the prior two, with the 5 broken builds fixed in the prior commit and the 26-entry baseline of intentional orphans. GRAPHITI_MARK comments added for all three so the Graphiti ingest picks them up. No code changes in this commit.
…release Three entries appended at the bottom of pmoves/docs/AGENTS/AGNOTE4482PHI.t1.md: 1. PR #2415 (RELEASE, 2026-08-05T20:08Z) — env.local stragglers across 3 hand-maintained compose overlays. Lost in the rebase/reset cycle that left main at 0ae6a99; re-adding so the AGNOTE chain reflects the actual merge history. 2. PR #2416 (RELEASE, 2026-08-05T20:39Z) — validate-composes + validate-tac ratchets. Same: lost in the rebase cycle, re-added. 3. PR #2417 (CLAIM, 2026-08-05T22:15Z) — this PR's own validate-dockerfile-paths ratchet. Third ratchet from the #2358 meta-callout, same shape as the prior two, with the 5 broken builds fixed in the prior commit and the 26-entry baseline of intentional orphans. GRAPHITI_MARK comments added for all three so the Graphiti ingest picks them up. No code changes in this commit.
…he dev touched ≠ file the runner builds" (#2358) Third pattern from the operator's #2358 meta-callout. Same shape as the validate-composes + validate-tac ratchets (PR #2416): no- network, ~100ms scan, baseline file committed and reviewable in PR diffs, count only goes DOWN over time. Two failure modes: 1. BROKEN_BUILD — a compose `build:` stanza points at a Dockerfile that doesn't exist on disk. (5 real instances fixed in the next commit: agent-zero in vps.override.yml, github-branch-naming + github-crossrepo-sync in docker-compose.yml + docker-compose.workers.yml.) 2. ORPHAN_DOCKERFILE — a Dockerfile in the repo that no compose references AND isn't in the operator-acknowledged baseline. Initial baseline has 26 entries with reasons. Files: - pmoves/tools/validate_dockerfile_paths.py (~280 lines, zero deps beyond pyyaml; same `!override` / `!reset` Compose-tag tolerance as validate_composes) - pmoves/tools/tests/test_validate_dockerfile_paths.py (11 tests, all pass; covers broken-build, baseline-aware orphan detection, sibling-submodule skipping, env-var default substitution, JSON output shape) - pmoves/configs/dockerfiles/_known_orphans.yaml (baseline with 26 entries + reasons; reduce-by-pr pattern) - .github/workflows/validate-dockerfile-paths-ratchet.yml (fires on any Dockerfile*, any docker-compose*.yml, the ratchet script, the ratchet tests, the baseline file, or the workflow itself) - pmoves/Makefile (3 new targets: validate-dockerfile-paths, validate-dockerfile-paths-json, validate-dockerfile-paths-list-orphans) The sibling-submodule / vendor / provisions paths are explicit- skip: the ratchet can't statically check external repos. Out-of- scope paths include anything under pmoves/vendor/, anything with "provisions" in the path, and the common sibling-submodule patterns (PMOVES-Archon, PMOVES.YT, Pmoves-cipher, etc.). The operator keeps those synced via `make submodules`. Counts at initial pass (on this PR): 90 dockerfiles in the scan dirs, 78 referenced by compose, 26 in baseline. Zero findings. 35/35 ratchet tests pass.
…release Three entries appended at the bottom of pmoves/docs/AGENTS/AGNOTE4482PHI.t1.md: 1. PR #2415 (RELEASE, 2026-08-05T20:08Z) — env.local stragglers across 3 hand-maintained compose overlays. Lost in the rebase/reset cycle that left main at 0ae6a99; re-adding so the AGNOTE chain reflects the actual merge history. 2. PR #2416 (RELEASE, 2026-08-05T20:39Z) — validate-composes + validate-tac ratchets. Same: lost in the rebase cycle, re-added. 3. PR #2417 (CLAIM, 2026-08-05T22:15Z) — this PR's own validate-dockerfile-paths ratchet. Third ratchet from the #2358 meta-callout, same shape as the prior two, with the 5 broken builds fixed in the prior commit and the 26-entry baseline of intentional orphans. GRAPHITI_MARK comments added for all three so the Graphiti ingest picks them up. No code changes in this commit.
…he dev touched ≠ file the runner builds' (#2358) (#2417) * feat(ratchets): validate-dockerfile-paths — durable answer to "file the dev touched ≠ file the runner builds" (#2358) Third pattern from the operator's #2358 meta-callout. Same shape as the validate-composes + validate-tac ratchets (PR #2416): no- network, ~100ms scan, baseline file committed and reviewable in PR diffs, count only goes DOWN over time. Two failure modes: 1. BROKEN_BUILD — a compose `build:` stanza points at a Dockerfile that doesn't exist on disk. (5 real instances fixed in the next commit: agent-zero in vps.override.yml, github-branch-naming + github-crossrepo-sync in docker-compose.yml + docker-compose.workers.yml.) 2. ORPHAN_DOCKERFILE — a Dockerfile in the repo that no compose references AND isn't in the operator-acknowledged baseline. Initial baseline has 26 entries with reasons. Files: - pmoves/tools/validate_dockerfile_paths.py (~280 lines, zero deps beyond pyyaml; same `!override` / `!reset` Compose-tag tolerance as validate_composes) - pmoves/tools/tests/test_validate_dockerfile_paths.py (11 tests, all pass; covers broken-build, baseline-aware orphan detection, sibling-submodule skipping, env-var default substitution, JSON output shape) - pmoves/configs/dockerfiles/_known_orphans.yaml (baseline with 26 entries + reasons; reduce-by-pr pattern) - .github/workflows/validate-dockerfile-paths-ratchet.yml (fires on any Dockerfile*, any docker-compose*.yml, the ratchet script, the ratchet tests, the baseline file, or the workflow itself) - pmoves/Makefile (3 new targets: validate-dockerfile-paths, validate-dockerfile-paths-json, validate-dockerfile-paths-list-orphans) The sibling-submodule / vendor / provisions paths are explicit- skip: the ratchet can't statically check external repos. Out-of- scope paths include anything under pmoves/vendor/, anything with "provisions" in the path, and the common sibling-submodule patterns (PMOVES-Archon, PMOVES.YT, Pmoves-cipher, etc.). The operator keeps those synced via `make submodules`. Counts at initial pass (on this PR): 90 dockerfiles in the scan dirs, 78 referenced by compose, 26 in baseline. Zero findings. 35/35 ratchet tests pass. * fix(compose): 5 real broken build targets the ratchet surfaced The validate-dockerfile-paths ratchet (P1 commit) caught these on its first run. Each one is a compose `build:` stanza that points at a Dockerfile that doesn't exist — the ratchet's exact "merged before the issue was addressed" pattern. 5 fixes (4 files): 1. pmoves/docker-compose.vps.override.yml:43-44 `agent-zero` had `build: { dockerfile: Dockerfile }` with no `context:`. Default context is `.` (pmoves/), so the resolved path was `pmoves/Dockerfile` which doesn't exist. Added `context: .` + `dockerfile: ./services/agent-zero/ Dockerfile` to match the build stanza in pmoves/docker- compose.yml:2911-2913. 2-3. pmoves/docker-compose.yml:5207-5211 (github-branch- naming) + 5247-5251 (github-crossrepo-sync) Both services had `build: { context: ./services/<name>, dockerfile: Dockerfile }` but the service directories don't exist in this repo (the implementation lives in separate `pmoves-github-branch-naming` / `pmoves-github- crossrepo-sync` repos). Dropped the `build:` block; the prebuilt `image: ${...IMAGE:-ghcr.io/...:pmoves-latest}` env var default is the canonical path. Matches the `publisher-discord` (line 3891) + `github-crossrepo-pr` (line 3891 neighborhood) pattern of prebuilt-only. 4-5. Same fix applied to pmoves/docker-compose.workers.yml: 695-699 (github-branch-naming) + 735-739 (github-crossrepo- sync) — these are a hand-curated split of the main file and had the same broken `build:` stanzas. After this commit: validate-dockerfile-paths reports 0 broken builds on the fleet. The next commit (docs) re-adds the missing AGNOTE entries for the prior ratchet PRs (#2415, #2416) plus the release marker for this PR. * docs(agnote): re-add #2415 + #2416 entries lost in rebase + new #2417 release Three entries appended at the bottom of pmoves/docs/AGENTS/AGNOTE4482PHI.t1.md: 1. PR #2415 (RELEASE, 2026-08-05T20:08Z) — env.local stragglers across 3 hand-maintained compose overlays. Lost in the rebase/reset cycle that left main at 0ae6a99; re-adding so the AGNOTE chain reflects the actual merge history. 2. PR #2416 (RELEASE, 2026-08-05T20:39Z) — validate-composes + validate-tac ratchets. Same: lost in the rebase cycle, re-added. 3. PR #2417 (CLAIM, 2026-08-05T22:15Z) — this PR's own validate-dockerfile-paths ratchet. Third ratchet from the #2358 meta-callout, same shape as the prior two, with the 5 broken builds fixed in the prior commit and the 26-entry baseline of intentional orphans. GRAPHITI_MARK comments added for all three so the Graphiti ingest picks them up. No code changes in this commit. * fix(compose): regenerate workers overlay after comment edits The ratchet PR updated inline comments in docker-compose.yml for github-branch-naming / github-crossrepo-sync (explaining why there's no build block) but didn't run `make -C pmoves compose-split` to propagate the changes to docker-compose.workers.yml. Validate Compose Files was correctly reporting drift. Verified idempotent (compose-split produces same output on re-run). 💘 Generated with Crush * fix(compose): regenerate overlays after rebase onto #2420 Rebase brought in #2420's json-file log rotation on tier anchors. Regenerate base/core/agents/apps overlays to match. Workers was already correct (compose-split on the pre-rebase branch fixed it). 💘 Generated with Crush * fix(ratchet): use is_file() not exists() for Dockerfile BROKEN_BUILD check exists() returns True for directories — a compose build.dockerfile that resolves to a directory (e.g. unset ${FOO} collapsing to the context dir) would pass the gate even though Docker can't use it. is_file() ensures only real files satisfy the check. Addresses Codex P2 on #2417. 💘 Generated with Crush --------- Co-authored-by: mavis <mavis@pmoves.ai> Co-authored-by: Agent Zero <agent.zero@pmoves.ai>
2 participants
Problem
Downloads failed with valid cookies: "No supported JavaScript runtime could be found" — YouTube now requires JS execution (deno/node) for player-response decoding, and the pmoves-yt image was a February build (yt-dlp 2026.02.04, no runtime). Investigating exposed three more wiring gaps that meant cookies never reached yt-dlp at all.
Fixes (each verified live on 5090)
DENO_VERSIONbuild-arg pinnable; rebuild picks up yt-dlp 2026.07.04. Proof:yt-dlp --cookies ... --simulateresolves title+format inside the fresh container.up-ytnever mounted cookies: it used$(DC)whose STACK_FILES omits the yt-cookies overlay — every historical bring-up ran pmoves-yt cookie-less. Now usesCOOKIES_DC+ theyt-cookiesprofile.rsharedbind with a B850-specific default path hard-fails Docker Desktop Windows and volume-backed-JuiceFS nodes. Moved to opt-indocker-compose.yt-media.yml, auto-included byup-ytwhenJUICEFS_HOST_MOUNTis set (B850 unaffected — it sets the var).root:root 0660; pmoves-yt runs as uid 65532. New_share_with_consumer()chowns to the consumer uid (env-overridable), applied at all three write sites.Follow-ups (not this PR)
-1expiry → cosmetic yt-dlp skip-warning for YSC; could clamp to 0 at export.up-yt— PO-token wiring validation is the next lane if downloads still hit bot-gate at scale.🤖 Generated with Claude Code