diff --git a/.agent/AGENTS.md b/.agent/AGENTS.md index 467531de..73e323d0 100644 --- a/.agent/AGENTS.md +++ b/.agent/AGENTS.md @@ -351,6 +351,8 @@ Subagents provide specialized capabilities. Read them when tasks require domain | `memory/` | Cross-session memory - SQLite FTS5 storage, /remember and /recall commands | README (system docs) | | `seo/` | Search optimization - keyword research, rankings, site audits, E-E-A-T scoring | dataforseo, serper, google-search-console, site-crawler, eeat-score, domain-research | | `content/` | Content creation - copywriting standards, editorial guidelines, tone of voice | guidelines | +| `tools/content/` | Content tools - summarization, extraction, processing | summarize | +| `tools/social-media/` | Social media tools - X/Twitter CLI, posting, reading | bird | | `tools/build-agent/` | Agent design - composing efficient agents, reviewing agent instructions | build-agent, agent-review | | `tools/build-mcp/` | MCP development - creating Model Context Protocol servers and tools | build-mcp, api-wrapper, server-patterns, transports, deployment | | `tools/ai-assistants/` | AI tool integration - configuring assistants, CAPTCHA solving, multi-modal agents | agno, capsolver, windsurf, configuration, status | @@ -434,6 +436,8 @@ For AI-assisted setup guidance, see `aidevops/setup.md`. | Agent design | `tools/build-agent/build-agent.md`, `tools/build-agent/agent-review.md` | | Database migrations | `workflows/sql-migrations.md` | | Framework internals | `aidevops/architecture.md` (when working in `~/Git/aidevops/`) | +| Content summarization | `tools/content/summarize.md` | +| X/Twitter automation | `tools/social-media/bird.md` | ## Security diff --git a/.agent/tools/content/summarize.md b/.agent/tools/content/summarize.md new file mode 100644 index 00000000..aeaa99c3 --- /dev/null +++ b/.agent/tools/content/summarize.md @@ -0,0 +1,353 @@ +--- +description: URL/YouTube/podcast summarization using steipete/summarize CLI +mode: subagent +tools: + read: true + write: false + edit: false + bash: true + glob: true + grep: true + webfetch: true + task: true +--- + +# Summarize CLI Integration + + + +## Quick Reference + +- **Purpose**: Summarize URLs, YouTube videos, podcasts, and files using AI +- **Install**: `npm i -g @steipete/summarize` or `brew install steipete/tap/summarize` +- **Repo**: https://github.com/steipete/summarize (726+ stars) +- **Website**: https://summarize.sh + +**Quick Commands**: + +```bash +# Summarize any URL +summarize "https://example.com" + +# YouTube video +summarize "https://youtu.be/dQw4w9WgXcQ" --youtube auto + +# Podcast RSS feed +summarize "https://feeds.npr.org/500005/podcast.xml" + +# Local file (PDF, images, audio/video) +summarize "/path/to/file.pdf" --model google/gemini-3-flash-preview + +# Extract content only (no summary) +summarize "https://example.com" --extract --format md +``` + +**Key Features**: + +- URLs, files, and media: web pages, PDFs, images, audio/video, YouTube, podcasts, RSS +- Real extraction pipeline: fetch -> clean -> Markdown (readability + markitdown) +- Transcript-first media flow: published transcripts when available, Whisper fallback +- Streaming TTY output with Markdown rendering +- Local, paid, and free models: OpenAI-compatible local endpoints, paid providers, OpenRouter free preset + +**Env Vars**: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, `OPENROUTER_API_KEY` + + + +## Installation + +### npm (recommended) + +```bash +# Global install +npm i -g @steipete/summarize + +# One-shot (no install) +npx -y @steipete/summarize "https://example.com" +``` + +### Homebrew (macOS Apple Silicon) + +```bash +brew install steipete/tap/summarize +``` + +### Requirements + +- Node.js 22+ +- Optional: `yt-dlp` for YouTube audio extraction +- Optional: `whisper.cpp` for local transcription +- Optional: `uvx markitdown` for enhanced preprocessing + +## Usage + +### Basic Summarization + +```bash +# Web page +summarize "https://example.com" + +# With specific model +summarize "https://example.com" --model openai/gpt-5-mini + +# Auto model selection (default) +summarize "https://example.com" --model auto +``` + +### YouTube Videos + +```bash +# Auto-detect transcript method +summarize "https://youtu.be/dQw4w9WgXcQ" --youtube auto + +# Supports youtube.com and youtu.be URLs +summarize "https://www.youtube.com/watch?v=dQw4w9WgXcQ" +``` + +### Podcasts + +```bash +# RSS feed (transcribes latest episode) +summarize "https://feeds.npr.org/500005/podcast.xml" + +# Apple Podcasts episode +summarize "https://podcasts.apple.com/us/podcast/2424-jelly-roll/id360084272?i=1000740717432" + +# Spotify episode (best-effort) +summarize "https://open.spotify.com/episode/5auotqWAXhhKyb9ymCuBJY" +``` + +### Local Files + +```bash +# PDF (Google models work best) +summarize "/path/to/document.pdf" --model google/gemini-3-flash-preview + +# Images +summarize "/path/to/image.png" + +# Audio/Video +summarize "/path/to/video.mp4" --video-mode transcript +``` + +### Output Control + +```bash +# Length presets: short, medium, long, xl, xxl +summarize "https://example.com" --length long + +# Character target +summarize "https://example.com" --length 20k + +# Hard token cap +summarize "https://example.com" --max-output-tokens 2000 + +# Output language +summarize "https://example.com" --language en +summarize "https://example.com" --lang auto # match source +``` + +### Content Extraction + +```bash +# Extract content without summarizing +summarize "https://example.com" --extract + +# Extract as Markdown +summarize "https://example.com" --extract --format md + +# Extract as plain text +summarize "https://example.com" --extract --format text +``` + +### Output Formats + +```bash +# JSON output with diagnostics +summarize "https://example.com" --json + +# Plain output (no ANSI/colors) +summarize "https://example.com" --plain + +# Disable streaming +summarize "https://example.com" --stream off +``` + +## Model Configuration + +### Supported Providers + +| Provider | Model ID Format | API Key | +|----------|-----------------|---------| +| OpenAI | `openai/gpt-5-mini` | `OPENAI_API_KEY` | +| Anthropic | `anthropic/claude-sonnet-4-5` | `ANTHROPIC_API_KEY` | +| Google | `google/gemini-3-flash-preview` | `GEMINI_API_KEY` | +| xAI | `xai/grok-4-fast-non-reasoning` | `XAI_API_KEY` | +| Z.AI | `zai/glm-4.7` | `Z_AI_API_KEY` | +| OpenRouter | `openrouter/openai/gpt-5-mini` | `OPENROUTER_API_KEY` | + +### Free Models via OpenRouter + +```bash +# Setup free model preset +OPENROUTER_API_KEY=sk-or-... summarize refresh-free --set-default + +# Use free models +summarize "https://example.com" --model free +``` + +### Configuration File + +Location: `~/.summarize/config.json` + +```json +{ + "model": { "id": "openai/gpt-5-mini" } +} +``` + +Or shorthand: + +```json +{ + "model": "openai/gpt-5-mini" +} +``` + +## Chrome Extension + +Summarize includes a Chrome Side Panel extension for one-click summarization. + +### Setup + +1. Install CLI: `npm i -g @steipete/summarize` +2. Build extension: `pnpm -C apps/chrome-extension build` +3. Load in Chrome: `chrome://extensions` -> Developer mode -> Load unpacked +4. Pick: `apps/chrome-extension/.output/chrome-mv3` +5. Open Side Panel -> copy install command +6. Run: `summarize daemon install --token ` + +### Daemon Commands + +```bash +# Check daemon status +summarize daemon status + +# Restart daemon +summarize daemon restart +``` + +## Advanced Features + +### Firecrawl Fallback + +For blocked or thin content, Firecrawl can be used as fallback: + +```bash +# Auto fallback (default) +summarize "https://example.com" --firecrawl auto + +# Force Firecrawl +summarize "https://example.com" --firecrawl always + +# Disable Firecrawl +summarize "https://example.com" --firecrawl off +``` + +Requires `FIRECRAWL_API_KEY` environment variable. + +### Whisper Transcription + +For audio/video without transcripts: + +```bash +# Force transcription mode +summarize "/path/to/video.mp4" --video-mode transcript + +# Environment variables +export SUMMARIZE_WHISPER_CPP_MODEL_PATH=/path/to/model.bin +export SUMMARIZE_WHISPER_CPP_BINARY=whisper-cli +export SUMMARIZE_DISABLE_LOCAL_WHISPER_CPP=1 # force remote +``` + +### Markdown Conversion + +```bash +# Readability mode (default) +summarize "https://example.com" --markdown-mode readability + +# LLM conversion +summarize "https://example.com" --markdown-mode llm + +# Auto (LLM when configured) +summarize "https://example.com" --markdown-mode auto + +# Disable +summarize "https://example.com" --markdown-mode off +``` + +## Common Flags + +| Flag | Description | +|------|-------------| +| `--model ` | Model to use (default: `auto`) | +| `--timeout ` | Request timeout (`30s`, `2m`, `5000ms`) | +| `--retries ` | LLM retry attempts (default: 1) | +| `--length ` | Output length control | +| `--language, --lang` | Output language (`auto` = match source) | +| `--max-output-tokens` | Hard cap for LLM output tokens | +| `--stream auto\|on\|off` | Stream LLM output | +| `--plain` | No ANSI/OSC Markdown rendering | +| `--no-color` | Disable ANSI colors | +| `--format md\|text` | Content format | +| `--extract` | Print extracted content and exit | +| `--json` | Machine-readable output | +| `--verbose` | Debug/diagnostics on stderr | +| `--metrics off\|on\|detailed` | Metrics output | + +## Integration with aidevops + +### Use Cases + +1. **Research**: Summarize articles, papers, documentation +2. **Content Curation**: Extract key points from multiple sources +3. **Podcast Notes**: Generate show notes from episodes +4. **Video Summaries**: Create text summaries of YouTube content +5. **Document Processing**: Summarize PDFs and reports + +### Example Workflow + +```bash +# Research a topic +summarize "https://docs.example.com/api" --length long --json > api-summary.json + +# Summarize multiple sources +for url in "${urls[@]}"; do + summarize "$url" --length medium >> research-notes.md +done + +# Extract and process +summarize "https://example.com" --extract --format md | process-content.sh +``` + +## Troubleshooting + +### Common Issues + +1. **Model not responding**: Check API key is set correctly +2. **YouTube transcript unavailable**: Install `yt-dlp` for audio extraction +3. **PDF extraction failing**: Use Google models for best PDF support +4. **Rate limiting**: Add `--timeout` and `--retries` flags + +### Debug Mode + +```bash +summarize "https://example.com" --verbose +``` + +## Resources + +- **GitHub**: https://github.com/steipete/summarize +- **Website**: https://summarize.sh +- **npm**: https://www.npmjs.com/package/@steipete/summarize +- **Docs**: https://github.com/steipete/summarize/tree/main/docs diff --git a/.agent/tools/social-media/bird.md b/.agent/tools/social-media/bird.md new file mode 100644 index 00000000..f5f00d01 --- /dev/null +++ b/.agent/tools/social-media/bird.md @@ -0,0 +1,437 @@ +--- +description: X/Twitter CLI for reading, posting, and replying using steipete/bird +mode: subagent +tools: + read: true + write: false + edit: false + bash: true + glob: true + grep: true + webfetch: true + task: true +--- + +# Bird CLI - X/Twitter Integration + + + +## Quick Reference + +- **Purpose**: Fast X/Twitter CLI for tweeting, replying, and reading +- **Install**: `npm i -g @steipete/bird` or `brew install steipete/tap/bird` +- **Repo**: https://github.com/steipete/bird (434+ stars) +- **Auth**: Uses browser cookies (Safari, Chrome, Firefox) + +**Quick Commands**: + +```bash +# Check logged-in account +bird whoami + +# Read a tweet +bird read https://x.com/user/status/1234567890123456789 + +# Post a tweet +bird tweet "Hello world!" + +# Reply to a tweet +bird reply 1234567890123456789 "Great post!" + +# Search tweets +bird search "from:steipete" -n 5 + +# Get mentions +bird mentions -n 5 +``` + +**Key Features**: + +- Tweet, reply, read via GraphQL API +- Thread and conversation viewing +- Search and mentions +- Bookmarks and likes management +- Following/followers lists +- Media uploads (images, GIFs, video) +- Browser cookie authentication (no API keys needed) + +**Auth Methods**: Browser cookies (Safari/Chrome/Firefox), or manual `--auth-token` and `--ct0` + + + +## Installation + +### npm (recommended) + +```bash +# Global install +npm i -g @steipete/bird + +# One-shot (no install) +bunx @steipete/bird whoami +``` + +### Homebrew (macOS Apple Silicon) + +```bash +brew install steipete/tap/bird +``` + +## Authentication + +Bird uses your existing X/Twitter web session via browser cookies. No API keys or passwords required. + +### Cookie Sources (in order of precedence) + +1. CLI flags: `--auth-token`, `--ct0` +2. Environment variables: `AUTH_TOKEN`, `CT0` (or `TWITTER_AUTH_TOKEN`, `TWITTER_CT0`) +3. Browser cookies via `@steipete/sweet-cookie` + +### Browser Cookie Locations + +| Browser | Cookie Path | +|---------|-------------| +| Safari | `~/Library/Cookies/Cookies.binarycookies` | +| Chrome | `~/Library/Application Support/Google/Chrome//Cookies` | +| Firefox | `~/Library/Application Support/Firefox/Profiles//cookies.sqlite` | + +### Verify Authentication + +```bash +# Check which account is logged in +bird whoami + +# Check available credentials +bird check +``` + +### Manual Cookie Override + +```bash +bird tweet "Hello" --auth-token YOUR_AUTH_TOKEN --ct0 YOUR_CT0_TOKEN +``` + +## Commands + +### Reading Tweets + +```bash +# Read a single tweet (URL or ID) +bird read https://x.com/user/status/1234567890123456789 +bird read 1234567890123456789 + +# Shorthand (just the URL/ID) +bird 1234567890123456789 + +# JSON output +bird read 1234567890123456789 --json + +# View thread/conversation +bird thread https://x.com/user/status/1234567890123456789 + +# View replies to a tweet +bird replies 1234567890123456789 +``` + +### Posting Tweets + +```bash +# Post a new tweet +bird tweet "Hello world!" + +# Tweet with media (up to 4 images or 1 video) +bird tweet "Check this out!" --media image.png --alt "Description" + +# Multiple images +bird tweet "Photos" --media img1.png --media img2.png --alt "First" --alt "Second" +``` + +### Replying + +```bash +# Reply to a tweet +bird reply 1234567890123456789 "Great post!" + +# Reply with URL +bird reply https://x.com/user/status/1234567890123456789 "Nice!" + +# Reply with media +bird reply 1234567890123456789 "Here's my response" --media response.png +``` + +### Search + +```bash +# Search tweets +bird search "from:steipete" -n 5 + +# Search with JSON output +bird search "AI tools" -n 10 --json +``` + +### Mentions + +```bash +# Your mentions +bird mentions -n 5 + +# Another user's mentions +bird mentions --user @steipete -n 5 + +# JSON output +bird mentions -n 10 --json +``` + +### Bookmarks + +```bash +# List bookmarks +bird bookmarks -n 5 + +# Specific bookmark folder +bird bookmarks --folder-id 123456789123456789 -n 5 + +# All bookmarks with pagination +bird bookmarks --all --max-pages 2 --json + +# Remove bookmark +bird unbookmark 1234567890123456789 +bird unbookmark https://x.com/user/status/1234567890123456789 +``` + +### Likes + +```bash +# List your liked tweets +bird likes -n 5 + +# JSON output +bird likes -n 10 --json +``` + +### Following/Followers + +```bash +# Who you follow +bird following -n 20 + +# Who follows you +bird followers -n 20 + +# Another user's following/followers (by user ID) +bird following --user 12345678 -n 10 +bird followers --user 12345678 -n 10 +``` + +### Utility Commands + +```bash +# Show help +bird help +bird help tweet + +# Refresh GraphQL query IDs cache +bird query-ids --fresh + +# Version info +bird --version +``` + +## Global Options + +| Option | Description | +|--------|-------------| +| `--auth-token ` | Set auth_token cookie manually | +| `--ct0 ` | Set ct0 cookie manually | +| `--cookie-source ` | Choose browser (safari, chrome, firefox) | +| `--chrome-profile ` | Chrome profile for cookies | +| `--firefox-profile ` | Firefox profile for cookies | +| `--cookie-timeout ` | Cookie extraction timeout | +| `--timeout ` | Request timeout | +| `--quote-depth ` | Max quoted tweet depth in JSON (default: 1) | +| `--plain` | Stable output (no emoji, no color) | +| `--no-emoji` | Disable emoji output | +| `--no-color` | Disable ANSI colors | +| `--media ` | Attach media file (repeatable, up to 4) | +| `--alt ` | Alt text for media (repeatable) | + +## Configuration + +### Config File (JSON5) + +Locations: +- Global: `~/.config/bird/config.json5` +- Project: `./.birdrc.json5` + +Example `~/.config/bird/config.json5`: + +```json5 +{ + // Cookie source order for browser extraction + cookieSource: ["firefox", "safari"], + firefoxProfile: "default-release", + cookieTimeoutMs: 30000, + timeoutMs: 20000, + quoteDepth: 1 +} +``` + +### Environment Variables + +```bash +# Timeout settings +BIRD_TIMEOUT_MS=20000 +BIRD_COOKIE_TIMEOUT_MS=30000 +BIRD_QUOTE_DEPTH=1 +``` + +## JSON Output Schema + +### Tweet Object + +| Field | Type | Description | +|-------|------|-------------| +| `id` | string | Tweet ID | +| `text` | string | Full tweet text | +| `author` | object | `{ username, name }` | +| `authorId` | string? | Author's user ID | +| `createdAt` | string | Timestamp | +| `replyCount` | number | Number of replies | +| `retweetCount` | number | Number of retweets | +| `likeCount` | number | Number of likes | +| `conversationId` | string | Thread conversation ID | +| `inReplyToStatusId` | string? | Parent tweet ID (if reply) | +| `quotedTweet` | object? | Embedded quote tweet | + +### User Object (following/followers) + +| Field | Type | Description | +|-------|------|-------------| +| `id` | string | User ID | +| `username` | string | Username/handle | +| `name` | string | Display name | +| `description` | string? | User bio | +| `followersCount` | number? | Followers count | +| `followingCount` | number? | Following count | +| `isBlueVerified` | boolean? | Blue verified flag | +| `profileImageUrl` | string? | Profile image URL | +| `createdAt` | string? | Account creation timestamp | + +## Media Uploads + +### Supported Formats + +- **Images**: jpg, jpeg, png, webp, gif +- **Video**: mp4, mov + +### Limits + +- Up to 4 images/GIFs, OR 1 video (no mixing) +- Video processing may take longer + +### Examples + +```bash +# Single image with alt text +bird tweet "Check this out!" --media photo.jpg --alt "A beautiful sunset" + +# Multiple images +bird tweet "Photo dump" \ + --media img1.png --alt "First image" \ + --media img2.png --alt "Second image" \ + --media img3.png --alt "Third image" + +# Video +bird tweet "Watch this!" --media video.mp4 +``` + +## GraphQL Query IDs + +X rotates GraphQL query IDs frequently. Bird handles this automatically: + +- Ships with baseline mapping in `src/lib/query-ids.json` +- Runtime cache at `~/.config/bird/query-ids-cache.json` +- Auto-recovery on 404 errors (refreshes and retries) +- TTL: 24 hours + +### Manual Refresh + +```bash +# Force refresh query IDs +bird query-ids --fresh + +# View current query IDs +bird query-ids --json +``` + +## Integration with aidevops + +### Use Cases + +1. **Social Media Automation**: Schedule and post content +2. **Engagement Monitoring**: Track mentions and replies +3. **Research**: Search and analyze tweets +4. **Content Curation**: Bookmark and organize content +5. **Analytics**: Export data for analysis + +### Example Workflows + +```bash +# Post announcement +bird tweet "New release v2.0 is out! Check the changelog: https://example.com/changelog" + +# Monitor mentions and respond +bird mentions -n 10 --json | jq '.[] | select(.text | contains("help"))' + +# Export bookmarks for analysis +bird bookmarks --all --json > bookmarks.json + +# Thread a long post +bird tweet "1/3 Here's a thread about..." +# Get the tweet ID from output, then: +bird reply "2/3 Continuing the thread..." +bird reply "3/3 Final thoughts..." +``` + +### Combining with summarize + +```bash +# Summarize a linked article and tweet about it +url="https://example.com/article" +summary=$(summarize "$url" --length short --plain) +bird tweet "Interesting read: $summary + +$url" +``` + +## Troubleshooting + +### Common Issues + +1. **Cookie extraction fails**: Ensure browser is logged into X/Twitter +2. **Rate limiting (429)**: Wait and retry, or use different account +3. **Query ID invalid (404)**: Run `bird query-ids --fresh` +4. **Error 226 (automated request)**: Bird auto-falls back to legacy endpoint + +### Debug Mode + +```bash +# Check credentials +bird check + +# Verify account +bird whoami + +# Test with plain output +bird --plain check +``` + +## Disclaimer + +This tool uses X/Twitter's undocumented web GraphQL API with cookie authentication. X can change endpoints, query IDs, and anti-bot behavior at any time - expect potential breakage. + +## Resources + +- **GitHub**: https://github.com/steipete/bird +- **npm**: https://www.npmjs.com/package/@steipete/bird +- **Changelog**: https://github.com/steipete/bird/blob/main/CHANGELOG.md diff --git a/TODO.md b/TODO.md index 305868bf..b81a1f58 100644 --- a/TODO.md +++ b/TODO.md @@ -82,8 +82,8 @@ Tasks with no open blockers - ready to work on. Use `/ready` to refresh this lis - [ ] t031 Company orchestration agent/workflow inspired by @DanielleMorrill #plan #agents #business ~4h (ai:2h test:1h read:1h) logged:2025-01-03 ref:https://x.com/DanielleMorrill/status/2007508036584341899 - [ ] t032 Create performance skill/subagent/command inspired by @elithrar #tools #performance ~3h (ai:1.5h test:1h read:30m) logged:2025-01-03 ref:https://x.com/elithrar/status/2007455910218871067 - [ ] t033 Add X/Twitter fetching via fxtwitter API (x.sh script) #tools #browser ~2h (ai:1h test:30m read:30m) logged:2025-01-03 ref:https://gist.github.com/marckohlbrugge/93bcf631c3317e793f0295e6155e6e7f -- [ ] t034 Add steipete/summarize for URL/YouTube/podcast summarization #tools #content ~2h (ai:1h test:30m read:30m) logged:2025-01-03 -- [ ] t035 Add steipete/bird CLI for X/Twitter reading and posting #tools #social-media ~2h (ai:1h test:30m read:30m) logged:2025-01-03 +- [ ] t034 Add steipete/summarize for URL/YouTube/podcast summarization #tools #content ~2h (ai:1h test:30m read:30m) logged:2025-01-03 started:2026-01-11T04:32Z +- [ ] t035 Add steipete/bird CLI for X/Twitter reading and posting #tools #social-media ~2h (ai:1h test:30m read:30m) logged:2025-01-03 started:2026-01-11T04:32Z - [ ] t036 Verify CodeRabbit CLI usage in code-review agents (coderabbit review --plain) #tools #code-review ~1h (ai:30m test:15m read:15m) logged:2025-01-03 - [ ] t037 Review ALwrity for SEO/marketing capabilities or inspiration #research #seo #marketing ~3h (ai:1.5h test:30m read:1h) logged:2025-01-03 - [ ] t038 Add CDN origin IP leak detection subagent (Cloudmare-inspired) #security #dns #hosting ~4h (ai:2h test:1h read:1h) logged:2025-01-03