Skip to content

feat(skills): add read-book and book-to-notes skills - #13671

Open
juancrfig wants to merge 1 commit into
NousResearch:mainfrom
juancrfig:feat/read-book-skills
Open

feat(skills): add read-book and book-to-notes skills#13671
juancrfig wants to merge 1 commit into
NousResearch:mainfrom
juancrfig:feat/read-book-skills

Conversation

@juancrfig

@juancrfig juancrfig commented Apr 21, 2026

Copy link
Copy Markdown

Add two complementary note-taking skills for deep reading and knowledge extraction from long-form texts, filling the missing ingestion layer for the LLM Wiki.

What does this PR do?

The llm-wiki skill provides an excellent architecture for persistent knowledge — raw sources, cross-linked concept pages, schema enforcement, and querying. But its ingestion flow assumes the agent can hold the entire source in context. This works for articles and papers, but fails for books: a 300-page text is ~100k tokens, which degrades quality and buries early chapters.

This PR adds the missing long-form ingestion layer:

read-book — A Socratic four-pass reading protocol (structural → emphatic → critical → syntopical) based on Adler's How to Read a Book. The agent does not summarize; it produces original understanding: reframed models, critiques, applications, or teachings.

book-to-notes — A Zettelkasten ingestion pipeline for books too long for single-context processing. Uses subagent delegation with disk-state pattern:

  • Parent agent never holds the book
  • Subagents extract atomic notes (one idea per file, sentence-titled, linked, source-provenanced)
  • Synthesis pass reads only the atomic notes to build wiki concept pages

Tested end-to-end on Flow: The Psychology of Optimal Experience (Csikszentmihalyi, 314 pages): 5 subagent chunks, 66 atomic notes, 39 preserved quotes, 12 cross-linked concept pages.

Both skills are vault-agnostic (default ~/notes/, configurable via BOOK_VAULT_PATH) and feed directly into the existing llm-wiki skill's concept/entity layer.

Related Issue

No existing issue. This is a new capability gap identified during real-world usage.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • skills/note-taking/read-book/SKILL.md — Deep reading protocol (Adler's four passes: structural, emphatic, critical, syntopical)
  • skills/note-taking/book-to-notes/SKILL.md — Zettelkasten extraction pipeline with subagent delegation, disk-state pattern, and architecture decisions
  • skills/note-taking/book-to-notes/scripts/split_book.py — Cross-platform chapter splitting utility (pure Python stdlib + pathlib)
  • skills/note-taking/book-to-notes/templates/reading-tool.md — Subagent quality checklist enforcing two-pass extraction, concept crystallization, and anti-parroting guards

How to Test

  1. Place a PDF or text book in a persistent directory (e.g., ~/books/)
  2. Run pdftotext book.pdf book.txt to extract text (Unix; on Windows use ocr-and-documents skill or pymupdf fallback)
  3. Ask Hermes: "Use the book-to-notes skill to read ~/books/flow.txt and extract concepts into ~/notes/"
  4. Verify subagents produce atomic notes in the output directory
  5. Verify synthesis generates concept pages linking atomic notes
  6. For read-book protocol: Ask "Use the read-book skill to help me understand [book]" and verify the four-pass structure (structural map → chapter digests → critical appraisal → knowledge integration)

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass (ran pytest tests/cli -q — 476 passed; skills are markdown files, not exercised by core test suite)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features) — N/A for skill addition; script tested manually
  • I've tested on my platform: Ubuntu 24.04

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guidesplit_book.py is pure Python stdlib + pathlib; pdftotext recommendation is Unix-only but skill documents EPUB and OCR fallbacks
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

  • This skill is broadly useful to most users (if bundled) — see Contributing Guide
  • SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
  • No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools) — split_book.py is pure stdlib. pdftotext is recommended but advisory; skill documents EPUB and OCR fallbacks
  • I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y" — skills load in hermes skills list and hermes chat; tested ingestion pipeline on 314-page book producing 66 atomic notes and 12 concept pages

Screenshots / Logs

Tested on Flow: The Psychology of Optimal Experience (Mihaly Csikszentmihalyi, 314 pages, 10 chapters)

Add two complementary note-taking skills for deep reading and
knowledge extraction:

- read-book: Socratic four-pass protocol (structural, emphatic,
  critical, syntopical) based on Adler's How to Read a Book.
  Produces original synthesis, not summaries.

- book-to-notes: Automated extraction pipeline for long texts.
  Uses subagent delegation with disk-state pattern to handle
  books up to ~400 pages without context overflow. Includes
  chapter splitting script and subagent quality checklist.

Both skills are vault-agnostic and configurable via BOOK_VAULT_PATH.
Tested end-to-end on a 314-page psychology text (Csikszentmihalyi,
Flow), producing 66 atomic notes and 12 concept pages.

Requires: terminal, files, delegate toolsets (declared in metadata).
@alt-glitch alt-glitch added type/feature New feature or request comp/tools Tool registry, model_tools, toolsets tool/skills Skills system (list, view, manage) labels Apr 21, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the long-form reading workflow and the reusable splitting helper. The capability remains distinct from current llm-wiki, but this needs a few corrections before salvage.

Problems

  • skills/note-taking/book-to-notes/SKILL.md:13 requires files and delegate, while current toolsets are file and delegation (toolsets.py:191-195, toolsets.py:245-248). The prompt builder hides a skill when any required toolset is absent (agent/prompt_builder.py:1418-1421).
  • SKILL.md:23 puts BOOK_VAULT_PATH in .env; AGENTS.md:888-914 requires behavioral settings in config.yaml. The existing skill-config mechanism is documented in agent/skill_utils.py:622-765.
  • scripts/split_book.py:7 advertises a non-capturing --pattern, but line 42 depends on captures being retained by re.split, so that documented form cannot yield title/content pairs.
  • The added skills do not meet current description, contributor-credit, and test requirements (AGENTS.md:888-900, AGENTS.md:926-950).

Suggested changes

  • Use [terminal, file, delegation], migrate vault configuration to metadata.hermes.config, and add focused skill/script tests.
  • Make custom chapter patterns work without capture groups, or validate the required capture group explicitly.

Automated hermes-sweeper review.

category: note-taking
related_skills: [read-book, ocr-and-documents, obsidian, llm-wiki]
requires_toolsets: [terminal, files, delegate]
---

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

files and delegate are not current toolset names (file and delegation are). Because required toolsets are an all-required visibility gate, this makes the skill hidden when normal toolset filtering is active. Please use [terminal, file, delegation] and add a visibility regression test.


This skill writes to a configurable vault. Set your preferred path via environment variable:

```bash

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a non-secret behavioral setting, so it must not be introduced through .env. Declare a metadata.hermes.config path setting instead; Hermes resolves those under skills.config.* and can prompt for it during setup.

return None

parts = re.split(pattern, text)
if len(parts) < 3:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documented --pattern "CHAPTER\\s+[0-9]+" has no capture group. re.split then drops the marker, while the loop below assumes alternating marker/content entries. Either split from finditer spans or validate and document a required capture group; add a test for the advertised command.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows tool/skills Skills system (list, view, manage) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants