Skip to content

Add daily memory files for per-day notes and context injection - #10

Merged
dylanneve1 merged 9 commits into
mainfrom
daily-memory
Apr 4, 2026
Merged

Add daily memory files for per-day notes and context injection#10
dylanneve1 merged 9 commits into
mainfrom
daily-memory

Conversation

@dylanneve1

@dylanneve1 dylanneve1 commented Apr 4, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds per-day memory files (~/.talon/workspace/memory/daily/YYYY-MM-DD.md) for the bot to write observations, learnings, corrections, and follow-ups throughout the day
  • These are the bot's own notes — separate from raw conversation logs and the long-term memory.md
  • The system prompt references the daily memory directory so the bot can read files on demand (not injected into the prompt, to preserve caching)

How it works

  • Real-time: Bot can write to today's daily file during conversations via the Write tool
  • Heartbeat (hourly): Reviews logs and updates today's daily memory via {{dailyMemoryFile}} template variable
  • Dream (12h): Consolidates daily memories into memory.md for long-term storage via {{dailyMemoryDir}} template variable
  • Cleanup: Daily memory files older than 30 days are pruned alongside conversation logs (timezone-aware, validated YYYY-MM-DD.md pattern)

Information lifecycle

Daily notes (short-term) → dream consolidation → memory.md (long-term) → daily files age out

Key design decisions

  • Daily memory is referenced, not injected into the system prompt — avoids prompt bloat and preserves prompt caching
  • All date computations are timezone-aware using toYMD() / todayAndYesterday() from time.ts
  • Cleanup validates the YYYY-MM-DD.md filename pattern before deleting

Files changed (12)

  • src/util/paths.ts — new dirs.dailyMemory constant
  • src/util/time.ts — export toYMD() and todayAndYesterday()
  • src/util/workspace.ts — create memory/daily/ directory on startup
  • src/util/config.ts — reference daily memory dir in system prompt
  • src/storage/daily-log.ts — timezone-aware 30-day cleanup for daily memory files
  • src/core/heartbeat.ts{{dailyMemoryFile}} template variable (timezone-aware)
  • src/core/dream.ts{{dailyMemoryDir}} template variable
  • prompts/heartbeat.md — daily notes in default tasks
  • prompts/dream.md — daily memory in consolidation stage
  • src/__tests__/heartbeat.test.ts — updated path mock
  • src/__tests__/dream.test.ts — updated path mock
  • src/__tests__/daily-log.test.ts — daily memory cleanup tests

Test plan

  • Type check passes
  • Full test suite: 1284 tests passing across 43 files
  • Prettier formatting clean
  • Fully additive — zero breaking changes to existing memory system

🤖 Generated with Claude Code

- New directory: ~/.talon/workspace/memory/daily/ for bot's daily notes
- Inject today's + yesterday's daily memory into system prompt at session start
- Add daily memory file reference to heartbeat prompt ({{dailyMemoryFile}})
- Add daily memory directory reference to dream prompt ({{dailyMemoryDir}})
- Clean up daily memory files older than 30 days alongside log cleanup
- Update workspace description in prompt to mention daily notes
- Update all test mocks with new dailyMemory directory path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sed-inserted lines had incorrect indentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a short-term “daily memory” layer (per-day markdown notes) alongside existing persistent memory.md, enabling heartbeat/dream agents to write daily summaries and injecting recent daily notes into the system prompt for better continuity.

Changes:

  • Introduces dirs.dailyMemory and ensures workspace/memory/daily/ is created on startup.
  • Injects today + yesterday daily notes into the assembled system prompt and documents the new daily note location in workspace instructions.
  • Extends retention cleanup to prune daily memory files older than 30 days; wires template variables into heartbeat/dream prompts.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/util/paths.ts Adds dirs.dailyMemory path constant for daily note storage.
src/util/workspace.ts Ensures the daily memory directory exists during workspace initialization.
src/util/config.ts Injects recent daily notes into the system prompt; updates workspace description.
src/storage/daily-log.ts Extends startup cleanup to prune old daily memory files alongside logs.
src/core/heartbeat.ts Adds {{dailyMemoryFile}} template variable to the heartbeat prompt.
src/core/dream.ts Adds {{dailyMemoryDir}} template variable to the dream prompt.
prompts/heartbeat.md Updates default heartbeat tasks to write to today’s daily memory file.
prompts/dream.md Instructs dream consolidation to also write per-day daily memory summaries.
src/tests/heartbeat.test.ts Updates mocked paths to include dailyMemory.
src/tests/dream.test.ts Updates mocked paths to include dailyMemory across test cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/util/config.ts Outdated
Comment thread src/util/config.ts Outdated
Comment thread src/core/heartbeat.ts Outdated
Comment thread src/storage/daily-log.ts Outdated
Covers old file deletion, missing directory handling, and retention
of recent files — addresses Copilot review comment on PR #10.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/storage/daily-log.ts
Prevents accidental deletion of non-daily files (e.g. 2020-summary.md)
that happen to sort before the cutoff string.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/storage/daily-log.ts
Comment thread src/__tests__/daily-log.test.ts
dylanneve1 and others added 2 commits April 4, 2026 22:52
Address Copilot review comments:
- Use toYMD() instead of UTC toISOString() for daily memory filenames
- Use todayAndYesterday() for timezone-aware date computation in config
- Add 10KB size cap per daily memory file in system prompt injection
- Export toYMD and todayAndYesterday from time.ts
- Use toYMD() for daily memory cleanup cutoff date computation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/util/config.ts Outdated
Daily memory files are no longer read and injected into the system
prompt. Instead, the bot is told where they are and can read them
on demand. This avoids prompt bloat and preserves prompt caching.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/storage/daily-log.ts
Comment thread prompts/heartbeat.md Outdated
Comment thread prompts/dream.md Outdated
- daily memory cleanup no longer depends on logs dir existing
- heartbeat/dream prompts say "read on demand" not "injected into context"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/__tests__/daily-log.test.ts
Comment thread src/util/paths.ts
- Daily memory tests use toYMD() instead of UTC toISOString()
- Add memory/daily/ to the paths.ts layout comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/util/time.ts:42

  • todayAndYesterday() derives "yesterday" by subtracting a fixed 86_400_000ms. Around DST transitions (and other offset changes), “24h ago” is not guaranteed to be the previous calendar day in the configured timezone, which can mislabel timestamps and any callers relying on true day boundaries. Consider computing yesterday as the previous calendar date in the configured timezone (e.g., step backward until toYMD() changes, or use a timezone-aware date arithmetic approach) rather than subtracting a fixed millisecond interval.
export function todayAndYesterday(): { today: string; yesterday: string } {
  const now = new Date();
  const today = toYMD(now);
  const yd = new Date(now.getTime() - 86_400_000);
  const yesterday = toYMD(yd);
  return { today, yesterday };

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dylanneve1
dylanneve1 merged commit 7628466 into main Apr 4, 2026
13 checks passed
dylanneve1 added a commit that referenced this pull request Apr 10, 2026
- Validate mcpServer.args elements are strings and reject empty command (#1)
- Shell-quote interpolated paths in dream bash commands (#2, #11)
- Replace CLI-based diary write with mempalace_diary_write MCP tool (#3)
- Make validation error message platform-agnostic (#4)
- Update mempalacePython comment for platform-dependent default (#5)
- Wrap mp.init() in Promise.race with 30s timeout (#6)
- Make init conditional on successful validation, pass actual config (#7)
- Move import mempalace check into validateConfig (#8)
- Replace execFileSync with async execFile in init() (#9)
- Document that registerPlugin does NOT call init (#10)
- Update dream prompt header from "4-stage" to "5-stage" (#12)
- Update getPluginMcpServers JSDoc to document mcpServer path (#13, #17)
- Add .min(1) to palacePath/pythonPath zod schemas (#14)
- Distinguish ENOENT/EACCES/EPERM from import failures in validation (#15)
- Fix test name from "logs warning" to match actual behavior (#16)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dylanneve1 added a commit that referenced this pull request Apr 10, 2026
* feat: integrate mempalace as built-in plugin for long-term memory

Adds mempalace (Python MCP server) as a first-class memory system.
When enabled, the agent gets semantic search, knowledge graph, and
verbatim memory storage via ChromaDB — all local, zero API calls.

Key changes:
- Extend plugin system with `mcpServer` field for non-Node MCP servers
- Add `registerPlugin()` for built-in plugin registration
- Create mempalace plugin (factory pattern, validates python venv)
- Wire mempalace into dream mode (Stage 5: mine logs into palace)
- Add `mempalace` config schema (enabled, palacePath, pythonPath)
- Add default paths for palace dir and python venv binary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(dream): mine daily notes instead of raw logs, add diary writing

Dream Stage 5 now mines memory/daily/ (curated observations) instead of
raw logs/ directory, eliminating junk chunks (tool JSON, df output, etc).
Added personal diary writing instruction — agent reflects on feelings,
state of mind, learnings, and loose threads after each dream run.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #27 review comments

- plugin.ts: validate mcpServer.args entries are strings and reject empty command
- dream.ts: quote interpolated paths in shell commands, replace mcp_server CLI diary with direct file write
- mempalace/index.ts: platform-agnostic error message for missing python binary
- paths.ts: update comment to reflect platform-dependent venv path
- bootstrap.ts: wrap mempalace init in 30s timeout to match loadSinglePlugin behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore mempalace CLI diary writer, keep path quoting

Copilot suggested removing the mcp_server CLI invocation for diary
writing but that's the intended mempalace interface. Restored it
with quoted paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add gating/validation tests for mempalace integration

- plugin.ts: test rejection of empty mcpServer.command and non-string args elements
- dream.ts: test mempalace section gating — verify mining/diary instructions
  only appear when mempalace is configured, skip message when not
- 1306 tests passing (4 new)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: upgrade mempalace system prompt with comprehensive tool docs

Adapted from mempalace SKILL.md (v3.1.0). Key improvements:
- Session protocol (verify before responding, invalidate stale facts)
- Full tool documentation including kg_timeline, traverse, find_tunnels,
  diary_read/write, delete_drawer, graph_stats, check_duplicate
- Semantic search tips (meaning-based, not keyword)
- Knowledge graph temporal validity guidance
- Tests updated to verify all tool names appear in prompt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract mempalace prompt to prompts/mempalace.md

Move system prompt instructions out of TypeScript into a .md file,
matching the pattern used by dream.md and other prompts. Plugin loads
and interpolates {{palacePath}} at runtime with graceful fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace pixi.intel.com registry URLs in lockfile with npmjs.org

Lockfile had resolved URLs pointing to pixi.intel.com (private/corporate
registry) for @Anthropic-AI packages, causing CI to fail with ENOTFOUND.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve all lint warnings and formatting issues

Remove unused imports, variables, and catch bindings across 14 files.
Add yield statements to generator function mocks. Fix prettier formatting.

0 lint warnings, 0 format issues, 1307 tests passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address all 17 Copilot review comments on PR #27

- Validate mcpServer.args elements are strings and reject empty command (#1)
- Shell-quote interpolated paths in dream bash commands (#2, #11)
- Replace CLI-based diary write with mempalace_diary_write MCP tool (#3)
- Make validation error message platform-agnostic (#4)
- Update mempalacePython comment for platform-dependent default (#5)
- Wrap mp.init() in Promise.race with 30s timeout (#6)
- Make init conditional on successful validation, pass actual config (#7)
- Move import mempalace check into validateConfig (#8)
- Replace execFileSync with async execFile in init() (#9)
- Document that registerPlugin does NOT call init (#10)
- Update dream prompt header from "4-stage" to "5-stage" (#12)
- Update getPluginMcpServers JSDoc to document mcpServer path (#13, #17)
- Add .min(1) to palacePath/pythonPath zod schemas (#14)
- Distinguish ENOENT/EACCES/EPERM from import failures in validation (#15)
- Fix test name from "logs warning" to match actual behavior (#16)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add 'mempalace' to LogComponent type

TypeScript type check was failing because 'mempalace' wasn't in the
LogComponent union type used by log/logError/logWarn functions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow MCP tools in dream prompt when required by Stage 5

Update tool access statement to permit MCP tools for mempalace
mining stage instead of blanket-blocking all MCP tools.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: duplicate guard in registerPlugin, pass MCP servers to dream

- registerPlugin now checks for duplicates before setting env vars
  or logging success, preventing misleading logs and env clobbering
- Dream agent now receives mempalace MCP servers when configured,
  so Stage 5 diary/mining tools actually work

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add selective MCP server loading via 'only' filter

getPluginMcpServers now accepts an optional plugin name filter:
- omitted = all plugins (backwards compatible for chat sessions)
- [] = none
- ["mempalace"] = only mempalace

Dream mode uses ["mempalace"] to load only what it needs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: load mempalace prompt from dirs.prompts, fix dream systemPrompt

- Mempalace prompt now loads from ~/.talon/prompts/mempalace.md
  (user-customisable, seeded on first run) instead of relative to
  source file. Consistent with heartbeat/dream prompt loading.
- Dream systemPrompt now permits MemPalace MCP tools when configured,
  preventing conflict with the markdown prompt's Stage 5 instructions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move duplicate check before validation, gate dream on plugin registration

- registerPlugin checks for duplicates before running validateConfig,
  avoiding expensive re-validation on accidental double registration
- Dream mempalace integration now gated on getPlugin("mempalace")
  instead of just config.mempalace.enabled, so failed validation
  or registration doesn't cause dream-time MCP tool failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mempalace): correct CLI arg order for status check

The --palace flag is a global option that must come before the
subcommand. Wrong order caused the init health check to always
fail with exit 2, logging a misleading "not yet initialized" warning.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address Copilot review round 8 — validation, error handling, unused param

- Validate `import mempalace.mcp_server` (actual spawned module) instead of
  just `import mempalace` in validateConfig
- Add timeout/killed error branching in validateConfig catch block
  (ETIMEDOUT, signal, killed) with specific messages instead of generic
  "not installed"
- Include stderr details in import failure messages for debugging
- Remove unused `config` from ProcessAndReplyParams and all processAndReply
  call sites (flushQueue, retry, callback handler)
- Replace `mempalace status` CLI smoke test in init() with a simple import
  check — fixes false "Palace not yet initialized" warning when palace IS
  initialized but CLI subcommand doesn't exist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove config from message queue chain, fix pythonPath comment

- Remove config from queue entry type, enqueueMessage signature, and
  all 3 call sites — completes the cleanup started in round 8
- Eliminates unnecessary TalonConfig reference (including botToken) from
  queue state
- Update mempalace plugin header comment to document platform-dependent
  pythonPath default (bin/python on Unix, Scripts/python.exe on Windows)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants