Skip to content

Fix catchup prompt loading all entries into memory#58

Merged
cmeans merged 1 commit into
mainfrom
fix/catchup-prompt-query
Mar 26, 2026
Merged

Fix catchup prompt loading all entries into memory#58
cmeans merged 1 commit into
mainfrom
fix/catchup-prompt-query

Conversation

@cmeans
Copy link
Copy Markdown
Owner

@cmeans cmeans commented Mar 26, 2026

Summary

  • The catchup prompt was calling get_knowledge() and get_active_alerts() without a since parameter, loading all entries into Python and filtering in-memory
  • Both methods already accept a since parameter that pushes the filter to SQL — this PR passes it through
  • Prevents linear performance degradation as the knowledge store grows

QA

Prerequisites

  • pip install -e ".[dev]"
  • Deploy to test instance on alternate port (AWARENESS_PORT=8421)

Manual tests (via MCP tools)

    • Seed entries at different ages
    remember(source="qa-test", description="Old entry created well before window", tags=["qa-catchup"])
    

    Wait a moment, then:

    remember(source="qa-test", description="Recent entry created within window", tags=["qa-catchup"])
    

    Expected: Two entries exist in the store with different updated timestamps

    • Verify catchup filters correctly
      Invoke the catchup prompt with a small time window (e.g., hours=1).
      Expected: Only entries updated within the last hour appear. The old entry (if created more than 1 hour ago) should not appear. If both are recent, both appear — the key is that the filtering happens server-side (verify via query logs or by checking that entries outside the window are excluded).
    • Verify catchup with default window
      Invoke the catchup prompt with default parameters (no hours argument).
      Expected: Returns entries updated in the last 24 hours, grouped by source, with [new] or [updated] markers.
    • Verify empty catchup
      Invoke the catchup prompt with hours=0.
      Expected: Returns "Nothing changed. You're up to date." since no entries were updated in the last 0 hours.
    • Cleanup
    delete_entry(source="qa-test", tags=["qa-catchup"], confirm=true)
    

    Expected: Test entries removed

🤖 Generated with Claude Code

Push since filter to get_knowledge() and get_active_alerts() instead
of fetching everything and filtering in Python. Prevents linear
degradation as the store grows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Owner Author

@cmeans cmeans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA Review — PR #58: Fix catchup prompt loading all entries into memory

Automated Tests

  • pytest: 333/333 passed
  • ruff: 0 violations
  • mypy: 0 errors

Code Review

Clean, minimal fix. since parameter correctly pushed to SQL for both get_knowledge() and get_active_alerts(). Eliminates O(n) in-memory filtering. No issues.

Manual QA (5/5 steps passed)

  1. Seed entries at different ages

    • Created two entries with 2s gap — both confirmed in store with different updated timestamps
  2. Catchup filters correctly (hours=1)

    • Returned 18 entries all within 1-hour window, grouped by source with [new]/[updated] markers
  3. Catchup with default window (24 hours)

    • Returned 18 entries, all marked [new], grouped by source. Correct format.
  4. Empty catchup (hours=0)

    • Output: "Nothing changed. You're up to date."
  5. Cleanup

    • delete_entry(source="qa-test", tags=["qa-catchup"], confirm=true) → trashed 2 entries ✓

Findings

None. Clean fix, all tests pass.

@cmeans cmeans added the QA Approved Manual QA testing completed and passed label Mar 26, 2026
@cmeans cmeans merged commit 0c0be8a into main Mar 26, 2026
9 checks passed
@cmeans cmeans deleted the fix/catchup-prompt-query branch March 26, 2026 10:37
cmeans-claude-dev Bot added a commit that referenced this pull request Apr 27, 2026
…400)

## Linked issue

Fixes # — operational fix; tracked via the cross-repo
`dependabot-pr-hygiene-playbook` awareness entry (logical_key
`dependabot-pr-hygiene-playbook`). Surfaced empirically by this repo's
two current open Dependabot PRs (#397 and #398) which both ship with
doubled-prefix titles and no CHANGELOG entries.

## Summary

Ports the cross-repo Dependabot pattern validated yesterday in
cmeans/mcp-synology (PRs #58 + #63 ordering fix) and cascaded to
cmeans/mcp-clipboard and cmeans/pypi-winnow-downloads. After this lands,
Dependabot PRs on this repo will:

- Open with single-prefix titles (`chore(deps): bump ...`, not
`chore(deps)(deps): ...`)
- Auto-prepend a `### Changed` CHANGELOG entry with populated version
arrows (`pkg X→Y, ...`)
- Pick up ecosystem labels (`python` / `github-actions` / `docker`) so
they sort filterably alongside manual dep work
- Re-fire required CI checks (`lint`, `typecheck`, `test`, `qa-gate`,
`scan`) on the bot's follow-up commit so main-protection doesn't block
merge

## Scope

```
.github/dependabot.yml                       | 24 ++++++++++++++++++------
.github/workflows/dependabot-changelog.yml   | new (224 lines)
CHANGELOG.md                                 |  6 ++++++
3 files changed, 248 insertions(+), 6 deletions(-)
```

No source code touched, no tests touched, no schema touched.

## What this changes

1. **New `.github/workflows/dependabot-changelog.yml`** —
`pull_request_target` workflow filtered to `dependabot[bot]`. Mints a
GitHub App installation token via
`actions/create-github-app-token@1b10c78c…` (v3.1.1, SHA-pinned), checks
out the PR head with that token, fetches Dependabot metadata via
`dependabot/fetch-metadata@25dd0e34…` (v3.1.0, SHA-pinned), composes a
one-line `### Changed` entry, prepends to `CHANGELOG.md`, commits as
`cmeans-claude-dev[bot]` (numeric user id `272174644` in the noreply
email so commits resolve to the bot account), pushes back via the App
token so required CI checks re-fire.

2. **`.github/dependabot.yml` commit-message prefix corrected** —
`chore(deps)` → `chore` in all four ecosystem blocks. Combined with
`include: scope`, bare `chore` produces the canonical `chore(deps): bump
foo`. The previous setting produced the doubled `chore(deps)(deps):` you
can see on PRs #397 and #398.

3. **Ecosystem labels added to dependabot.yml** — `python` for pip,
`github-actions` for actions, `docker` for both Docker and
docker-compose. The four labels (`dependencies`, `python`,
`github-actions`, `docker`) were created on the repo separately via `gh
label create` so Dependabot doesn't silently skip them.

## Playbook gotchas folded in

- **`pull_request_target` not `pull_request`** — Dependabot's
`GITHUB_TOKEN` is read-only on `pull_request`; the workflow couldn't
push back.
- **GitHub App token not `secrets.GITHUB_TOKEN`** — pushes by
`GITHUB_TOKEN` don't trigger downstream `pull_request` workflows
(anti-loop policy), so required checks never run on the bot's commit and
main-protection blocks merge.
- **`dependabot/fetch-metadata@v3.1.0` not v2.x** — v2.x returns empty
`prevVersion`/`newVersion` on grouped updates, producing entries like
`Bump foo →, bar →` with empty arrows.
- **Numeric `BOT_USER_ID` (`272174644`) not `APP_ID` (`3223881`)** in
the commit author noreply email so commits resolve back to
`cmeans-claude-dev[bot]`. Wrong id breaks `require_last_push_approval`.
- **Keep-a-Changelog v1.1.0 subsection ordering** (Added → Changed →
Deprecated → Removed → Fixed → Security) — when creating a fresh `###
Changed`, walks forward to insert before the first later-sorting
subsection or the next `## ` release heading rather than placing at
`unreleased_idx + 1`. Reference fix: cmeans/mcp-synology PR #63.

## Adjustment for this repo's CHANGELOG style

This repo uses bracketed Keep-a-Changelog headings (`## [Unreleased]`,
`## [0.18.3] - 2026-04-24`); cmeans/mcp-synology uses unbracketed (`##
Unreleased`). The detection logic in this port accepts both forms
(`is_unreleased(line)` returns `True` for either); the fresh-create path
uses the bracketed form to match this repo's existing release headings.
The improvement should be folded back into the canonical
`dependabot-pr-hygiene-playbook` so future cascades don't have to
re-discover it — followup tracked in
[#401](#401), not
blocking.

## AI-assistance disclosure

- [ ] No AI used in producing this PR
- [x] AI assisted with code generation (e.g., Copilot, Cursor, Claude
Code)
- [x] AI assisted with review / suggestions during authoring
- [x] AI assisted with the PR body or commit messages

## QA

### Prerequisites

- `BOT_APP_ID` + `BOT_APP_PRIVATE_KEY` repo secrets configured (operator
confirmed today).
- Repo labels `dependencies`, `python`, `github-actions`, `docker` exist
(verified via `gh label list -R cmeans/mcp-awareness`).
- This is an operational change — verification is GitHub-side, not
local. No `pip install` or test deps required.

### Manual tests (via MCP tools and GitHub state)

The playbook's verification gate is end-to-end live data, not local unit
tests. Walk it after merge.

1. - [ ] **Workflow runs on `@dependabot recreate` of PR #397** — after
this PR merges to `main`, post `@dependabot recreate` on
#397. Within ~30s GitHub
Actions should fire `Dependabot CHANGELOG` on the recreated PR. Expected
outcome on the recreated PR:
- Title rewritten to `chore(deps): bump the github-actions group with N
updates` (single prefix, not doubled)
- A new bot-authored commit appears on the branch authored by
`cmeans-claude-dev[bot]` adding a CHANGELOG entry under `[Unreleased]` /
`### Changed`
- The CHANGELOG entry reads roughly `- **Bump github-actions group:
actions/foo X→Y, actions/bar X→Y** (#<recreated-PR-number>)` with
populated version arrows
- Required CI checks (`lint`, `typecheck`, `test (3.10)` … `test
(3.14)`, `qa-gate`, `scan`) all run and pass on the bot's HEAD SHA
   - `dependencies` and `github-actions` labels are present on the PR

2. - [ ] **Workflow runs on `@dependabot recreate` of PR #398** — same
flow, on #398. Expected:
- Title rewritten to `chore(deps): bump ollama/ollama from 0.21.0 to
0.21.2 in the docker-compose group`
   - CHANGELOG entry with populated `0.21.0→0.21.2` arrow
   - `dependencies` and `docker` labels present

3. - [ ] **Loop guard** — the bot's CHANGELOG-adding commit must NOT
re-trigger the workflow. After step 1 lands, check `gh run list -R
cmeans/mcp-awareness --workflow dependabot-changelog.yml --limit 5`.
Expected: exactly one run per Dependabot PR commit; no infinite-loop
runs.

4. - [ ] **Idempotency guard** — if step 1's PR is recreated again
(e.g., a force-push), the workflow should skip rather than duplicate the
CHANGELOG entry. Optional smoke test if you happen to recreate twice.

### Failure modes to watch for

- Workflow fails at `Mint GitHub App installation token` step → secrets
not configured or App not installed on the repo.
- Workflow runs but pushes nothing → loop guard / idempotency guard
tripped (check workflow logs for the `skipping` message).
- CHANGELOG entry has empty arrows (`Bump foo →,bar →`) →
`fetch-metadata` regression; verify the SHA pin still resolves to
v3.1.0.
- Required CI checks don't run on the bot's commit → App-token mint
failed and the workflow fell back to `GITHUB_TOKEN`; check the App-token
step output.

## Checklist

- [x] `CHANGELOG.md` entry added under `[Unreleased]` in
Keep-a-Changelog format (both `### Added` and `### Changed`)
- [x] `README.md` and `docs/data-dictionary.md` updated if affected —
not affected (no schema or tool count change)
- [x] No secrets, credentials, API tokens, signing keys, or `.env`
contents included in the diff
- [x] `ruff check`, `mypy`, and `pytest` pass locally (1014 passed, 7
skipped — unchanged from main)
- [x] CLA — bot commit; CLA bypass workflow handles
`cmeans-claude-dev[bot]` per `.github/cla-bot-allowlist`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: cmeans-claude-dev[bot] <272174644+cmeans-claude-dev[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

QA Approved Manual QA testing completed and passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant