Skip to content

fix(capture): skip "Review the conversation above" internal prompts - #1517

Closed
SonicBotMan wants to merge 41 commits into
MemTensor:mainfrom
SonicBotMan:fix/review-conversation-prompt-1776871798
Closed

fix(capture): skip "Review the conversation above" internal prompts#1517
SonicBotMan wants to merge 41 commits into
MemTensor:mainfrom
SonicBotMan:fix/review-conversation-prompt-1776871798

Conversation

@SonicBotMan

Copy link
Copy Markdown

Summary

Add a content-based filter in captureMessages() to skip "Review the conversation above..." internal agent prompts before they reach storage.

Files changed: apps/memos-local-openclaw/src/capture/index.ts

Changes

+const REVIEW_CONVERSATION_RE = /^Review the conversation above/i;

     if (BOOT_CHECK_RE.test(msg.content.trim())) {
       continue;
     }
+    if (role === "user" && REVIEW_CONVERSATION_RE.test(msg.content.trim())) {
+      continue;
+    }

Why

Internal Hermes → Agent review prompts ("Review the conversation above, consider saving...") are injected with role: "user", bypassing existing filters. They accumulate as noise in the memory chunk store (~37 records per production instance).

See commit: SonicBotMan@512d45a

Skip internal Hermes → Agent review prompts (role:user) at capture time
to prevent them from polluting the memory chunk store.

Added REVIEW_CONVERSATION_RE pattern and corresponding skip check
in captureMessages(), following the same approach as BOOT_CHECK_RE.
…ble probe

When the user provides an endpoint that already includes the full path
(e.g. https://open.bigmodel.cn/api/paas/v4/chat/completions for
Zhipu AI / GLM models), the probe would previously append another
/v1/chat/completions segment, resulting in a malformed URL.

This fix checks if the endpoint already ends with the target path
(/chat/completions or /embeddings) before appending, making the
probe compatible with endpoints that specify the full URL.

Affects both probeChat and probeEmbedding for openai_compatible provider.
@Memtensor-AI
Memtensor-AI changed the base branch from main to dev-20260604-v2.0.19 June 10, 2026 15:39
Memtensor-AI and others added 8 commits June 14, 2026 17:24
docs(memos-local-plugin): clarify install path and stale dir names (MemTensor#1540)

The README's 'Quick start' section told users to use install.sh instead
of npm install, but the warning was buried and users still tried
'npm install -g @memtensor/memos-local-plugin' first. The reporter in
MemTensor#1540 encountered this on a Hermes deployment.

This change:

- Promotes the 'do not run npm install -g' notice to a prominent
  IMPORTANT callout explaining why global install is wrong (no
  agent-home deploy, no config.yaml, no bridge/viewer) and that the
  tarball intentionally ships built artifacts only.
- Adds a Troubleshooting subsection covering the two specific symptoms
  in the bug report: the 'package not found' misread, and the stale
  web/ and site/ directory names (web/ is now viewer/, site/ was
  removed by commit 26e7e3d).
- Mentions install.ps1 for Windows alongside install.sh.
- CHANGELOG: record the docs fix and reference MemTensor#1540.

Documentation-only change; no code or runtime behavior touched.

Co-authored-by: MemOS AutoDev <autodev@memtensor.ai>
Co-authored-by: Matthew <heimixiaozhuang@zju.edu.cn>
…_() got an unexpected keyword a (MemTensor#1889)

fix: remove invalid chunker parameter from SystemParser test instantiation

- SystemParser.__init__() signature changed to (embedder, llm=None)
- Test was still passing chunker=None causing TypeError
- Fixes all 5 failing tests in test_system_parser.py

Fixes MemTensor#1888

Co-authored-by: MemOS AutoDev <autodev@memos.ai>
Co-authored-by: Matthew <heimixiaozhuang@zju.edu.cn>
…tributeError when given None (MemTensor#1884)

* test: add comprehensive tests for clean_json_response (issue MemTensor#1525)

- Add test suite in tests/mem_os/test_format_utils.py
- Cover None input ValueError with diagnostic message
- Cover markdown removal, whitespace stripping, edge cases
- Verify fix for AttributeError when LLM returns None

* style: format clean_json_response tests

---------

Co-authored-by: MemOS AutoDev <autodev@memos.ai>
Co-authored-by: Matthew <heimixiaozhuang@zju.edu.cn>
…date_cube_access — fails for ev (MemTensor#1903)

fix: validate current user not target in share_cube_with_user (MemTensor#1901)

share_cube_with_user(cube_id, target_user_id) called
_validate_cube_access(cube_id, target_user_id), but the validator
signature is (user_id, cube_id). The cube_id therefore landed in the
user_id slot and _validate_user_exists raised
"User '<cube_id>' does not exist or is inactive" for every well-formed
call, making the API unusable.

The in-code comment "Validate current user has access to this cube"
already documented the correct intent: the sharing user (self.user_id)
must have access to the cube being shared, not the target. Switch the
call to self._validate_cube_access(self.user_id, cube_id). The target
user's existence is independently checked on the next line via
validate_user(target_user_id), so that path is unchanged.

Add regression tests in tests/mem_os/test_memos_core.py that pin down:
- validate_user_cube_access is consulted with (self.user_id, cube_id),
- add_user_to_cube is called with (target_user_id, cube_id) on success,
- a missing target raises "Target user '<id>' does not exist".

Closes MemTensor#1901

Co-authored-by: MemOS AutoDev Bot <autodev@memtensor.local>
Co-authored-by: Matthew <heimixiaozhuang@zju.edu.cn>
runReflect's orphan-steps fallback was inserting new trace rows whenever
a recovered episode's snapshot timestamps didn't match existing DB rows.
This caused trace_ids_json to grow on every bridge restart, keeping
reward.traceCount != traceIds.length and looping forever — generating
48k+ empty traces and continuous OpenRouter traffic.

Guard: when meta.recoveryReason === "dirty_reward_rescore", log and skip
the insert instead of writing new rows. Legitimate orphan handling (test
paths, genuinely dropped events) is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
daemon_manager.py: extend ensure_viewer_daemon health-poll deadline
from 15s to 45s to cover cold Node.js starts (tsx compile + SQLite
open + FTS warmup). After the first 15s, back off probes from 0.5s
to 2s to avoid hammering a slow-starting daemon.

__init__.py: increase _open_session timeout in init from 30s to 60s
so session.open doesn't time out before the daemon finishes starting.

Add a 1s double-spawn guard: when probe_viewer_status() returns
"free", wait 1s and re-probe before falling through to
ensure_viewer_daemon(). Eliminates the race where a second gateway
session spawns a second stdio bridge because the daemon from the
first session hasn't bound the port yet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The RPC endpoint was unconditionally exempt from session-cookie auth
to allow the local Python adapter to connect without a browser session.
On hub-mode deployments (bindHost=0.0.0.0) this left the endpoint
reachable from the network without any auth when password protection
is enabled.

Now the exemption only applies when remoteAddress is 127.0.0.1, ::1,
or ::ffff:127.0.0.1. Standard installs (bindHost=127.0.0.1) are
unaffected; the Python adapter always connects from loopback and
continues to work. Network callers on hub deployments must hold a
valid session cookie.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- auth.ts: add reverse-proxy assumption comment to loopback RPC exemption
- capture.ts + memory-core.ts: extract magic string to RECOVERY_REASONS.DIRTY_REWARD_RESCORE in new recovery-constants.ts
- memory-core.test.ts: update assertions to use shared constant
- __init__.py: extract _connect_http_bridge() helper; fix unbound http_bridge on constructor exception; conditional cold-start sleep
- daemon_manager.py: guard negative sleep in backoff loop; add probe_viewer_status() + startup_lock_active() helpers
@Memtensor-AI
Memtensor-AI changed the base branch from dev-20260604-v2.0.19 to dev-v2.0.22 July 1, 2026 13:17
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

Automated Test Results: PASSED

Cloud test-engine rerun against dev-v2.0.22 completed successfully after adding focused scope mapping for OpenClaw capture changes.

  • Run: tr-dfd2b3d4-fd3 on cloud test-engine 10011
  • Scope: memos_local_openclaw/unit
  • Result: 11 passed, 0 failed, 0 skipped

This replaces the earlier broad OpenClaw full-suite failure, which was unrelated to this PR's src/capture/index.ts change. Manual code review is still required before merge.

@CarltonXiang
CarltonXiang deleted the branch MemTensor:main July 3, 2026 07:25
@syzsunshine219 syzsunshine219 reopened this Jul 3, 2026
@syzsunshine219
syzsunshine219 changed the base branch from dev-v2.0.22 to main July 3, 2026 08:24
chiefmojo and others added 6 commits July 4, 2026 07:25
…ategyStructMemReader (MemTensor#1794)

Co-authored-by: Memtensor-AI <project@memtensor.cn>
* docs: clarify API product model descriptions

* docs: normalize product model descriptions

---------

Co-authored-by: Lucas-FManager <luong.nguyen188@gmail.com>
Co-authored-by: Lucas-FManager <265058144+Lucas-FManager@users.noreply.github.com>
Co-authored-by: Memtensor-AI <project@memtensor.cn>
…or#1931)

Fixes two occurrences of "monitering" → "monitoring" in
APIADDRequest and APIFeedbackRequest task_id field descriptions
in src/memos/api/product_models.py.

Co-authored-by: Memtensor-AI <project@memtensor.cn>
* docs(cn): translate remaining English text in dream.md

* chore: update github workflow stale.yaml

* fix: quote OpenClaw FTS query terms

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
@Memtensor-AI Memtensor-AI added area:memory 记忆存储、检索、更新、召回逻辑 area:plugin OpenClaw & Hermes labels Jul 8, 2026
@Memtensor-AI
Memtensor-AI requested a review from bittergreen July 8, 2026 11:44
…mOSClient signatures (MemTensor#1919)

* docs(cn): translate remaining English text in dream.md

* docs: align Quick Start examples with MemOSClient signatures and add EN translations

Fix the Chat and Message API reference docs where Quick Start code
examples used parameters not present in the actual MemOSClient:

- chat/chat.md: replace readable_cube_ids/writable_cube_ids/mode with
  conversation_id/knowledgebase_ids to match client.chat() signature
- message/feedback.md: replace history/writable_cube_ids/retrieved_memory_ids/
  corrected_answer with conversation_id to match client.add_feedback()
- message/get_suggestion_queries.md: replace MemOSClient.get_suggestions()
  (method does not exist) with raw HTTP POST request; remove duplicate H1 heading

Also add English translations for all four files under docs/en/.

* chore: update github workflow stale.yaml

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: jiachengzhen <jiacz@memtensor.cn>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
Co-authored-by: Memtensor-AI <project@memtensor.cn>
RerankerGuo and others added 11 commits July 9, 2026 10:32
* docs(cn): translate remaining English text in dream.md

* chore: update github workflow stale.yaml

* fix: load bridge package version without require

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
* docs(cn): translate remaining English text in dream.md

* chore: update github workflow stale.yaml

* fix: log llm error response bodies

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
…ponse (MemTensor#2064)

* fix(skill-crystallize): retry with context when LLM returns empty response

* chore: add retry_succeeded log so successful rescues are visible
When the correction retry rescues a previously-failed crystallization,
log a warning with policyId and the original error so operators can
distinguish first-try successes from retry-rescued ones.

* fix: allow retry when raw response is empty string (not just null)
The condition  treats empty string as falsy, so when
the LLM returns blank output the correction retry was never triggered.
Changed to  so empty responses also get a retry.

---------

Co-authored-by: 34262315716 <34262315716@users.noreply.github.com>
* docs(cn): translate remaining English text in dream.md

* chore: update github workflow stale.yaml

* fix: avoid blocking OpenClaw registration rebuild

* fix(openclaw): alias pluginDir to moduleDir after detectPluginDir removal

OCR review on MemTensor#2044 flagged that after removing detectPluginDir(moduleDir),
the local 'pluginDir' variable is still referenced on lines 270 and 273
(path.join(pluginDir, 'package.json') and new Telemetry(..., pluginDir)),
which would cause a ReferenceError at runtime.

Alias pluginDir = moduleDir at the same scope so the existing
references keep working without re-introducing the removed helper.

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
Co-authored-by: Ziyang Guo <121015044+RunMarshal@users.noreply.github.com>
…th (MemTensor#1817)

## Summary

Three targeted fixes from companion infrastructure stabilization:

### 1. Skip orphan trace insert during dirty-reward recovery
(`fc7252fc`)
When `dirtyFlag` triggers a rescore on episodes with no matching turns,
the capture path inserts a trace with `r_reward=0` but no valid
`idParent`. This creates orphan rows that inflate the database and break
FTS indexing. This fix checks for the orphan condition and skips the
trace insert entirely, logging a warning instead.

### 2. Bridge startup race (`ef311ebb`)
Hermes gateway restarts can race with the MemOS daemon warm-up (~30s).
The default 15s poll deadline and 30s `session.open` timeout cause
cascading failures when the bridge isn't ready. This extends the
deadline to 45s, adds exponential backoff health probes, and ups the
session timeout to 60s. Double-spawn prevention via PID file singleton
guard ensures only one bridge process survives a race.

### 3. RPC loopback auth (`a459a0a5`)
The `/api/v1/rpc` session exemption was overly broad — any caller could
bypass auth. This restricts the exemption to loopback callers only
(`127.0.0.1` / `::1`).

## Testing

All three fixes have been running on three companion instances
(neuromancer, wintermute, case) for 48+ hours with zero bridge failures,
no new orphan traces, and clean RPC auth.
…t actually skip evolution pipel (MemTensor#2074)

* fix(memos-local-plugin): actually skip evolution pipeline when lightweightMemory is enabled (MemTensor#2063)

`buildPipelineSubscribers` used to attach reward / L2 / L3 / skill /
feedback subscribers unconditionally, so `algorithm.lightweightMemory.enabled: true`
only affected `flush()` — every `episode.finalized` still cascaded
through the LLM-heavy evolution chain.

Startup + periodic recovery compounded the bug: `recoverOpenEpisodesAsSessionEnd`
and `recoverDirtyClosedEpisodes` re-emitted `episode.finalized` for
legacy episodes that lack `meta.lightweightMemory === true`, producing
a large backlog of `skill_generate` / `world_model_generate` /
`policy_evolve` calls after every bridge restart.

- Gate subscriber attachment in `buildPipelineSubscribers` and return
  no-op stubs for the runner/handle shape callers depend on.
- Treat every orphan / stale / dirty-closed episode as lightweight in
  the recovery paths of `memory-core.init`, `autoFinalizeStaleTasks`,
  and `autoRescoreDirtyClosedEpisodes` when the flag is on.
- Regression coverage in `tests/unit/pipeline/lightweight-mode.test.ts`.

* fix(memos-local-plugin): address open code review findings on lightweight mode

- deps.ts: return throwLightweight(...) in rewardSubscription.runManually and
  l2.runOnce so control-flow analysis marks subsequent statements unreachable,
  matching the pattern used by every other throwing stub in the file.
- memory-core.ts (autoFinalizeStaleTasks lightweight path): write the
  lightweight meta BEFORE close() so a failing close() cannot leave an
  episode without meta.lightweightMemory=true (which would let the periodic
  non-lightweight rescan pick it up). Also rename the meta fields to
  closeReason="lightweight_stale" + closedAtMs so observability tooling does
  not mistake a periodic close for startup recovery.
- memory-core.ts (init lightweight orphan close): replace the nested ternary
  for recoveryReason with an explicit if/else.

* fix(memos-local-plugin): address round-2 open code review on init lightweight close

Three issues in the init() lightweight-orphan-close loop:

1. Ordering: close() was called BEFORE updateMeta(). If close() succeeded
   but updateMeta() threw, the episode ended up closed without
   meta.lightweightMemory=true, so autoRescoreDirtyClosedEpisodes would
   later pick it up and feed it into the reward/L2/L3 pipeline. Now the
   meta write happens first — matching the periodic stale-topic path.

2. Error isolation: the loop had no per-episode try/catch, so a single
   DB lock / constraint violation would abort the whole loop and be
   swallowed by the outer scan-level try/catch. Now each episode has its
   own try/catch that emits init.lightweight_close_error at debug.

3. Backfill semantics: legacy rows being backfilled were tagged
   closeReason="finalized" even though they were never actually
   finalized. They now get closeReason="lightweight_backfill", matching
   the existing recoveryReason="lightweight_startup_close_backfill"
   distinction so downstream analytics can tell the two apart.

---------

Co-authored-by: autodev <autodev@memtensor.local>
* docs(cn): translate remaining English text in dream.md

* chore: update github workflow stale.yaml

* feat: add configurable CJK keyword tokenization

---------

Co-authored-by: harvey_xiang <harvey_xiang22@163.com>
Co-authored-by: sonimwang <17816198144@163.com>
Co-authored-by: HarveyXiang <harvey_xiang@163.com>
Summary
- Fixes MemTensor#1421.
- Add optional `inputType`, `queryInputType`, and `documentInputType`
embedding config fields.
- Route document embeddings through `documentInputType` and query
embeddings through `queryInputType`, falling back to `inputType` when a
specific value is absent.
- Forward `input_type` to OpenAI-compatible and OpenClaw host embedding
requests when configured.
- Retry the viewer embedding model probe once with `input_type: "query"`
when a provider explicitly reports that `input_type` is required.

Validation
- static embedding input_type checks passed
- npm test -- --run tests/embedding-input-type.test.ts (blocked: vitest:
not found)
- npm run build (blocked: tsc: not found)
- npm run lint (blocked: eslint: not found)
- make format (blocked: poetry: Command not found)
…1474) (MemTensor#1475)

feat(memos-local): add data export endpoint and UI (MemTensor#1474)

- SqliteStore.exportAll(): dump all memories/tasks/skills as JSON
- SqliteStore.exportMemoriesAsCsv(): memories-only CSV with ISO timestamps
- GET /api/export?format=json|csv endpoint in ViewerServer
- Export buttons in Settings → General tab (i18n: en + zh)
- exportData() JS helper triggers file download via <a> click

Closes MemTensor#1474

Co-authored-by: zhaxi <syzsunshine219@gmail.com>
…ble probe (MemTensor#1663)

## Problem

When configuring `openai_compatible` provider with an endpoint that
already includes the full API path (e.g. Zhipu AI / GLM models):

```yaml
endpoint: https://open.bigmodel.cn/api/paas/v4/chat/completions
```

The probe function would append another `/v1/chat/completions` segment,
resulting in a malformed URL:

```
https://open.bigmodel.cn/api/paas/v4/chat/completions/v1/chat/completions
```

This causes the model test to fail for any provider whose endpoint
already specifies the full path.

## Solution

Check if the endpoint already ends with the target path
(`/chat/completions` or `/embeddings`) before appending. The URL
resolution now handles three cases:

1. Endpoint ends with `/chat/completions` → use as-is
2. Endpoint ends with `/v1` → append `/chat/completions`
3. Otherwise → append `/v1/chat/completions`

The same fix was applied symmetrically to both `probeChat` and
`probeEmbedding`.

## Files Changed

- `apps/memos-local-plugin/server/routes/models.ts`

## Backward Compatibility

Fully backward compatible — existing endpoint formats (`/v1` or base
URL) continue to work as before. This only adds support for the
already-full-path case.
@Memtensor-AI Memtensor-AI added the area:memory 记忆存储、检索、更新、召回逻辑 label Jul 9, 2026
…ist (MemTensor#1669)

Co-authored-by: zhaxi <syzsunshine219@gmail.com>
# Conflicts:
#	src/memos/api/config.py
@Memtensor-AI Memtensor-AI removed the area:memory 记忆存储、检索、更新、召回逻辑 label Jul 9, 2026
@syzsunshine219
syzsunshine219 changed the base branch from main to dev-v2.0.23 July 9, 2026 03:47
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: NO TEST SCOPE

Automated tests were not run because the changed files do not map to an executable test scope.

Details: No executable test scope maps to the changed files. Automated tests were not run; add env.yaml source_mapping + execution for this path family, then rerun. Changed files: (none detected)
Manual review or env.yaml source_mapping/execution coverage is required before merge.

Branch: fix/review-conversation-prompt-1776871798

bittergreen and others added 5 commits July 9, 2026 11:48
## Description

fix: Disable thinking for qwen3 preference extractor

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

- Not applicable

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [ ] I have created related documentation issue/PR in
[MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) |
我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档
issue/PR(如果适用)
- [x] I have linked the issue to this PR (if applicable) | 我已将 issue
链接到此 PR(如果适用)
- [x] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人

## Reviewer Checklist
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] Made sure Checks passed
- [ ] Tests have been provided
…rable (MemTensor#1483)

* feat(memos-local): make auto-recall Phase 1 parameters configurable

Auto-recall Phase 1 was using hardcoded maxResults=10 and minScore=0.45
for the quick local search pass before LLM dedup. This makes those
values configurable via the existing recall.* config namespace:

  recall.autoRecallMaxResults  (default: 3)
  recall.autoRecallMinScore    (default: 0.50)

Lowering Phase 1 defaults from 10→3 and 0.45→0.50 improves auto-recall
precision and reduces token usage during the dedup step, especially for
users with large memory stores.

* fix: align auto-recall config with base

---------

Co-authored-by: 庸人i <yong@yongrenideMac-mini.local>
Co-authored-by: jiachengzhen <jiacz@memtensor.cn>
@syzsunshine219
syzsunshine219 changed the base branch from dev-v2.0.23 to main July 9, 2026 06:35
@shinetata

Copy link
Copy Markdown
Collaborator

Thanks for this fix, @SonicBotMan! 🙏

The exact same issue (#1518) was already resolved and merged via #2042 — the REVIEW_CONVERSATION_RE filter in apps/memos-local-openclaw/src/capture/index.ts is now on main, so this change is already live.

This branch has also drifted onto the dev-v2.0.x line (now ~41 commits / 63 files against main), and the capture fix itself no longer shows in the diff after it was aligned with main. Rather than merge the unrelated churn into main, I'm closing this as superseded by #2042. Really appreciate the contribution — please open fresh, single-purpose branches off the latest main for future fixes.

@shinetata shinetata closed this Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes

Projects

None yet

Development

Successfully merging this pull request may close these issues.