Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ jobs:
close-pr-message: 'This PR has been automatically closed due to inactivity.'
days-before-stale: 30 # Days of inactivity before marking as stale
days-before-close: 7 # Days of inactivity before closing stale issues/PRs
stale-issue-label: 'stale'
stale-pr-label: 'stale'
exempt-issue-labels: 'do not close'
exempt-pr-labels: 'do not close'
stale-issue-label: 'status:stale'
stale-pr-label: 'status:stale'
exempt-issue-labels: 'status:do not close'
exempt-pr-labels: 'status:do not close'
remove-stale-when-updated: true
2 changes: 1 addition & 1 deletion docs/cn/open_source/modules/dream.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Dream 模拟这一点:从*未完成的内在动机*出发,而不是从原始
```json
{
"motive_id": "motive:dream_memory_strategy_alignment",
"description": "Several conversations failed for the same hidden reason: weekly reporting, future planning, and filter design were treated as separate tasks, while the user needed a shared strategic narrative.",
"description": "几次对话失败,背后是同一个隐藏原因:周报、未来规划和 filter 设计被当成三个独立任务,而用户需要的是一条统一的战略叙事。",
"memory_ids": ["weekly_report_thread", "future_planning_thread", "filter_design_thread"]
}
```
Expand Down
48 changes: 48 additions & 0 deletions docs/en/open_source/open_source_api/help/error_codes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: Error Codes
---

| Error Code | Meaning | Recommended Action |
| :--- | :--- | :--- |
| **Parameter Errors** | | |
| 40000 | Invalid request parameters | Check whether parameter names, types, and formats meet the requirements. |
| 40001 | Requested data does not exist | Check whether the resource ID, such as `memory_id`, is correct. |
| 40002 | Required parameter is empty | Add the missing required field. |
| 40003 | Parameter is empty | Check whether the provided list or object is empty. |
| 40006 | Unsupported type | Check the value of the `type` field. |
| 40007 | Unsupported file type | Upload only allowed formats: `.pdf`, `.docx`, `.doc`, `.txt`. |
| 40008 | Invalid Base64 content | Check whether the Base64 string contains invalid characters. |
| 40009 | Invalid Base64 format | Check whether the Base64 encoding format is correct. |
| 40010 | User ID is too long | `user_id` must not exceed 100 characters. |
| 40011 | Conversation ID is too long | `conversation_id` must not exceed 100 characters. |
| 40020 | Invalid project ID | Confirm that the Project ID format is correct. |
| **Authentication and Permission Errors** | | |
| 40100 | API Key authentication required | Add a valid API Key to the request header. |
| 40130 | API Key authentication required | Add a valid API Key to the request header. |
| 40132 | API Key is invalid or expired | Check the API Key status or regenerate it. |
| **Quota and Rate Limit Errors** | | |
| 40300 | API call limit exceeded | <a href="/memos_cloud/limit#_4-领取更多额度" target="_blank">Get more quota</a>. |
| 40301 | Request token call limit exceeded | Reduce input content or get more quota. |
| 40302 | Response token call limit exceeded | Shorten the expected output or get more quota. |
| 40303 | Single conversation length exceeds the limit | Reduce the length of a single input or output. |
| 40304 | Account API call quota exhausted | <a href="/memos_cloud/limit#_4-领取更多额度" target="_blank">Get more quota</a>. |
| 40305 | Input exceeds the single-request token limit | Reduce input content. |
| 40306 | Delete memory authorization failed | Confirm that you have permission to delete the memory. |
| 40307 | Memory to delete does not exist | Check whether `memory_id` is valid. |
| 40308 | User for the memory to delete does not exist | Check whether `user_id` is correct. |
| **System and Service Errors** | | |
| 50000 | Internal system exception | The server is busy or encountered an exception. Contact support if it persists. |
| 50002 | Operation failed | Check the operation logic or retry later. |
| 50004 | Memory service temporarily unavailable | Retry memory write or retrieval later. |
| 50005 | Search service temporarily unavailable | Retry memory search later. |
| **Knowledge Base and Operation Errors** | | |
| 50103 | File count exceeds the limit | Upload no more than 20 files in one request. |
| 50104 | Single file size exceeds the limit | Ensure each file is no larger than 100 MB. |
| 50105 | Total file size exceeds the limit | Ensure the total upload size is no larger than 300 MB. |
| 50107 | File upload format does not meet requirements | Check and replace the file format. |
| 50120 | Knowledge base does not exist | Confirm that the knowledge base ID is correct. |
| 50123 | Knowledge base is not associated with this project | Confirm that the knowledge base is authorized for the current project. |
| 50131 | Task does not exist | Check whether `task_id` is correct. This is common when querying processing status. |
| 50143 | Failed to add memory | The algorithm service encountered an exception. Retry later. |
| 50144 | Failed to add message | Failed to save chat history. |
| 50145 | Failed to save feedback and write memory | An exception occurred while processing feedback. |
99 changes: 99 additions & 0 deletions docs/en/open_source/open_source_api/scheduler/get_status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
title: Scheduler Status
desc: Monitor the lifecycle of MemOS asynchronous tasks, including task progress, queue backlog, and system load.
---

**API Paths**:
* **System overview**: `GET /product/scheduler/allstatus`
* **Task status query**: `GET /product/scheduler/status`
* **User queue metrics**: `GET /product/scheduler/task_queue_status`

**Description**: These endpoints provide observability for the asynchronous memory-production pipeline. You can track a specific task, monitor Redis queue backlog, and inspect global scheduler metrics.

## 1. Core Mechanism: MemScheduler

In the open-source architecture, **MemScheduler** handles time-consuming background work such as LLM memory extraction and vector index construction:

* **Status transitions**: A task moves through states such as `waiting`, `in_progress`, `completed`, or `failed`.
* **Queue monitoring**: Task distribution is based on Redis Stream. Monitoring `pending` and `remaining` counts helps estimate processing pressure.
* **Multilevel observability**: Inspect status from three perspectives: a single task, a single user's queue, or a system-wide summary.

## 2. Endpoint Details

### 2.1 Task Status Query (`/status`)

Use this endpoint to track the current execution stage of a specific asynchronous task.

| Parameter | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **`user_id`** | `str` | Yes | Unique identifier of the user whose task status is being queried. |
| `task_id` | `str` | No | Optional. If provided, only this task is queried. |

**Status values**:
* `waiting`: The task has entered the queue and is waiting for an available worker.
* `in_progress`: A worker is extracting memory with an LLM or writing to storage.
* `completed`: Memory has been persisted and vector indexes have been synchronized.
* `failed`: The task failed.

### 2.2 User Queue Metrics (`/task_queue_status`)

Use this endpoint to monitor the Redis task backlog for a specific user.

| Parameter | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **`user_id`** | `str` | Yes | User ID whose queue status should be queried. |

**Core metrics**:
* `pending_tasks_count`: Number of tasks delivered to workers but not yet acknowledged.
* `remaining_tasks_count`: Number of tasks still queued and waiting for assignment.
* `stream_keys`: Redis Stream keys matched by the query.

### 2.3 System Overview (`/allstatus`)

Use this endpoint to retrieve global scheduler health, typically for an admin dashboard.

* **Core response fields**:
* `scheduler_summary`: Current scheduler load and health.
* `all_tasks_summary`: Aggregated statistics for running and queued tasks.

## 3. How It Works (SchedulerHandler)

When you send a status request, **SchedulerHandler** performs the following operations:

1. **Cache lookup**: Reads the latest progress for `task_id` from Redis status cache.
2. **Queue inspection**: For queue metrics, calls Redis commands such as `XLEN` and `XPENDING` to analyze Stream state.
3. **Metric aggregation**: For global status, aggregates metrics from all active nodes into a system-level summary.

## 4. Quick Start

Poll task status with the SDK until completion:

```python
from memos.api.client import MemOSClient
import time

client = MemOSClient(api_key="...", base_url="...")

# 1. System overview: inspect overall MemOS health.
global_res = client.get_all_scheduler_status()
if global_res:
print(f"System summary: {global_res.data['scheduler_summary']}")

# 2. Queue metrics: inspect backlog for a specific user.
queue_res = client.get_task_queue_status(user_id="dev_user_01")
if queue_res:
print(f"Remaining tasks: {queue_res.data['remaining_tasks_count']}")
print(f"Pending tasks: {queue_res.data['pending_tasks_count']}")

# 3. Task progress: poll a specific task until it finishes.
task_id = "task_888999"
while True:
res = client.get_task_status(user_id="dev_user_01", task_id=task_id)
if res and res.code == 200:
current_status = res.data[0]['status'] # data is a status list
print(f"Task {task_id} status: {current_status}")

if current_status in ['completed', 'failed', 'cancelled']:
break
time.sleep(2)
```
76 changes: 76 additions & 0 deletions docs/en/open_source/open_source_api/scheduler/wait.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: Advanced Task Synchronization
desc: Provides blocking waits and streaming progress observation so asynchronous tasks for a user are completed before follow-up operations run.
---

**API Paths**:
* **Blocking wait**: `POST /product/scheduler/wait`
* **Real-time progress stream (SSE)**: `GET /product/scheduler/wait/stream`

**Description**: Automation scripts, data migrations, and integration tests often need to ensure that all asynchronous memory extraction tasks, such as LLM fact extraction and vector writes, have fully completed. These endpoints let a client hold the request open until the scheduler detects that the target user's queue is empty.

## 1. Core Mechanism: Scheduler Idle Detection

The system uses **SchedulerHandler** to monitor the underlying **MemScheduler** state in real time:

* **Queue checks**: The system checks Redis Stream tasks for the user, including pending and remaining tasks.
* **Idle detection**: A user is considered idle only when queue counts are zero and no worker is currently processing that user's tasks.
* **Timeout protection**: Set `timeout_seconds` to avoid blocking forever. If the timeout is reached before tasks finish, the endpoint returns the current status and stops waiting.

## 2. Key Parameters

Both endpoints share the following query parameters:

| Parameter | Type | Required | Default | Description |
| :--- | :--- | :--- | :--- | :--- |
| **`user_name`** | `str` | Yes | - | Target user name or ID. |
| `timeout_seconds` | `num` | No | - | Maximum wait time in seconds. The request returns after this limit. |
| `poll_interval` | `num` | No | - | How often the queue state is checked, in seconds. |

## 3. Response Modes

### 3.1 Blocking Mode (`/wait`)

* **Behavior**: Standard HTTP response. The connection stays open until tasks clear or the request times out.
* **Use Cases**: Automation scripts or ensuring data has been written before calling `search`.

### 3.2 Streaming Mode (`/wait/stream`)

* **Behavior**: Uses **Server-Sent Events (SSE)**.
* **Use Cases**: Admin dashboards that display a dynamic progress bar as the queue drains.

## 4. Quick Start

Use the open-source SDK for a blocking wait:

```python
from memos.api.client import MemOSClient

client = MemOSClient(api_key="...", base_url="...")
user_name = "dev_user_01"

# Scenario A: blocking wait, commonly used in Python automation scripts.
print(f"Waiting for user {user_name}'s task queue to drain...")
res = client.wait_until_idle(
user_name=user_name,
timeout_seconds=300,
poll_interval=2
)
if res and res.code == 200:
print("All tasks have completed.")

# Scenario B: streaming progress, commonly used by frontend progress bars.
print("Listening to the live task progress stream...")
# The SSE endpoint usually returns a generator from the SDK.
progress_stream = client.stream_scheduler_progress(
user_name=user_name,
timeout_seconds=300
)

for event in progress_stream:
# Print the remaining queued tasks in real time.
print(f"Remaining queued tasks: {event['remaining_tasks_count']}")
if event['status'] == 'idle':
print("Scheduler is idle")
break
```
7 changes: 7 additions & 0 deletions docs/en/open_source/open_source_api/start/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Project Configuration
---

For detailed configuration instructions for the MemOS open-source API server, including LLM engines, storage backends, and environment variables, see:

[**REST API Server Configuration Guide**](../../../getting_started/rest_api_server.md)
55 changes: 55 additions & 0 deletions docs/en/open_source/open_source_api/start/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: Overview
---

## 1. API Introduction

The MemOS open-source project provides a high-performance REST API service built with **FastAPI**. The system follows a **Component + Handler** architecture, and core capabilities such as memory extraction, semantic search, and asynchronous scheduling are available through standard REST endpoints.

![MemOS Architecture](https://cdn.memtensor.com.cn/img/memos_run_server_success_compressed.png)
<div style="text-align: center; margin-top: 10px">Overview of the MemOS REST API service architecture</div>

### Core Capabilities

* **Multidimensional memory production**: Use `AddHandler` to process conversations, text, or documents and convert them into structured memories.
* **Physical isolation with MemCube**: Use Cube IDs to isolate data and indexes across users or knowledge bases.
* **End-to-end chat loop**: Use `ChatHandler` to orchestrate retrieval, generation, and asynchronous storage.
* **Asynchronous task scheduling**: Use the built-in `MemScheduler` engine to smooth large memory-production workloads and track task status.
* **Self-correction workflow**: Use feedback endpoints to correct or mark stored memories with natural language.

## 2. Getting Started

Integrate memory capabilities into your application with two core steps:

* [**Add Memory**](../core/add_memory.md): Use `POST /product/add` to write raw message streams into a target MemCube.
* [**Search Memory**](../core/search_memory.md): Use `POST /product/search` to retrieve relevant context from one or more Cubes by semantic similarity.

## 3. API Categories

MemOS APIs are grouped into the following categories:

* **[Core Memory](../core/add_memory.md)**: Atomic operations for creating, deleting, updating, and querying memories.
* **[Chat](../chat/chat.md)**: Streaming or full-response chat with memory augmentation.
* **[Message Management](../message/feedback.md)**: User feedback, suggestion queries, and related interaction APIs.
* **[Scheduler](../scheduler/get_status.md)**: Monitor background memory extraction tasks and queue status.
* **[Tools](../tools/check_cube.md)**: Utility APIs such as Cube existence checks and reverse memory ownership lookup.

## 4. Authentication and Context

### Authentication

In the open-source environment, every API request must include the `Authorization` header.

* **Development**: Define `API_KEY` in your local `.env` file or configuration.
* **Production**: Extend `RequestContextMiddleware` for OAuth2 or stronger identity checks.

### Request Context

* **user_id**: Required in the request body and used by handlers for identity tracking.
* **MemCube ID**: The core isolation unit in the open-source edition. Use `readable_cube_ids` or `writable_cube_ids` to precisely control physical read and write boundaries.

## 5. Next Steps

* [**System Configuration**](./configuration.md): Configure your LLM provider and vector database engine.
* [**Add Your First Memory**](../core/add_memory.md): Submit your first conversation messages through the SDK or curl.
* [**Common Error Codes**](../help/error_codes.md): Learn API status codes and how exceptions are handled.
49 changes: 49 additions & 0 deletions docs/en/open_source/open_source_api/tools/check_cube.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: Check Cube Existence
desc: Verify whether a specified MemCube ID has been initialized and is available.
---

**Endpoint**: `POST /product/exist_mem_cube_id`
**Description**: This endpoint verifies whether a specified `mem_cube_id` already exists in the system. It acts as a guard for data consistency and is recommended before dynamically creating knowledge bases or allocating storage for new users.

## 1. Core Mechanism: Cube Index Validation

In the MemOS architecture, MemCube existence determines whether subsequent memory operations are valid:

* **Logical validation**: **MemoryHandler** checks the underlying storage index to confirm whether the ID is registered.
* **Cold-start guard**: In on-demand Cube creation scenarios, this endpoint helps decide whether an initial `add` operation is needed to activate the memory space.

## 2. Key Parameters

Request body:

| Parameter | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **`mem_cube_id`** | `str` | Yes | Unique MemCube identifier to validate. |

## 3. How It Works (MemoryHandler)

1. **Index passthrough**: **MemoryHandler** receives the request and calls the metadata query interface of the underlying **naive_mem_cube**.
2. **Status lookup**: The system searches persistent storage for the configuration or database record associated with the ID.
3. **Boolean feedback**: The response does not include memory content. It reports whether the Cube is active through `code` or `data`.

## 4. Quick Start

Check the target Cube status with the SDK:

```python
from memos.api.client import MemOSClient

client = MemOSClient(api_key="...", base_url="...")

# Scenario: confirm that the target knowledge base exists before importing documents.
kb_id = "kb_finance_2026"
res = client.exist_mem_cube_id(mem_cube_id=kb_id)

if res and res.code == 200:
# Assume data returns a boolean value or an existence object.
if res.data.get('exists'):
print(f"MemCube '{kb_id}' is ready.")
else:
print(f"MemCube '{kb_id}' has not been initialized.")
```
Loading
Loading