From afe3df9981c2224b7ebbc6296b401a437b8eb180 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 20:06:39 +0900 Subject: [PATCH 01/13] docs: rewrite README for customers and operators --- README.md | 815 +++++++++++++----------------------------------------- 1 file changed, 186 insertions(+), 629 deletions(-) diff --git a/README.md b/README.md index ff64840e9..a34c69d21 100644 --- a/README.md +++ b/README.md @@ -1,688 +1,245 @@ -# Naruon AI Email Workspace +# Naruon [![Application CI](https://github.com/ContextualWisdomLab/naruon/actions/workflows/app-ci.yml/badge.svg)](https://github.com/ContextualWisdomLab/naruon/actions/workflows/app-ci.yml) [![Bandit Security Scan](https://github.com/ContextualWisdomLab/naruon/actions/workflows/bandit.yml/badge.svg)](https://github.com/ContextualWisdomLab/naruon/actions/workflows/bandit.yml) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/naruon) -Full-stack AI workspace with a FastAPI backend, Next.js frontend, vector search, -AI summaries, hardened email threading, and relay/proxy contracts for external -mail/calendar/file systems. +**Naruon is a self-hostable work hub for customer-owned email, calendar, files, tasks, projects, search, and AI-assisted workflows.** +It connects to systems an organization already controls, keeps bounded workspace metadata and provenance in PostgreSQL, and gives users one place to review work, find context, create follow-up actions, and operate integrations without turning Naruon into the authoritative mailbox, calendar, or file server. -## Quick Links -- [Installation & Setup](#five-minute-local-path) -- [Architecture](docs/architecture/) -- [Topic-intelligence documentation set](docs/topic-intelligence/README.md) -- [Architecture decisions](docs/adr/README.md) -- [Contributing](CONTRIBUTING.md) -- [Code of Conduct](CODE_OF_CONDUCT.md) -- [Security Policy](SECURITY.md) +## What Naruon provides + +| Workspace | Customer outcome | +| --- | --- | +| Today | See pending replies, judgment points, and the next work that needs attention. | +| Mail | Import and thread messages, inspect conversation history, summarize context, and prepare replies. | +| Calendar | Review calendar evidence, detect conflicts, and create explicit writeback intents. | +| Tasks and Projects | Turn source-linked email or document evidence into trackable work without losing provenance. | +| Context Search | Search indexed messages, attachments, documents, and relationship context. | +| AI Hub | Configure approved model providers and use AI-backed flows when an operator enables them. | +| Data | Review document ingestion, parsing, embeddings, and data-quality state. | +| Security | Inspect source-backed access, policy, connector, and audit evidence. | +| Settings | Manage connected accounts, provider configuration, and self-hosted connector registration. | + +Naruon labels simulated, deferred, pending, and completed actions differently. A generated intent or local payload check is not presented as proof that a customer-owned provider accepted a write. + +## Product boundary + +### Naruon owns + +- the web workspace and API control plane; +- authentication and authorization enforcement; +- bounded metadata, indexes, preferences, task records, and audit evidence; +- canonical email threading and source-linked work provenance; +- server-authoritative action intent and connector dispatch policy; and +- explicit failure, conflict, and abstention states. + +### Customer systems remain authoritative + +- IMAP, POP3, SMTP, and hosted mailbox data; +- CalDAV and CardDAV calendars and contacts; +- WebDAV files and provider revisions; +- enterprise identity and account lifecycle; and +- credentials and provider-side delivery or write status. + +Naruon is **not** an SMTP or IMAP server, an MX host, a mailbox-capacity provider, or a general-purpose credential relay. Private-network integrations belong behind the outbound-only self-hosted connector boundary. + +## Hub architecture and optional contracts + +Naruon is the CWL ecosystem hub, but it is not a source-code aggregator or a mandatory bundle of sibling products. + +The core application runs from this repository with: -## North-star scope contract - -- Naruon is not an SMTP server, IMAP server, MX host, or mailbox capacity - provider. It is a web client/control plane that works through member-configured - providers and customer-owned systems. -- Customer mail, CalDAV/CardDAV, and WebDAV accounts remain the source of truth; - Naruon stores bounded metadata, indexes, preferences, and auditable action - intent rather than replacing those systems. -- Private-network protocols use an outbound-only self-hosted connector to - `naruon.net`; GitHub self-hosted runners are CI smoke infrastructure, not the - production connector itself. -- Calendar/file/contact writeback is opt-in, server-authoritative, and - conflict-aware through source capabilities, provenance, ETags/If-Match, and - audit logs. -- Access control is universal RBAC plus ABAC: data-region, consent, workspace, - group, source capability, and customer-policy denies take precedence over broad - role allows. A permitted `platform_admin` can cross organization and resource - ownership boundaries for platform operations, but not data-region or consent - denies. -- Keycloak is the default enterprise OIDC evaluation target; Casdoor remains a - lighter alternative. Traefik and OpenTelemetry are evaluated for edge policy and - open-source observability. -- PR automation is metadata-only inside this repository and uses current-head - robot-review evidence plus required checks. Human approval is not awaited by - default under repo policy. -- OpenCode Review, Strix Security Scan, and PR Review Merge Scheduler are - supplied by the ContextualWisdomLab central required workflows from - `ContextualWisdomLab/.github`. This repository does not carry repo-local - OpenCode, Strix, or merge-scheduler workflow copies; branch updates, - auto-merge, and mechanical merge actions run as the target repository's - `github-actions[bot]` through the central workflow. Pending CodeRabbit or - required-check evidence is a wait state, not a hard blocker. -- Topic intelligence is not currently a live Naruon capability. The lexical - `keyword_extractor` is metadata only; Naruon fails closed rather than present - keyword, embedding, or LLM labels as Structural Topic Modeling. The product, - technical, architecture, contract, security, UML, conceptual ERD, test, and - operability records are indexed in - [`docs/topic-intelligence/`](docs/topic-intelligence/README.md). -- Security governance is source-backed through signed - `/api/security/access-surface`. The endpoint reads scoped WebDAV, CalDAV, and - connector evidence plus durable `security_audit_events`, reuses the deny-first - RBAC/ABAC policy engine, and returns no sequential account ids, browser-facing - source/event/decision identifiers, provider-write execution flags, raw - credentials, legacy unscoped audit rows, or fake security posture claims. - HMAC fallback sessions are not accepted as authoritative workspace-membership - evidence for this security posture surface; enterprise OIDC/JWKS or an - explicit server-side membership path must establish the workspace boundary. -- Data quality is source-backed through signed `/api/data/quality-surface`. - The endpoint summarizes scoped repositories, workspace documents, recent - email-attachment file assets, ingestion inventory, embedding coverage, - quality checks, and connector evidence from existing rows, returns - `provider_write_executed=false`, and does not expose provider credentials, - raw usernames, server URLs, message bodies, raw message/thread ids, or - sequential ids. The Data workspace lets operators upload a signed-session - workspace document, request document reparse, embedding regeneration intent, - HWP conversion intent, and explicit WebDAV document materialization. The - materialization route re-reads the selected `document_id` from the signed - workspace, derives the provider path and Markdown content server-side, and - dispatches `write_webdav` only when the caller explicitly requests - `execute_provider=true`. -- Projects are source-backed through signed `/api/webdav/folders` and - `/api/tasks`. The workspace derives project boundaries from customer-owned - WebDAV folders, task progress from opaque public ticket ids, and labels - provider writes as deferred intent work. -- Custom LLM provider `base_url` calls fail closed unless the host is - exact-allowlisted, HTTPS-only, and resolved to global addresses. Runtime calls - use a pinned-address `httpx` transport so DNS is not resolved a second time - after validation. -- OIDC issuer and JWKS URLs follow the same outbound fetch posture: - exact-allowlisted HTTPS hosts must resolve only to global addresses, and JWKS - preload fetches connect to the validated pinned address while keeping TLS/SNI - on the allowlisted hostname. -- Session authority is assigned by the verified HMAC or OIDC code path, not by a - `_session_verifier` JWT payload claim supplied inside the token. - -## Agentic Ontology & Auto-Organization - -- **Sender ontology**: The backend classifies sender relationships and returns a - deterministic next-action hint, such as reply/task tracking for colleagues or - summary-first handling for newsletters. Relationship graph reads can be - filtered by source message/thread ids so the Search workspace can show the - sender DAG beside the originating mail context. If no relationship exists for - the selected search result, the browser can call signed - `/api/ontology/relationships/capture-source`; the backend re-reads the source - email under owner/organization scope and derives the thread provenance - server-side before storing the relationship. -- **Self-sent knowledge capture**: IMAP-imported emails sent from a user to the - same address now create one idempotent, source-linked `self_sent_knowledge` - ticket task with a plain-text memo title. The Tasks workspace can request a - signed WebDAV/Notes materialization intent for that task and shows the planned - customer-owned target with `provider_write_executed=false`; connector-side - WebDAV/CalDAV PUT adapters now enforce `If-Match`, and - `execute_provider=true` dispatches the signed materialization command to an - active outbound runner. Durable retry queues and extended execution audit - workflows remain future work. -- **Pending reply dashboard**: the Today dashboard reads signed - `/api/emails/pending-replies?limit=3` data and shows sent-mail reply waits in - Home KPIs and judgment points. Pending replies are calculated from - customer-owned mailbox metadata; Naruon does not host the mailbox or fabricate - provider writes. -- **Overdue reply follow-up**: Home and Tasks can call signed - `POST /api/tasks/reply-sla-escalations` to convert overdue pending sent-mail - replies into opaque, source-linked `reply_sla` ticket tasks. Escalation reuses - server-side reply tracking, keeps generated titles plain text, and does not - mutate the customer's email provider. - -## Five-minute local path +- a Next.js frontend; +- a FastAPI backend; +- PostgreSQL with pgvector; and +- an operator-selected OpenAI-compatible model path when AI features are enabled. + +Sibling repositories connect only through **optional, versioned contracts**. A sibling integration must use a released package, API, event schema, or OCI artifact pinned by version or digest. It must not become an undeclared startup dependency, mutable branch dependency, copied code fork, or direct database dependency. + +| Optional contract | Role | Default behavior when absent | +| --- | --- | --- | +| `newsdom-api` | PDF-to-DOM recognition sidecar, enabled with the `newsdom` Compose profile | PDFs remain pending or use other configured paths; the core stack still starts. | +| Enterprise OIDC/JWKS provider | Production identity and membership authority | Local HMAC sessions remain a compatibility path, not production membership proof. | +| OpenAI-compatible provider | Summaries, drafting, embeddings, and model-backed workflows | AI-dependent actions are unavailable or fail explicitly; non-AI workspace functions remain usable. | +| Other CWL siblings | Specialized identity, orchestration, document, catalog, or analysis capabilities | Disabled until an operator accepts and configures the published contract. | + +See [Architecture](ARCHITECTURE.md) for the detailed system and trust boundaries. + +## Five-minute local deployment + +### Prerequisites + +- Docker Engine or a compatible Compose runtime; +- Python 3 for local secret generation; and +- enough disk space for PostgreSQL, images, and the local model runtime. + +### 1. Create local configuration ```bash cp .env.example .env python3 - <<'PY' from pathlib import Path import base64 +import re import secrets -env_path = Path(".env") -env_values = {} -for line in env_path.read_text().splitlines(): - if "=" not in line or line.lstrip().startswith("#"): - continue - key, value = line.split("=", 1) - env_values[key] = value - -db_password = secrets.token_urlsafe(32) -env_values.update( - { - "POSTGRES_DB": "ai_email", - "POSTGRES_USER": "naruon_local", - "POSTGRES_PASSWORD": db_password, - "DATABASE_URL": ( - "postgresql+asyncpg://naruon_local:" - f"{db_password}@localhost:5432/ai_email" - ), - "AUTH_SESSION_HMAC_SECRET": secrets.token_urlsafe(48), - "ENCRYPTION_KEY": base64.urlsafe_b64encode(secrets.token_bytes(32)).decode(), - } -) - -existing_lines = env_path.read_text().splitlines() -existing_keys = { - line.split("=", 1)[0] - for line in existing_lines - if "=" in line and not line.lstrip().startswith("#") +path = Path(".env") +text = path.read_text() +password = secrets.token_urlsafe(32) +values = { + "POSTGRES_DB": "ai_email", + "POSTGRES_USER": "postgres", + "POSTGRES_PASSWORD": password, + "DATABASE_URL": ( + "postgresql+asyncpg://postgres:" + f"{password}@127.0.0.1:15432/ai_email" + ), + "AUTH_SESSION_HMAC_SECRET": secrets.token_urlsafe(48), + "ENCRYPTION_KEY": base64.urlsafe_b64encode(secrets.token_bytes(32)).decode(), } -required_lines = [f"{key}={value}" for key, value in env_values.items() if key not in existing_keys] -env_path.write_text("\n".join(existing_lines + required_lines) + "\n") + +for key, value in values.items(): + pattern = rf"(?m)^{re.escape(key)}=.*$" + replacement = f"{key}={value}" + if re.search(pattern, text): + text = re.sub(pattern, replacement, text) + else: + text += f"\n{replacement}" + +path.write_text(text.rstrip() + "\n") PY -./scripts/naruon_compose.sh up -d --build -./scripts/naruon_compose.sh exec backend python import_fixtures.py -curl -s http://localhost:8000/api/emails -python3 -m webbrowser http://localhost:3000 ``` -### Apple Silicon / MLX local path (OS별 로컬 API 모델 서버 사용) +Keep `.env` local. Never commit mailbox exports, provider credentials, session secrets, encryption keys, or customer content. -기본 `docker-compose.yml`는 Linux Ollama 컨테이너를 그대로 유지합니다. Apple Silicon -로컬 실 테스트(또는 외부 MLX/OpenAI-compatible 서비스)만 분리하려면 임시 오버라이드 파일을 붙여 실행합니다. +### 2. Start the core stack ```bash -# 다음 블록은 로컬 실사용 검증용 샘플입니다. 민감한 쿼리로 대체할 수 있지만, -# 현재 실검증에서는 아래 두 키워드로 테스트합니다. -cat > .env.mlx <<'EOF' - -# 기존 보안값은 그대로 두고, 로컬 모델 경로만 오버라이드 -OPENAI_API_KEY=mlx -ALLOWED_LLM_BASE_URL_HOSTS=localhost,127.0.0.1,host.docker.internal -ALLOW_LOCAL_LLM_PROVIDERS=true -OPENAI_BASE_URL=http://host.docker.internal:11434/v1 -OPENAI_EMBEDDING_MODEL=embeddinggemma -OPENAI_MODEL=gemma4:e2b-it-qat -# 포트 충돌이 있으면 아래 두 값으로 변경 -NARUON_FRONTEND_HOST_PORT=127.0.0.1:3000 -NARUON_BACKEND_HOST_PORT=127.0.0.1:8000 -# Linux에서만 host-gateway가 필요합니다. -NARUON_MLX_EXTRA_HOSTS=host-gateway -NARUON_MLX_ALLOWED_LLM_BASE_URL_HOSTS=localhost,127.0.0.1,host.docker.internal -NARUON_MLX_OPENAI_API_KEY=mlx -NARUON_MLX_BASE_URL=http://host.docker.internal:11434/v1 -NARUON_MLX_EMBEDDING_MODEL=embeddinggemma -NARUON_MLX_LLM_MODEL=gemma4:e2b-it-qat -EOF - -# 로컬에서만 쓰는 compose 오버라이드는 임시 파일로 만들고 커밋하지 않습니다. -# OS 분기 없이 환경변수 하나로 host.docker.internal 매핑을 제어합니다. -# Linux에서 host-gateway가 필요한 환경이면 .env.mlx에서 NARUON_MLX_EXTRA_HOSTS를 덮어씁니다. -# Apple Silicon 검증 기준: 백엔드는 host.docker.internal:11434의 MLX(OpenAI-compatible) -# 엔드포인트로 바로 연결해 Ollama 컨테이너 의존을 피합니다. -mlx_compose_override="$(mktemp "${TMPDIR:-/tmp}/docker-compose.mlx.XXXXXX.yml")" -cat > "$mlx_compose_override" <<'EOF' -services: - backend: - depends_on: - db: - condition: service_healthy - environment: - ALLOW_LOCAL_LLM_PROVIDERS: "true" - ALLOWED_LLM_BASE_URL_HOSTS: ${NARUON_MLX_ALLOWED_LLM_BASE_URL_HOSTS:-localhost,127.0.0.1,host.docker.internal} - OPENAI_API_KEY: ${NARUON_MLX_OPENAI_API_KEY:-mlx} - OPENAI_BASE_URL: ${NARUON_MLX_BASE_URL:-http://host.docker.internal:11434/v1} - OPENAI_EMBEDDING_MODEL: ${NARUON_MLX_EMBEDDING_MODEL:-embeddinggemma} - OPENAI_MODEL: ${NARUON_MLX_LLM_MODEL:-gemma4:e2b-it-qat} - extra_hosts: - - "host.docker.internal:${NARUON_MLX_EXTRA_HOSTS:-host.docker.internal}" - ports: - - "${NARUON_BACKEND_HOST_PORT:-127.0.0.1:8000}:8000" - frontend: - ports: - - "${NARUON_FRONTEND_HOST_PORT:-127.0.0.1:3000}:3000" -EOF - -NARUON_ENV_FILE=.env.mlx \ -docker compose --env-file .env.mlx -f docker-compose.yml -f "$mlx_compose_override" up -d --build - -# 혹시 모델 엔드포인트 미노출이 있을 경우는 위 명령 직전에 로컬 MLX 서버/게이트웨이를 -# 먼저 확인합니다. (호스트는 본인 환경별로 달라질 수 있음) -curl -sf http://127.0.0.1:11434/v1/models >/dev/null && \ - echo "MLX/OpenAI-compatible server is reachable" || \ - echo "MLX endpoint is not reachable on 127.0.0.1:11434" +./scripts/naruon_compose.sh up -d --build +./scripts/naruon_compose.sh exec backend python import_fixtures.py ``` -실 메일 임포트 + 요약/초안 검증: +The default Compose profile starts PostgreSQL, Ollama, the backend, and the frontend. + +### 3. Verify the deployment ```bash -# 아래 두 --query는 실 사용자 공개 테스트 키워드입니다. -MAIL_DIR="/Users/seonghobae/Library/Mobile Documents/com~apple~CloudDocs/Downloads/mail" -if [ ! -r "$MAIL_DIR" ]; then - echo "ERROR: cannot read $MAIL_DIR (Apple CloudDocs 권한 또는 path 접근 권한 점검 필요)" - echo "대체: 실 메일 파일을 별도 로컬 폴더에 복사한 뒤 MAIL_DIR을 교체해 재실행" - exit 1 -fi - -AUTH_SESSION_HMAC_SECRET="$(grep -E '^AUTH_SESSION_HMAC_SECRET=' .env | cut -d= -f2-)" -python3 backend/scripts/private_mail_http_smoke.py \ - --mail-dir "$MAIL_DIR" \ - --base-url http://127.0.0.1:3000 \ - --frontend-base-url http://127.0.0.1:3000 \ - --api-base-url http://127.0.0.1:8000 \ - --session-secret "$AUTH_SESSION_HMAC_SECRET" \ - --query "중공업 전력PU 회의록" \ - --query "중공업 기전PU 회의록" \ - --match-mode all-terms \ - --limit 20 \ - --batch-size 6 \ - --require-browser-visible \ - --llm-smoke \ - --print-session-token +./scripts/naruon_compose.sh ps +curl -fsS http://127.0.0.1:8000/ +python3 -m webbrowser http://127.0.0.1:3000 ``` -`--print-session-token`이 켜진 경우 스크립트가 같은 토큰을 브라우저로 전파하는 -`/auth/session` 호출 예시를 출력합니다. 위 출력의 JS 한 줄을 앱 콘솔에서 실행하면 -`naruon_session` 쿠키가 갱신되어 API로 임포트한 메일이 브라우저와 동일 세션에서 보입니다. -`session_check=ok` 로그는 세션 클레임이 브라우저에서 확인되었음을 뜻하고, -`session_check=failed(...)`는 토큰 검증/클레임 파싱 문제가 있음을 뜻합니다. -`--require-browser-visible`은 동일 토큰을 `Cookie: naruon_session=...`로 주입해 -`/api/emails` 응답을 조회해 브라우저 프록시 경로까지 반영 확인이 되도록 합니다. +The API root should return an `ok` status. The fixture import creates a small synthetic conversation so the Mail and threading path can be checked without real customer data. -동기화 지연이 큰 환경에서는 재시도 옵션을 조정할 수 있습니다. +### Optional PDF recognition ```bash - --search-retry-attempts 5 \ - --search-retry-delay-seconds 1.2 \ - --inbox-retry-attempts 5 \ - --inbox-retry-delay-seconds 1.2 +COMPOSE_PROFILES=newsdom ./scripts/naruon_compose.sh up -d --build ``` -실제 브라우저 검증 순서: +The `newsdom` service stays on the internal Compose network and is not published to the host. Enable it deliberately and review the pinned sidecar revision before production use. -1) 브라우저에서 `http://127.0.0.1:3000` 접속 후 `"/mail"`로 이동 -2) 방금 입력한 키워드 중 하나로 검색 -3) `/mail` 결과 목록에서 임포트된 메일을 열어 상세가 정상 표시되는지 확인 -4) 동일 이메일 상세 화면에서 요약/초안 버튼이 작동하고(`llm=ok`, `draft=ok` 또는 UI 동작), - 브라우저 세션 값(`session_check=ok`)이 스크립트 출력에 남아있는지 확인 - - 브라우저에서 동일 이메일을 선택한 뒤 LLM 요약/초안 버튼 동작 확인 -5) 세션 불일치 의심 시 `session_check=failed(...)` 또는 `session_check=skipped(...)`가 - 출력되면 `--print-session-token`의 콘솔 스니펫을 다시 실행하고 새로고침 후 2~4단계를 반복 +## Required operator configuration -실행 전 체크(빠른 사전 진단): +| Setting | Purpose | +| --- | --- | +| `POSTGRES_PASSWORD` | PostgreSQL runtime password. | +| `AUTH_SESSION_HMAC_SECRET` | Local/control-plane session signing bootstrap secret. | +| `ENCRYPTION_KEY` | Root key used to protect encrypted credential records. | +| `DATABASE_URL` | Required for manual backend execution outside the Compose network. | -```bash -# Podman/Docker 런타임 연결 확인 -podman system connection ls +Production deployments should inject bootstrap secrets from an approved secret manager rather than a shared environment file. Tenant provider and account credentials belong in the encrypted server-side registry and must not be exposed to the browser or copied into documentation. -# MLX(OpenAI-compatible) 엔드포인트 노출 확인 -curl -sf http://127.0.0.1:11434/v1/models | head +## Identity and tenant safety -# 기존 웹 서비스(Nginx/프록시)가 3000/8000/11434를 가로채고 있지 않은지 확인 -lsof -iTCP:3000 -sTCP:LISTEN -lsof -iTCP:8000 -sTCP:LISTEN -lsof -iTCP:11434 -sTCP:LISTEN -``` +- Browser traffic uses same-origin `/api/*`; the Next.js server-side proxy converts the HttpOnly session cookie into the backend bearer session. +- Public identity headers such as `X-User-Id` and `X-Organization-Id` are not trusted as authentication. +- Local HMAC sessions are suitable for local and compatibility testing. Production workspace membership should be established by verified OIDC/JWKS or an explicit server-side membership authority. +- Before mixing real multi-user data in one database, operators must complete and audit mailbox-owner and organization backfills for historical rows. +- Authorization is deny-first RBAC plus ABAC. Data-region, consent, workspace, group, source capability, and customer-policy denies take precedence over broad role grants. -백엔드 API를 바로 확인하려면(필요 시): +See [Authentication and key management](docs/operations/auth-key-management.md) and the [Security policy](SECURITY.md). -```bash -curl -s http://127.0.0.1:3000/api/emails?limit=10 -# 아래는 동일 샘플로 API 직접 점검하는 예시입니다. -curl -s -X POST http://127.0.0.1:3000/api/search \ - -H 'Content-Type: application/json' \ - -d '{"query": "중공업 전력PU 회의록", "limit": 3}' -``` +## Connector and writeback operations -세션이 다르게 보이면 `/auth/session` 동기화 콘솔 코드를 다시 실행한 뒤 새로고침 합니다. - -What you should see: the fixture import loads a three-message `Quarterly plan` -conversation. `/api/emails` returns one threaded inbox item with `reply_count` -greater than 1, and the frontend shows conversation history oldest to newest. -First-run frontend sessions open the Today execution dashboard by default, with -explicit entry points to the email workspace and calendar-first workspace. - -The fixture importer uses real OpenAI embeddings only when `OPENAI_API_KEY` is -set. With the default empty key it writes local zero-vector embeddings so the -threading proof path works offline. - -Backend settings read environment variables first, then `.env`, `../.env`, and -`~/.env`. `DATABASE_URL`, `AUTH_SESSION_HMAC_SECRET`, and `ENCRYPTION_KEY` still -have no code defaults; Compose and Kubernetes must inject them explicitly before -runtime. `docker compose build backend frontend` is intentionally allowed to -parse without local secrets because image builds do not need database or session -credentials. `docker compose up` still fails closed inside the database/backend -startup path when `POSTGRES_PASSWORD`, `AUTH_SESSION_HMAC_SECRET`, or -`ENCRYPTION_KEY` are missing. For Compose, `./scripts/naruon_compose.sh` reads -`${NARUON_ENV_FILE}` when set, otherwise uses `~/.env` if present, and falls back -to the project `.env`. It passes that file to Docker Compose only as an -interpolation source so the backend service receives the whitelisted variables -in `docker-compose*.yml`, not every local secret present in `~/.env`. The -backend image starts through -`python scripts/start_backend.py`, which checks the same required settings before -`uvicorn` imports the app. A direct `docker run` therefore still needs explicit -environment injection through `--env`, an orchestrator secret, or a minimal -Naruon-specific env file containing only the backend settings needed by the -container. - -## Manual development path - -Backend: +For customer-owned mail, calendar, contact, and file systems: -```bash -cd backend -python3 -m pip install -r requirements.txt -python3 scripts/migrate_db.py -python3 -m pytest -q -uvicorn main:app --reload -``` +1. register the source and its capabilities server-side; +2. scope the source to the authenticated organization and workspace; +3. verify the current provider revision, such as ETag/If-Match where applicable; +4. create an explicit action intent; +5. set provider execution explicitly when the workflow is ready; and +6. confirm the resulting connector and provider evidence before reporting completion. + +A conflict, missing capability, absent runner, stale ETag, unapproved destination, or unavailable credential fails closed. Naruon does not silently overwrite customer-owned state. + +Read: -Frontend: +- [Email relay and proxy boundary](docs/operations/email-relay-proxy-boundary.md) +- [Source-of-truth and writeback sovereignty](docs/operations/source-of-truth-and-writeback-sovereignty.md) + +## Routine operator checks ```bash -cd frontend -npm install -npm test -npm run lint -npm run build -npm run dev +./scripts/naruon_compose.sh ps +./scripts/naruon_compose.sh logs --tail=200 db backend frontend +curl -fsS http://127.0.0.1:8000/ ``` -`next.config.ts` applies a best-effort local guard for build worker fan-out and -static generation concurrency (`NEXT_BUILD_CPUS=2`, -`NEXT_STATIC_GENERATION_MAX_CONCURRENCY=2`, -`NEXT_STATIC_GENERATION_MIN_PAGES_PER_WORKER=50`) so constrained CI/build -machines do not fan out excessive Node/PostCSS workers. Treat the Next.js CPU -knob as experimental and enforce authoritative limits through CI, Docker, or the -runner. Raise those values only with explicit build evidence. - -## Threading proof points - -- Canonical thread IDs are assigned in `backend/services/threading_service.py`. -- Parser output preserves raw `Message-ID`, `In-Reply-To`, `References`, and - `Reply-To` headers. -- Importers persist the canonical service-assigned `thread_id`; they do not - recompute their own thread IDs. -- Signed email file imports accept `.eml`, `.zip`, and `.mbox` uploads through - `/api/emails/import-files`; imported email and attachment vectors use the - active organization embedding model such as local `embeddinggemma` when an - LLM provider is configured. -- Duplicate ZIP/forward candidates can be checked through signed - `/api/emails/unique-thread-intent`. The intent uses normalized Message-ID and - strong body fingerprint matches, returns canonical thread metadata, and does - not execute provider writes or irreversible DB merges. -- IMAP imports store the strong body fingerprint when message body content is - available, preserving the older lightweight fingerprint only as a fallback. -- Subject-only `Fwd:` or `Re:` matching is not a valid duplicate/thread merge - signal. Forwarded threading must come from Message-ID, References, - In-Reply-To, or future persisted duplicate provenance. -- Replies include `In-Reply-To` and `References` headers in the send payload. -- Development sends are explicit simulations unless a real SMTP path is wired. - -## API smoke examples - -The backend accepts only signed bearer sessions. For local smoke tests, generate -a local-only `AUTH_SESSION_HMAC_SECRET`, start the API with that exact value, and -then mint a short-lived fixture token from the same shell: +Also verify in the application: -```bash -export AUTH_SESSION_HMAC_SECRET="$(python3 - <<'PY' -import secrets +- connected-account readiness in Settings; +- active self-hosted connector and recent heartbeat evidence; +- pending or conflicted writeback intents; +- failed ingestion or embedding jobs; +- security-policy denials and recent audit events; and +- storage growth, database backup status, and restore readiness. -print(secrets.token_urlsafe(48)) -PY -)" -export NARUON_DEV_BEARER="$(python3 - <<'PY' -import base64, hashlib, hmac, json, os, time - -secret = os.environ["AUTH_SESSION_HMAC_SECRET"].encode() -payload = { - "ver": 1, - "iss": "naruon-control-plane", - "aud": "naruon-api", - "sub": "default", - "role": "organization_admin", - "org": "default", - "groups": [], - "workspace": "default", - "exp": int(time.time()) + 300, -} -enc = lambda raw: base64.urlsafe_b64encode(raw).rstrip(b"=").decode() -header = enc(json.dumps({"alg": "HS256", "typ": "JWT"}).encode()) -body = enc(json.dumps(payload, separators=(",", ":"), sort_keys=True).encode()) -sig = enc(hmac.new(secret, f"{header}.{body}".encode(), hashlib.sha256).digest()) -print(f"{header}.{body}.{sig}") -PY -)" -``` +Prometheus metrics are opt-in through `ENABLE_PROMETHEUS_METRICS`; do not expose operational telemetry publicly without an authenticated observability boundary. -```bash -curl -s http://localhost:8000/api/emails \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - | jq '.emails[] | {subject, thread_id, reply_count}' -curl -s http://localhost:8000/api/emails/thread/thread-root@example.com \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - | jq '.thread[] | {message_id, in_reply_to, references}' - -# Requires a tenant OpenAI key because search generates a query embedding. -curl -s -X POST http://localhost:8000/api/search \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - -H 'content-type: application/json' \ - -d '{"query":"Quarterly plan"}' - -# Send remains honest in local/dev mode: missing SMTP config returns 400. -curl -s -X POST http://localhost:8000/api/emails/send \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - -H 'content-type: application/json' \ - -d '{"to":"alice@example.com","subject":"Re: Quarterly plan","body":"Thanks"}' - -# Convert email-derived execution items into source-linked ticket tasks. -TASK_BODY="$(cat <<'JSON' -{ - "source_email_id": "", - "thread_id": "thread-root@example.com", - "items": ["담당자 확인"] -} -JSON -)" -curl -s -X POST http://localhost:8000/api/tasks/from-email \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - -H 'content-type: application/json' \ - -d "$TASK_BODY" - -# Request a customer-owned calendar writeback intent. This selects a trusted -# server-side source and returns no provider secret. Provider execution is -# explicit opt-in and requires If-Match/ETag evidence. -curl -s http://localhost:8000/api/calendar/writeback-sources \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" -curl -s -X POST http://localhost:8000/api/calendar/writeback-intent \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - -H 'content-type: application/json' \ - -d '{"action":"update","summary":"담당자 확인 회의","target_source_id":"caldav-primary","execute_provider":true}' - -# Review source-backed Security governance without exposing provider secrets. -curl -s http://localhost:8000/api/security/access-surface \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - | jq '{scope_kind, sources, policy_decisions}' - -# Review source-backed Data repository, ingestion, embedding, and quality state. -curl -s http://localhost:8000/api/data/quality-surface \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - | jq '{workspace_id, audit_event, repositories, quality_checks}' -curl -s -X POST http://localhost:8000/api/data/documents \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" \ - -H 'content-type: application/json' \ - -d '{"document_name":"decision-note.md","document_type":"text/markdown","document_content":"# Decision note"}' -curl -s -X POST http://localhost:8000/api/data/documents/doc_example/reparse \ - -H "Authorization: Bearer $NARUON_DEV_BEARER" -``` +## Backup, upgrade, and recovery -## Error-message contract - -Errors should tell a contributor what failed and avoid leaking internals: - -- SMTP not configured: `400 {"detail":"SMTP is not configured"}`. Create a - tenant config with SMTP host, port, and username before testing real send. -- Local simulated send: `{"status":"simulated","simulated":true}`. Treat as - payload/header verification only, not delivery proof. -- Search without OpenAI key: `400 {"detail":"OpenAI API key not configured"}`. - Add a tenant OpenAI key or skip search smoke locally. -- Search backend failure: `500 {"detail":"Search failed"}`. Check backend logs; - raw exceptions are intentionally not returned to clients. -- Missing thread: `404 {"detail":"Thread not found"}`. Re-import fixtures or - verify the URL uses the normalized thread id. -- Task creation from a missing or unauthorized source email: - `404 {"detail":"Source email not found"}`. -- Task creation without usable execution items: - `422 {"detail":"At least one execution item is required"}`. -- Calendar writeback with no trusted customer-owned CalDAV/CardDAV/WebDAV source: - `422 {"detail":"No customer-owned writeback source is available"}`. The - frontend must show this as a writeback-intent failure, not as a completed - provider calendar write. - -## Current scope contract - -Runtime auth no longer trusts public `X-User-*`, `X-Organization-*`, -`X-Group-*`, or `X-Dev-Auth-Token` headers. Email rows now carry a nullable -`user_id` owner key, and email/search/network graph endpoints scope reads to the -authenticated user. Local bootstrap and fixture imports default that owner to -`default`; production -multi-user use still needs a verified OIDC provider plus an audited -mailbox-owner migration/backfill before real tenant data is mixed. - -The current frontend shell now exposes the north-star workspace map in the -primary and mobile menus: Today dashboard, Mail, Calendar, Tasks, Projects, -Context Search, AI Hub, Data, Security, and Settings. The `/mail`, `/search`, -`/tasks`, `/calendar`, `/projects`, `/ai-hub`, `/data`, `/security`, and -`/settings` destinations must render real work-detail surfaces rather than -static placeholder copy: calendar month/week/detail/coordination and CalDAV -writeback queues, ticket task boards and source-linked details, integrated -search result/detail graph timelines, source-backed project folders and -decision-evidence logs, document -repository/ingestion/embedding/quality queues, security dashboards and policy -screens, and operational settings. Provider write execution and enterprise -identity remain future connector/auth slices until source-backed integrations -exist. Browser writes to signed backend routes use the HttpOnly -`naruon_session` cookie; the same-origin Next.js `/api/*` proxy converts that -server-readable cookie into the backend `Authorization: Bearer` session and -strips public identity headers such as `X-User-Id` and `X-Organization-Id`, -including group and dev-token variants, rather than forwarding development -identity fallbacks. -Settings connected-account workflow reads and saves `/api/accounts/config` -through the same signed-session path and scopes provider settings by the signed -`user_id + organization_id` owner. It displays SMTP, IMAP, POP3, OAuth, -CalDAV/CardDAV, and WebDAV readiness from masked account fields and source -registry APIs, preserves stored credential secrets when the user leaves -replacement fields blank, and keeps Naruon framed as a web client/relay proxy -rather than an email host. Settings also exposes organization-admin -self-hosted connector token rotation through `/api/runner-config/rotate`; the -one-time token is shown only after rotation and is not included in the connector -manifest. -Mail worker logs and raised errors use generic account-configuration wording for -missing POP3 credentials so operational logs do not reveal credential-type -details. - -Email-derived work is tracked through `/api/tasks/from-email`. Created ticket -tasks retain an internal source-email foreign key, expose source message/thread -provenance, sanitize NUL bytes from LLM/email-derived titles, and return opaque -public task ids instead of exposing database integer surrogates. The new -`ticket_tasks` table keeps database names two-word `snake_case` such as -`task_id`, `task_title`, `status_code`, and `priority_code`. - -Calendar actions in `EmailDetail` now request `/api/calendar/writeback-intent` -for each extracted execution item and display the selected trusted source -provenance. Calendar source selection now reads opaque -`calendar_writeback_sources.source_uid` rows instead of exposing sequential -CalDAV account ids, and the Calendar workspace loads those rows through signed -`/api/calendar/writeback-sources` before posting an opaque `target_source_id`. -The workspace now presents those sources as explicit selectable writeback -targets and shows the selected source ETag/capability state before intent -creation. Provider execution is opt-in through `execute_provider=true`; the API -dispatches a signed `write_caldav` command to an active outbound runner only -after server-authoritative source selection and If-Match evidence are available. -The browser no longer claims `/api/calendar/sync` success from the mail-detail -action path; direct browser provider writes stay deferred. Connector-side DAV -adapters can execute ETag/If-Match-guarded PUTs, and the WebDAV -materialization endpoint can dispatch signed commands to an active outbound -runner. Durable queueing, retry, and broader UI dispatch controls remain -connector workflow follow-ups. -WebDAV writeback and self-sent knowledge materialization use -`webdav_accounts.source_uid` as the browser-visible source id, scope lookup by -the signed session organization, honor persisted `writeback_enabled` -eligibility, surface `webdav_accounts.etag_value` as source-safe If-Match -evidence, reject legacy `target_account_id` payloads, and keep sequential -`account_id` values internal-only. The Data workspace exposes the WebDAV source -as an explicit selected target and treats `409` If-Match/ETag responses as -conflicts instead of generic failures, so UI copy never implies a provider write -overwrote customer-owned files. Provider URLs, usernames, and credentials stay -server-side. Project folder listings are scoped by the signed session -organization and expose opaque `project_folders.folder_uid` values instead of -sequential `folder_id` values, and the `/dav` PUT skeleton fails closed until -provider-backed source, capability, and ETag/If-Match checks exist. -Data repository, ingestion, embedding, and quality status is loaded from signed -`/api/data/quality-surface`. Workspace document uploads use signed -`POST /api/data/documents`, while reparse, embedding-regeneration, and HWP -conversion controls call the scoped document action endpoints and keep -`provider_write_executed=false`. Workspace document WebDAV materialization uses -signed `POST /api/data/documents/{document_id}/webdav-materialization-intent` -with an opaque selected WebDAV `source_uid`; the backend derives the target path -and content server-side and dispatches connector execution only on -`execute_provider=true`. The UI must not reintroduce static ingestion logs, fake -vector counts, unsupported embedding model names, fake quality totals, or inert -permanent ready-soon controls; use source-backed rows or explicit pending -states. - -## Operations and release docs - -- `docs/operations/release-deployment-architecture.md`: release, CI, GHCR, and - live E2E evidence path. -- `docs/operations/open-source-apm.md`: OpenTelemetry, Prometheus, Grafana, Loki, - Tempo/Jaeger adoption plan. Settings calls signed - `/api/observability/operational-signals` to show server-observed connector - registration, active runner connection state, recent durable heartbeat - history, Prometheus, and OpenTelemetry configuration while provider execution - remains future connector work. -- `docs/operations/email-relay-proxy-boundary.md`: Naruon is a web client - relay/proxy, not an email server. -- `docs/operations/source-of-truth-and-writeback-sovereignty.md`: customer-owned - source-of-truth, connector, writeback, and audit rules. -- `docs/operations/postgresql-physical-replication.md`: physical replication, - WAL, restore, and read-routing plan. -- `docs/operations/auth-key-management.md`: auth boundary, Fernet key management, - and Keycloak/Casdoor evaluation. -- `docs/operations/traefik-evaluation.md`: Traefik versus current NGINX ingress - evaluation. -- `docs/development/merge-gate-policy.md`: metadata-only PR governance, - current-head CodeRabbit evidence, and required-check behavior. - -## Verification used for this hardening pass +Before an upgrade: -```bash -./scripts/verify_threading.sh +1. review the target release notes and image provenance; +2. take and verify a PostgreSQL backup; +3. confirm object or file artifacts are covered by the deployment backup plan; +4. apply database migrations through the supported startup or migration path; +5. verify the API root, signed-session access, Mail, Search, and configured connectors; and +6. retain a rollback point until the new deployment has passed operator smoke checks. -# Equivalent manual checks: -cd backend && python3 -m pytest -q -cd frontend && npm test && npm run lint && npm run build -``` +Operational references: -Additional focused checks for the current workspace/task/governance slice: +- [Release and deployment architecture](docs/operations/release-deployment-architecture.md) +- [Container provenance contract](docs/operations/container-provenance-contract.md) +- [PostgreSQL replication and recovery](docs/operations/postgresql-physical-replication.md) +- [Open-source observability](docs/operations/open-source-apm.md) +- [Latest release](https://github.com/ContextualWisdomLab/naruon/releases/latest) -```bash -cd backend && \ - PYTHONDONTWRITEBYTECODE=1 DISABLE_BACKGROUND_WORKERS=1 \ - pytest tests/test_tasks_api.py -q -cd frontend && npm test -- \ - src/lib/api-client.test.ts \ - src/lib/workspace-preferences.test.ts \ - src/components/DashboardLayout.test.tsx \ - src/app/calendar/page.test.tsx \ - src/app/tasks/page.test.tsx \ - src/app/search/page.test.tsx \ - src/app/projects/page.test.tsx \ - src/app/data/page.test.tsx \ - src/app/security/page.test.tsx \ - src/app/page.test.tsx \ - src/components/EmailDetail.test.tsx -cd frontend && LIVE_BASE_URL=http://127.0.0.1:18081 npm run test:e2e -- tests/e2e/dashboard-branding.spec.ts -bash scripts/ci/test_pr_governance_gate.sh -``` +## Current limitations + +- Naruon does not host customer mailboxes or provide inbound MX service. +- Production multi-user operation requires verified identity plus an audited historical ownership migration. +- Provider write intent is not delivery proof; some workflows require an active self-hosted connector and source revision evidence. +- Topic-intelligence documentation is a design and governance record, not a live Structural Topic Modeling feature. Naruon fails closed when an accepted fitted-model authority is unavailable. +- Optional sibling services are not enabled merely because their repositories exist. Operators must select, pin, configure, and validate each contract. + +## Documentation + +- [Architecture](ARCHITECTURE.md) +- [Architecture detail](docs/architecture/) +- [Architecture decisions](docs/adr/README.md) +- [Threading contract](docs/threading-contract.md) +- [Topic-intelligence boundary](docs/topic-intelligence/README.md) +- [Operations](docs/operations/) +- [Contributing](CONTRIBUTING.md) +- [Code of Conduct](CODE_OF_CONDUCT.md) +- [Security Policy](SECURITY.md) -Known local warnings: backend tests emit dependency/toolchain deprecation warnings -from Starlette multipart and compiled SWIG metadata. They are not caused by -threading code. +Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. -## Phase 10 development rules +## License -- **Stepwise execution**: Each phase requires an atomic PR, GitHub PR Tracking, Push, and Robot Review. A phase only ends when merged. Do not proceed without merge. -- **TDD + DDD**: Practice TDD, micro TDD, nano TDD, Domain Driven Development, and Context Driven Development. -- **API Wiring**: Always work with API wiring completed. -- **Collaboration**: Respect other agents' concurrent work; do not overwrite or dismiss unfamiliar changes. -- **Subagent Delegation**: Actively delegate tasks to Subagents. -- **UI/Browser Testing**: Use a real browser for testing (do not rely on assumptions). -- **Strict Errors**: Treat `Timeout`, `Fatal`, `Warn`, and `Denied` outputs as hard failures. -- **Goal**: Actively manage tasks to ensure open PR counts converge to 0. +See [LICENSE](LICENSE). From 805721fff5c3924f98d83c9f635911d642abb325 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 20:07:03 +0900 Subject: [PATCH 02/13] docs: move contributor procedures out of README --- CONTRIBUTING.md | 97 ++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 18 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1df13d718..a7ea04844 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,41 +1,102 @@ # Contributing +Thank you for improving Naruon. Keep each change focused, preserve customer-owned source boundaries, and make verification reproducible for the exact code under review. + ## Setup 1. Copy `.env.example` to `.env`. -2. Prefer `docker compose up -d --build` for full-stack local work. -3. For manual backend work, run commands from `backend/`. -4. For frontend work, run `npm install` before lint/build/test. +2. Prefer `./scripts/naruon_compose.sh up -d --build` for full-stack local work. +3. Use synthetic fixtures only. Do not commit real email, calendar, contact, file, credential, or customer data. + +## Manual development paths + +Backend: + +```bash +cd backend +python3 -m pip install -r requirements.txt +python3 scripts/migrate_db.py +python3 -m pytest -q +uvicorn main:app --reload +``` -## Verification before opening a PR +Frontend: + +```bash +cd frontend +npm install +npm test +npm run lint +npm run build +npm run dev +``` + +## Verification before opening or updating a PR + +Run the checks that cover the changed surface. The repository-wide threading verification remains the default starting point: ```bash ./scripts/verify_threading.sh ``` -For backend-only changes, run `cd backend && python3 -m pytest -q`. For frontend changes, run `cd frontend && npm test && npm run lint && npm run build`. +For focused changes, also run the relevant paths: + +```bash +cd backend && python3 -m pytest -q +cd frontend && npm test && npm run lint && npm run build +``` + +UI changes require a real-browser check of the changed user flow. Do not infer browser behavior only from unit tests or source inspection. + +## Pull request scope + +- Use a descriptive title such as `fix: preserve thread provenance during import`. +- Keep one logical product or infrastructure change per PR. +- State the customer or operator outcome, changed boundary, validation, and known limitations. +- Do not mix unrelated dependency churn, workflow repair, feature work, or sibling-repository implementation in one PR. +- When a sibling integration is needed, change Naruon's adapter or contract here and use a separate PR in the owning sibling repository. Do not copy sibling source into Naruon. ## Threading changes - Add or update tests before production code changes. -- Keep `backend/services/threading_service.py` as the only canonical thread assignment owner. +- Keep `backend/services/threading_service.py` as the only canonical thread-assignment owner. - Keep fixtures in `backend/tests/fixtures` small and synthetic; do not commit real email data. - Preserve honest send semantics: simulated local send is not delivery proof. -- Do not claim production multi-user email isolation until historical - `emails.user_id` values are audited/backfilled against verified mailbox - owners and the scoped query tests are kept green. +- Do not claim production multi-user email isolation until historical `emails.user_id` and organization ownership have been audited and backfilled against verified mailbox owners. + +## Source and writeback boundaries + +- Customer mail, calendar, contact, and file systems remain authoritative. +- Browser input must not choose provider credentials, private server URLs, or unscoped database identifiers. +- Provider writes require server-authoritative source lookup, ownership and capability checks, explicit execution intent, and conflict evidence such as ETag/If-Match when supported. +- Pending, simulated, deferred, or locally validated operations must not be described as completed provider writes. ## Secrets and data -Never commit `.env`, real mailbox exports, SMTP credentials, OAuth secrets, OpenAI keys, or user email content. Use synthetic `.eml` fixtures only. +Never commit: + +- `.env` files or secret-manager exports; +- mailbox archives or real `.eml` content; +- SMTP, IMAP, POP3, CalDAV, CardDAV, or WebDAV credentials; +- OAuth, OIDC, model-provider, or connector tokens; +- session-signing or encryption keys; or +- customer message, attachment, document, calendar, or contact data. + +Use synthetic fixtures and opaque identifiers in tests and documentation. + +## Automation and concurrent work + +The normative procedures for Phase 10 delivery, exact-head CI evidence, stacked PRs, workflow operation, and repository writer ownership are in: + +- [`docs/development/automation-and-collaboration.md`](docs/development/automation-and-collaboration.md) +- [`docs/development/merge-gate-policy.md`](docs/development/merge-gate-policy.md) -## Communication Guidelines +Read both before changing a PR branch, acting on a dependent PR, or interpreting required checks. -To keep our project organized and easy to navigate for everyone, please adhere to the following communication guidelines: +## Communication guidelines -* **Issues:** Use the provided Issue Templates (`Bug Report` or `Feature Request`). Search existing issues before creating a new one to prevent duplicates. -* **Pull Requests:** Ensure your PR title is descriptive (e.g., `fix: correct email parsing bug`). Fill out the PR template entirely. Keep PRs focused on a single logical change. -* **Review Process:** - * Ensure all CI checks pass (linting, tests, etc.) before requesting a review. - * Respond to reviewer comments promptly. - * Be respectful and constructive in your code reviews. +- Search existing issues before creating a new one. +- Use the issue templates for bugs and feature requests. +- Fill out the PR template completely. +- Respond to review findings with a fix, evidence-backed rebuttal, or explicit supersession. +- Keep review comments respectful, specific, and actionable. From 336f39d8316de71de9a8cd9ce43526b6d6377956 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 20:07:32 +0900 Subject: [PATCH 03/13] docs: document automation stacking and writer boundaries --- .../automation-and-collaboration.md | 118 ++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/development/automation-and-collaboration.md diff --git a/docs/development/automation-and-collaboration.md b/docs/development/automation-and-collaboration.md new file mode 100644 index 000000000..7c486d51d --- /dev/null +++ b/docs/development/automation-and-collaboration.md @@ -0,0 +1,118 @@ +# Development Automation and Collaboration + +This document contains repository-internal delivery rules. It is intentionally separate from the customer and operator README. + +## Scope + +These rules apply to human contributors, coding agents, review agents, and repository automation working on Naruon. Live branch protection and organization rulesets remain authoritative. Detailed merge-gate semantics are maintained in [`merge-gate-policy.md`](merge-gate-policy.md). + +## Phase 10 delivery procedure + +1. **Stepwise execution** — Each dependent phase uses an atomic PR with explicit tracking, a pushed branch, and applicable review evidence. A dependent phase does not land before its parent phase. +2. **TDD and domain boundaries** — Use test-driven development at the smallest useful unit and preserve domain and context boundaries. +3. **Complete API wiring** — A product slice is not complete while its public UI, API, persistence, or adapter path is knowingly disconnected. +4. **Concurrent-work safety** — Do not overwrite, dismiss, or silently absorb unfamiliar changes from another active writer. +5. **Delegation** — Delegate independent, read-only investigation and bounded implementation work when that reduces shared-state contention. +6. **Real browser verification** — UI behavior must be checked in a browser for the changed flow. +7. **Strict failure handling** — `Timeout`, `Fatal`, `Warn`, and `Denied` output from governed validation is not clean evidence. Investigate and resolve the cause or record the external blocker. +8. **Queue convergence** — Resolve valid open work in dependency order and remove only work that is demonstrably duplicate, superseded, or invalid. + +## Exact-head CI evidence + +Merge evidence belongs to one immutable PR head SHA. + +- Read the current `headRefOid` before evaluating checks or reviews. +- Required checks, security evidence, review evidence, and thread-resolution state must correspond to that same head. +- A success from an earlier commit, base branch, merge simulation, or sibling PR is not evidence for the current head. +- A new push invalidates predecessor-head evidence and starts a new evidence cycle. +- Queued, pending, requested, waiting, or in-progress checks are wait states, not successes and not automatically defects. +- Failed, cancelled, timed-out, action-required, unknown, or unverifiable required states fail closed. +- Conditional skipped jobs satisfy a gate only when the live ruleset and workflow contract explicitly make them non-required for that head. +- Review comments and requested changes must be resolved or superseded on the current diff. +- Before a merge action, re-read the live rulesets, current head, required contexts, reviews, and unresolved threads. Do not rely on remembered evidence. + +Use the commands and robot-review contract in [`merge-gate-policy.md`](merge-gate-policy.md) for the detailed evidence procedure. + +## PR stacking + +A stacked PR is a dependency graph, not a shortcut around the protected base. + +1. Declare the immediate parent PR and base branch in the child PR description. +2. Keep each child limited to the delta above its declared parent. +3. Run required security and CI workflows on every PR base, including intermediate stacked bases. +4. Land or close the parent before the dependent child is considered for integration. +5. Never use a child's checks, reviews, or mergeability as evidence that its parent is safe. +6. After the parent lands, retarget or rebase the child onto the live protected `develop` tip, resolve conflicts, and push the same child branch. +7. Re-run the complete exact-head evidence cycle for the rewritten child head. +8. Reinspect the final diff for duplicated parent commits, obsolete compatibility code, and unrelated scope before requesting integration. +9. Do not merge children out of dependency order or bypass a blocked parent with an equivalent hidden change. + +Independent PRs may proceed in parallel when they do not share an uncoordinated writer surface or dependency edge. + +## Repository writer boundary + +Each branch and overlapping path set has one active writer at a time. + +- A contributor or agent owns only the branch explicitly assigned to that work. +- Do not push to another contributor's PR branch, edit its commits, resolve its threads, or change its base without an explicit handoff. +- Review, analysis, and evidence gathering are read-only unless the branch owner requests implementation help. +- On handoff, record the current branch, head SHA, intended files, unresolved findings, and validation state before the new writer changes anything. +- If two changes overlap, serialize them, split the files or contracts, or create a declared stack. Do not race writes and repair the history afterward. +- Unknown files or commits are preserved until ownership and intent are established. +- Central `.github` governance and sibling CWL repositories are read-only dependencies from a Naruon PR unless that repository has a separately assigned writer and PR. + +### Sibling integration boundary + +Naruon is the ecosystem hub, but every sibling owns its implementation and release lifecycle. + +- Naruon may add an adapter, generated client, event consumer, feature flag, or contract test for a published sibling contract. +- Naruon must not copy a sibling implementation, write directly to the sibling database, or depend on an unpinned mutable branch. +- A required sibling change is delivered in a separate PR in the owning repository, then consumed through a released package, API, event schema, or OCI digest. +- The Naruon core must start and report a clear degraded or disabled state when an optional sibling is unavailable. + +### Customer-provider writer boundary + +Repository write ownership is distinct from provider write authority. Mail, calendar, contact, and file writes require server-authoritative source selection, scoped credentials, explicit execution intent, capability checks, conflict evidence, and an auditable connector result. A browser or model proposal cannot self-authorize a provider mutation. + +## Workflow operation boundary + +Routine delivery must not manipulate GitHub Actions through a maintainer's personal account. + +- Do not force-cancel queued or running workflows to free capacity. +- Do not manually rerun an entire workflow or failed jobs merely to seek a different result. +- Prefer the natural trigger created by the relevant code push, review event, scheduled controller, or approved bot-owned workflow contract. +- When a run is waiting, continue independent non-conflicting work or record the external wait state; do not convert waiting into artificial evidence. +- A genuine infrastructure failure may be retried only through the repository's authorized automation path and documented policy, never as an unexplained personal-account intervention. +- Do not use admin merge, branch-protection bypass, review dismissal, required-check suppression, or temporary policy weakening for routine delivery. + +Creating or updating the code under review may naturally start new checks. That is different from cancelling or replaying an existing run. + +## Review and merge responsibilities + +- Review agents inspect and report; they do not silently become branch writers. +- Repair agents modify only the explicitly assigned branch and scope. +- Metadata controllers may publish idempotent status or blocker information within their documented permissions. +- Merge automation acts only after the live protected-branch contract is satisfied. +- The author of a change does not manufacture independent approval evidence. + +## Failure and recovery + +When a gate fails: + +1. identify the exact head SHA and failing context; +2. obtain the failing log, annotation, or reproducible local evidence; +3. map the failure to the smallest owned source surface; +4. add or update a regression test when applicable; +5. push the narrow fix to the same assigned branch; +6. let the new head trigger its own evidence cycle; and +7. verify that no stacked child or concurrent branch was silently invalidated. + +When the cause is outside the repository, record the external dependency, current head, observed state, and next non-destructive action. Do not report a waiting or unavailable external service as a passing gate. + +## Related documents + +- [`merge-gate-policy.md`](merge-gate-policy.md) +- [`../../AGENTS.md`](../../AGENTS.md) +- [`../../ARCHITECTURE.md`](../../ARCHITECTURE.md) +- [`../operations/source-of-truth-and-writeback-sovereignty.md`](../operations/source-of-truth-and-writeback-sovereignty.md) +- [`../operations/release-deployment-architecture.md`](../operations/release-deployment-architecture.md) From 432aad7fe7ed900bad449e02d2551c6629871b41 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 21:02:58 +0900 Subject: [PATCH 04/13] docs: preserve README governance contracts --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a34c69d21..f1f8948ae 100644 --- a/README.md +++ b/README.md @@ -128,7 +128,7 @@ The default Compose profile starts PostgreSQL, Ollama, the backend, and the fron ```bash ./scripts/naruon_compose.sh ps curl -fsS http://127.0.0.1:8000/ -python3 -m webbrowser http://127.0.0.1:3000 +python3 -m webbrowser http://localhost:3000 ``` The API root should return an `ok` status. The fixture import creates a small synthetic conversation so the Mail and threading path can be checked without real customer data. @@ -238,7 +238,7 @@ Operational references: - [Code of Conduct](CODE_OF_CONDUCT.md) - [Security Policy](SECURITY.md) -Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. +Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. ## License From 58a5a0addfd903d606c9205ffacdcfb219735432 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 21:19:13 +0900 Subject: [PATCH 05/13] docs: retain central workflow compatibility statement --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f1f8948ae..62e3e0ebd 100644 --- a/README.md +++ b/README.md @@ -238,7 +238,7 @@ Operational references: - [Code of Conduct](CODE_OF_CONDUCT.md) - [Security Policy](SECURITY.md) -Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. +Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local copies of those governed review, security, or merge workflows. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. ## License From da5952563a4d6c47f122feab1e341f73365e47a7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 21:27:06 +0900 Subject: [PATCH 06/13] docs: preserve central automation contract wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 62e3e0ebd..be3a0ec74 100644 --- a/README.md +++ b/README.md @@ -238,7 +238,7 @@ Operational references: - [Code of Conduct](CODE_OF_CONDUCT.md) - [Security Policy](SECURITY.md) -Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local copies of those governed review, security, or merge workflows. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. +Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local OpenCode, Strix, or merge-scheduler workflow copies; branch updates, auto-merge, and mechanical merge actions are governed by those central workflows under the protected-branch policy. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. ## License From 3428cd9e6272f6238d1ce8ae7d81b0780048e74e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 21:36:06 +0900 Subject: [PATCH 07/13] docs: address contributor review findings --- CONTRIBUTING.md | 42 ++++++++++++++++++++++++++++++------------ 1 file changed, 30 insertions(+), 12 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a7ea04844..a12503f7a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -23,12 +23,13 @@ uvicorn main:app --reload Frontend: ```bash +corepack enable pnpm cd frontend -npm install -npm test -npm run lint -npm run build -npm run dev +pnpm install --frozen-lockfile +pnpm test +pnpm run lint +pnpm run build +pnpm run dev ``` ## Verification before opening or updating a PR @@ -39,20 +40,34 @@ Run the checks that cover the changed surface. The repository-wide threading ver ./scripts/verify_threading.sh ``` -For focused changes, also run the relevant paths: +For focused changes, run each path from the repository root so one directory change cannot affect the next command: ```bash -cd backend && python3 -m pytest -q -cd frontend && npm test && npm run lint && npm run build +(cd backend && python3 -m pytest -q) +(cd frontend && pnpm test && pnpm run lint && pnpm run build) ``` -UI changes require a real-browser check of the changed user flow. Do not infer browser behavior only from unit tests or source inspection. +UI changes require a real-browser check of the changed user flow. The supported full-product smoke path mirrors [Application CI](.github/workflows/app-ci.yml) and requires Node.js 24, Corepack/pnpm, installed frontend dependencies, and Playwright Chromium: + +```bash +corepack enable pnpm +(cd frontend && pnpm install --frozen-lockfile) +(cd frontend && pnpm exec playwright install --with-deps chromium) +( + cd frontend + NARUON_FULL_PRODUCT_BASE_URL=http://127.0.0.1:3001 \ + NARUON_FULL_PRODUCT_SCREENSHOT_DIR=/tmp/naruon-full-product-smoke \ + pnpm run full:smoke +) +``` + +Do not infer browser behavior only from unit tests or source inspection. Add a narrower Playwright target when the changed flow has a focused test, and record the exact command and result in the PR body. ## Pull request scope - Use a descriptive title such as `fix: preserve thread provenance during import`. - Keep one logical product or infrastructure change per PR. -- State the customer or operator outcome, changed boundary, validation, and known limitations. +- State the customer or operator outcome, changed boundary, exact focused verification commands and results, and known limitations. - Do not mix unrelated dependency churn, workflow repair, feature work, or sibling-repository implementation in one PR. - When a sibling integration is needed, change Naruon's adapter or contract here and use a separate PR in the owning sibling repository. Do not copy sibling source into Naruon. @@ -86,12 +101,15 @@ Use synthetic fixtures and opaque identifiers in tests and documentation. ## Automation and concurrent work -The normative procedures for Phase 10 delivery, exact-head CI evidence, stacked PRs, workflow operation, and repository writer ownership are in: +Before making changes, read the instructions that govern the repository and the delivery surface: +- [`AGENTS.md`](AGENTS.md) - [`docs/development/automation-and-collaboration.md`](docs/development/automation-and-collaboration.md) - [`docs/development/merge-gate-policy.md`](docs/development/merge-gate-policy.md) -Read both before changing a PR branch, acting on a dependent PR, or interpreting required checks. +For frontend work, also read [`frontend/AGENTS.md`](frontend/AGENTS.md). After installing frontend dependencies, read the relevant version-matched Next.js guide under `frontend/node_modules/next/dist/docs/` before relying on framework APIs or conventions. + +These documents are mandatory before changing a PR branch, acting on a dependent PR, interpreting required checks, or modifying frontend behavior. ## Communication guidelines From a968586c0e48959abc0541b40f66e0929427a9d7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 21:37:03 +0900 Subject: [PATCH 08/13] docs: address README review findings --- README.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index be3a0ec74..372da0600 100644 --- a/README.md +++ b/README.md @@ -156,9 +156,9 @@ Production deployments should inject bootstrap secrets from an approved secret m - Browser traffic uses same-origin `/api/*`; the Next.js server-side proxy converts the HttpOnly session cookie into the backend bearer session. - Public identity headers such as `X-User-Id` and `X-Organization-Id` are not trusted as authentication. -- Local HMAC sessions are suitable for local and compatibility testing. Production workspace membership should be established by verified OIDC/JWKS or an explicit server-side membership authority. -- Before mixing real multi-user data in one database, operators must complete and audit mailbox-owner and organization backfills for historical rows. -- Authorization is deny-first RBAC plus ABAC. Data-region, consent, workspace, group, source capability, and customer-policy denies take precedence over broad role grants. +- Local HMAC sessions are suitable for local and compatibility testing. Production workspace membership should use verified OIDC/JWKS or another explicit server-side membership authority. [OpenID Connect Core 1.0](https://openid.net/specs/openid-connect-core-1_0-18.html#IDTokenValidation) requires clients to validate token issuer, audience, signature, and expiry rather than trusting unverified claims. +- Before mixing real multi-user data in one database, operators must complete and audit mailbox-owner and organization backfills for historical rows. This product-specific migration requirement is defined in the [data and tenancy architecture](ARCHITECTURE.md#data-and-tenancy-boundary) and the [authentication and key-management runbook](docs/operations/auth-key-management.md). +- Authorization is deny-first RBAC plus ABAC. [NIST SP 800-162](https://csrc.nist.gov/pubs/sp/800/162/upd2/final) defines ABAC decisions over subject, object, operation, and environment attributes; the [OWASP Authorization Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) recommends deny-by-default behavior and permission validation on every request. Naruon's data-region, consent, workspace, group, source-capability, and customer-policy denies therefore take precedence over broad role grants. See [Authentication and key management](docs/operations/auth-key-management.md) and the [Security policy](SECURITY.md). @@ -238,8 +238,13 @@ Operational references: - [Code of Conduct](CODE_OF_CONDUCT.md) - [Security Policy](SECURITY.md) -Repository delivery and security automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local OpenCode, Strix, or merge-scheduler workflow copies; branch updates, auto-merge, and mechanical merge actions are governed by those central workflows under the protected-branch policy. Developer, automation, review, and collaboration procedures are maintained under `CONTRIBUTING.md` and `docs/development/`, separate from this customer and operator guide. - ## License See [LICENSE](LICENSE). + + From 9290769056cc781f9b0171062a594e64b8ceabb1 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 22:33:52 +0900 Subject: [PATCH 09/13] docs: codify recurring-pattern and research requirements --- CONTRIBUTING.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a12503f7a..34d656d95 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -68,6 +68,8 @@ Do not infer browser behavior only from unit tests or source inspection. Add a n - Use a descriptive title such as `fix: preserve thread provenance during import`. - Keep one logical product or infrastructure change per PR. - State the customer or operator outcome, changed boundary, exact focused verification commands and results, and known limitations. +- When a fix exposes a recurring bug pattern or delivery anti-pattern, record the prevention rule in `AGENTS.md` and update every affected test, mock, fixture, and document in the same PR so the defect cannot survive in a parallel contract. +- Substantive feature and process PRs must follow the [`AGENTS.md` research-grounding policy](AGENTS.md#research-grounding-attach-paper-pdfs): include relevant academic literature with complete citations, commit paper PDFs only when redistribution is permitted, and otherwise provide source links and concise evidence summaries. - Do not mix unrelated dependency churn, workflow repair, feature work, or sibling-repository implementation in one PR. - When a sibling integration is needed, change Naruon's adapter or contract here and use a separate PR in the owning sibling repository. Do not copy sibling source into Naruon. From 82751aa9720701fc2bfbbd323e1e6232adc21052 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 19 Aug 2026 16:46:30 +0900 Subject: [PATCH 10/13] docs: remove internal automation from README --- README.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/README.md b/README.md index 372da0600..1848008da 100644 --- a/README.md +++ b/README.md @@ -241,10 +241,3 @@ Operational references: ## License See [LICENSE](LICENSE). - - From 7e930419a72736d75f78a085a82b6f6e4a8e0482 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 19 Aug 2026 20:34:49 +0900 Subject: [PATCH 11/13] docs: retain central workflow contract --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 1848008da..9bb6a7de6 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,8 @@ It connects to systems an organization already controls, keeps bounded workspace Naruon labels simulated, deferred, pending, and completed actions differently. A generated intent or local payload check is not presented as proof that a customer-owned provider accepted a write. +Review and merge automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local OpenCode, Strix, or merge-scheduler workflow copies; branch updates, auto-merge, and mechanical merge actions run through the central workflow as the target repository's `github-actions[bot]`. + ## Product boundary ### Naruon owns From f3731c3e4a6d153225ca3c1be9e92cc3bec4e406 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 21 Aug 2026 05:00:04 +0900 Subject: [PATCH 12/13] docs: keep delivery tooling out of operator README --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 9bb6a7de6..1848008da 100644 --- a/README.md +++ b/README.md @@ -24,8 +24,6 @@ It connects to systems an organization already controls, keeps bounded workspace Naruon labels simulated, deferred, pending, and completed actions differently. A generated intent or local payload check is not presented as proof that a customer-owned provider accepted a write. -Review and merge automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local OpenCode, Strix, or merge-scheduler workflow copies; branch updates, auto-merge, and mechanical merge actions run through the central workflow as the target repository's `github-actions[bot]`. - ## Product boundary ### Naruon owns From c0ac8c01d58473680b89a225107366fec4fae986 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 21 Aug 2026 09:41:14 +0900 Subject: [PATCH 13/13] docs: restore central workflow contract --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 1848008da..9bb6a7de6 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,8 @@ It connects to systems an organization already controls, keeps bounded workspace Naruon labels simulated, deferred, pending, and completed actions differently. A generated intent or local payload check is not presented as proof that a customer-owned provider accepted a write. +Review and merge automation is supplied by the ContextualWisdomLab central required workflows. This repository does not carry repo-local OpenCode, Strix, or merge-scheduler workflow copies; branch updates, auto-merge, and mechanical merge actions run through the central workflow as the target repository's `github-actions[bot]`. + ## Product boundary ### Naruon owns