Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/ops/AGENT-OBSERVABILITY-PRIORITY-DECISION.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ The observability stack is reorganized by dependency rather than by product cate

**P1 — parallel evaluation and reproduction surfaces**

- Temporal (self-host FOSS first; optional Cloud Free Tier): durable Workflow/Activity substrate for long-running agents. Capability-gated; not a dual-gate dependency. Docker Compose under `mcp-docker/temporal/`; local `temporal server start-dev`.
- LangSmith Trajectories: readable flat chronological path over multi-turn sessions (thread projection). Online evals, annotation queues, dataset/SFT export. Observational adapter only — never canonical corpus. Official Temporal Python `LangSmithPlugin` bridges Worker boundaries.
- Langfuse: optional experiment/evaluation adapter over canonical JSONL/OTEL evidence.
- Phoenix: parallel open-source experiment/evaluation adapter, with Docker as one possible reproducible lab substrate.
- Lizard: fast multi-language CCN/NLOC/token/parameter feature provider.
Expand Down Expand Up @@ -55,7 +57,7 @@ Join execution events to GitHub run attempts, SHAs, PRs, Action→Effect events,

### Phase D — parallelize/reproduce

Run equivalent telemetry through Langfuse and Phoenix adapters, and use Docker/Codespaces for distinct reproduction/experiment purposes. Compare completeness, decision quality, cost, privacy surface, and variance.
Run equivalent telemetry through Langfuse, Phoenix, and (optional) LangSmith Trajectories adapters. Temporal self-host may host durable multi-turn execution for those cohorts. Use Docker/Codespaces for distinct reproduction/experiment purposes. Compare completeness, decision quality, cost, privacy surface, and variance.

### Phase E — compete

Expand Down
127 changes: 127 additions & 0 deletions docs/ops/TEMPORAL-LANGSMITH-ADAPTER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Temporal (self-host) + LangSmith Trajectories Adapter

**Status:** P1 observational · **Self-host first** · **Date:** 2026-09-24
**Proposal:** `docs/proposals/active/temporal-langsmith-adapter/`
**Canonical evidence remains:** OpenTelemetry + GitHub JSONL / ATES. This adapter never replaces them.

## Why

Long-running agents produce nested traces that are hard to read. LangSmith **Trajectories** project a thread into a flat chronological path (human / AI / tool messages once each). Temporal provides durable Workflows/Activities so those sessions survive process death, retries, and Worker boundaries. The official Python `LangSmithPlugin` keeps a single connected trajectory across those boundaries.

## Priority placement

| Layer | Priority | Role |
|-------|----------|------|
| OpenTelemetry | P0 | Neutral transport |
| ATES / JSONL | P0 | Canonical corpus |
| Temporal self-host | P1 | Durable execution substrate (optional) |
| LangSmith Trajectories | P1 | Readable path + online evals + dataset export (adapter) |
| Langfuse / Phoenix | P1 | Parallel eval adapters (unchanged) |

## Self-host first (available surfaces)

### 1. Local single-binary (fastest)

```bash
# Install Temporal CLI (operator machine / Codespace)
# https://docs.temporal.io/cli
temporal server start-dev --ui-port 8080
# UI: http://localhost:8080
# Frontend: localhost:7233
```

### 2. Docker Compose (repo surface)

```bash
cd mcp-docker/temporal
docker compose up -d
# UI: http://localhost:8088
# Frontend gRPC: localhost:7233
```

Same separation rule as `mcp-docker/github-mcp/`: **not** folded into Vercel mcp-hub.

### 3. Codespaces agent lane

Use existing Codespace agent lane (`docs/ops/CODESPACE-AGENT-LANE.md`) to run:

- Temporal dev server or Compose stack
- Worker process from `scripts/temporal/`
- Optional LangSmith tracing when `LANGSMITH_API_KEY` is present in the Codespace secret store

### 4. Temporal Cloud Free Tier (optional later)

Only after operator confirms signup and injects secrets. Not required for dual-gate or local smoke.

## LangSmith Trajectories (concepts)

| Concept | Shape | Use when |
|---------|-------|----------|
| Run | Single unit of work (span-like) | Debug one step |
| Trace | Tree of runs for one operation | Full execution detail |
| Thread | Sequence of traces (multi-turn) | Session linkage |
| **Trajectory** | Flat ordered messages across the thread | Read the path the agent took |

Enable tracing (capability-gated):

```bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=lsv2_... # external secret only
# optional: LANGSMITH_PROJECT=termux-monorepo-agents
```

Works with LangChain / LangGraph / Deep Agents and with SDK-style agents. Trajectories support online evaluators, annotation queues, and export to SFT datasets.

## Temporal ↔ LangSmith plugin (Python)

Experimental official plugin propagates context across Workers and avoids duplicate traces on replay.

```bash
pip install 'temporalio[langsmith]' # or uv add temporalio[langsmith]
```

```python
from temporalio.client import Client
from temporalio.contrib.langsmith import LangSmithPlugin

client = await Client.connect(
"localhost:7233",
plugins=[LangSmithPlugin(project_name="termux-monorepo-agents")],
)
```

Optional: `add_temporal_runs=True` to also surface Temporal operations in LangSmith. Default keeps application logic only.

## Repo smoke

```bash
# Terminal A: temporal server start-dev OR docker compose -f mcp-docker/temporal/docker-compose.yml up
# Terminal B:
python3 scripts/temporal/hello_workflow.py
```

Env (all optional for structural smoke):

| Variable | Purpose |
|----------|---------|
| `TEMPORAL_ADDRESS` | default `localhost:7233` |
| `TEMPORAL_NAMESPACE` | default `default` |
| `TEMPORAL_TASK_QUEUE` | default `termux-agent` |
| `LANGSMITH_TRACING` | `true` to enable |
| `LANGSMITH_API_KEY` | external only |
| `LANGSMITH_PROJECT` | project name |

## Non-goals

- No hard CI dependency on Temporal or LangSmith for `repo_gate` / `termux_smoke`.
- No secrets or Class 3/4 artifacts in git.
- No replacement of OTEL/ATES as source of truth.
- Dual-gate green + verified outcome before promote.

## References

- LangSmith Trajectories: https://www.langchain.com/blog/langsmith-trajectories-tracing
- Observability concepts: https://docs.langchain.com/langsmith/observability-concepts
- Temporal self-host: https://docs.temporal.io/self-hosted-guide
- Temporal Python + LangSmith plugin: https://github.com/temporalio/sdk-python/tree/main/temporalio/contrib/langsmith
- Local CLI: https://docs.temporal.io/cli
15 changes: 15 additions & 0 deletions docs/proposals/active/temporal-langsmith-adapter/ITEMS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# ITEMS — temporal-langsmith-adapter

| ID | Item | Status | Notes |
|----|------|--------|-------|
| TLS-001 | RECON LangSmith Trajectories + Temporal FOSS/Free Tier + official LangSmithPlugin | **done** | 2026-09-24 session |
| TLS-002 | Proposal MANIFEST + registry row | **done** | this PR |
| TLS-003 | Update AGENT-OBSERVABILITY-PRIORITY-DECISION (P1 Temporal + LangSmith Trajectories) | **done** | this PR |
| TLS-004 | Update AGENT-OBSERVABILITY-RESEARCH-MATRIX rows | **done** | this PR |
| TLS-005 | Ops runbook `docs/ops/TEMPORAL-LANGSMITH-ADAPTER.md` | **done** | self-host 1st |
| TLS-006 | Docker Compose self-host under `mcp-docker/temporal/` | **done** | official temporalio images |
| TLS-007 | Minimal Python smoke (Workflow + Activity + optional LangSmithPlugin) | **done** | scripts/temporal/ |
| TLS-008 | Integration Graph Matrix row | **done** | this PR |
| TLS-009 | Dual-gate green on tip before promote | backlog | adaptive-wait; no YOLO merge |
| TLS-010 | Optional Codespace worker runbook note | backlog | after TLS-009 |
| TLS-011 | Optional Free Tier Cloud path (operator-confirmed only) | backlog | secrets external |
52 changes: 52 additions & 0 deletions docs/proposals/active/temporal-langsmith-adapter/MANIFEST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
id: temporal-langsmith-adapter
title: "Temporal self-host + LangSmith Trajectories adapter (P1 observational)"
author: grok
posted_at: 2026-09-24
source: operator directive — LangSmith Trajectories blog + Temporal FOSS Free Tier; self-host first
status: executing
priority: P1
reviewers:
- id: grok
role: author+executor
status: executing
- id: timerloggedout-spec
role: operator-authorizer
status: requested
related_prs: []
related_branches:
- feat/temporal-langsmith-adapter
gates_required: [repo-gate, termux-smoke]
---

# MANIFEST — Temporal + LangSmith Trajectories Adapter

## Summary

Upgrade agent observability and durable-execution posture with:

1. **Temporal** (MIT OSS) — self-host first (`temporal server start-dev` → Docker Compose under `mcp-docker/temporal/`) using existing surfaces (Docker, Codespaces agent lane, local). Free Tier Cloud is optional later, not required.
2. **LangSmith Trajectories** — readable chronological path over multi-turn agent sessions (flat ordered messages). P1 observational adapter parallel to Langfuse/Phoenix. Never replaces OTEL + GitHub JSONL canonical evidence.

Official Temporal Python `LangSmithPlugin` connects Worker-boundary traces so trajectories remain coherent across durable Activities.

## Boundary

- Self-host path is primary. Cloud Free Tier only after operator confirms account.
- Capability-gated: no hard dependency for dual-gate paths.
- No secrets committed (`LANGSMITH_API_KEY`, Temporal Cloud keys stay external).
- OTEL + ATES remain P0 canonical; LangSmith/Temporal are adapters/substrate.
- Extract-only; no mega-PR fold into mcp-hub / Vercel.

## Surfaces used

| Surface | Role |
|---------|------|
| Local CLI | `temporal server start-dev` |
| Docker Compose | `mcp-docker/temporal/` (same separation pattern as github-mcp) |
| Codespaces agent lane | Interactive reproduction / Worker runs |
| GitHub Actions | Optional smoke later; not a merge gate dependency |

## Evidence

See `ITEMS.md`, `docs/ops/TEMPORAL-LANGSMITH-ADAPTER.md`, matrix updates.
21 changes: 19 additions & 2 deletions docs/proposals/registry.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,27 @@
# ArchW1z proposal registry — agents read this first
version: 1
updated_at: "2026-09-23T18:15:00Z"
updated_at: "2026-09-24T16:40:00Z"
updated_by: grok-administrator

proposals:
- id: temporal-langsmith-adapter
title: "Temporal self-host + LangSmith Trajectories adapter (P1 observational)"
author: grok
status: executing
priority: P1
path: active/temporal-langsmith-adapter/
reviewers:
- id: grok
role: author+executor
status: executing
- id: timerloggedout-spec
role: operator-authorizer
status: requested
related_prs: []
related_branches:
- feat/temporal-langsmith-adapter
gates_required: [repo-gate, termux-smoke]

- id: approxination-integration
title: "Approxination skill search/create/contribute + A/B/C/D evaluation cohort"
author: grok
Expand Down Expand Up @@ -44,7 +62,6 @@ proposals:
- https://github.com/timerloggedout-spec/bifrost-benchmarking_fork
gates_required: [repo-gate, termux-smoke]


- id: auditengine-adapt
title: "RinDig AuditEngine (Ethics Engine) adapt pass"
author: timerloggedout-spec
Expand Down
38 changes: 38 additions & 0 deletions mcp-docker/temporal/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Temporal self-host (Docker)

Separated lane — **not** part of Vercel mcp-hub.

## Quick start

```bash
docker compose up -d
# gRPC frontend: localhost:7233
# UI: http://localhost:8088
```

Prefer single-binary for pure local smoke:

```bash
temporal server start-dev --ui-port 8080
```

## Smoke worker

```bash
export TEMPORAL_ADDRESS=localhost:7233
python3 ../../scripts/temporal/hello_workflow.py
```

Optional LangSmith (external secret):

```bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=lsv2_...
export LANGSMITH_PROJECT=termux-monorepo-agents
```

## Boundary

- Dev/lab only until dual-gate path is proven.
- No secrets in this directory.
- Canonical telemetry remains OTEL + ATES/JSONL.
62 changes: 62 additions & 0 deletions mcp-docker/temporal/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
version: "3.8"

# Temporal self-host lane (official images).
# Separated from Vercel mcp-hub — same rule as mcp-docker/github-mcp.
# Dev / lab surface only. Not a dual-gate dependency.

services:
postgresql:
image: postgres:16-alpine
container_name: temporal-postgresql
environment:
POSTGRES_PASSWORD: temporal
POSTGRES_USER: temporal
volumes:
- temporal_pg_data:/var/lib/postgresql/data
networks:
- temporal-net
healthcheck:
test: ["CMD-SHELL", "pg_isready -U temporal"]
interval: 5s
timeout: 5s
retries: 10

temporal:
image: temporalio/auto-setup:1.25.2
container_name: temporal
depends_on:
postgresql:
condition: service_healthy
environment:
- DB=postgres12
- DB_PORT=5432
- POSTGRES_USER=temporal
- POSTGRES_PWD=temporal
- POSTGRES_SEEDS=postgresql
- DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
ports:
- "7233:7233"
networks:
- temporal-net
volumes:
- ./dynamicconfig:/etc/temporal/config/dynamicconfig

temporal-ui:
image: temporalio/ui:2.31.2
container_name: temporal-ui
depends_on:
- temporal
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CORS_ORIGINS=http://localhost:8088
ports:
- "8088:8080"
networks:
- temporal-net

networks:
temporal-net:
driver: bridge

volumes:
temporal_pg_data:
10 changes: 10 additions & 0 deletions mcp-docker/temporal/dynamicconfig/development-sql.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Minimal dynamic config for local Temporal auto-setup (dev only).
limit.blobSize.error:
- value: 2097152
constraints: {}
limit.blobSize.warn:
- value: 524288
constraints: {}
system.forceSearchAttributesCacheRefreshOnRead:
- value: true
constraints: {}
30 changes: 30 additions & 0 deletions scripts/temporal/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# scripts/temporal

Minimal Temporal smoke for the self-host lane.

## Prerequisites

1. Temporal frontend up:
- `temporal server start-dev`, or
- `docker compose -f mcp-docker/temporal/docker-compose.yml up -d`
2. Python package: `pip install temporalio`
Optional trajectories bridge: `pip install 'temporalio[langsmith]'`

## Run

```bash
export TEMPORAL_ADDRESS=localhost:7233
python3 scripts/temporal/hello_workflow.py
# → hello, termux-monorepo
```

With LangSmith Trajectories (external secret only):

```bash
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=lsv2_...
export LANGSMITH_PROJECT=termux-monorepo-agents
python3 scripts/temporal/hello_workflow.py
```

See `docs/ops/TEMPORAL-LANGSMITH-ADAPTER.md`.
Loading
Loading