Skip to content
This repository was archived by the owner on Aug 24, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
permissions: {contents: read}

jobs:
validate:
gate: # check 名 = job id;ruleset required check = gate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
Expand Down
7 changes: 7 additions & 0 deletions decisions/ADR-0002-llm-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@
- status: accepted
- date: 2026-08-18

## 修订记录

### rev1 (2026-08-18)
- Gateway 部署配置落盘位置由"openjiuwen 私有仓 deploy/llm-gateway"改为 **本仓 `deploy/llm-gateway/`**:与 models.yaml 同仓,validate.py 强制 gateway config 的 model_name 集合与 models.yaml alias 集合一致(防声明与网关漂移)。
- openjiuwen **不 fork 私有仓**:上游官方镜像 `openJiuwen-ai/jiuwenswarm`(A2X 注册中心 = `openJiuwen-ai/agent-protocol`),REPOS.yaml `external_upstreams` 段声明,部署渲染时 clone + pin tag。
- 不使用 submodule 引用上游:agent-registry 的消费者(validate/渲染)不需要框架源码随仓分发;submodule 指针随上游高频提交过期,且 recursive clone 膨胀。需要源码的场景按需 clone 固定 tag。

## 背景

模型选型在实现中不是一个名字,而是三件事:接 API、用量监控、多节点路由(单节点用量耗尽自动切换)。且 provider key 不能进仓库,也不能要求每次搭 agent swarm 时人工粘贴。
Expand Down
6 changes: 6 additions & 0 deletions deploy/llm-gateway/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# 复制为 .env(不进 git),填真实值后: docker compose up -d
# provider key(一处集中,agent 侧永远不接触)
PROVIDER_KEY_A=sk-xxxxxxxxxxxxxxxx
PROVIDER_KEY_B=sk-xxxxxxxxxxxxxxxx
# 管理面 master key,生成: openssl rand -hex 32
LITELLM_MASTER_KEY=sk-xxxxxxxxxxxxxxxx
40 changes: 40 additions & 0 deletions deploy/llm-gateway/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# LLM Gateway 部署指南(ADR-0002 rev1)

回答"要不要起服务器":**要一台常驻机器**,但极轻(1C/512M 即可)——家里的盒子/NAS/VPS 都行;起步阶段用你日常开发机常驻也可。所有 agent(无论在哪台机器、哪个 swarm 实例)都只连它。

## 三步部署

```bash
cd deploy/llm-gateway
cp .env.example .env && vim .env # ① 填 provider key + master key
sed -i 's/<UPSTREAM_MODEL_A>/你的真实模型名/' config.yaml # ② 把 4 个 TODO 占位换成真实 provider/模型
docker compose up -d # ③ 起服务(常驻)
curl -s http://localhost:4000/health/liveliness # 验证
```

## 发 per-team key(用量计量/配额的单位)

```bash
curl -X POST http://localhost:4000/key/generate \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"models": ["coder-fast","reviewer"], "max_budget": 50, "tpm": 200000, "team_id": "dev-wave"}'
```
返回的 `key` 即该团队专属 `LLM_GATEWAY_KEY`(配额尽自动限流;failover 在组内自动发生,团队无感)。

## agent 侧接线(一次性,之后全自动)

运行 agent swarm 的机器只设两个 env(secret manager / 部署脚本注入,不落仓库):

```
LLM_GATEWAY_ENDPOINT=http://<gateway机器>:4000
LLM_GATEWAY_KEY=sk-<该团队的 virtual key>
```

之后任何 swarm 启动即自动接入;换模型/加节点/调配额只改本目录 config.yaml(走 PR),全部 agent 立即生效,声明零改动。

## 上游运行时(openjiuwen)

不 fork、不用 submodule(ADR-0002 rev1):上游官方镜像 `openJiuwen-ai/jiuwenswarm`,
部署渲染器执行时 `git clone --depth 1 -b <pinned-tag>` 锁定版本;pin 关系随渲染器配置版本化。
A2X 注册中心(分布式 Team 控制面)同源上游 `openJiuwen-ai/agent-protocol`。
47 changes: 47 additions & 0 deletions deploy/llm-gateway/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# LLM Gateway 配置(LiteLLM proxy)
# ADR-0002 rev1:与 registry/models.yaml 同仓;validate.py 强制别名对齐。
# 规则:model_name = 对外别名(与 models.yaml alias 完全一致);
# 同一 model_name 写多条 = 路由组(自动 failover / 用量轮转)。
model_list:
# ── flash-pool: coder-fast ──────────────────────────────
- model_name: coder-fast
litellm_params:
model: openai/<UPSTREAM_MODEL_A> # TODO 部署时替换为真实 provider/模型名
api_key: os.environ/PROVIDER_KEY_A
# 加第二节点即自动组成 failover 组(注释示例):
# - model_name: coder-fast
# litellm_params:
# model: deepseek/deepseek-chat
# api_key: os.environ/PROVIDER_KEY_B

# ── flagship-pool: coder-deep ───────────────────────────
- model_name: coder-deep
litellm_params:
model: openai/<UPSTREAM_MODEL_B> # TODO
api_key: os.environ/PROVIDER_KEY_A

# ── flagship-pool: reviewer ─────────────────────────────
- model_name: reviewer
litellm_params:
model: openai/<UPSTREAM_MODEL_C> # TODO
api_key: os.environ/PROVIDER_KEY_A

# ── embed-pool: embed-default ───────────────────────────
- model_name: embed-default
litellm_params:
model: openai/<EMBED_MODEL> # TODO
api_key: os.environ/PROVIDER_KEY_A

router_settings:
routing_strategy: usage-based-routing-v2 # 按用量路由:单节点配额尽/失败自动切组内下一节点
num_retries: 2
timeout: 600
allowed_fails: 3 # 连续失败 3 次拉黑该节点
cooldown_time: 60 # 拉黑 60s

litellm_settings:
drop_params: true
request_timeout: 600

general_settings:
master_key: os.environ/LITELLM_MASTER_KEY # 管理面 key(生成 per-team key 用);绝不发给 agent
14 changes: 14 additions & 0 deletions deploy/llm-gateway/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
services:
litellm:
image: ghcr.io/berriai/litellm:main-v1.74.9-patch-stable # pin 固定 tag,升级=改这里走 PR
ports:
- "4000:4000"
volumes:
- ./config.yaml:/app/config.yaml:ro
env_file: .env # 不进 git(.gitignore 已含 *.local/密钥模式,本文件手建)
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4000/health/liveliness"]
interval: 30s
timeout: 5s
retries: 3
5 changes: 3 additions & 2 deletions registry/models.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@
# 规则(GOVERNANCE AR-3):
# 1. agent 声明只引用 alias;alias → 实际 provider/节点的解析只发生在 LLM Gateway 侧
# 2. 一切 key 只存 gateway 的 secret store(部署时注入 env),任何仓库/声明/agent 配置不得出现明文
# 3. 用量按 per-team/per-agent 的 gateway key 计量;路由策略(failover/配额)为网关部署配置,落 openjiuwen 仓
# 3. 用量按 per-team/per-agent 的 gateway key 计量;路由策略(failover/配额)为网关部署配置,落本仓 deploy/llm-gateway
version: 1
gateway:
endpoint: env:LLM_GATEWAY_ENDPOINT # 如 http://llm-gateway.internal:4000
auth: env:LLM_GATEWAY_KEY
deployment_ref: "Cloudbird-Software/openjiuwen deploy/llm-gateway" # 部署与路由配置版本化位置
deployment_ref: "Cloudbird-Software/agent-registry deploy/llm-gateway" # 本仓(ADR-0002 rev1);validate 强制别名对齐
upstream_runtime: {repo: openJiuwen-ai/jiuwenswarm, policy: deploy-time-pin} # 上游官方镜像,不 fork 不 submodule
strategies: [failover, quota-routing] # 节点用量耗尽/不可达 → 自动切路由组内下一节点
models:
- alias: coder-fast
Expand Down
8 changes: 8 additions & 0 deletions scripts/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ def frontmatter(path: Path) -> dict:
models = (load_yaml(REG / "models.yaml") or {}).get("models", [])
model_aliases = {m.get("alias") for m in models}

# ---- gateway 配置对齐(ADR-0002 rev1):别名集合与 models.yaml 完全一致 ----
GW_CFG = ROOT / "deploy" / "llm-gateway" / "config.yaml"
if GW_CFG.exists():
gwc = load_yaml(GW_CFG) or {}
gw_aliases = {m.get("model_name") for m in gwc.get("model_list", []) or []}
if gw_aliases != model_aliases:
fail(f"deploy/llm-gateway/config.yaml 的别名 {sorted(gw_aliases)} 与 models.yaml {sorted(model_aliases)} 不一致(ADR-0002 rev1)")

OK = {"approved", "deprecated"} # deprecated 仍可被既有声明引用,但新引用报警
ACTIVE = {"approved", "active"}

Expand Down