Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion site/quartz/quartz.config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ configuration:
enableSPA: true
enablePopovers: true
analytics: null
locale: en-US
locale: zh-CN
baseUrl: www.justinchuby.com/onnx-genai
ignorePatterns:
- private
Expand Down
132 changes: 77 additions & 55 deletions wiki/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,75 +6,97 @@ tags:
- wiki
- index
status: maintained
lang: zh-CN
created: 2026-08-17
updated: 2026-08-19
---

# onnx-genai Wiki

This directory is an Obsidian-compatible knowledge base for explanatory notes,
learning paths, and links between implementation concepts. It does not replace
the specifications, measured evidence, or accepted designs in `docs/`.
本目录是一个兼容 Obsidian 的知识库,收录解释性笔记、学习路径,以及实现概念之间的
关联。它**不替代** `docs/` 下的规范、实测证据与已接受的设计文档。

Published readers start at [[index|onnx-genai Knowledge Base]].
发布后的读者请从 [[index|onnx-genai Knowledge Base]] 开始。

> [!important] Source precedence
> When a wiki note disagrees with formal documentation or code, use this order:
> 1. Current code and reproducible measurements
> 2. Authoritative documents under `docs/`
> 3. Accepted design decisions
> 4. Explanatory wiki notes
> [!important] 来源优先级
> 当 wiki 笔记与正式文档或代码不一致时,按以下顺序采信:
> 1. 当前代码与可复现的实测数据
> 2. `docs/` 下的权威文档
> 3. 已接受的设计决策
> 4. 解释性的 wiki 笔记

## Maps of content
## 内容地图

- **Start here:** [[start/Repository Map]]
- **Architecture:** [[architecture/Crate Architecture]]
- **Runtime flow:** [[architecture/Inference Request Lifecycle]]
- **Execution:** [[execution/Execution Backends]]
- **EP contract:** [[execution/Execution Provider Contract]]
- **从这里开始:** [[start/Repository Map]]
- **架构:** [[architecture/Crate Architecture]]
- **运行时流程:** [[architecture/Inference Request Lifecycle]]
- **执行:** [[execution/Execution Backends]]
- **EP 契约:** [[execution/Execution Provider Contract]]
- **CPU EP:** [[execution/CPU Execution Provider]]
- **CUDA EP:** [[execution/CUDA Execution Provider]]
- **Plugin EPs:** [[execution/Plugin Execution Providers]]
- **Memory:** [[memory/Memory Management for Beginners]]
- **KV cache virtual memory:** [[memory/Virtual Memory for KV Cache]]
- **MoE router skew:** [[memory/MoE Router Skew and Always-On Experts]]
- **Chat templates:** [[prompting/Chat Templates]]
- **Tracing:** [[observability/Tracing and Profiling]]
- **Performance engineering:** [[performance/Performance Engineering Playbook]]
- **Chunked prefill:** [[performance/Chunked Prefill]]
- **API design:** [[api/API Design Principles]]
- **Contracts:** [[contracts/Runtime Contracts]]
- **Formal verification:** [[contracts/Formal Verification with TLA+]]
- **Testing and verification:** [[development/Testing and Verification]]
- **Metadata:** [[metadata/Metadata Driven Runtime]]
- **Model packages:** [[metadata/Model Packages and Variants]]
- **Documentation:** [[start/Documentation Guide]]
- **Wiki maintenance:** [[meta/Using this Wiki]]
- **插件 EP:** [[execution/Plugin Execution Providers]]
- **内存:** [[memory/Memory Management for Beginners]]
- **KV cache 虚拟内存:** [[memory/Virtual Memory for KV Cache]]
- **MoE 路由倾斜:** [[memory/MoE Router Skew and Always-On Experts]]
- **对话模板:** [[prompting/Chat Templates]]
- **追踪:** [[observability/Tracing and Profiling]]
- **性能工程:** [[performance/Performance Engineering Playbook]]
- **分块 prefill:** [[performance/Chunked Prefill]]
- **API 设计:** [[api/API Design Principles]]
- **契约:** [[contracts/Runtime Contracts]]
- **形式化验证:** [[contracts/Formal Verification with TLA+]]
- **测试与验证:** [[development/Testing and Verification]]
- **元数据:** [[metadata/Metadata Driven Runtime]]
- **模型包:** [[metadata/Model Packages and Variants]]
- **文档:** [[start/Documentation Guide]]
- **Wiki 维护:** [[meta/Using this Wiki]]

## Note conventions
## 笔记写作规范

Every note should:
每篇笔记都应当:

1. Use an English filename and `title` so links remain stable across languages.
2. Include YAML frontmatter with `title`, `aliases`, `tags`, `status`,
`created`, and `updated`.
3. Begin with a short statement of the question the note answers.
4. Answer one primary question and usually remain readable in roughly 5–10 minutes.
5. Keep a longer tutorial when shortening it would force a beginner to chase
prerequisite explanations across other files.
6. Use `[[wikilinks]]` instead of duplicating explanations across notes.
7. Use Obsidian callouts for invariants, warnings, examples, and context.
8. Make the note self-contained for its intended reader. Links to `docs/` and code
are evidence and implementation detail, not required homework.
9. Clearly label proposed behavior; never present a target design as implemented.
1. 使用**英文文件名**和**英文 `title`**,使链接在跨语言时保持稳定。
2. 包含 YAML frontmatter,字段为 `title`、`aliases`、`tags`、`status`、`lang`、
`created`、`updated`。
3. 开篇先用一句话说明这篇笔记回答的是什么问题。
4. 只回答一个主要问题,通常保持在 5–10 分钟可读完的篇幅。
5. 当缩短会迫使初学者跑到其他文件里去补前置知识时,保留较长的教程体例。
6. 用 `[[wikilinks]]` 链接到其他笔记,而不是在多篇之间重复同一段解释。
7. 用 Obsidian callout 标注不变量、警告、示例与背景。
8. 对目标读者做到自包含。指向 `docs/` 和代码的链接是**证据和实现细节**,不是必须
先读完的功课。
9. 明确标注"提议中"的行为,**绝不**把目标设计写成已实现的样子。

Notes may be written in the language that best serves their audience. English
titles and paths are required even when the body is written in another language.
## 语言

> [!note] Creation and modification dates
> Obsidian can display the version-controlled `created` and `updated` properties
> in the Properties view. Obsidian also knows local filesystem creation and
> modification times, but those are not durable across clones, checkouts, and
> rebases. Core Obsidian does not automatically maintain custom `updated`
> frontmatter on every edit; update it with the note, or use a configured
> automation/plugin. See [[meta/Using this Wiki]].
> [!important] Wiki 正文用中文写,标题保留英文
> 本 wiki 的**正文使用简体中文**。以下内容**保持英文**:
>
> - **文件名与目录路径** —— 例如 `execution/CUDA Execution Provider.md`
> - **frontmatter 的 `title` 字段**
> - **页面的一级标题(H1)** —— 与 `title` 保持一致
> - **`tags`** —— 标签始终用英文
> - **`[[wikilinks]]` 的链接目标** —— 需要中文显示文本时用 `[[目标|显示文本]]`,
> 绝不改动竖线左边的目标本身
> - **代码、标识符、crate 名、文件路径、环境变量、函数名、命令行**
> - **Obsidian callout 的类型关键字** —— 如 `> [!important]`,这是语法
>
> 章节标题(H2 及以下)、正文、表格内容、callout 的标题文字都应译为中文。
>
> `aliases` 可以同时包含中英文条目 —— 它的作用是让人用任一语言都能搜到这篇笔记,
> 因此中文别名是鼓励的,但**不要**删除既有的英文别名,那会破坏已有链接。
>
> 技术术语首次出现时保留英文原词并在括号中给出中文,例如
> "execution provider(执行提供者,EP)";此后可只用英文缩写。当英文术语本身就是
> 业界通用称呼时(如 kernel、arena、allocator),直接沿用英文,不必强译。

`lang` 字段会被 Quartz 直接用作发布页面的 `<html lang>` 属性
(`renderPage.tsx`),因此它影响的是真实的可访问性与搜索引擎行为,不只是元数据。
中文笔记写 `lang: zh-CN`。

> [!note] 创建与修改日期
> Obsidian 可以在 Properties 视图里显示受版本控制的 `created` 与 `updated` 属性。
> Obsidian 同样知道本地文件系统的创建与修改时间,但那些时间在 clone、checkout 和
> rebase 之后并不可靠。Obsidian 核心功能**不会**在每次编辑时自动维护自定义的
> `updated` frontmatter;请随笔记一起更新它,或配置自动化/插件来处理。参见
> [[meta/Using this Wiki]]。
123 changes: 61 additions & 62 deletions wiki/api/API Design Principles.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,103 +9,102 @@ tags:
- ffi
- compatibility
status: maintained
lang: zh-CN
created: 2026-08-17
updated: 2026-08-17
updated: 2026-08-19
---

# API Design Principles

> [!summary] Question answered
> How should one behavior remain coherent across Rust, CLI, HTTP, Python and C without forcing every surface to expose identical mechanics?
> [!summary] 本篇回答的问题
> 一种行为如何在 Rust、CLI、HTTP、Python 与 C 之间保持一致,而不强迫每个表面都
> 暴露完全相同的机制?

## Layered surfaces
## 分层表面

| Surface | Primary audience | Design emphasis |
| 表面 | 主要受众 | 设计侧重 |
|---|---|---|
| Rust facade | Rust applications | Typed ownership and composability |
| Engine API | Advanced integrations | Full generation policy/control |
| CLI/REPL | Humans and scripts | Discoverability, stable stdout, actionable errors |
| HTTP server | OpenAI-compatible clients | Wire compatibility, streaming, sessions |
| Python | Python/NumPy users | Familiar session API, DLPack where explicit |
| C ABI | Cross-language hosts | Opaque handles, value/vtable ABI, explicit ownership |
| Rust facade | Rust 应用 | 带类型的所有权与可组合性 |
| Engine API | 高级集成方 | 完整的生成策略/控制 |
| CLI/REPL | 人类与脚本 | 可发现性、稳定的 stdout、可操作的错误 |
| HTTP server | 兼容 OpenAI 的客户端 | wire 兼容、streaming、会话 |
| Python | Python/NumPy 用户 | 熟悉的 session API,显式处的 DLPack |
| C ABI | 跨语言 host | 不透明 handle、value/vtable ABI、显式所有权 |

The surfaces share semantics, not necessarily identical signatures.
这些表面共享的是语义,而不一定是完全相同的签名。

## One semantic core
## 单一语义内核

Prompt processing, generation options, finish reasons, stop behavior and sampling
should converge on engine types. Bindings marshal into that core instead of
reimplementing generation policy.
prompt 处理、生成选项、finish reason、stop 行为与采样都应收敛到引擎的类型上。
binding 把数据 marshal 进这个内核,而不是重新实现生成策略。

For example, a foreign sampler replaces terminal token selection while the engine
still applies the configured processor/constraint chain.
例如,一个外部 sampler 替换的是终端 token 选择,而引擎仍然施加所配置的
processor/constraint 链。

## Actionable failure
## 可操作的失败

Every surface should preserve:
每个表面都应保留:

- what operation failed;
- which argument/node/shape/dtype/path/device was involved;
- why it was rejected;
- how the caller can fix it;
- the underlying cause where safe.
- 是哪个操作失败了;
- 涉及了哪个 argument/node/shape/dtype/path/device;
- 为什么被拒绝;
- 调用方如何修复;
- 在安全的前提下给出底层原因。

Rust uses typed crate errors; orchestration may add `anyhow::Context`; C uses
machine-readable status plus a rich message; Python/HTTP map without erasing
diagnostic detail.
Rust 使用带类型的 crate 错误;编排层可以加上 `anyhow::Context`;C 使用机器可读的
状态码加上一条丰富的消息;Python/HTTP 在映射时不擦除诊断细节。

## Ownership by language
## 按语言划分的所有权

### Rust

Use ownership, borrowing and newtypes to make invalid states difficult to express.
Fallible device/runtime work returns `Result`.
利用所有权、borrowing 与 newtype 让非法状态难以表达。可失败的设备/运行时工作
返回 `Result`。

### C

- opaque handles have matching create/release functions;
- null is checked before dereference;
- every panic is caught at the ABI boundary;
- status/message ownership is explicit;
- caller and library never free each other's unspecified heap allocations.
- 不透明 handle 有配对的 create/release 函数;
- 在解引用之前检查 null;
- 每个 panic 都在 ABI 边界处被捕获;
- 状态/消息的所有权是显式的;
- 调用方与库绝不释放对方未指明的 heap 分配。

### Python

- copied NumPy output is the safe default;
- DLPack/zero-copy paths are explicit;
- thread/reentrancy behavior is stated;
- iterator/callback completion semantics must distinguish token events from final
result state.
- 拷贝出的 NumPy 输出是安全的默认;
- DLPack/zero-copy 路径是显式的;
- thread/可重入行为被明确说明;
- iterator/callback 的完成语义必须区分 token event 与最终结果状态。

### HTTP

- request/response shapes follow the intended OpenAI-compatible contract;
- SSE streaming preserves completion/error semantics;
- debug/admin endpoints are opt-in;
- payloads and secrets are not logged by default.
- request/response 的形状遵循目标中兼容 OpenAI 的契约;
- SSE streaming 保持 completion/error 语义;
- debug/admin 端点是 opt-in 的;
- 默认不记录 payload 与 secret。

## Compatibility policy
## 兼容性策略

The repository is pre-release for its own Rust/product APIs: reshape them cleanly
and update all callers instead of accumulating aliases and deprecations.
对仓库自有的 Rust/产品 API 而言,它处于 pre-release 阶段:干净地重塑它们并更新
所有调用方,而不是不断堆积别名与弃用。

That freedom does not apply to:
这种自由不适用于:

- ONNX semantics/opsets;
- documented model metadata;
- stable C/plugin ABI versions;
- OpenAI-compatible wire behavior promised to users;
- supported Python wheel ABI.
- ONNX 语义/opset;
- 已文档化的模型 metadata;
- 稳定的 C/plugin ABI 版本;
- 向用户承诺的兼容 OpenAI 的 wire 行为;
- 受支持的 Python wheel ABI。

## Explicit behavior beats hidden convenience
## 显式行为胜过隐藏的便利

- no silent CPU fallback unless explicitly enabled;
- no implicit cross-device transfer in eager APIs;
- capability negotiation before using optional behavior;
- no model-name dispatch where metadata/graph structure is required;
- report a constrained fallback instead of pretending the requested mode ran.
- 除非显式启用,否则不做静默的 CPU 回退;
- eager API 中不做隐式的跨设备传输;
- 在使用可选行为之前先做 capability 协商;
- 在需要 metadata/图结构之处,不按模型名做 dispatch;
- 报告一个受约束的回退,而不是假装请求的模式已经运行。

## Formal sources
## 正式来源

- [`RULES.md`](../../RULES.md)
- [`onnx-genai` facade](../../crates/onnx-genai/src/lib.rs)
Expand All @@ -115,7 +114,7 @@ That freedom does not apply to:
- [`onnx-genai-capi`](../../crates/onnx-genai-capi/src/lib.rs)
- [`ERROR_AND_LOGGING_CONVENTIONS.md`](../../docs/architecture/ERROR_AND_LOGGING_CONVENTIONS.md)

## Related notes
## 相关笔记

- [[contracts/Runtime Contracts]]
- [[metadata/Metadata Driven Runtime]]
Expand Down
Loading
Loading