Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
4a948ac
refactor(omni): migrate settings to design namespace (processing/deli…
LaZzyMan Aug 6, 2026
11feda7
feat(omni): surface bitRate/sampleRateHz/channels from ffprobe
LaZzyMan Aug 6, 2026
9c328a4
feat(omni): add media-policy tool protocol (execution origin, descrip…
LaZzyMan Aug 6, 2026
edd3517
feat(omni): add staging/quarantine storage areas and recovery sweeps
LaZzyMan Aug 6, 2026
9a75fcd
feat(omni): add fixed-policy when-condition DSL evaluator and validator
LaZzyMan Aug 6, 2026
7a66122
feat(omni): add three-modality degradation media-policy tools
LaZzyMan Aug 7, 2026
e2cc1a4
feat(omni): add degradation result cache keyed by policy fingerprint
LaZzyMan Aug 7, 2026
0bb99b7
feat(omni): add fixed-policy orchestrator and reorder the delivery pi…
LaZzyMan Aug 7, 2026
dc43481
feat(omni): normalize omni.processing config with system default poli…
LaZzyMan Aug 7, 2026
5c4b8cf
feat(omni): enforce policy budgets, transport-guard passes and quaran…
LaZzyMan Aug 7, 2026
b639f5b
fix(omni): declare the disclosure text output in degradation tool des…
LaZzyMan Aug 7, 2026
d25f3fc
fix(omni): harden policy pipeline per review (cache validation, permi…
LaZzyMan Aug 7, 2026
5cb041c
fix(omni): harden policy pipeline after self-review round
LaZzyMan Aug 7, 2026
14662dd
fix(omni): drop system-default fixedPolicies; guard is the only default
LaZzyMan Aug 8, 2026
eabf697
feat(omni): add four local stage B policy tools (keyframes/extract-au…
LaZzyMan Aug 8, 2026
f39e0b5
feat(omni): add omni_transcribe_audio with transcript delivery protocol
LaZzyMan Aug 8, 2026
00345b6
feat(omni): fill request/session condition namespaces and wire client…
LaZzyMan Aug 8, 2026
1747100
fix(omni): deliver every media deliverable of multi-output fixed poli…
LaZzyMan Aug 8, 2026
f5e5d4b
fix(core): detect mkv/avi/flac/aac media missing from mime/lite stand…
LaZzyMan Aug 9, 2026
8b9e331
fix(omni): harden policy pipeline per self-review correctness findings
LaZzyMan Aug 9, 2026
775ba8e
refactor(omni): consolidate media-policy tool plumbing
LaZzyMan Aug 9, 2026
317870e
refactor(omni): share the transcript materializer across delivery con…
LaZzyMan Aug 9, 2026
ae3d356
perf(omni): parallelize orchestrator validation and promotion
LaZzyMan Aug 9, 2026
bdcf4ae
fix(omni): add display-name translations for the media-policy tools
LaZzyMan Aug 9, 2026
7b9cbce
fix(omni): open the download .part fd before the pipeline starts
LaZzyMan Aug 9, 2026
7c73835
feat(omni): switch when-condition DSL to expression arrays
LaZzyMan Aug 10, 2026
cbef4a3
feat(omni): degrade delivered media and retry when the server rejects…
LaZzyMan Aug 10, 2026
97e3aec
feat(omni): inject progressive media understanding guidance into the …
LaZzyMan Aug 10, 2026
acc5c18
fix(omni): extract keyframes across the full video duration (bucketed…
LaZzyMan Aug 10, 2026
006928d
fix(omni): chunk long-audio transcription and collapse repetition deg…
LaZzyMan Aug 10, 2026
4f801d8
docs(omni): add media policy orchestration usage guide
LaZzyMan Aug 10, 2026
a913c10
fix(omni): key pure-transcript guard resolution on this pass's file d…
LaZzyMan Aug 11, 2026
04a9b21
fix(omni): render the primary disclosure on pure-transcript deliveries
LaZzyMan Aug 11, 2026
3f9f509
fix(omni): prune non-object entry values when loading a JSON cache file
LaZzyMan Aug 11, 2026
ae5ab29
fix(omni): harden degradation-cache hits against tampered entries
LaZzyMan Aug 11, 2026
803ade3
fix(omni): include the hours field when a rounded duration reaches 3600s
LaZzyMan Aug 11, 2026
cb03f21
fix(omni): cap chunked transcription at 512 segments (fail closed)
LaZzyMan Aug 11, 2026
f1339ce
fix(omni): disable the omni pipeline in bare mode
LaZzyMan Aug 11, 2026
7807e87
fix(omni): degrade media nested in functionResponse.parts on reactive…
LaZzyMan Aug 11, 2026
8aec3ff
fix(omni): detect animated images whose headers carry no frame count
LaZzyMan Aug 11, 2026
e04b7e9
docs(omni): align tunable descriptions with the implemented semantics
LaZzyMan Aug 11, 2026
07bac6e
fix(omni): keep a raw "__proto__" policy id as an ordinary own key
LaZzyMan Aug 11, 2026
deb0740
fix(omni): validate transportGuard.maxEstimatedTokens at startup
LaZzyMan Aug 11, 2026
989fa8a
fix(omni): raise quarantine retention/budget schema minimums to 1
LaZzyMan Aug 11, 2026
57ef473
docs(omni): correct the fixedPolicies merge/tombstone description
LaZzyMan Aug 11, 2026
9b2903e
fix(omni): expose io paths to the AUTO-mode permission classifier
LaZzyMan Aug 11, 2026
45f9d1c
fix(omni): reject unknown keys in policyTools entries (§13 #1)
LaZzyMan Aug 11, 2026
31bddf2
fix(omni): reject parameterSchema overrides that loosen native constr…
LaZzyMan Aug 11, 2026
7d637c1
fix(omni): clamp downsample-audio targets to the probed source
LaZzyMan Aug 11, 2026
e59046f
fix(omni): reject full-span no-op clips instead of re-encoding
LaZzyMan Aug 11, 2026
f54bd40
fix(omni): disclose partial bucket coverage in keyframe extraction
LaZzyMan Aug 11, 2026
bcdb004
fix(omni): only claim alpha loss for alpha-capable JPEG conversion so…
LaZzyMan Aug 11, 2026
e1ac704
fix(omni): build the downscale-video loss clause from actual drops
LaZzyMan Aug 11, 2026
aca7b21
fix(omni): stop calling a degraded delivery "full resolution" in the …
LaZzyMan Aug 11, 2026
ffff2f3
fix(omni): localize the fixed-only tool description suffix
LaZzyMan Aug 11, 2026
8f26dec
fix(omni): unify the downscale-video display name as 降采样视频
LaZzyMan Aug 11, 2026
c31fa2e
test(omni): match the actual .tmp suffix in cache-file cleanup assert…
LaZzyMan Aug 11, 2026
48623f7
test(omni): pin the repetition-collapse rep threshold independently o…
LaZzyMan Aug 11, 2026
ddd3843
fix(omni): cascade corrupt-object deletion into the degradation cache
LaZzyMan Aug 11, 2026
b75c3a0
refactor(omni): resolve role selectors via a role→output-spec map
LaZzyMan Aug 11, 2026
ba1d359
test(omni): pin the when-condition round-trip through normalization (D7)
LaZzyMan Aug 11, 2026
3814256
test(omni): pin the runtime.timeoutMs / staging-grace boundary (recov…
LaZzyMan Aug 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 43 additions & 70 deletions docs/design/2026-07-29-omni-multimodal-policy-orchestration.md
Original file line number Diff line number Diff line change
Expand Up @@ -502,20 +502,25 @@ priority 排序。

### 8.3 匹配条件

`when` 使用受限条件 DSL,不接受任意 JavaScript 或 JSONPath。条件由 `all`、`any`
和 comparison 递归组成;comparison 支持 `gt`、`gte`、`lt`、`lte`、`eq`,左右两侧
都可以是字段或字面量:
`when` 使用受限的表达式条件 DSL(Mapbox style-spec 风格的表达式数组),不接受
任意 JavaScript 或 JSONPath。表达式形如 `[operator, ...operands]`:comparison
支持 `>`、`>=`、`<`、`<=`、`==`、`!=`,恰好两个操作数,操作数是
`["field", "<namespace.field>"]` 字段引用或裸字面量;组合子 `["all", <expr>, ...]`、
`["any", <expr>, ...]`、`["!", <expr>]` 递归嵌套:

```ts
type ConditionOperand =
| { field: FixedPolicyField }
| { value: number | string | boolean };

interface ComparisonCondition {
left: ConditionOperand;
operator: 'gt' | 'gte' | 'lt' | 'lte' | 'eq';
right: ConditionOperand;
}
type ConditionOperand = ['field', FixedPolicyField] | number | string | boolean;

type ComparisonCondition = [
'>' | '>=' | '<' | '<=' | '==' | '!=',
ConditionOperand,
ConditionOperand,
];

type FixedPolicyCondition =
| ComparisonCondition
| ['all' | 'any', ...FixedPolicyCondition[]]
| ['!', FixedPolicyCondition];
```

可读字段分为三个自然命名空间:
Expand Down Expand Up @@ -547,20 +552,15 @@ availableContextTokens = max(

```jsonc
{
"when": {
"all": [
{
"left": { "field": "resource.estimatedTokenCount" },
"operator": "gt",
"right": { "field": "session.availableContextTokens" },
},
{
"left": { "field": "session.contextWindowTokens" },
"operator": "gte",
"right": { "value": 131072 },
},
"when": [
"all",
[
">",
["field", "resource.estimatedTokenCount"],
["field", "session.availableContextTokens"],
],
},
[">=", ["field", "session.contextWindowTokens"], 131072],
],
}
```

Expand Down Expand Up @@ -1204,27 +1204,14 @@ normalizer,并用配置 snapshot 测试防止两处示例漂移。
"mediaTypes": ["image"],
},
// 定义触发该 Fixed Policy 的 metadata 条件。
"when": {
"when": [
// 任意一个条件成立即可执行。
"any": [
{
// 检查图片宽度。
"left": { "field": "resource.width" },
// 使用大于比较。
"operator": "gt",
// 宽度超过 2000 像素时命中。
"right": { "value": 2000 },
},
{
// 检查图片高度。
"left": { "field": "resource.height" },
// 使用大于比较。
"operator": "gt",
// 高度超过 2000 像素时命中。
"right": { "value": 2000 },
},
],
},
"any",
// 图片宽度超过 2000 像素时命中。
[">", ["field", "resource.width"], 2000],
// 图片高度超过 2000 像素时命中。
[">", ["field", "resource.height"], 2000],
],
// 执行图片降采样的 Tool 名。
"tool": "omni_downsample_image",
// 固定调用时使用的降采样参数。
Expand Down Expand Up @@ -1264,33 +1251,19 @@ normalizer,并用配置 snapshot 测试防止两处示例漂移。
"mediaTypes": ["video"],
},
// 比较本轮全部媒体的预估 token 与当前 session 可用上下文。
"when": {
"when": [
// 所有条件均满足时才执行。
"all": [
{
// 左值是本轮待发送媒体的预估 token 总量。
"left": {
"field": "request.totalEstimatedMediaTokens",
},
// 当预估媒体 token 大于可用上下文时命中。
"operator": "gt",
// 可用上下文由窗口、已用 prompt 和预留输出 token 计算。
"right": {
"field": "session.availableContextTokens",
},
},
{
// 确认当前视频本身具有可用的 token 估算值。
"left": {
"field": "resource.estimatedTokenCount",
},
// 大于零表示该资源已完成有效估算。
"operator": "gt",
// 零是估算值有效性的下界。
"right": { "value": 0 },
},
"all",
// 本轮待发送媒体的预估 token 总量大于可用上下文时命中;
// 可用上下文由窗口、已用 prompt 和预留输出 token 计算。
[
">",
["field", "request.totalEstimatedMediaTokens"],
["field", "session.availableContextTokens"],
],
},
// 确认当前视频本身具有可用的 token 估算值(大于零表示有效)。
[">", ["field", "resource.estimatedTokenCount"], 0],
],
// 条件依赖值不可用时跳过,不误判为条件不成立。
"onConditionUnavailable": "skip",
// 执行视频关键帧提取的 Tool 名。
Expand Down
1 change: 1 addition & 0 deletions docs/users/features/_meta.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,5 @@ export default {
'status-line': 'Status Line',
'scheduled-tasks': 'Scheduled Tasks',
tips: 'Contextual Tips',
'omni-media-policies': 'Omni Media Policies',
};
Loading
Loading