(MOT-4080) feat(provider-kimi): add Moonshot provider - #522
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
skill-check — worker0 verified, 45 skipped (no docs/).
Four for four. Nicely done. |
📝 WalkthroughWalkthroughAdds a complete Rust ChangesKimi provider worker
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Router
participant KimiWorker
participant Moonshot
participant ModelCatalog
Router->>KimiWorker: resolve provider and invoke stream
KimiWorker->>Moonshot: POST /v1/chat/completions
Moonshot-->>KimiWorker: SSE reasoning, text, tools, usage
KimiWorker-->>Router: framed assistant events
Router->>KimiWorker: refresh_models
KimiWorker->>Moonshot: GET /v1/models
Moonshot-->>KimiWorker: live model list
KimiWorker->>ModelCatalog: filter and enrich models
KimiWorker-->>Router: reconciled catalog
**Poem**
> A rabbit hops through Kimi’s stream,
> With thinking sparks and JSON gleam.
> Models bloom in router rows,
> Tokens rest where state-flow goes.
> “Done!” the bunny thumps with cheer—
> Moonshot’s worker now is here!
</details>
<!-- walkthrough_end -->
<!-- pre_merge_checks_walkthrough_start -->
<details>
<summary>🚥 Pre-merge checks | ✅ 5</summary>
<details>
<summary>✅ Passed checks (5 passed)</summary>
| Check name | Status | Explanation |
| :------------------------: | :------- | :-------------------------------------------------------------------------------------------------- |
| Description Check | ✅ Passed | Check skipped - CodeRabbit’s high-level summary is enabled. |
| Title check | ✅ Passed | The title clearly and concisely summarizes the addition of the new provider-kimi Moonshot provider. |
| Docstring Coverage | ✅ Passed | Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. |
| Linked Issues check | ✅ Passed | Check skipped because no linked issues were found for this pull request. |
| Out of Scope Changes check | ✅ Passed | Check skipped because no linked issues were found for this pull request. |
</details>
</details>
<!-- pre_merge_checks_walkthrough_end -->
<!-- finishing_touch_checkbox_start -->
<details>
<summary>✨ Finishing Touches</summary>
<details>
<summary>📝 Generate docstrings</summary>
- [ ] <!-- {"checkboxId": "7962f53c-55bc-4827-bfbf-6a18da830691"} --> Create stacked PR
- [ ] <!-- {"checkboxId": "3e1879ae-f29b-4d0d-8e06-d12b7ba33d98"} --> Commit on current branch
</details>
<details>
<summary>🧪 Generate unit tests (beta)</summary>
- [ ] <!-- {"checkboxId": "f47ac10b-58cc-4372-a567-0e02b2c3d479", "radioGroupId": "utg-output-choice-group-unknown_comment_id"} --> Create PR with unit tests
- [ ] <!-- {"checkboxId": "6ba7b810-9dad-11d1-80b4-00c04fd430c8", "radioGroupId": "utg-output-choice-group-unknown_comment_id"} --> Commit unit tests in branch `feat/provider-kimi`
</details>
</details>
<!-- finishing_touch_checkbox_end -->
<!-- tips_start -->
---
Thanks for using [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=iii-hq/workers&utm_content=522)! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
<details>
<summary>❤️ Share</summary>
- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
</details>
<sub>Comment `@coderabbitai help` to get the list of available commands.</sub>
<!-- tips_end -->
|
andersonleal
approved these changes
Jul 16, 2026
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@provider-kimi/src/curated.rs`:
- Around line 16-18: Update the Kimi catalog entries in
provider-kimi/src/curated.rs with verified context_window, max_output_tokens,
and pricing values from platform.moonshot.ai before release. Remove the
placeholder disclaimer once the rows reflect authoritative platform data,
preserving the existing catalog structure and downstream field usage.
In `@provider-kimi/src/discovery.rs`:
- Around line 19-23: Update models_url to return an error when api_url does not
end with "/chat/completions" instead of falling back to the hardcoded Moonshot
host. Propagate the resulting error through the model-discovery flow and its
callers, including the code around the additional referenced range, while
preserving derived same-host URLs for recognized endpoints.
- Around line 38-53: Update parse_live_models to return a parse error when the
response lacks an array-valued data field or is otherwise malformed, instead of
producing an empty catalog. Propagate that error through refresh_models and
classify it as transient so the existing catalog remains unchanged when a
malformed 200 response is received.
In `@provider-kimi/src/register.rs`:
- Around line 137-140: Coordinate the boot registration path and the
RegisterFunction router-ready callback through a single shared
declare_and_refresh task, mutex, or cancellation handle. Update the surrounding
registration state so repeated RouterReadyEvent notifications and boot
initialization reuse or replace the existing loop instead of spawning
independent declare_with_backoff tasks and duplicate refreshes.
In `@provider-kimi/src/request.rs`:
- Around line 23-50: Update build_response_format and its use from build_body to
preserve a requested json_schema when the target Kimi model supports it, rather
than always returning json_object. For unsupported models, retain the existing
degradation warning behavior and add the requested schema guidance to the prompt
while continuing to request json_object; use the available model and
message-building symbols to determine support without changing unrelated request
fields.
In `@provider-kimi/src/sse.rs`:
- Around line 281-315: The tool-call SSE handling around the `OpenBlock::Call`
state currently tracks only one active index, causing interleaved calls to emit
duplicate lifecycle events. Update the `tool_calls` processing and related
lifecycle state so each call index is started and ended exactly once while still
accumulating its `id`, function name, and arguments; add a regression test
covering interleaved chunks for two indices.
In `@provider-kimi/src/state.rs`:
- Around line 12-22: Update load_token to return Result<Option<String>, Error>,
preserving None only when state::get succeeds without a string value and
propagating trigger failures. Update every caller involved in registration,
resolve, and reconcile flows to explicitly handle the Result error rather than
treating failures as an absent token.
In `@provider-kimi/src/stream_fn.rs`:
- Around line 86-90: Update the stream handling around the thinking_level
comment in the Kimi stream flow to emit a report-and-continue warning whenever
thinking_level is provided but cannot be mapped to Moonshot’s implicit thinking
behavior. Reuse the existing warning/report propagation mechanism used for the
response-format warning so the warning reaches the final message without
interrupting streaming.
In `@provider-kimi/src/upstream.rs`:
- Around line 139-144: Update the EOF handling in the upstream stream flow so a
connection closing before receiving the `[DONE]` marker is treated as incomplete
rather than emitting `AssistantMessageEvent::Done`. Preserve the accumulated
partial state and return or propagate the established transient stream error
path, while keeping `Done` emission exclusively for an explicit `[DONE]` event.
- Around line 31-37: Update the SSE parsing flow around data_line to buffer raw
response bytes and frame complete events using CRLF or LF delimiters before
decoding once with strict UTF-8 handling, avoiding from_utf8_lossy on split
multibyte sequences. Preserve all data: fields in each block by concatenating
them in order, rather than returning only the last payload.
In `@provider-kimi/tests/integration.rs`:
- Line 385: Require the success-stream pump at
provider-kimi/tests/integration.rs:385-385 to complete without timeout or join
failure before inspecting frames, and apply the same handling to the
error-stream pump at provider-kimi/tests/integration.rs:425-425. Replace the
discarded timeout results with an assertion or equivalent propagation that fails
the test unless each pump finishes successfully.
- Around line 335-346: Update the registration-token assertion after boot_stack
to poll state::get for the provider-kimi registration_token until it becomes
non-empty or the existing catalog-assertion deadline is reached. Preserve the
current failure message and final token validation, using the same polling
pattern and timing symbols already used by the catalog assertions.
- Around line 213-221: Update the request-reading logic inside the tokio::spawn
handler to accumulate data until the HTTP request line is complete, stopping
once a newline or full header terminator is received. Select STUB_MODELS only
after reliably identifying the GET /v1/models request, while preserving
messages_response for other requests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2244b812-c0f8-44c0-b0ae-3f82a19b6a0e
⛔ Files ignored due to path filters (1)
provider-kimi/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (32)
provider-kimi/Cargo.tomlprovider-kimi/README.mdprovider-kimi/build.rsprovider-kimi/config.yamlprovider-kimi/iii-permissions.yamlprovider-kimi/iii.worker.yamlprovider-kimi/prompts/identity.txtprovider-kimi/src/config.rsprovider-kimi/src/curated.rsprovider-kimi/src/discovery.rsprovider-kimi/src/errors.rsprovider-kimi/src/lib.rsprovider-kimi/src/main.rsprovider-kimi/src/manifest.rsprovider-kimi/src/register.rsprovider-kimi/src/request.rsprovider-kimi/src/router_client.rsprovider-kimi/src/sse.rsprovider-kimi/src/state.rsprovider-kimi/src/stream_fn.rsprovider-kimi/src/surface.rsprovider-kimi/src/upstream.rsprovider-kimi/src/wire/messages.rsprovider-kimi/src/wire/mod.rsprovider-kimi/src/wire/names.rsprovider-kimi/src/wire/tools.rsprovider-kimi/tests/golden/schemas/provider.kimi.on_router_ready.jsonprovider-kimi/tests/golden/schemas/provider.kimi.refresh_models.jsonprovider-kimi/tests/golden/schemas/provider.kimi.stream.jsonprovider-kimi/tests/integration.rsprovider-kimi/tests/schemas.rsprovider-kimi/tests/support/mod.rs
Comment on lines
+16
to
+18
| //! NOTE: context windows and pricing below are best-effort PLACEHOLDERS — | ||
| //! verify against platform.moonshot.ai before any release. A wrong row only | ||
| //! degrades display polish and cost enrichment, never routing. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== provider-kimi/src/curated.rs ==\n'
cat -n provider-kimi/src/curated.rs | sed -n '1,220p'
printf '\n== search for context_window/max_output_tokens/price usage in provider-kimi ==\n'
rg -n "context_window|max_output_tokens|price\(" provider-kimi -g '!**/target/**'
printf '\n== locate family_meta / pretty_name references ==\n'
rg -n "family_meta|pretty_name|curated" provider-kimi -g '!**/target/**'Repository: iii-hq/workers
Length of output: 14883
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== provider-kimi/src/config.rs ==\n'
cat -n provider-kimi/src/config.rs | sed -n '1,240p'
printf '\n== provider-kimi/src/stream_fn.rs ==\n'
cat -n provider-kimi/src/stream_fn.rs | sed -n '1,220p'
printf '\n== provider-kimi/src/discovery.rs ==\n'
cat -n provider-kimi/src/discovery.rs | sed -n '1,220p'
printf '\n== search for config_from_resolve and any context-window enforcement ==\n'
rg -n "config_from_resolve|context_window|max_output_tokens|input_limit|truncat|budget|reserve" provider-kimi -g '!**/target/**'Repository: iii-hq/workers
Length of output: 26590
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== repo-wide context_window / max_output_tokens usage ==\n'
rg -n "\.context_window\b|context_window:|\.max_output_tokens\b|max_output_tokens:" . -g '!**/target/**'
printf '\n== repo-wide Model field construction / access around curated metadata ==\n'
rg -n "supports_structured_output|thinking_budgets|supports_cache|supports_tools|supports_vision" . -g '!**/target/**'Repository: iii-hq/workers
Length of output: 41364
Update the Kimi placeholder limits and pricing before release
These rows feed the catalog and downstream budget calculations, not just the UI: context_window is used in context sizing, and max_output_tokens drives the effective output budget. Leaving placeholder values here risks wrong truncation/budgeting and misleading cost estimates.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/curated.rs` around lines 16 - 18, Update the Kimi catalog
entries in provider-kimi/src/curated.rs with verified context_window,
max_output_tokens, and pricing values from platform.moonshot.ai before release.
Remove the placeholder disclaimer once the rows reflect authoritative platform
data, preserving the existing catalog structure and downstream field usage.
Comment on lines
+19
to
+23
| pub fn models_url(api_url: &str) -> String { | ||
| match api_url.strip_suffix("/chat/completions") { | ||
| Some(base) => format!("{base}/models"), | ||
| None => "https://api.moonshot.ai/v1/models".to_string(), | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Never fall back to a different host for model discovery.
An unrecognized custom api_url causes discovery to query the public Moonshot endpoint and send the resolved bearer credential there. This ignores the configured upstream and may disclose proxy-specific credentials. Return an error when the models endpoint cannot be safely derived instead of changing hosts.
Also applies to: 149-164
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/discovery.rs` around lines 19 - 23, Update models_url to
return an error when api_url does not end with "/chat/completions" instead of
falling back to the hardcoded Moonshot host. Propagate the resulting error
through the model-discovery flow and its callers, including the code around the
additional referenced range, while preserving derived same-host URLs for
recognized endpoints.
Comment on lines
+38
to
+53
| pub fn parse_live_models(json: &Value) -> Vec<Model> { | ||
| let ids: Vec<String> = json | ||
| .get("data") | ||
| .and_then(Value::as_array) | ||
| .map(|rows| { | ||
| rows.iter() | ||
| .filter_map(|raw| { | ||
| let id = raw | ||
| .get("id") | ||
| .and_then(Value::as_str) | ||
| .filter(|s| !s.is_empty())?; | ||
| is_chat_model(id).then(|| id.to_string()) | ||
| }) | ||
| .collect() | ||
| }) | ||
| .unwrap_or_default(); |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not reconcile malformed responses as an empty catalog.
Missing or non-array data becomes Ok(vec![]); refresh_models then replaces the existing catalog with an empty slice. Return a parse error and classify it as transient so malformed 200 responses preserve the previous catalog.
Also applies to: 93-95, 193-197
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/discovery.rs` around lines 38 - 53, Update
parse_live_models to return a parse error when the response lacks an
array-valued data field or is otherwise malformed, instead of producing an empty
catalog. Propagate that error through refresh_models and classify it as
transient so the existing catalog remains unchanged when a malformed 200
response is received.
Comment on lines
+137
to
+140
| RegisterFunction::new_async(move |_event: RouterReadyEvent| { | ||
| let (iii, http) = (iii_ready.clone(), http_ready.clone()); | ||
| async move { | ||
| tokio::spawn(declare_and_refresh(iii, http)); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
Single-flight registration across boot and router-ready events.
Both paths spawn independent declare_with_backoff loops. During router outages or token mismatches, repeated ready notifications can accumulate permanent tasks and duplicate declarations/model refreshes. Coordinate these paths through one shared task, mutex, or cancellation handle.
Also applies to: 154-155
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/register.rs` around lines 137 - 140, Coordinate the boot
registration path and the RegisterFunction router-ready callback through a
single shared declare_and_refresh task, mutex, or cancellation handle. Update
the surrounding registration state so repeated RouterReadyEvent notifications
and boot initialization reuse or replace the existing loop instead of spawning
independent declare_with_backoff tasks and duplicate refreshes.
Comment on lines
+23
to
+50
| /// Moonshot supports only JSON mode (`{"type":"json_object"}`) — there is no | ||
| /// strict `json_schema` constrained-decoding mode like OpenAI's. A requested | ||
| /// schema is therefore advisory: we still ask for json_object (and stream_fn | ||
| /// emits a report-and-continue warning so the caller knows the schema was not | ||
| /// enforced). json_object mode requires the word "JSON" somewhere in the | ||
| /// messages — the caller's contract per spec § Model capabilities. | ||
| pub fn build_response_format(_rf: &ResponseFormat) -> Value { | ||
| json!({ "type": "json_object" }) | ||
| } | ||
|
|
||
| /// The classic `max_tokens` param (Moonshot accepts it directly). No | ||
| /// `reasoning_effort`: Moonshot has no such knob. No `temperature`: the API | ||
| /// default applies. | ||
| pub fn build_body(args: &BodyArgs) -> Value { | ||
| let mut body = json!({ | ||
| "model": args.model, | ||
| "max_tokens": args.max_tokens, | ||
| "messages": to_wire_messages(&args.messages, &args.system_prompt), | ||
| "stream": true, | ||
| "stream_options": { "include_usage": true }, | ||
| }); | ||
| let wire_tools = functions_to_wire(&args.tools); | ||
| if !wire_tools.is_empty() { | ||
| body["tools"] = Value::Array(wire_tools); | ||
| } | ||
| if let Some(rf) = &args.response_format { | ||
| body["response_format"] = build_response_format(rf); | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Preserve the requested schema instead of always downgrading to json_object.
This path drops the schema entirely. If the target Kimi model supports json_schema, pass it through; otherwise keep the degradation warning and inject the schema guidance into the prompt.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/request.rs` around lines 23 - 50, Update
build_response_format and its use from build_body to preserve a requested
json_schema when the target Kimi model supports it, rather than always returning
json_object. For unsupported models, retain the existing degradation warning
behavior and add the requested schema guidance to the prompt while continuing to
request json_object; use the available model and message-building symbols to
determine support without changing unrelated request fields.
Comment on lines
+31
to
+37
| /// Last `data: ` payload in an SSE block, if any. | ||
| fn data_line(block: &str) -> Option<&str> { | ||
| block | ||
| .lines() | ||
| .filter_map(|l| l.strip_prefix("data: ")) | ||
| .next_back() | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the file and inspect the relevant sections
ast-grep outline provider-kimi/src/upstream.rs --view expanded || true
echo '--- file size ---'
wc -l provider-kimi/src/upstream.rs
echo '--- lines 1-220 ---'
cat -n provider-kimi/src/upstream.rs | sed -n '1,220p'Repository: iii-hq/workers
Length of output: 10450
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ast-grep outline provider-kimi/src/upstream.rs --view expanded || true
echo '--- file size ---'
wc -l provider-kimi/src/upstream.rs
echo '--- lines 1-220 ---'
cat -n provider-kimi/src/upstream.rs | sed -n '1,220p'Repository: iii-hq/workers
Length of output: 10450
🏁 Script executed:
#!/bin/bash
set -euo pipefail
cat -n provider-kimi/src/upstream.rs | sed -n '1,220p'Repository: iii-hq/workers
Length of output: 9532
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "from_utf8_lossy|lines\\(|split\\(|data:" provider-kimi/src/upstream.rsRepository: iii-hq/workers
Length of output: 1009
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- lines 220-285 ---'
cat -n provider-kimi/src/upstream.rs | sed -n '220,285p'
echo '--- search SSE helpers ---'
rg -n "handle_chunk|build_final|build_partial|PartialState|synthetic_error_event" -S .Repository: iii-hq/workers
Length of output: 35418
Use a byte-safe SSE parser. from_utf8_lossy can mangle split multibyte characters, and buf.find("\n\n") only recognizes LF-delimited events. Buffer raw bytes until a complete SSE block is framed, then decode once and concatenate all data: lines; data_line() currently drops everything except the last payload.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/upstream.rs` around lines 31 - 37, Update the SSE parsing
flow around data_line to buffer raw response bytes and frame complete events
using CRLF or LF delimiters before decoding once with strict UTF-8 handling,
avoiding from_utf8_lossy on split multibyte sequences. Preserve all data: fields
in each block by concatenating them in order, rather than returning only the
last payload.
Comment on lines
+139
to
+144
| // Stream ended without [DONE] (connection close framing): still terminal. | ||
| let _ = tx | ||
| .send(AssistantMessageEvent::Done { | ||
| message: build_final(&state, &args.model), | ||
| }) | ||
| .await; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
sed -n '1,260p' provider-kimi/src/upstream.rs && printf '\n---\n' && rg -n "\[DONE\]|finish_reason|AssistantMessageEvent::Done|transient|EOF|connection close" provider-kimi -SRepository: iii-hq/workers
Length of output: 13310
🏁 Script executed:
sed -n '252,310p' provider-kimi/src/upstream.rs && printf '\n---\n' && sed -n '1,220p' provider-kimi/src/sse.rsRepository: iii-hq/workers
Length of output: 8636
🌐 Web query:
Kimi streaming output feature [DONE] completion semantics finish_reason docs
💡 Result:
In the Kimi API, when using streaming output (stream=True), the completion status and semantics are governed by specific patterns in the Server-Sent Events (SSE) response [1][2]. Completion Semantics and [DONE]: The definitive marker for the completion of a streaming response is the receipt of the data: [DONE] message [1]. Official documentation explicitly advises developers to rely on this [DONE] signal to determine if the transmission is complete, rather than relying solely on the finish_reason field [1]. Even if a chunk contains a non-null finish_reason, the stream should be considered incomplete until the [DONE] data chunk is received [1]. finish_reason Values: The finish_reason field appears only in the final data chunk and indicates why the model stopped generating [2][3]. Common values include: - stop: The model finished generating naturally [1]. - length: The output was truncated because it reached the max_completion_tokens limit [4][5][6]. - tool_calls: The model has requested a tool execution [7][3]. Usage in Streaming: During streaming, the finish_reason field is null for all chunks until the final one [1][2]. When processing streaming responses, especially those involving tool calls, developers should note that tool_calls will follow any content output [7]. Because the finish_reason arrives only in the last chunk, it is recommended to check for the existence of specific fields (like delta.tool_calls) to identify intermediate states such as tool invocation [7]. Additionally, if using models with thinking capabilities (like Kimi K3 or K2 thinking models), reasoning_content will typically be streamed before the final content [8][9].
Citations:
- 1: https://platform.kimi.ai/docs/guide/utilize-the-streaming-output-feature-of-kimi-api
- 2: https://platform.kimi.ai/docs/api/chat.md
- 3: https://platform.kimi.ai/docs/api/chat
- 4: https://platform.kimi.ai/docs/guide/faq?utm=
- 5: https://platform.kimi.ai/docs/guide/use-json-mode-feature-of-kimi-api
- 6: https://platform.kimi.ai/docs/guide/use-partial-mode-feature-of-kimi-api
- 7: https://platform.kimi.ai/docs/guide/use-kimi-api-to-complete-tool-calls
- 8: https://platform.kimi.ai/docs/guide/kimi-k3-quickstart
- 9: https://platform.kimi.ai/docs/guide/use-kimi-k2-thinking-model
Treat EOF before [DONE] as incomplete. Kimi uses data: [DONE] as the completion marker; finish_reason alone does not mean the stream is complete. Emitting Done here can turn truncated text or tool arguments into a successful final message instead of a transient error that preserves the partial state.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/src/upstream.rs` around lines 139 - 144, Update the EOF
handling in the upstream stream flow so a connection closing before receiving
the `[DONE]` marker is treated as incomplete rather than emitting
`AssistantMessageEvent::Done`. Preserve the accumulated partial state and return
or propagate the established transient stream error path, while keeping `Done`
emission exclusively for an explicit `[DONE]` event.
Comment on lines
+213
to
+221
| tokio::spawn(async move { | ||
| let mut buf = vec![0u8; 65536]; | ||
| let n = sock.read(&mut buf).await.unwrap_or(0); | ||
| let head = String::from_utf8_lossy(&buf[..n]); | ||
| let response = if head.starts_with("GET /v1/models") { | ||
| STUB_MODELS | ||
| } else { | ||
| messages_response | ||
| }; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Read through the HTTP request line before selecting a stub response.
A TCP read may return only part of GET /v1/models; that request then receives the SSE response and makes discovery tests flaky. Buffer until at least the first newline or the complete header terminator.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/tests/integration.rs` around lines 213 - 221, Update the
request-reading logic inside the tokio::spawn handler to accumulate data until
the HTTP request line is complete, stopping once a newline or full header
terminator is received. Select STUB_MODELS only after reliably identifying the
GET /v1/models request, while preserving messages_response for other requests.
Comment on lines
+335
to
+346
| // the registration token was persisted to the provider's state scope | ||
| let token = call( | ||
| &provider_iii, | ||
| "state::get", | ||
| json!({ "scope": "provider-kimi", "key": "registration_token" }), | ||
| ) | ||
| .await | ||
| .unwrap(); | ||
| assert!( | ||
| token.as_str().is_some_and(|t| !t.is_empty()), | ||
| "token persisted, got {token}" | ||
| ); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Poll for token persistence instead of racing registration.
boot_stack returns once the router lists Kimi, but declare_once stores the token only after registration returns. This immediate state::get can therefore observe the pre-write state. Poll with a deadline, as the catalog assertions already do.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/tests/integration.rs` around lines 335 - 346, Update the
registration-token assertion after boot_stack to poll state::get for the
provider-kimi registration_token until it becomes non-empty or the existing
catalog-assertion deadline is reached. Preserve the current failure message and
final token validation, using the same polling pattern and timing symbols
already used by the catalog assertions.
| "cost filled: {res}" | ||
| ); | ||
|
|
||
| let _ = tokio::time::timeout(Duration::from_secs(5), pump).await; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Require channel pumps to finish before inspecting frames. Both tests discard timeout and join failures, allowing assertions against incomplete output.
provider-kimi/tests/integration.rs#L385-L385: require the success-stream pump to complete successfully.provider-kimi/tests/integration.rs#L425-L425: apply the same check to the error-stream pump.
📍 Affects 1 file
provider-kimi/tests/integration.rs#L385-L385(this comment)provider-kimi/tests/integration.rs#L425-L425
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@provider-kimi/tests/integration.rs` at line 385, Require the success-stream
pump at provider-kimi/tests/integration.rs:385-385 to complete without timeout
or join failure before inspecting frames, and apply the same handling to the
error-stream pump at provider-kimi/tests/integration.rs:425-425. Replace the
discarded timeout results with an assertion or equivalent propagation that fails
the test unless each pump finishes successfully.
ytallo
added a commit
that referenced
this pull request
Jul 17, 2026
validate_worker.py hard-blocks any worker's manifest version decreasing between merged commits once its source has changed, and main already has provider-kimi at 1.0.0 from the original PR #522 merge. Releasing at 1.0.0 instead of introducing an exemption to that CI gate.
ytallo
added a commit
that referenced
this pull request
Jul 17, 2026
…527) * chore(provider-kimi): wire release CD, set initial version to 0.1.0 Adds provider-kimi to create-tag.yml's worker options and release.yml's tag-trigger patterns per docs/sops/new-worker.md §6, and drops the manifest version from the placeholder 1.0.0 to 0.1.0 for its first release. * fix(provider-kimi): emit slim (partial: None) delta frames provider-kimi predates #525's llm-router!: slim streaming delta frames change and still built full AssistantMessage snapshots on every delta, which no longer compiles against the new Option<AssistantMessage> contract. Mirrors the pattern already applied to the other six providers. * chore(provider-kimi): sync Cargo.lock llm-router pin to 1.2.0 * chore(provider-kimi): keep initial release at 1.0.0 validate_worker.py hard-blocks any worker's manifest version decreasing between merged commits once its source has changed, and main already has provider-kimi at 1.0.0 from the original PR #522 merge. Releasing at 1.0.0 instead of introducing an exemption to that CI gate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Moonshot Kimi models to the shared model router so agent conversations can use Kimi with the same provider lifecycle and streaming contract as the existing providers.
What changed
provider-kimibinary workerMOONSHOT_API_KEYConfiguration
Set
MOONSHOT_API_KEYfor the router, then install and startprovider-kimi. The provider registers itself and refreshes the available Kimi model catalog automatically.Validation
cargo fmt --all -- --checkcargo clippy --all-targets -- -D warningscargo test(62 unit and binary tests, 5 engine-backed integration tests, 4 schema tests, doc tests)Fixes MOT-4080
Summary by CodeRabbit
New Features
Tests