feat: flare-proxy crate — Anthropic to OpenAI free-provider proxy - #261
Merged
Conversation
…ate instead of hand-rolled id
…r heuristic tool injection before message_stop, buffer SSE lines across chunk boundaries, gate proxy behind optional shared-secret token
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughAdds the ChangesFlare proxy
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant Client
participant v1_messages_handler
participant ProviderConfig
participant proxy_request
participant reqwest
Client->>v1_messages_handler: POST /proxy/v1/messages
v1_messages_handler->>ProviderConfig: resolve model route and provider
v1_messages_handler->>proxy_request: forward Anthropic JSON
proxy_request->>reqwest: POST upstream /chat/completions
reqwest-->>proxy_request: streaming OpenAI SSE chunks
proxy_request-->>Client: translated Anthropic SSE response
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (3)
src/dashboard/server.rs (1)
230-230: 🔒 Security & Privacy | 🔵 TrivialMerge looks correct; operational reminder.
Route mounting and ordering relative to
fallback(static_handler)is correct. Operationally: this endpoint spends the operator's provider quota and is now reachable wherever the dashboard is bound (including non-localhost, per the existing warning further down this file) unlessAGENTFLARE_PROXY_TOKENis set — worth documenting that operators binding off-localhost should set this env var.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/dashboard/server.rs` at line 230, Document near the dashboard bind-address warning or proxy route setup that operators exposing the dashboard beyond localhost should set AGENTFLARE_PROXY_TOKEN, since the mounted flare_proxy route consumes provider quota and is otherwise publicly reachable.crates/flare-proxy/src/heuristic.rs (1)
31-73: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winRegexes recompiled on every call.
extract_invoke_meal,extract_code_fence_json, andextract_json_tool_blockeach callRegex::newinline every timetry_extract_tool_callruns, which is a well-known Rust anti-pattern (compilation isn't free). Prefer compiling once viastd::sync::LazyLock/once_cell::sync::Lazy.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/flare-proxy/src/heuristic.rs` around lines 31 - 73, The regexes in extract_invoke_meal, extract_code_fence_json, and extract_json_tool_block are recompiled on every invocation. Define each pattern once using std::sync::LazyLock or the project’s established once_cell::sync::Lazy approach, then reuse the initialized regexes inside these functions while preserving their existing matching behavior.crates/flare-proxy/src/shape_xlat.rs (1)
187-251: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low valueRemove or wire up
chat_to_messages
No runtime caller in this crate references it; only the unit tests do. If there’s no external use, drop it to avoid dead code and drift from the streaming translation path.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/flare-proxy/src/shape_xlat.rs` around lines 187 - 251, Remove the unused chat_to_messages function and its associated tests, since no runtime code in the crate calls it. Verify imports or helpers used only by this function are also removed, while leaving the streaming translation path unchanged.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/flare-proxy/src/forward.rs`:
- Around line 179-188: The current post-processing block in the streaming flow
only computes stripped text after raw deltas have already been emitted, so
requires_think_parsing routes still expose think tags. Update the logic around
openai_chunk_to_anthropic_sse and strip_think_tags to remove or transform
think-tag content before sending each client-visible SSE event, preserving
non-thought text and the existing route behavior; do not rely on the discarded
_clean result after is_finish.
- Around line 61-64: Update the request flow around reqwest::Client in the
forwarding function to reuse a client stored in AppState instead of constructing
one per call. Configure that shared client with an explicit timeout during
AppState initialization, then use it for the existing POST request while
preserving the current request payload and endpoint.
- Around line 104-120: Update the SSE buffering logic in the stream closure
around line_buf so raw chunk bytes are accumulated before UTF-8 decoding. Track
complete lines at the byte level, decode only each assembled complete line after
removing it from the byte buffer, and preserve partial multibyte sequences and
trailing partial lines across chunks.
In `@crates/flare-proxy/src/heuristic.rs`:
- Around line 98-104: Update needs_heuristic_tools to normalize the model
identifier to a consistent case before checking the existing model-family
substrings, so mixed-case identifiers trigger the same heuristic detection as
lowercase values. Preserve the current substring list and boolean behavior.
- Around line 56-94: Fix extract_json_tool_block by slicing from the regex
match’s opening-brace position rather than from the "name" field, so
find_balanced_brace receives the complete outer object and serde_json can parse
it. Update find_balanced_brace to ignore braces occurring inside JSON string
literals, including escaped quotes, and add coverage for bare JSON tool blocks
and brace characters inside string values.
- Around line 31-54: Update the regex patterns in extract_invoke_meal and
extract_code_fence_json to enable dot-all matching so their .*? sections span
newlines, preserving existing capture behavior. Add or update tests to verify
both extractors parse pretty-printed multiline JSON payloads.
In `@crates/flare-proxy/src/lib.rs`:
- Around line 34-42: Update the proxy token validation around
AGENTFLARE_PROXY_TOKEN so a configured empty value cannot disable
authentication: treat an empty expected token as invalid configuration or reject
requests rather than comparing it against the missing-header default. Preserve
the existing missing/invalid token response, and use constant-time token
comparison if the project already provides suitable support.
In `@crates/flare-proxy/src/shape_xlat.rs`:
- Around line 74-124: Update translate_user_content to extract data and
tool_use_id as strings before formatting, avoiding JSON-quoted values in image
URLs and tool-result markers. Validate the text field with the same ?-based
string extraction pattern used for other fields, and skip blocks whose text is
missing or not a string instead of serializing null.
- Around line 8-10: Update the Anthropic system-prompt handling in the
messages-to-chat conversion to prepend a system-role message with the prompt
content to the front of messages, rather than assigning body["system"]. Preserve
existing message conversion, and update test_messages_to_chat_with_system to
assert the prepended message shape.
In `@crates/flare-proxy/src/think.rs`:
- Around line 27-29: Update needs_think_parsing to use the shared model-family
detection logic from heuristic.rs, avoiding duplicated case-sensitive substring
checks and ensuring the supported families include mistral and mixtral alongside
the existing families.
---
Nitpick comments:
In `@crates/flare-proxy/src/heuristic.rs`:
- Around line 31-73: The regexes in extract_invoke_meal,
extract_code_fence_json, and extract_json_tool_block are recompiled on every
invocation. Define each pattern once using std::sync::LazyLock or the project’s
established once_cell::sync::Lazy approach, then reuse the initialized regexes
inside these functions while preserving their existing matching behavior.
In `@crates/flare-proxy/src/shape_xlat.rs`:
- Around line 187-251: Remove the unused chat_to_messages function and its
associated tests, since no runtime code in the crate calls it. Verify imports or
helpers used only by this function are also removed, while leaving the streaming
translation path unchanged.
In `@src/dashboard/server.rs`:
- Line 230: Document near the dashboard bind-address warning or proxy route
setup that operators exposing the dashboard beyond localhost should set
AGENTFLARE_PROXY_TOKEN, since the mounted flare_proxy route consumes provider
quota and is otherwise publicly reachable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 419c4cd1-4a0f-47ce-9a2f-f38b66c9c4f8
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
Cargo.tomlcrates/flare-proxy/Cargo.tomlcrates/flare-proxy/src/forward.rscrates/flare-proxy/src/heuristic.rscrates/flare-proxy/src/lib.rscrates/flare-proxy/src/providers.rscrates/flare-proxy/src/shape_xlat.rscrates/flare-proxy/src/think.rssrc/dashboard/server.rs
Comment on lines
+179
to
+188
| // Think tag stripping on accumulated text | ||
| if needs_think && !accumulated_text.is_empty() { | ||
| let (_clean, thoughts) = crate::think::strip_think_tags(&accumulated_text); | ||
| if !thoughts.is_empty() { | ||
| // We've already streamed the text with think tags. | ||
| // In a real implementation, we'd buffer and re-stream. | ||
| // For v1, we strip in post-processing of accumulated text. | ||
| // The SSE events already went out; this is best-effort cleanup. | ||
| } | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Think-tag stripping is a no-op on the actual streamed output.
Raw text deltas (including <think> tags) are already emitted via openai_chunk_to_anthropic_sse before is_finish is reached. The strip_think_tags call here only computes thoughts from accumulated_text and discards it — for requires_think_parsing: true routes, think-tag content will still leak into the client-visible stream, contrary to what the route flag implies.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/flare-proxy/src/forward.rs` around lines 179 - 188, The current
post-processing block in the streaming flow only computes stripped text after
raw deltas have already been emitted, so requires_think_parsing routes still
expose think tags. Update the logic around openai_chunk_to_anthropic_sse and
strip_think_tags to remove or transform think-tag content before sending each
client-visible SSE event, preserving non-thought text and the existing route
behavior; do not rely on the discarded _clean result after is_finish.
Comment on lines
+27
to
+29
| pub fn needs_think_parsing(model: &str) -> bool { | ||
| model.contains("deepseek") || model.contains("qwen") || model.contains("llama") | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Model-family detection is case-sensitive and duplicated from heuristic.rs.
Same lowercase-substring pattern as needs_heuristic_tools in heuristic.rs, but with a different list (missing "mistral"/"mixtral" here). See consolidated comment.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/flare-proxy/src/think.rs` around lines 27 - 29, Update
needs_think_parsing to use the shared model-family detection logic from
heuristic.rs, avoiding duplicated case-sensitive substring checks and ensuring
the supported families include mistral and mixtral alongside the existing
families.
cargo-deny licenses failed: flare-proxy had no license field, unlike every other workspace crate. clippy --locked failed: Cargo.lock was never regenerated after nanoid was added as a dependency.
- shape_xlat: prepend system prompt as a system message instead of setting body[\system\], which the OpenAI chat-completions schema does not have; system prompts were being silently dropped on every request (critical - breaks the proxy for real agent traffic). - shape_xlat: translate_user_content stringified image data and tool_use_id via Value's Display impl, embedding JSON quotes into base64 data URLs and tool_result markers; extract as str instead. - heuristic: extract_json_tool_block sliced from the \name\ substring instead of the enclosing brace, so find_balanced_brace matched the nested arguments object and the extractor always returned None; slice from the regex match start instead. find_balanced_brace now also tracks string state so braces inside string values do not miscount. - heuristic/think: add (?s) so .*? spans newlines, matching pretty-printed JSON tool calls; make model-family detection case-insensitive; think.rs now also matches mistral/mixtral. - lib: reject a configured-but-empty AGENTFLARE_PROXY_TOKEN instead of silently disabling the auth gate. - forward: reuse one reqwest::Client with a timeout via AppState instead of building a new client per request; buffer raw bytes across SSE chunk boundaries before UTF-8 decoding instead of lossy-decoding each raw chunk, which could mangle a multibyte sequence split across a chunk boundary. Think-tag suppression during streaming is left as a known limitation (flagged Heavy lift by review): raw deltas are already streamed before the accumulated-text pass runs, so it cannot retroactively strip tags from what the client already received. Fixing that requires buffering deltas and delaying emission, tracked separately.
3 tasks
4 tasks
5 tasks
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
crates/flare-proxy/: proxies/proxy/v1/messages(Anthropic Messages API) to free OpenAI-compatible providers (NVIDIA NIM, OpenRouter, LM Studio), mounted on the existingagentflare servedashboard server.shape_xlat.rs), heuristic tool-call extraction for providers without native function calling, and<think>tag stripping.Authorizationheader was missing theBearerscheme, breaking the default free-tier route entirely.content_block_stop). Buffer now tracks open indices and allocates fresh ones.message_stop— reordered so it closes before the message ends./proxy/v1/messagesgated behind an opt-inAGENTFLARE_PROXY_TOKENshared-secret check (dashboard server can be bound off-localhost).Content-Type: application/json; heuristic tool-call IDs use the workspace's existingnanoidcrate instead of a hand-rolled generator.Test plan
cargo test -p flare-proxy— 12/12 pass (11 original + 1 new regression test for the index-collision fix)cargo clippy -p flare-proxy --all-features -- -D warnings— cleancargo build --workspace— clean (1 pre-existing unrelated warning insrc/components.rs)cargo test --bin agentflare dashboard— 11/11 pass, confirms dashboard server wiring unaffectedSummary by CodeRabbit
<think>...</think>stripping.