feat: add Tinfoil private inference provider - #62
Conversation
Add a dedicated Tinfoil LLM backend (`LLM_BACKEND=tinfoil`) for Tinfoil's private inference service (https://tinfoil.sh). The existing `openai_compatible` backend cannot be used with Tinfoil because rig-core 0.30.0 defaults to the OpenAI Responses API (`/v1/responses`), which Tinfoil does not support — it only implements the Chat Completions API (`/v1/chat/completions`), returning 403 "shim: path not allowed" when hit on the responses endpoint. Rather than changing `openai_compatible` to use Chat Completions (which would break users expecting the Responses API), this adds a dedicated provider that explicitly uses rig's `.completions_api()` client. This also lays the groundwork for integrating Tinfoil's privacy wrapper client (enclave attestation, TLS certificate pinning) once their Rust SDK is available. The provider implementation can be swapped to use the Tinfoil Rust client without changing the LlmProvider interface. Configuration: LLM_BACKEND=tinfoil TINFOIL_API_KEY=tk_... TINFOIL_MODEL=kimi-k2-5 # optional, default
Summary of ChangesHello @lwlee2608, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces Tinfoil as a new, dedicated LLM backend provider, enabling users to leverage its private inference capabilities. The integration addresses Tinfoil's specific API requirements by using the Chat Completions API, differentiating it from other OpenAI-compatible services. The changes encompass adding necessary configuration structures, environment variable parsing, and a client adapter to facilitate interaction with Tinfoil's services, while also preparing the codebase for future SDK enhancements. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces support for the Tinfoil private inference provider. The implementation is solid, adding the necessary configuration, provider creation logic, and updating the relevant enums and structs. The changes align well with the existing architecture for LLM providers. I have one minor suggestion to enhance configuration consistency.
There was a problem hiding this comment.
Pull request overview
This PR adds support for Tinfoil, a private inference provider, as a new LLM backend option. Tinfoil is specifically designed for privacy-focused model inference and requires the Chat Completions API rather than the Responses API that rig-core 0.30.0 defaults to for OpenAI-compatible endpoints. This necessitates a dedicated provider implementation rather than using the generic openai_compatible backend.
Changes:
- Added
Tinfoilvariant toLlmBackendenum with correspondingFromStrandDisplayimplementations - Created
TinfoilConfigstruct and environment variable resolution logic (TINFOIL_API_KEY, TINFOIL_MODEL) - Implemented
create_tinfoil_provider()function that explicitly uses rig's Chat Completions API client via.completions_api() - Updated setup wizard to initialize
tinfoil: NoneinLlmConfigconstruction
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/config.rs | Adds Tinfoil backend enum variant, configuration struct, and environment variable resolution with "kimi-k2-5" as default model |
| src/llm/mod.rs | Implements Tinfoil provider creation using OpenAI client with explicit Chat Completions API and hardcoded base URL |
| src/setup/wizard.rs | Initializes tinfoil field to None in wizard's LlmConfig construction |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
serrrfirat
left a comment
There was a problem hiding this comment.
PR #62 Review: feat: add Tinfoil private inference provider
Author: lwlee2608 | Reviewed: 2026-02-13T12:36:00+04:00
Verdict: APPROVE ✅
Clean, well-structured PR that follows established patterns from other provider integrations (OpenRouter, OpenAI-compatible). No significant issues found.
Findings
P3 - Minor: Hardcoded base URL (informational)
File: src/llm/mod.rs:144
const TINFOIL_BASE_URL: &str = "https://inference.tinfoil.sh/v1";The base URL is hardcoded. Other providers like OpenAI-compatible allow configurable endpoints. For a specific provider like Tinfoil this is acceptable, but consider adding an optional TINFOIL_BASE_URL env override for flexibility (e.g., self-hosted instances or staging).
P3 - Minor: Default model choice
File: src/config.rs:434
.unwrap_or_else(|| "kimi-k2-5".to_string());Default model is kimi-k2-5. This is fine but worth confirming it's a valid model on Tinfoil's inference API.
Positives
- Follows the exact same pattern as other provider implementations
- Proper use of
SecretStringfor API key handling - Good error messages with provider context
- Config parsing correctly gated behind
LlmBackend::Tinfoilcheck - Wizard properly initialized with
tinfoil: None - Uses the existing
RigAdaptervia OpenAI-compatible client — clean reuse
Security Assessment
- ✅ API key stored in
SecretString, accessed viaexpose_secret() - ✅ No secrets logged (only model name in tracing::info)
- ✅ HTTPS endpoint hardcoded
- ✅ No user input passed unsanitized
|
@copilot review again |
PR Review: feat: add Tinfoil private inference providerPR Author: Jason Lee SummaryThis PR adds support for Tinfoil, a private inference provider, to the IronClaw framework. It implements a new The changes are minimal and focused: adding the backend enum variant, configuration struct, environment variable parsing, provider creation logic, and a field in the setup wizard. Pros
Concerns
Suggestions
Overall AssessmentStatus: 🟢 Approve with minor suggestions This is a clean, well-structured implementation that follows the project's existing patterns. The code is minimal, uses appropriate Rust idioms, and handles secrets correctly. The primary concern is the lack of test coverage, which should be addressed before merge. The hardcoded base URL is acceptable for an initial implementation but should be considered for future flexibility. The PR successfully adds Tinfoil as a new LLM backend with proper configuration, error handling, and integration with the existing provider infrastructure. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/config.rs:362
- The documentation comment for
LlmConfiglists examples of backend values but does not include the newly addedtinfoilbackend. Consider updating the examples to includetinfoilfor completeness, e.g.,(e.g. 'openai', 'anthropic', 'ollama', 'openai_compatible', 'tinfoil').
/// NEAR AI remains the default backend. Users can switch to other providers
/// by setting `LLM_BACKEND` (e.g. `openai`, `anthropic`, `ollama`).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Thanks @serrrfirat , CI fixed |
|
Hi @serrrfirat - just to clarify clearly: this CI failure was not caused by the Tinfoil feature changes in this PR. It was from a pre-existing rustfmt mismatch in src/llm/session.rs on the merge ref. I pushed a formatting-only fix in commit 02cce46 to unblock Code Style. Thank you! |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| })?; | ||
|
|
||
| // Tinfoil currently only supports the Chat Completions API and not the newer Responses API, | ||
| // so we must explicitly select the completions API here (unlike other OpenAI-compatible providers). |
There was a problem hiding this comment.
The comment phrase "unlike other OpenAI-compatible providers" is ambiguous. Consider rephrasing to "unlike the openai_compatible backend" or "unlike the default behavior for OpenAI client" to clarify that you're referring to the openai_compatible backend in this codebase, which relies on the default Responses API behavior.
| // so we must explicitly select the completions API here (unlike other OpenAI-compatible providers). | |
| // so we must explicitly select the completions API here, unlike the openai_compatible backend, | |
| // which relies on the default OpenAI Responses API behavior. |
* feat: add Tinfoil private inference provider Add a dedicated Tinfoil LLM backend (`LLM_BACKEND=tinfoil`) for Tinfoil's private inference service (https://tinfoil.sh). The existing `openai_compatible` backend cannot be used with Tinfoil because rig-core 0.30.0 defaults to the OpenAI Responses API (`/v1/responses`), which Tinfoil does not support — it only implements the Chat Completions API (`/v1/chat/completions`), returning 403 "shim: path not allowed" when hit on the responses endpoint. Rather than changing `openai_compatible` to use Chat Completions (which would break users expecting the Responses API), this adds a dedicated provider that explicitly uses rig's `.completions_api()` client. This also lays the groundwork for integrating Tinfoil's privacy wrapper client (enclave attestation, TLS certificate pinning) once their Rust SDK is available. The provider implementation can be swapped to use the Tinfoil Rust client without changing the LlmProvider interface. Configuration: LLM_BACKEND=tinfoil TINFOIL_API_KEY=tk_... TINFOIL_MODEL=kimi-k2-5 # optional, default * style: fix rustfmt formatting in Tinfoil provider * style: remove unnecessary tin_foil alias for Tinfoil backend * Update src/llm/mod.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: add tinfoil field to LlmConfig test fixture * style: fix rustfmt output in session manager --------- Co-authored-by: firat.sertgoz <f@nuff.tech> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
* feat: add Tinfoil private inference provider Add a dedicated Tinfoil LLM backend (`LLM_BACKEND=tinfoil`) for Tinfoil's private inference service (https://tinfoil.sh). The existing `openai_compatible` backend cannot be used with Tinfoil because rig-core 0.30.0 defaults to the OpenAI Responses API (`/v1/responses`), which Tinfoil does not support — it only implements the Chat Completions API (`/v1/chat/completions`), returning 403 "shim: path not allowed" when hit on the responses endpoint. Rather than changing `openai_compatible` to use Chat Completions (which would break users expecting the Responses API), this adds a dedicated provider that explicitly uses rig's `.completions_api()` client. This also lays the groundwork for integrating Tinfoil's privacy wrapper client (enclave attestation, TLS certificate pinning) once their Rust SDK is available. The provider implementation can be swapped to use the Tinfoil Rust client without changing the LlmProvider interface. Configuration: LLM_BACKEND=tinfoil TINFOIL_API_KEY=tk_... TINFOIL_MODEL=kimi-k2-5 # optional, default * style: fix rustfmt formatting in Tinfoil provider * style: remove unnecessary tin_foil alias for Tinfoil backend * Update src/llm/mod.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: add tinfoil field to LlmConfig test fixture * style: fix rustfmt output in session manager --------- Co-authored-by: firat.sertgoz <f@nuff.tech> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com>
Summary
LLM_BACKEND=tinfoilprovider for Tinfoil private inferenceopenai_compatiblebackend because rig-core 0.30.0 defaults to the OpenAI Responses API (/v1/responses), which Tinfoil does not support — it only implements the Chat Completions API (/v1/chat/completions), returning403 "shim: path not allowed"openai_compatibleto use Chat Completions (which would break users expecting the Responses API), this adds a dedicated provider that explicitly uses rig's.completions_api()clientLlmProviderinterfaceConfiguration
LLM_BACKEND=tinfoil TINFOIL_API_KEY=tk_... TINFOIL_MODEL=kimi-k2-5 # optional, defaultFiles changed
src/config.rs—Tinfoilvariant inLlmBackend,TinfoilConfigstruct, env var resolutionsrc/llm/mod.rs—create_tinfoil_provider()using rig's Chat Completions clientsrc/setup/wizard.rs— Addtinfoil: Noneto wizard'sLlmConfiginitializerTest plan
LLM_BACKEND=tinfoilwithkimi-k2-5model responds successfully