Add `local-inference` feature flag to gate llama-cpp-2 dependency by r0x0d · Pull Request #7648 · aaif-goose/goose

r0x0d · 2026-03-04T17:56:29Z

Summary

Gate the local LLM inference provider behind a local-inference feature flag (default ON) so downstream builds can opt out of the heavy llama-cpp-2 native dependency and its libclang build requirement.

When disabled, the LocalInferenceProvider is not registered, the server local_inference routes are excluded, and the CLI local-models command is removed.

Made-with: Cursor

Type of Change

AI Assistance

This PR was created or reviewed with AI assistance

Testing

Tested locally with cargo test --no-default-features and with cargo test

Related Issues

Relates to #5716
Discussion: LINK (if any)

Screenshots/Demos (for UX changes)

Before:

After:

r0x0d · 2026-03-04T17:57:31Z

This is the first of a series of changes to add more feature flags to goose to enable disabling certain aspects during build time. We have discussed that in the past in Discord, so I will link it up in the description

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5279e0d3a6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f6bb6b81b4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

jh-block · 2026-03-04T19:53:30Z

Thanks for this. Please make sure that all the relevant crates (goose-server, the ACP server, the CLI) have a local-inference feature to pass through to the goose crate's one, and also on goose make sure that the cuda feature is forwarded as llama-cpp-2?/cuda to avoid having the cuda feature trigger pulling in the llama-cpp-2 dependency.

Also, the frontend will need to handle 404 on the local inference endpoints and hide/disable that provider & the local inference tab in config.

r0x0d · 2026-03-04T19:56:14Z

Thanks for this. Please make sure that all the relevant crates (goose-server, the ACP server, the CLI) have a local-inference feature to pass through to the goose crate's one, and also on goose make sure that the cuda feature is forwarded as llama-cpp-2?/cuda to avoid having the cuda feature trigger pulling in the llama-cpp-2 dependency.

Also, the frontend will need to handle 404 on the local inference endpoints and hide/disable that provider & the local inference tab in config.

For sure! I will take care of this, thanks!

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9e987dd7f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8abd8cdab

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8e78b414c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-03-12T12:51:52Z

+#[cfg(feature = "code-mode")]
 pub async fn run_prompt_codemode<C: Connection>() {


Gate all code-mode ACP tests consistently

Adding #[cfg(feature = "code-mode")] here removes run_prompt_codemode from no-code-mode builds, but crates/goose-acp/tests/provider_test.rs still imports and invokes it unconditionally (use ... run_prompt_codemode and test_prompt_codemode). That makes cargo test -p goose-acp --no-default-features fail to compile with an unresolved symbol, so the no-default-features path is currently broken.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-12T12:51:52Z

+        const activeProviders = providersResponse.filter(
+          (provider) => provider.is_configured && (provider.name !== 'local' || localInferenceAvailable)
+        );


Reset stale local provider when filtering unavailable providers

This filter can remove local from activeProviders while the modal state still holds provider === 'local' (for example, a user with persisted local config opening a build without local inference). In that state, submit still uses the stale provider and getProviderMetadata('local', getProviders) throws (modelInterface.ts), so model switching fails instead of gracefully falling back to a valid provider.

Useful? React with 👍 / 👎.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26875402e1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-03-12T13:31:16Z

+        const activeProviders = providersResponse.filter(
+          (provider) => provider.is_configured && (provider.name !== 'local' || localInferenceAvailable)
+        );


Reset stale local selection when local provider is filtered

Removing local from activeProviders when localInferenceAvailable is false does not clear the modal's existing provider/model state, so users with persisted local settings can still submit provider === "local" even though it is no longer selectable. Fresh evidence: submit later calls getProviderMetadata(provider || '', getProviders) in this same component; in no-local-inference builds the providers API omits local, causing that lookup to throw and the model-switch flow to fail.

Useful? React with 👍 / 👎.

Gate the local LLM inference provider behind a `local-inference` feature flag (default ON) so downstream builds can opt out of the heavy llama-cpp-2 native dependency and its libclang build requirement. When disabled, the LocalInferenceProvider is not registered, the server local_inference routes are excluded, and the CLI `local-models` command is removed. Made-with: Cursor Relates to aaif-goose#5716 Signed-off-by: Rodolfo Olivieri <rodolfo.olivieri3@gmail.com> Co-authored-by: Claude (Anthropic) <noreply@anthropic.com>

Signed-off-by: Rodolfo Olivieri <rodolfo.olivieri3@gmail.com>

r0x0d · 2026-03-23T17:14:54Z

closing this since #7976 got merged first.

chatgpt-codex-connector Bot reviewed Mar 4, 2026

View reviewed changes

Comment thread crates/goose/Cargo.toml

r0x0d force-pushed the feature/local-inference-feature-flag branch from 5279e0d to f6bb6b8 Compare March 4, 2026 18:27

chatgpt-codex-connector Bot reviewed Mar 4, 2026

View reviewed changes

Comment thread crates/goose/src/providers/init.rs

chatgpt-codex-connector Bot reviewed Mar 5, 2026

View reviewed changes

Comment thread ui/desktop/src/hooks/useLocalInferenceAvailable.ts Outdated

Comment thread ui/desktop/src/components/settings/localInference/LocalInferenceSettings.tsx Outdated

r0x0d force-pushed the feature/local-inference-feature-flag branch from d9e987d to d8abd8c Compare March 5, 2026 11:56

chatgpt-codex-connector Bot reviewed Mar 5, 2026

View reviewed changes

Comment thread ui/desktop/src/components/settings/SettingsView.tsx

r0x0d force-pushed the feature/local-inference-feature-flag branch from d8abd8c to e1f7f96 Compare March 6, 2026 11:05

DOsinga assigned alexhancock Mar 9, 2026

r0x0d force-pushed the feature/local-inference-feature-flag branch 2 times, most recently from 4cb7a8b to e4fea42 Compare March 12, 2026 12:28

jh-block assigned jh-block and unassigned alexhancock Mar 12, 2026

r0x0d force-pushed the feature/local-inference-feature-flag branch 2 times, most recently from ef27049 to d8e78b4 Compare March 12, 2026 12:41

chatgpt-codex-connector Bot reviewed Mar 12, 2026

View reviewed changes

r0x0d force-pushed the feature/local-inference-feature-flag branch 2 times, most recently from 54e85d2 to 2687540 Compare March 12, 2026 13:21

chatgpt-codex-connector Bot reviewed Mar 12, 2026

View reviewed changes

r0x0d and others added 2 commits March 12, 2026 10:36

Fix comments from code review

8632e4d

Signed-off-by: Rodolfo Olivieri <rodolfo.olivieri3@gmail.com>

r0x0d force-pushed the feature/local-inference-feature-flag branch from 2687540 to 8632e4d Compare March 12, 2026 13:36

DOsinga added the needs_human label to set when a robot looks at a PR and can't handle it label Mar 21, 2026

r0x0d closed this Mar 23, 2026

r0x0d deleted the feature/local-inference-feature-flag branch March 23, 2026 17:14

		#[cfg(feature = "code-mode")]
		pub async fn run_prompt_codemode<C: Connection>() {

Conversation

r0x0d commented Mar 4, 2026

Summary

Type of Change

AI Assistance

Testing

Related Issues

Screenshots/Demos (for UX changes)

Uh oh!

r0x0d commented Mar 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

jh-block commented Mar 4, 2026

Uh oh!

r0x0d commented Mar 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

r0x0d commented Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants