Skip to content

test(providers): isolate environment-proxy test in its own binary - #11262

Merged
DOsinga merged 1 commit into
aaif-goose:mainfrom
asiantuntija:providers-isolate-proxy-env-test
Aug 20, 2026
Merged

DOsinga merged 1 commit into
aaif-goose:mainfrom
asiantuntija:providers-isolate-proxy-env-test

Conversation

@asiantuntija

Copy link
Copy Markdown
Contributor

Problem

cargo test -p goose-providers fails deterministically on this machine (12 cores): 65 consecutive runs, 9 failures each.

---- anthropic::tests::fetch_supported_models_propagates_auth_error ----
expected Authentication error, got: NetworkError("Network error — check your network connection and try again.")

---- api_client::tests::loopback_transport_does_not_use_environment_proxy ----
assertion failed: tokio::time::timeout(Duration::from_millis(100), proxy.accept()).await.is_err()

azure_foundry::tests::canonical_claude_deployment_uses_canonical_output_limit also failed on some runs and not others, which is the scheduling dependence showing.

Root cause

loopback_transport_does_not_use_environment_proxy sets HTTP_PROXY/http_proxy process-wide, pointing at a throwaway listener. The env-lock guard serializes it against other env-lock callers, but the wiremock-based provider tests don't use env-lock — they just build clients:

Their helpers call ApiClient::new_with_tls(...), which uses TransportPolicy::Default. Unlike TransportPolicy::LoopbackHttp, that branch never calls .no_proxy().
reqwest reads the proxy environment when the client is built, so requests to the loopback mock server are routed to the throwaway proxy and fail as NetworkError.
That listener then accepts the stray connections, so the proxy test's own "the proxy should never be contacted" assertion fails too.

One env mutation, nine failures. anthropic.rs, ollama.rs, openai.rs, and azure_foundry.rs all use MockServer, so which ones fail on a given run depends on scheduling.

Same class as #11059 (process-global env mutation leaking into parallel tests); this instance is in goose-providers, with HTTP_PROXY rather than GOOSE_PATH_ROOT.

Fix

Move that one test into crates/goose-providers/tests/loopback_proxy_env.rs. Integration test files each get their own process, so the mutation can't reach tests that read it. No production code changes, and no other test needs annotating.

Verification
before: 65/65 runs of cargo test -p goose-providers failed
after: every run green, repeatedly
cargo test -p goose-providers --test loopback_proxy_env passes in its own binary, so the no-proxy behaviour is still genuinely exercised
--no-default-features --features rustls-tls and --features native-tls both pass, matching the CI matrix
cargo clippy -p goose-providers --tests and cargo fmt --check clean

Since CI runs plain cargo test on this crate with no per-test process isolation, this may be surfacing there as intermittent failures — runners have fewer cores, so less of the suite is in flight at once and the overlap is easier to miss.

Alternatives considered

#[serial] on the proxy test doesn't help: serial_test and env-lock are separate locks, so unannotated victims are unaffected — every test that builds an ApiClient would need annotating, effectively serializing the suite. Adding .with_loopback_http_only() to the wiremock helpers would fix today's victims but leaves the trap for future tests.

Found while packaging goose for nixpkgs, where this shows up reliably in the sandboxed build.

Disclosure: analysis and description prepared with assistance from Claude Opus 5 (Anthropic); I reproduced the failure, verified the diagnosis and the fix locally, and reviewed everything here.

asiantuntija pushed a commit to asiantuntija/nixpkgs that referenced this pull request Aug 15, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
asiantuntija pushed a commit to asiantuntija/nixpkgs that referenced this pull request Aug 15, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
asiantuntija pushed a commit to asiantuntija/nixpkgs that referenced this pull request Aug 15, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
asiantuntija pushed a commit to asiantuntija/nixpkgs that referenced this pull request Aug 15, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
caniko pushed a commit to caniko/nixpkgs that referenced this pull request Aug 15, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
@DOsinga DOsinga self-assigned this Aug 19, 2026
@DOsinga
DOsinga added this pull request to the merge queue Aug 19, 2026
Merged via the queue into aaif-goose:main with commit fe83492 Aug 20, 2026
23 checks passed
michaelneale added a commit that referenced this pull request Aug 20, 2026
* origin/main:
  fix: bound retry command diagnostics (#11365)
  Disable thinking for tool call labels (#11207)
  fix(review): contain REVIEW.md discovery (#11367)
  fix(acp): preserve tool result audience metadata (#11375)
  test(providers): isolate environment-proxy test in its own binary (#11262)
  fix(security): bind Foundry API keys to request origin (#11347)
  fix: canonicalize mangled tool names before permission inspection (follow-up to #10230) (#10285)
  feat(providers): add Lynkr as a declarative OpenAI-compatible provider (#11372)
  fix: sanitize Pi imported output (#10990)
  fix(otel): honor RUST_LOG directives for logs (#11360)
  feat(skills): add web-search and browser-use built-in skills (#11233)
  feat(dictation): add model-native audio transcription provider (#10589)
  feat(aws_bedrock): route OpenAI GPT-5.6 (sol/terra/luna) via Bedrock … (#10502)
  refactor(goose-local-inference): move mlx deps under macos (#11328)
  feat(providers): add PleumRouter declarative provider (#10479)
jbg added a commit that referenced this pull request Aug 20, 2026
* origin/main: (50 commits)
  chore(deps): bump the ui-minor-and-patch group across 1 directory with 53 updates (#11386)
  fix(security): bound call graph traversal (#11193)
  fix: pin arrayref to known-good commit (#11389)
  feat(providers): add SayGM as declarative OpenAI-compatible provider (#11267)
  feat(providers): custom provider cost fields drive cost tracking (config-declared pricing fallback) (#11220)
  fix(deps): repair dangling syn reference in Cargo.lock (#11385)
  fix(flake): add cudaforge hash for git dependency (#10910)
  feat: auto-focus chat input when user starts typing (#11184)
  fix(security): fail closed on invalid Codex ACP mode (#11362)
  fix(mcp): keep stdio extensions alive across worker exits (#10364)
  feat(ui): collapse scheduled job sessions into accordion in chat history (#11265)
  fix: bound retry command diagnostics (#11365)
  Disable thinking for tool call labels (#11207)
  fix(review): contain REVIEW.md discovery (#11367)
  fix(acp): preserve tool result audience metadata (#11375)
  test(providers): isolate environment-proxy test in its own binary (#11262)
  fix(security): bind Foundry API keys to request origin (#11347)
  fix: canonicalize mangled tool names before permission inspection (follow-up to #10230) (#10285)
  feat(providers): add Lynkr as a declarative OpenAI-compatible provider (#11372)
  fix: sanitize Pi imported output (#10990)
  ...

# Conflicts:
#	crates/goose/src/agents/state_machine/tests/hooks_lifecycle.rs
jbg added a commit that referenced this pull request Aug 20, 2026
* origin/main: (59 commits)
  chore(deps): bump the ui-minor-and-patch group across 1 directory with 53 updates (#11386)
  fix(security): bound call graph traversal (#11193)
  fix: pin arrayref to known-good commit (#11389)
  feat(providers): add SayGM as declarative OpenAI-compatible provider (#11267)
  feat(providers): custom provider cost fields drive cost tracking (config-declared pricing fallback) (#11220)
  fix(deps): repair dangling syn reference in Cargo.lock (#11385)
  fix(flake): add cudaforge hash for git dependency (#10910)
  feat: auto-focus chat input when user starts typing (#11184)
  fix(security): fail closed on invalid Codex ACP mode (#11362)
  fix(mcp): keep stdio extensions alive across worker exits (#10364)
  feat(ui): collapse scheduled job sessions into accordion in chat history (#11265)
  fix: bound retry command diagnostics (#11365)
  Disable thinking for tool call labels (#11207)
  fix(review): contain REVIEW.md discovery (#11367)
  fix(acp): preserve tool result audience metadata (#11375)
  test(providers): isolate environment-proxy test in its own binary (#11262)
  fix(security): bind Foundry API keys to request origin (#11347)
  fix: canonicalize mangled tool names before permission inspection (follow-up to #10230) (#10285)
  feat(providers): add Lynkr as a declarative OpenAI-compatible provider (#11372)
  fix: sanitize Pi imported output (#10990)
  ...
caniko pushed a commit to caniko/nixpkgs that referenced this pull request Aug 24, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
SuperSandro2000 pushed a commit to caniko/nixpkgs that referenced this pull request Aug 25, 2026
Changelog: https://github.com/aaif-goose/goose/releases/tag/v1.46.0

- Upstream removed the goose-server crate; the goosed binary is no
  longer built or installed.
- Token counting moved to tiktoken-rs; drop the pre-fetched Xenova
  tokenizer files and the preBuild linking.
- Add git to nativeCheckInputs; the plugin tests create git-backed
  fixture repositories.
- Prune checkFlags entries for tests removed upstream, mostly along
  with goose-server.
- Skip tests that race on process-global state under parallel
  execution; see aaif-goose/goose#11059 and aaif-goose/goose#11262.

Assisted-by: Claude (claude.ai, Claude Opus 5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants