Skip to content

feat(tinyfish): make search provider permissive, attribute errors - #31411

Open
ChenluJi wants to merge 7 commits into
BerriAI:litellm_internal_stagingfrom
ChenluJi:feat/tinyfish-search-mirror-api-surface
Open

feat(tinyfish): make search provider permissive, attribute errors#31411
ChenluJi wants to merge 7 commits into
BerriAI:litellm_internal_stagingfrom
ChenluJi:feat/tinyfish-search-mirror-api-surface

Conversation

@ChenluJi

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #30634. Reshapes the TinyFish search provider so LiteLLM mirrors the TinyFish Search API surface instead of maintaining a parallel cherry-pick.

Request side

  1. Drop the request TypedDict. Was a runtime no-op and misleadingly listed include_thumbnail. Removed; readers should refer to TinyFish's own docs.
  2. Stop sending max_results on the wire. TinyFish doesn't honor it server-side; clamp to [1, 10] (TinyFish's natural SERP cap) and thread the caller value through self for client-side response truncation.
  3. Guard max_results="abc". Previously raised a bare ValueError; now logs a warning and treats as unset.
  4. Auto-JSON-encode dict params; lowercase bool serialization. Callers naturally pass fetch={"format": "html"} (dict) and include_thumbnail=True (bool); serialize both before urlencode so the existing strict adapter and ux-labs' literal-"true"/"false" validator accept them.

Response side

  1. Drop both Pydantic response models. Parse directly into LiteLLM's SearchResponse. Per-result extras (position, site_name, fetch, fetch_error, etc.) flow through automatically via SearchResult's extra="allow".
  2. Default missing per-result title/url/snippet to "". A degraded result flows through instead of failing the whole call.
  3. Read TinyFish's parameter_warnings field when present and re-fire each entry as a verbose_logger.warning. Defensive on every malformed shape (non-list field, non-dict entry, missing required keys, non-string types). Pre-wired for a future TinyFish-side rollout; no-op today.

Error handling

  1. Inline attributed error wrapping (_wrap_error). Three call sites in transform_search_response (non-2xx, JSONDecodeError, ValidationError) wrap errors with "TinyFish Search: <msg>. See https://docs.tinyfish.ai/search-api for details." Network failures and "no response after retries" flow through LiteLLM core's _handle_error → base get_error_class and inherit any future upstream improvements; their attribution is implicit via the hostname in the bare error message.
  2. Dispatch non-2xx responses through _wrap_error at the top of transform_search_response. AsyncHTTPHandler.get doesn't call raise_for_status, so 4xx/5xx bodies were previously parsed as if successful — a pre-existing bug that silently returned SearchResponse(results=[]) on auth failures, rate limits, etc.
  3. Wrap json.JSONDecodeError on 200 bodies through the same path.
  4. Wrap pydantic.ValidationError for envelope-shape mismatches so callers see "TinyFish Search: Response shape does not match LiteLLM's SearchResponse schema: ..." instead of an opaque pydantic traceback.

Behavior changes worth flagging

  • 4xx/5xx responses now raise an attributed BaseLLMException instead of silently returning SearchResponse(results=[]). Pre-existing bug fix.

Test plan

  • 58 unit + integration tests pass (tests/test_litellm/llms/tinyfish/, tests/search_tests/test_tinyfish_search.py)
  • Full LLM-provider suite passes (sibling providers unaffected — 7170+ passing; pre-existing bedrock/custom_httpx flakes unrelated)
  • Ruff clean
  • No new basedpyright errors beyond the 4 pre-existing in this file (new errors all in low-priority budget categories with ample slack)
  • Live-tested against the production TinyFish endpoint: 47 cases covering happy paths, all unified params, all TinyFish-side params (page, recency_minutes, after_date, before_date, fetch, include_thumbnail, language, location), boundary values, cross-field validation, length limits, malformed inputs, auth failures, network failures, and combinations. All wrap or pass as expected.

Reshapes the TinyFish search provider so LiteLLM mirrors the TinyFish
Search API surface instead of maintaining a parallel cherry-pick.

Request side:
- Drop misleading request TypedDict
- Stop sending max_results on wire (TinyFish ignores it); clamp to [1,10]
  client-side via self-threaded state
- Guard non-numeric max_results from bare ValueError
- Auto-JSON-encode dict params; lowercase bool serialization for ux-labs

Response side:
- Drop both Pydantic response models; parse directly into SearchResponse
  so per-result extras flow through via extra="allow"
- Default missing title/url/snippet to "" instead of failing the call
- Read top-level parameter_warnings and re-fire as verbose_logger.warning
  (pre-wired for upcoming TinyFish-side rollout; no-op today)

Error handling:
- Attributed _wrap_error helper at 3 call sites in transform_search_response
  ("TinyFish Search: <msg>. See https://docs.tinyfish.ai/search-api for
  details.")
- Dispatch non-2xx responses through _wrap_error (fixes pre-existing bug
  where 4xx/5xx silently returned empty SearchResponse)
- Wrap json.JSONDecodeError on 200 bodies
- Wrap pydantic.ValidationError for envelope-shape mismatches

Bug fix worth flagging: 4xx/5xx responses now raise an attributed
BaseLLMException instead of silently returning SearchResponse(results=[]).

Follow-up to BerriAI#30634.
@CLAassistant

CLAassistant commented Jun 26, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@ChenluJi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR reshapes the TinyFish search provider to mirror TinyFish's actual API surface: removes a now-redundant TypedDict and two Pydantic response models, stops sending max_results on the wire (TinyFish ignores it server-side), and adds inline attributed error wrapping so 4xx/5xx responses raise a BaseLLMException instead of silently returning empty results.

  • Request side: transform_search_request now threads max_results through self._caller_max_results for client-side slicing (clamped to TinyFish's 10-result SERP ceiling), auto-JSON-encodes dict params (e.g. fetch), and lowercases Python bool params to satisfy ux-labs' validator.
  • Response side: Parses directly into SearchResponse (leveraging extra="allow" for per-result extras), defaults missing title/url/snippet to "" in-place before validation, and reads parameter_warnings defensively with per-entry validation.
  • Error handling: _wrap_error unwraps ux-labs' {"error": {"message": ...}} envelope and raises BaseLLMException with a consistent "TinyFish Search: … See docs …" prefix for non-2xx, JSONDecodeError, and ValidationError paths.

Confidence Score: 5/5

Safe to merge — the changes fix a real bug (silent 4xx/5xx swallowing), align the wire format with TinyFish's actual behavior, and are covered by a thorough test suite with no weakened assertions.

The transformation logic is correct: state threading via _caller_max_results is safe given per-call instantiation, _default_missing_result_fields correctly defaults before pydantic validation, _wrap_error handles all known ux-labs envelope shapes defensively, and _emit_parameter_warnings is fully guarded against malformed shapes. The only finding is a stale test helper in the integration test file that doesn't affect any current tests.

tests/search_tests/test_tinyfish_search.py — the _make_mock_response helper was not updated to set mock.text and mock.headers, which the production code now accesses in error paths. Harmless today (no error-path tests in that file), but worth aligning.

Important Files Changed

Filename Overview
litellm/llms/tinyfish/search/transformation.py Comprehensive refactor: removes custom TypedDicts/Pydantic models, adds proper non-2xx error attribution via _wrap_error, threads max_results through instance state for client-side truncation, auto-encodes dict/bool params, and adds defensive parameter_warnings forwarding
tests/test_litellm/llms/tinyfish/test_tinyfish_search.py Well-updated unit test suite: _make_mock_response helper now supports text, headers, and json_data=None; new error-handling tests cover 4xx, 429, 5xx, non-JSON, and schema-mismatch paths; new request-side tests cover dict/bool serialization, max_results wire exclusion, and per-param pass-through
tests/search_tests/test_tinyfish_search.py Integration-style test file: adds test_fetch_param_round_trip for end-to-end dict param serialization, but _make_mock_response is not updated to set mock.text or mock.headers, which are both now accessed by the production code in error paths

Reviews (5): Last reviewed commit: "fix(tinyfish): reduce transform_search_r..." | Re-trigger Greptile

Comment thread litellm/llms/tinyfish/search/transformation.py Outdated
@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…clamp

- Run ruff format on the touched files (CI lint job rejected the prior
  commit's formatting).
- Add OverflowError to the except clause in the max_results clamp so
  callers passing math.inf (or other non-finite floats) get the same
  warn-and-ignore behavior as other malformed values. Greptile spotted
  this in the first-pass review.
- Add test_max_results_infinity_float_warns_and_skips covering the
  inf case.
@ChenluJi

Copy link
Copy Markdown
Contributor Author

@greptileai review

CI uses 'ruff format --check --line-length 88'; my prior format pass
used the default line length, leaving several lines unwrapped. No
behavior change — purely whitespace.
@ChenluJi

Copy link
Copy Markdown
Contributor Author

@greptileai review

CI's ruff strict-rule budget rejected the prior commit with:
- C901: transform_search_response complexity 16 > 10 (cap exceeded by 1)
- I001: import sort violation (cap exceeded by 1)

Extract two module-level helpers from transform_search_response to drop
its cyclomatic complexity:
- _default_missing_result_fields: in-place title/url/snippet defaulting
- _emit_parameter_warnings: defensive parameter_warnings reader

Auto-fix the import sort via ruff --fix.

No behavior change; the 59 existing tests still pass.
@ChenluJi

Copy link
Copy Markdown
Contributor Author

@greptileai review

@ChenluJi
ChenluJi marked this pull request as ready for review June 26, 2026 17:19
…ields

Codecov flagged 97.61% patch coverage (2 lines missing). The uncovered
lines were the non-dict raw_json and non-dict per-result item early-exits
in _default_missing_result_fields. Add two unit tests on the helper
directly to bring patch coverage to 100%.
@tin-berri
tin-berri enabled auto-merge (squash) July 2, 2026 22:35
@tin-berri
tin-berri disabled auto-merge July 2, 2026 22:47
ChenluJi added a commit to ChenluJi/litellm that referenced this pull request Jul 7, 2026
Follow-up to BerriAI#31411 (superseded and merged as BerriAI#31997). Two related fixes
so LiteLLM callers see what TinyFish actually returns.

## Top-level response extras (query, total_results, page, future fields)

transform_search_response was constructing a fresh SearchResponse from
just `results`, silently dropping every top-level extra that
SearchResponse.model_validate(raw_json) had captured via extra='allow'.
Reproduced twice against production — resp.query, resp.total_results,
resp.page all absent, model_extra empty.

Fix: spread parsed.model_extra at construction time.

    return SearchResponse(
        results=list(parsed.results[:max_results]),
        **(parsed.model_extra or {}),
    )

Future-proof: any new top-level field TinyFish adds rides through with
no LiteLLM code change (SearchResponse's extra='allow' captures every
non-declared field into parsed.model_extra automatically).

## Response headers surfaced on _hidden_params

TinyFish sets useful response headers (X-Request-ID on every response;
Retry-After and X-RateLimit-Limit on 429s). These were only accessible
via BaseLLMException.headers on error paths; on the success path they
were dropped entirely.

Fix: stash headers on both LiteLLM-conventional channels, matching the
convention used by Gemini/Volcengine/Manus/ChatGPT/OpenAI responses
providers.

    response._hidden_params['headers'] = dict(raw_response.headers)
    response._hidden_params['additional_headers'] = (
        process_response_headers(dict(raw_response.headers))
    )

- 'headers' — raw dict for debugging.
- 'additional_headers' — passed through process_response_headers so
  downstream LiteLLM consumers get sanitized keys (strips any
  x-litellm-* a misbehaving provider might set).

Future-proof: no allowlist, no filtering. Any future response header
flows through automatically.

## Tests

Adds 5 new mock-only unit tests:
- test_top_level_extras_flow_through
- test_top_level_future_extras_flow_through
- test_response_headers_stashed_on_hidden_params
- test_response_headers_future_headers_flow_through
- test_response_headers_strips_x_litellm_spoof

66 unit + integration tests pass locally. Live-tested against production
TinyFish (23/23 regression checks). No request-side changes. No behavior
change for per-result extras (already correct). Error paths unchanged.
ChenluJi added a commit to ChenluJi/litellm that referenced this pull request Jul 7, 2026
Follow-up to BerriAI#31411 (superseded and merged as BerriAI#31997). Two related fixes
so LiteLLM callers see what TinyFish actually returns.

## Top-level response extras (query, total_results, page, future fields)

transform_search_response was constructing a fresh SearchResponse from
just `results`, silently dropping every top-level extra that
SearchResponse.model_validate(raw_json) had captured via extra='allow'.
Reproduced twice against production — resp.query, resp.total_results,
resp.page all absent, model_extra empty.

Fix: spread parsed.model_extra at construction time.

    return SearchResponse(
        results=list(parsed.results[:max_results]),
        **(parsed.model_extra or {}),
    )

Future-proof: any new top-level field TinyFish adds rides through with
no LiteLLM code change (SearchResponse's extra='allow' captures every
non-declared field into parsed.model_extra automatically).

## Response headers surfaced on _hidden_params

TinyFish sets useful response headers (X-Request-ID on every response;
Retry-After and X-RateLimit-Limit on 429s). These were only accessible
via BaseLLMException.headers on error paths; on the success path they
were dropped entirely.

Fix: stash headers on both LiteLLM-conventional channels, matching the
convention used by Gemini/Volcengine/Manus/ChatGPT/OpenAI responses
providers.

    response._hidden_params['headers'] = dict(raw_response.headers)
    response._hidden_params['additional_headers'] = (
        process_response_headers(dict(raw_response.headers))
    )

- 'headers' — raw dict for debugging.
- 'additional_headers' — passed through process_response_headers so
  downstream LiteLLM consumers get sanitized keys (strips any
  x-litellm-* a misbehaving provider might set).

Future-proof: no allowlist, no filtering. Any future response header
flows through automatically.

## Tests

Adds 5 new mock-only unit tests:
- test_top_level_extras_flow_through
- test_top_level_future_extras_flow_through
- test_response_headers_stashed_on_hidden_params
- test_response_headers_future_headers_flow_through
- test_response_headers_strips_x_litellm_spoof

66 unit + integration tests pass locally. Live-tested against production
TinyFish (23/23 regression checks). No request-side changes. No behavior
change for per-result extras (already correct). Error paths unchanged.
ChenluJi added a commit to ChenluJi/litellm that referenced this pull request Jul 7, 2026
Follow-up to BerriAI#31411 (superseded and merged as BerriAI#31997). Two related fixes
so LiteLLM callers see what TinyFish actually returns.

## Top-level response extras (query, total_results, page, future fields)

transform_search_response was constructing a fresh SearchResponse from
just `results`, silently dropping every top-level extra that
SearchResponse.model_validate(raw_json) had captured via extra='allow'.
Reproduced twice against production — resp.query, resp.total_results,
resp.page all absent, model_extra empty.

Fix: spread parsed.model_extra at construction time.

    return SearchResponse(
        results=list(parsed.results[:max_results]),
        **(parsed.model_extra or {}),
    )

Future-proof: any new top-level field TinyFish adds rides through with
no LiteLLM code change (SearchResponse's extra='allow' captures every
non-declared field into parsed.model_extra automatically).

## Response headers surfaced on _hidden_params

TinyFish sets useful response headers (X-Request-ID on every response;
Retry-After and X-RateLimit-Limit on 429s). These were only accessible
via BaseLLMException.headers on error paths; on the success path they
were dropped entirely.

Fix: stash headers on both LiteLLM-conventional channels, matching the
convention used by Gemini/Volcengine/Manus/ChatGPT/OpenAI responses
providers.

    response._hidden_params['headers'] = dict(raw_response.headers)
    response._hidden_params['additional_headers'] = (
        process_response_headers(dict(raw_response.headers))
    )

- 'headers' — raw dict for debugging.
- 'additional_headers' — passed through process_response_headers so
  downstream LiteLLM consumers get sanitized keys (strips any
  x-litellm-* a misbehaving provider might set).

Future-proof: no allowlist, no filtering. Any future response header
flows through automatically.

## Tests

Adds 5 new mock-only unit tests:
- test_top_level_extras_flow_through
- test_top_level_future_extras_flow_through
- test_response_headers_stashed_on_hidden_params
- test_response_headers_future_headers_flow_through
- test_response_headers_strips_x_litellm_spoof

66 unit + integration tests pass locally. Live-tested against production
TinyFish (23/23 regression checks). No request-side changes. No behavior
change for per-result extras (already correct). Error paths unchanged.
ChenluJi added a commit to ChenluJi/litellm that referenced this pull request Jul 8, 2026
Follow-up to BerriAI#31411 (superseded and merged as BerriAI#31997). Two related fixes
so LiteLLM callers see what TinyFish actually returns.

## Top-level response extras (query, total_results, page, future fields)

transform_search_response was constructing a fresh SearchResponse from
just `results`, silently dropping every top-level extra that
SearchResponse.model_validate(raw_json) had captured via extra='allow'.
Reproduced twice against production — resp.query, resp.total_results,
resp.page all absent, model_extra empty.

Fix: spread parsed.model_extra at construction time.

    return SearchResponse(
        results=list(parsed.results[:max_results]),
        **(parsed.model_extra or {}),
    )

Future-proof: any new top-level field TinyFish adds rides through with
no LiteLLM code change (SearchResponse's extra='allow' captures every
non-declared field into parsed.model_extra automatically).

## Response headers surfaced on _hidden_params

TinyFish sets useful response headers (X-Request-ID on every response;
Retry-After and X-RateLimit-Limit on 429s). These were only accessible
via BaseLLMException.headers on error paths; on the success path they
were dropped entirely.

Fix: stash headers on both LiteLLM-conventional channels, matching the
convention used by Gemini/Volcengine/Manus/ChatGPT/OpenAI responses
providers.

    response._hidden_params['headers'] = dict(raw_response.headers)
    response._hidden_params['additional_headers'] = (
        process_response_headers(dict(raw_response.headers))
    )

- 'headers' — raw dict for debugging.
- 'additional_headers' — passed through process_response_headers so
  downstream LiteLLM consumers get sanitized keys (strips any
  x-litellm-* a misbehaving provider might set).

Future-proof: no allowlist, no filtering. Any future response header
flows through automatically.

## Tests

Adds 5 new mock-only unit tests:
- test_top_level_extras_flow_through
- test_top_level_future_extras_flow_through
- test_response_headers_stashed_on_hidden_params
- test_response_headers_future_headers_flow_through
- test_response_headers_strips_x_litellm_spoof

66 unit + integration tests pass locally. Live-tested against production
TinyFish (23/23 regression checks). No request-side changes. No behavior
change for per-result extras (already correct). Error paths unchanged.
ChenluJi added a commit to ChenluJi/litellm that referenced this pull request Jul 8, 2026
Follow-up to BerriAI#31411 (superseded and merged as BerriAI#31997). Two related fixes
so LiteLLM callers see what TinyFish actually returns, plus small
correctness cleanups.

## Response headers surfaced on _hidden_params

TinyFish sets useful response headers (x-request-id on every response,
retry-after and x-ratelimit-limit on 429s). Previously these were only
accessible via BaseLLMException.headers on error paths; on the success
path they were dropped entirely.

Fix: stash headers on both LiteLLM-conventional channels, matching the
pattern used by Gemini / Volcengine / Manus / ChatGPT / OpenAI-responses
providers.

- `_hidden_params["headers"]` -- raw dict from httpx, all keys lowercased.
- `_hidden_params["additional_headers"]` -- passed through
  process_response_headers, which prefixes any x-litellm-* provider
  header with `llm_provider-` so downstream LiteLLM code that trusts
  bare x-litellm-* markers can't be spoofed (values still survive
  under the prefixed key for observability).

## Top-level response extras (query, total_results, page, future fields)

transform_search_response was building a fresh SearchResponse from just
`results`, silently dropping every top-level field TinyFish's response
carries beyond `results` / `object`.

Fix: mutate parsed.results to its truncated slice and return the same
SearchResponse instance rather than reconstructing. Every field pydantic
populated during model_validate -- declared attributes AND extras
(query, total_results, page, parameter_warnings, and any future TinyFish
additions) -- survives regardless of which storage bucket holds it.
Robust against upstream schema evolution: if LiteLLM later promotes a
field from extras to declared, this code needs no change.

## Code cleanup

- List-valued custom params JSON-encoded on the wire (matching the
  existing dict handling), so callers can pass a natural Python list
  for JSON-array wire params.
- URL-encodable-params adapter accepts float in addition to
  str / int / bool; server-side rejection of a wrong-typed float now
  surfaces cleanly with `TinyFish Search:` attribution + docs link.
- Assorted comment / docstring / test-fixture hygiene (no logic changes).

## Tests

70 unit + integration tests pass locally. Live-tested against
production TinyFish with 6 diverse queries (basic / max_results /
country=US / language=ja / domain filter / fetch={"format":"html"}) --
all 6 pass every expected-behavior check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants