Skip to content

chore: Enable more ruff rules (list taken from any-agnet) and fix err… - #123

Merged
njbrake merged 5 commits into
mainfrom
more-ruff-rules
Aug 11, 2025
Merged

njbrake merged 5 commits into
mainfrom
more-ruff-rules

Conversation

@daavoo

@daavoo daavoo commented Aug 11, 2025

Copy link
Copy Markdown
Contributor

…ors.

@daavoo
daavoo requested a review from njbrake August 11, 2025 10:45
@codecov

codecov Bot commented Aug 11, 2025 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 71.25000% with 92 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/any_llm/providers/cerebras/cerebras.py 41.17% 10 Missing ⚠️
src/any_llm/providers/cohere/utils.py 10.00% 9 Missing ⚠️
src/any_llm/tools.py 42.85% 7 Missing and 1 partial ⚠️
src/any_llm/api.py 40.00% 4 Missing and 2 partials ⚠️
src/any_llm/providers/cohere/cohere.py 57.14% 6 Missing ⚠️
src/any_llm/providers/openai/base.py 62.50% 4 Missing and 2 partials ⚠️
src/any_llm/providers/azure/utils.py 16.66% 5 Missing ⚠️
src/any_llm/providers/together/utils.py 28.57% 5 Missing ⚠️
src/any_llm/provider.py 78.94% 4 Missing ⚠️
src/any_llm/providers/anthropic/utils.py 55.55% 4 Missing ⚠️
... and 12 more
Files with missing lines Coverage Δ
src/any_llm/__init__.py 100.00% <100.00%> (ø)
src/any_llm/exceptions.py 100.00% <ø> (ø)
src/any_llm/providers/anthropic/anthropic.py 88.88% <100.00%> (+0.20%) ⬆️
src/any_llm/providers/aws/aws.py 86.36% <100.00%> (+0.20%) ⬆️
src/any_llm/providers/deepseek/deepseek.py 65.00% <100.00%> (+1.84%) ⬆️
src/any_llm/providers/fireworks/fireworks.py 42.22% <100.00%> (ø)
src/any_llm/providers/fireworks/utils.py 20.00% <100.00%> (ø)
src/any_llm/providers/groq/utils.py 15.00% <100.00%> (ø)
src/any_llm/providers/huggingface/huggingface.py 71.73% <100.00%> (ø)
src/any_llm/providers/lmstudio/lmstudio.py 100.00% <100.00%> (ø)
... and 50 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@njbrake
njbrake marked this pull request as ready for review August 11, 2025 16:44
@njbrake
njbrake merged commit e719132 into main Aug 11, 2025
@njbrake
njbrake deleted the more-ruff-rules branch August 11, 2025 17:08
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->
It was supported all along, turns out it was my API key that didn't have
enough permissions, which caused the initial setting it to False

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->
Added `list_models` method to `OllamaProvider` for fetching available
models.


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
Convert mistral list_models to the any-llm spec


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
)

## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 20, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 21, 2025
## Description
Google doesn't have the max_tokens param


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 21, 2025
## Description
Demo chat interface to show how easily to toggle between providers and
models


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 21, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

📚 Documentation

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake pushed a commit that referenced this pull request Aug 22, 2025
## Description

Migrate AWS provider to async.

NOTE: boto3 doesn't support async yet, so following guidance from
https://stackoverflow.com/questions/78348036/how-to-call-aws-bedrock-asynchronously/79452953#79452953


PR Type: 🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

Fixes #202

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 22, 2025
## Description

- Now only async implementations
- Added wrapper to turn asynciterable into synciterable
- Removed the sync integration tests
- switched all the tests unit and integration over to using async 


## PR Type

<!-- Delete the types that don't apply --!>
💅 Refactor

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Aug 26, 2025
## Description
Better parsing for model id string


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Sep 2, 2025
## Description
<!-- What does this PR do? -->
In order to reduce confusion, any-llm won't prevent people from making a
call to mistral with a tool message with no assistant message before it.
Then, if Mistral throws an exception it is clear that it is not any-llm
that has the limitation

## PR Type

<!-- Delete the types that don't apply --!>

🐛 Bug Fix
💅 Refactor

## Relevant issues

<!-- e.g. "Fixes #123" -->
Relates to mozilla-ai/any-agent#787

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Sep 4, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

💅 Refactor

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Sep 4, 2025
## Description
<!-- What does this PR do? -->
Sometime you only make a change for one provider and you want to run the
integration test just on that feature or provider.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Sep 5, 2025
## Description
<!-- What does this PR do? -->
We had the "google" provider grouping both gemini and vertex ai. This PR
splits them into being separate providers

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
💅 Refactor

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Sep 20, 2025
…s to complete (#487)

## Description
Fix the error when running tasks in threads

## PR Type

<!-- Delete the types that don't apply --!>

🐛 Bug Fix

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 16, 2025
- Updated the CohereProvider to support reasoning in completion
responses.
- Modified the response conversion methods to handle V2ChatResponse and
reasoning content.

## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 16, 2025
## Description
<!-- What does this PR do? -->
Azure requires the JsonSchemaFormat in all cases, it doesn't accept json

## PR Type

<!-- Delete the types that don't apply --!>

🐛 Bug Fix

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```

---------

Co-authored-by: David de la Iglesia Castro <daviddelaiglesiacastro@gmail.com>
njbrake added a commit that referenced this pull request Oct 16, 2025
## Description
<!-- What does this PR do? -->
The base openai `_normalize_reasoning_on_message` already handles
Deepseek, all we needed to do is flip it on and add the test

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 16, 2025
- Updated the CerebrasProvider to support completion reasoning.
- Modified response conversion methods to include reasoning content.
- Added Cerebras model mapping in tests for improved compatibility.

## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 16, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
While we wait for keys, YOLO and hopefully someone in the community
using databricks can help us with whether this works or not!

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
Works with our existing logic!

To set up llama cpp I did

```bash
 brew install llama.cpp
llama-server -hf ggml-org/Qwen3-1.7B-GGUF --jinja --port 8090
```

Make sure you don't have LLAMA_API_KEY set, otherwise it'll add auth to
the llama cpp server which we don't want.
That's an annoying thing, the Meta Llama API and Llama.cpp share the
same API key name.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
Llamafile doesn't do anything to parse out the reasoning content, so I
put a little post processing to look for the xml tags.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
Fireworks team is actively working to fix the issue, once they fix it,
running the integration tests from this branch should pass.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
Sambanova is another provider that doesn't do any preprocessing of their
thinking content, which makes extracting the reasoning content of a
streaming response a bit tricky. Refactored a bit so that huggingface
and sambanova share some logic, since they're both in the same
predicament.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->
Until we have a clear indication that support for watsonx is needed by a
user who can help us debug the intermittent rate limiting and model
failures, disabling them to avoid having distracting and irrelevant CI
test failures.

## PR Type

<!-- Delete the types that don't apply --!>

🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 17, 2025
…pis (#568)

## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

📚 Documentation

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 20, 2025
## Description
<!-- What does this PR do? -->
Portkey docs show that it supports returning:


https://portkey.ai/docs/product/ai-gateway/multimodal-capabilities/thinking-mode#understanding-response-format

I need to make some changes to support and test this but I'm waiting for
our portkey account to support a model that outputs thinking, that way I
can edit the portkey response as needed.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 22, 2025
## Description
<!-- What does this PR do? -->
hf expects max_tokens and no longer accepts max_new_tokens...

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 22, 2025
## Description
<!-- What does this PR do? -->
Integration tests didn't like my max_tokens setting
nthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error':
{'type': 'invalid_request_error', 'message': '`max_tokens` must be
greater than `thinking.budget_tokens`

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 22, 2025
## Description
<!-- What does this PR do? -->
Getting integration tests happy is quite the task.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 22, 2025
## Description
<!-- What does this PR do? -->
Found that gemini/vertexai didn't have usage data being added to their
completionchunks.

## PR Type

<!-- Delete the types that don't apply --!>

🐛 Bug Fix

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 29, 2025
## Description
<!-- What does this PR do? -->
Initial support for batch. This PR implements only openai batch, but
sets up repo for more implementations. I added an experimental decorator
to make it clear that the Batch API may need a few changes to the
interface as we expand the implementations and usage.

The `any-llm` batch API requires you to pass a **path to a local JSONL
file** containing your batch requests. The provider implementation
automatically handles uploading and file management as needed.

Different providers handle batch processing differently:

- **OpenAI**: Requires uploading a file first, then creating a batch
with the file ID
- **Anthropic** (future): Expects file content passed directly in the
request
- **Other providers**: May have their own unique requirements

By accepting a local file path, `any-llm` abstracts these provider
differences and handles the implementation details automatically.


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 30, 2025
## Description
<!-- What does this PR do? -->
(untested) Eventually I think we would be able to add it to the
integration test suite, once the gateway is a public image so that we
can easily run it in a github action without needing to configure
authentication etc.

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```

---------

Co-authored-by: David de la Iglesia Castro <daviddelaiglesiacastro@gmail.com>
njbrake added a commit that referenced this pull request Oct 30, 2025
## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [x] I have added unit tests that prove my fix/feature works
- [x] New and existing tests pass locally
- [x] Documentation was updated where necessary
- [x] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 30, 2025
## Description
<!-- What does this PR do? -->
(Waiting for an API key to verify behavior)

## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 30, 2025
#590)

… about how people use databricks

## Description
<!-- What does this PR do? -->


## PR Type

<!-- Delete the types that don't apply --!>

🆕 New Feature
🐛 Bug Fix
💅 Refactor
📚 Documentation
🚦 Infrastructure

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
njbrake added a commit that referenced this pull request Oct 30, 2025
#589)

## Description

Claude Code helped me with this refactor and the nice explanation. I've
never used the github 'actions' folder before but this seems like a nice
way to organize the logic to avoid the workflow file blowing up. I also
opted to put it into the action.yml instead of elevating it to a
separate shell script because it seemed like logic that was only
relevant for github actions, and wouldn't be used by people outside of
that env.

<!-- What does this PR do? -->
## Problem

The GitHub Actions workflow filtering logic was causing integration
tests to be skipped incorrectly when using the `filter` input parameter
with `workflow_dispatch`.

### Root Cause

The workflow used simple substring matching with `contains()` to
determine whether jobs should run:

```yaml
if: github.event.inputs.filter == '' || contains(needs.expected-providers.outputs.providers, github.event.inputs.filter)
```

This approach had a critical issue: **test function filters were
broken**. When passing pytest test filters like `"test_completion"` or
`"test_streaming"`, the jobs would be skipped entirely because these
strings don't exist in the provider list. The jobs would skip before
pytest ever had a chance to run.

The workflow conflated two different types of filtering:
- **Job-level**: "Should we set up infrastructure for these providers?"
- **pytest-level**: "Which specific test functions should run?"

### Example Failure Scenario

```bash
# User wants to run all completion tests across all providers
gh workflow run tests-integration.yaml --field filter="test_completion"
```

**Expected**: Both test jobs run, pytest filters to only completion
tests
**Actual**: Both jobs skip because
`contains("anthropic,bedrock,...,minimax", "test_completion")` returns
`false`

## Solution

Created a reusable composite action (`.github/actions/determine-jobs`)
that intelligently analyzes the filter to determine which test jobs
should run:

1. **Provider matching**: Checks if the filter contains any provider
name from either the integration or local provider lists

2. **Smart fallback**: If the filter doesn't match any provider name,
assumes it's a test function filter and runs all relevant jobs, letting
pytest handle the filtering
   - `"test_completion"` → runs both jobs, pytest filters the tests ✅
   - `"test_streaming"` → runs both jobs, pytest filters the tests ✅

3. **Provider-specific optimization**: When a provider is explicitly
named, only the relevant job runs
   - `"minimax"` → only runs integration tests job ✅
   - `"ollama"` → only runs local tests job ✅

## Testing Scenarios

| Filter Input | Integration Job | Local Job | Behavior |
|-------------|----------------|-----------|----------|
| _(empty)_ | ✅ Runs | ✅ Runs | All tests |
| `minimax` | ✅ Runs | ❌ Skips | Only minimax tests |
| `ollama` | ❌ Skips | ✅ Runs | Only ollama tests |
| `test_completion` | ✅ Runs | ✅ Runs | All completion tests |
| `test_streaming and minimax` | ✅ Runs | ❌ Skips | Minimax streaming
tests |
| `llama` | ✅ Runs | ❌ Skips | All llama provider tests |

## Changes

- **Created** `.github/actions/determine-jobs/action.yml` - Composite
action with smart filter analysis logic
- **Updated** `determine-jobs-to-run` job to use the new composite
action
- **Updated** `run-integration-tests` job condition (already correctly
configured)
- **Updated** `run-local-integration-tests` job condition (already
correctly configured)

### Benefits

- ✅ pytest test function filters now work correctly
- ✅ Provider name filters work correctly
- ✅ Jobs only run when they need to, saving CI resources
- ✅ Logic is extracted into a reusable, testable composite action
- ✅ Workflow YAML is cleaner and easier to maintain
- ✅ The workflow is more intuitive and matches user expectations

## PR Type

<!-- Delete the types that don't apply --!>

🐛 Bug Fix

## Relevant issues

<!-- e.g. "Fixes #123" -->

## Checklist
- [ ] I have added unit tests that prove my fix/feature works
- [ ] New and existing tests pass locally
- [ ] Documentation was updated where necessary
- [ ] I have read and followed the [contribution
guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md)```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants