Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generators: update openai for o1 models #922

Merged
merged 10 commits into from
Sep 23, 2024
Merged

generators: update openai for o1 models #922

merged 10 commits into from
Sep 23, 2024

Conversation

leondz
Copy link
Collaborator

@leondz leondz commented Sep 20, 2024

The OpenAI "o1" reasoning models have different API expectations from other Chat API models, in terms of params and defaults, and so a new class is added to account for these, and some switching hints placed in the code.

Verification

  • List the steps needed to make sure this thing works
export OPENAI_API_KEY=
python -m pytest tests/generators/test_openai.py
python -m garak -m openai -n o1-mini -p test.Test -g1
python -m garak -m openai.OpenAIReasoningGenerator -n o1-mini -p test.Test -g1
python -m garak -m openai.OpenAIReasoningGenerator -n o1-mini -p test.Test --parallel_requests 10

@leondz leondz added the generators Interfaces with LLMs label Sep 20, 2024
Copy link
Collaborator

@jmartin-tech jmartin-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested refactor is untested, let me know if the pattern does not seem reasonable.

garak/generators/openai.py Outdated Show resolved Hide resolved
garak/generators/openai.py Outdated Show resolved Hide resolved
garak/generators/openai.py Outdated Show resolved Hide resolved
garak/generators/openai.py Outdated Show resolved Hide resolved
garak/generators/openai.py Outdated Show resolved Hide resolved
tests/generators/test_openai.py Show resolved Hide resolved
@jmartin-tech
Copy link
Collaborator

To support o1 model this also needs to update requirements.txt and pyproject.toml to openai>=1.45.0,<2 as the new max_completion_tokens parameter was added in 1.45.0

garak/generators/openai.py Outdated Show resolved Hide resolved
garak/generators/openai.py Outdated Show resolved Hide resolved
@leondz
Copy link
Collaborator Author

leondz commented Sep 23, 2024

thanks, missed that one

@jmartin-tech jmartin-tech merged commit bcd5b69 into main Sep 23, 2024
10 checks passed
@jmartin-tech jmartin-tech deleted the update/openai_o1 branch September 23, 2024 19:51
@github-actions github-actions bot locked and limited conversation to collaborators Sep 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
generators Interfaces with LLMs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants