Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Gemini bug when system role is provided #1048

Merged
merged 2 commits into from
Dec 29, 2024

Conversation

winsonluk
Copy link
Contributor

Gemini doesn't support role=system.

When "SMART_LLM": "google_vertexai:gemini-1.5-flash-002" for example, the current report generation fails with "Error in generate_report".

This fix adds a fallback. It first attempts the original chat completion:

            messages=[                                                                              
                {"role": "system", "content": f"{agent_role_prompt}"},                              
                {"role": "user", "content": content},                                               
            ],

Upon failure, it retries with this new chat completion:

                messages=[                                                                          
                    {"role": "user", "content": f"{agent_role_prompt}\n\n{content}"},               
                ],

This results in a successful report generation when "SMART_LLM": "google_vertexai:gemini-1.5-flash-002".

@winsonluk
Copy link
Contributor Author

@assafelovic short fix for when SMART_LLM fails with Gemini, thanks

Copy link
Owner

@assafelovic assafelovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this!

@assafelovic assafelovic merged commit eae2cde into assafelovic:master Dec 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants