This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Add Google Gemini API Support #105
Comments
|
import os llm = ChatGoogleGenerativeAI(model="gemini-pro",verbose = True,temperature = 0.1,google_api_key="GEMINI-API-KEY") Define your agents with roles and goalsresearcher = Agent( writer = Agent( task1 = Task( task2 = Task( Instantiate your crew with a sequential processcrew = Crew( Get your crew to work!result = crew.kickoff() print("######################") ##Thank you for the update |
|
@janda-datascience Thank you so much! Really excited to get CrewAI working with Google Gemini. I ran into one issue with your code, it appers /crewai/agent.py is still looking for openAI and its key for its functions and won't run without an OpenAI key with the error: "Did not find openai_api_key, please add an environment variable I assume settings need to be changed in /crewai/agent.py to point to the Gemini Pro API instead of OpenAI for all agent functions, so it doesn't bill the OpenAI API? It looks like this is the section, could you please help provide the proper updates to agent.py . In contrast, If I just add the OpenAI API, it start running agent tasks and billing me on openai api. Really appreciate your help. agent.py section to modify? (Lines 57-60 in agent.py) llm: Optional[Any] = Field( Error: agent.py calling for OpenAI still: File "/home/aivirtual/miniconda3/envs/crewai/lib/python3.10/site-packages/langchain_core/load/serializable.py", line 107, in init |
|
@mindwellsolutions by default the agent uses gpt-4, if you want to use your gemini model you can provide the gemini llm inside the agent definition: eg: |
|
@punitchauhan771 Thank you so much, that worked! It successfully runs CrewAI completely free with Gemini Pro API! Going to spend some time testing it out. Really appreciate the help here. I had to add one small update to the solution punitchauhan provided to get it working, thought I'd share it with everyone. You need to define the google_api_key directly within the llm = line of code. (Working Update to llm = code for gemini api key) Last Question: Google Gemini API (Free) has a limit of 60 API uses per minute. Is there a way to throttle the agent activity, so it doesn't perform more than 59 API requests per minute, so we can avoid going over free limits when dispatching large agent swarms? Really appreciate everything, ty. |
|
Hi @mindwellsolutions, at the end of the agent activity it will give you some response like : hope this helps. |
|
@punitchauhan771 Thank you and everyone for the help, such an amazing community. This would help a lot, but I'm using Google Gemini API rather than OpenAI with from langchain_google_genai import ChatGoogleGenerativeAI. Is there a callback for ChatGoogleGenerativeAI similar to the OpenAI callback you posited "from langchain.callbacks import get_openai_callback". Really appreciate the help. |
|
Hi @mindwellsolutions the solution I provided works for Gemini as well,though it doesn't count tokens 🙂. |
|
@punitchauhan771 Thanks again. Unfortunately using your solution after 2 agents complete all their tasks successfully with Gemini API its still showing "Successful Request: 0". As you mentioned simply being able to monitor the amount of Successful Requests is all Gemini Pro (Free Tier) needs since there are no costs to track associated with token usage :) The Output I get after Gemini clearly performs multiple tasks: |
|
@mindwellsolutions if possible can you provide me the code snippet? |
|
@punitchauhan771 Actually. I closed the terminal window in VS Code and ran it again and it worked perfect. Really appreciate all your help, you've gotten everything running perfectly for me. Ty |
|
The performance of the Gemini Pro model using tools is not good. There are big problems with the logic and results of using the tools, and the final results are often hallucinatory. |
|
@edisonzf2020 Thanks for your comment. I got time to test out gemini api in crewai further over the weekend and as you mentioned it seems to be having issues using tools like duckduckgosearch. I tested openai API and Ollama (zephyr) and those both worked perfectly, while it looks like Gemini is pulling responses from its internal knowledgebase rather than relaying the data from DuckDuckGo. Are there any possible fixes moving forward to get Gemini to play well with CrewAI? The scalability of using a free API for personal use will provide incredible power of scalability without having to run local models on local GPU resources. |
|
Hey folks, catching up to this issue! great comments, so glad you were able to get Gemini working, we are adding new docs that will have instructions for all the major models so stay tuned for that. I'll do some testing with Gemini models specifically to see how we could make that better! |
|
@joaomdmoura Thank you. This is incredibly appreciated. We realized that Gemini API has significant problems currently using CrewAI's tools like DuckDuckGoSearch. It appears none of the data from the tools make it back to Gemini, and Gemini always generates answers from its internal knowledge base rather than the research done by the tool. If there is a way to fix this so Gemini works like other LLMs that would be amazing. I've been using Zephyr 7b as a local model and that runs well, but the power of decent size scalability of swarms that would require significant power to run locally, can be run from any device using Gemini's free API - which is going to be extremely valuable. Especially if each Agent is assigned it's own Free Gemini API key in concert with each other, the ability to built large API swarms for free will be significant. Thanks for all that you do. Really loving the functionality of CrewAI. |
Hi @mindwellsolutions, response: when i tried google search the response was: and when I tried duckduckgoSearch the response was: also when i tried the same using langchain agent, the response were same, but when i just used the basic llm, the response was: Also it doesn't allow questionable prompt as mentioned in the gemini doc: |
|
@punitchauhan771 thanks for your detailed research. The test I did that leads me to believe its using internal knowledge, is when I ask it to only research information from the year 2024 for any topic with Google Gemini Pro Free with Duckduckgosearch I get the response "I am sorry, but I cannot find any information on CrewAI in the context of LLM agents specifically from the year 2024. My access to data is limited to information available up until April 2023. I recommend checking more up-to-date sources or reaching out to CrewAI directly for more information." If I run the same CrewAI script using an OpenAI Key or Zephyr 7b as a local model, it does relay only 2024 research back using duckduckgo. Duckduckgosearch does run as an "Action:", so it appears there is still an issue with Gemini where it cannot relay the information found from DuckDuckgoSearch back to Gemini, and Gemini resorts to its internal knowledge up to April 2023. Full Example of my Test: I used your exact Researcher Agent settings you posted above, but modified your task to mention only research from 2024 as the only change. "research on crewai in context of llm agents only from 2024!". You could do the same for "research new breakthroughs in AI llm agents only from 2024!" and it will show the april 2023 cutoff etc.. (See second code area below) 2024 general AI breakthroughs only task = 'research on AI breakthroughs only from 2024!' |
|
Hi @mindwellsolutions, response: and when I used duckduckgo search: response: and when I used your prompt : research new breakthroughs in AI llm agents only from 2024! response: I am not sure, but I believe the agent hallucinates and believes it does not have internet access, I ran the query multiple times till I got the response that I wanted. |
|
Hey! Sincerely, Eddie |
|
@punitchauhan771 Interesting. Really appreciate all the help. I wonder if I'm setting up the duckduckgo_search function properly. In your DuckDuckGo response, it says "In 2024, breakthroughs in AI LLMs are expected to" and only references 2023 sources, which seems like the April 2023 KB still. Maybe it's specifically an error with DuckDuckGo & Gemini API. Where I would need to get BrowserTools.xx and SearchTools.xx working instead of duckduck. DuckDuckGo_Search Error: "I am sorry, but I do not have access to real-time information and my knowledge cutoff is April 2023. Therefore, I cannot provide you with a detailed tech report for CES 2024 hosted in Las Vegas" @punitchauhan771 Could you please share your entire .py code including the imports, please. I'm also having trouble running BrowserTools.xx and SearchTools.xx. The script couldn't find "BrowserTools" nor "SearchTools". I must be setting up the imports and functions incorrectly. "NameError: name 'BrowserTools' is not defined", "NameError: name 'SearchTools' is not defined" Would be amazing if you could share your entire py file with support for 1) Duckduckgosearch 2) BrowserTools.scrape_and_summarize_website 3) SearchTools.search_internet 4) SearchTools.search_news Thanks in advance :) Here is my Current Code For Reference: |
|
Hi @mindwellsolutions, importing necessary modulesstoring all the necessary keysconfiguring gemini LLMCustom toolscreating agents and crewyou can get more tools code in crew-ai example codebase. |
|
Hi @FeelsDaumenMan, you can use @janda-datascience code 🙂. |
You leaked ur api key |
|
@FeelsDaumenMan Thanks. I removed it, can you delete your quote that also has the key. @punitchauhan771 |
|
@mindwellsolutions, |
|
@FeelsDaumenMan I ran into the same issue on Windows. Don't import DuckDuck from langchain, instead install duckduckgo_search directly. Then remove "from langchain_community.tools import DuckDuckGoSearchRun" from your code. How to best Install duckduckgo_search for CrewAI: |
|
@punitchauhan771 Thought of a simple solution to get around the glitch where every couple API requests gemini gets the no internet connection error, even though duckduckgo_search completes properly. If CrewAI, when using Gemini prints "I apologize, but I do not have access to real-time information or the ability to browse the internet" then CrewAI should automatically reprocess the task (hopefully where it left off after the successful duckduckgo_search response it can't find. Until it doesn't receive that error. This will automatically rerun the LLM's processing of duckduckgo's research until it works properly. |
hmmm, weird. Tried it all. Could you send your exact code that worked on windows? Would appreciate it. |
|
@FeelsDaumenMan , I ran into the same error and should be able to trace back the steps of how I resolved if with that error details. Here is a simple crewai python script that works for me with gemini. Although, every 3-4 times it says it can't find internet - just run it again. If its still an issue. Can you share the full errors you are getting with "from langchain_community.tools" saying community tools is depreciated again. I can't find them. |
how do you create the SearchTools and BrowserTools? |
|
I can run Define Agentsemail_author = Agent( content_specialist = Agent( Define Taskemail_task = Task( Create a Single Crewemail_crew = Crew( Execution Flowprint("Crew: Working on Email Task") I am getting the error langchain_google_genai.chat_models:Retrying langchain_google_genai.chat_models._chat_with_retry.._chat_with_retry in 2.0 seconds as it raised FailedPrecondition: 400 User location is not supported for the API use.. |
Hi @souvikcs , |
|
You could also just use this new proxy-solution to do it :) (i didn't test it, please report success, if you try it) |
|
I got Google Gemini Pro API working substantially better than before thanks to joaomdmoura's post on another issue thread that provides a new DuckDuckGoSearch function that seems to play better with gemini pro api now. However the issue with Gemini-Pro Free API is google throttles the usage and queries based on their server load. This means that any query or string of queries that is sent to gemini-pro appears to have the potential to be denied with a non explicit or 504/503 error from google, which in turn fails to process portions of the crewai sequences, thus creates issues. It may be a limitation on the free version google has, that dictates the inconsistency that gemini-pro api responds to queries. How to use Gemini-Pro API with CrewAI & new DuckDuckGoSearch Function: You will need to likely reinstall newest version of crewai from scratch: Fix: New better functioning code using crewai_tools version of duckduckgosearch Full Code: gemini-pro API with CrewAI (Basic Crew AI Code Example) |
|
I am using this code: import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import tool
from langchain_community.tools import DuckDuckGoSearchRun
from langchain.callbacks import get_openai_callback
from langchain.agents import load_tools
# pip install --upgrade --quiet langchain-google-genai
from langchain_google_genai import ChatGoogleGenerativeAI
os.environ["GOOGLE_API_KEY"] = ""
os.environ["LANGCHAIN_API_KEY"] = ""
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_PROJECT"] = "Human Feed-Back Gemini"
llm = ChatGoogleGenerativeAI(
model="gemini-pro",
convert_system_message_to_human=True,
verbose=True,
temperature=0.1,
)
# Loading Tools
human_tools = load_tools(["human"], llm=llm)
# search_tool = DuckDuckGoSearchRun()
@tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchRun().run(search_query)
# Define your agents with roles, goals, and tools
researcher = Agent(
role="Senior Research Analyst",
goal="Uncover cutting-edge developments in AI and data science",
backstory=(
"You are a Senior Research Analyst at a leading tech think tank."
"Your expertise lies in identifying emerging trends and technologies in AI and data science."
"You have a knack for dissecting complex data and presenting actionable insights."
),
verbose=True,
allow_delegation=False,
tools=[search_tool] + human_tools, # Passing human tools to the agent
llm=llm,
)
writer = Agent(
role="Tech Content Strategist",
goal="Craft compelling content on tech advancements",
backstory=(
"You are a renowned Tech Content Strategist, known for your insightful and engaging articles on technology and innovation."
"With a deep understanding of the tech industry, you transform complex concepts into compelling narratives."
),
verbose=True,
allow_delegation=True,
llm=llm,
)
# Create tasks for your agents
task1 = Task(
description=(
"Conduct a comprehensive analysis of the latest advancements in AI in 2024."
"Identify key trends, breakthrough technologies, and potential industry impacts."
"Compile your findings in a detailed report."
"Make sure to check with a human if the draft is good before finalizing your answer."
),
expected_output="A comprehensive full report on the latest AI advancements in 2024, leave nothing out",
agent=researcher,
)
task2 = Task(
description=(
"Using the insights from the researcher's report, develop an engaging blog post that highlights the most significant AI advancements."
"Your post should be informative yet accessible, catering to a tech-savvy audience."
"Aim for a narrative that captures the essence of these breakthroughs and their implications for the future."
),
expected_output="A compelling 3 paragraphs blog post formatted as markdown about the latest AI advancements in 2024",
agent=writer,
)
# Instantiate your crew with a sequential process
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2,
process=Process.sequential,
)
# Get your crew to work!
with get_openai_callback() as cb:
result = crew.kickoff()
print(result)
print(cb)And I get this error. The without human tools, I got this error: What is the reason for the DuckDuckGoSearch error search_tool = DuckDuckGoSearchRun()and @tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchRun().run(search_query)Also, Is there a way to use human tools with Gemini? Thanks. |
Hi @OmarAlsaqa Did you try increasing the temperature? |
|
Thanks @punitchauhan771 for your reply, I tried different temperatures with the same problem. I got the same issue with ChatGPT 4.0 today, maybe I am doing something wrong or there is an issue with DuckDuckGo Api. I used it before but a while ago and it was working with GPT 4.0 and crewAI. |
|
It worked, after updating pip install --upgrade --quiet duckduckgo-searchAlso, human tools are working with it, with the same code. |
|
Using The full code to use Gemini with human input: import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import tool
from langchain_community.tools import DuckDuckGoSearchRun, HumanInputRun
from langchain_community.callbacks import get_openai_callback
from langchain_google_genai import ChatGoogleGenerativeAI
os.environ["GOOGLE_API_KEY"] = ""
os.environ["LANGCHAIN_API_KEY"] = ""
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_PROJECT"] = "Human Feed-Back Gemini"
llm = ChatGoogleGenerativeAI(
model="gemini-pro",
convert_system_message_to_human=True,
verbose=True,
temperature=0.8,
)
# Loading Tools
@tool('HumanInputTool')
def human_input_tool(query: str):
"""Human Input as a tool"""
return HumanInputRun().run(query)
@tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchRun().run(search_query)
# Define your agents with roles, goals, and tools
researcher = Agent(
role="Senior Research Analyst",
goal="Uncover cutting-edge developments in AI and data science",
backstory=(
"You are a Senior Research Analyst at a leading tech think tank."
"Your expertise lies in identifying emerging trends and technologies in AI and data science."
"You have a knack for dissecting complex data and presenting actionable insights."
),
verbose=True,
allow_delegation=False,
tools=[search_tool, human_input_tool], # Passing human tools to the agent
llm=llm,
)
writer = Agent(
role="Tech Content Strategist",
goal="Craft compelling content on tech advancements",
backstory=(
"You are a renowned Tech Content Strategist, known for your insightful and engaging articles on technology and innovation."
"With a deep understanding of the tech industry, you transform complex concepts into compelling narratives."
),
verbose=True,
allow_delegation=True,
llm=llm,
)
# Create tasks for your agents
task1 = Task(
description=(
"Conduct a comprehensive analysis of the latest advancements in AI in 2024."
"Identify key trends, breakthrough technologies, and potential industry impacts."
"Compile your findings in a detailed report."
"Make sure to check with a human if the draft is good before finalizing your answer."
),
expected_output="A comprehensive full report on the latest AI advancements in 2024, leave nothing out",
agent=researcher,
)
task2 = Task(
description=(
"Using the insights from the researcher's report, develop an engaging blog post that highlights the most significant AI advancements."
"Your post should be informative yet accessible, catering to a tech-savvy audience."
"Aim for a narrative that captures the essence of these breakthroughs and their implications for the future."
),
expected_output="A compelling 3 paragraphs blog post formatted as markdown about the latest AI advancements in 2024",
agent=writer,
)
# Instantiate your crew with a sequential process
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2,
process=Process.sequential,
)
# Get your crew to work!
with get_openai_callback() as cb:
result = crew.kickoff()
print(result)
print(cb) |
Hi ref code: |
|
To be honest I am not sure. I see (in the log) that the action is
Maybe someone can confirm for both of us. |
|
can you ask your agent to add reference urls for your research papers that they went through or used (duckduckgo reference links) and check whether they are valid links or not? i did that in my code as mentioned above it gave me wrong urls as references and the output was also based on gemini's output, since according to the report the latest llm model was gpt 4 and didn't mentioned anything about claude or dbrx etc |
|
I replaced @tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchRun().run(search_query)to be: @tool('DuckDuckGoSearch')
def search_tool(search_query: str):
"""Search the web for information on a given topic"""
return DuckDuckGoSearchResults().run(search_query)Here is a screenshot of the output: the log as text: As you can see it retrieved the URL. |
api key still needs to be removed from the comment, just in case you're still using the same key. |
|
I have set all possible places where llm can be set to gemini-pro in the code, but when running the code, it still calls gpt4.0. Could you please help me identify the issue? The complete code is posted below. import os
import sys
from crewai import Agent, Task, Crew, Process
from langchain_google_genai import ChatGoogleGenerativeAI
class PromptImproveAgent:
def __init__(self):
default_llm = ChatGoogleGenerativeAI(
model="gemini-pro",
verbose=True,
temperature=0.1,
google_api_key=os.environ["GOOGLE_API_KEY"],
)
self.improver = Agent(
role="提示优化助理",
backstory="""你是一位资深的Gemini提示工程师，为用户提供优化的效果更好的ChatGPT提示语""",
goal="""帮助用户为用户的需求制定最佳Gemini提示语。""",
allow_delegation=False,
verbose=True,
llm=default_llm,
function_calling_llm=default_llm,
)
self.expect = Agent(
role="各领域专家",
goal="针对用户需求，提出专业的解决方案或答案",
verbose=True,
memory=True,
backstory="针对用户的要求，提出专业的解决方案，方案要精炼，不要进行太多解释。",
llm=default_llm,
function_calling_llm=default_llm,
)
self.improve_prompt_task = Task(
description=f"""优化用户提供的Gemini提示词{input}""",
agent=self.improver,
expected_output="修订提示（提供您重写的提示。它应该清晰、简洁，并且您可以轻松理解），没有其他任何描述和说明",
human_input=False,
)
self.answer_question_task = Task(
description=f"""根据优化过的提示词{input}，提供专业的解决方案，它应该清晰、简洁，不需要任何解释""",
agent=self.expect,
expected_output="解决方案，没有任何多余解释",
human_input=False,
)
self.crew = Crew(
agents=[self.improver, self.expect],
tasks=[self.improve_prompt_task, self.answer_question_task],
process=Process.sequential,
memory=True,
max_rpm=100,
share_crew=True,
manager_llm=default_llm,
function_calling_llm=default_llm,
)
def process(self, input: str):
result = self.crew.kickoff(inputs={"input": input})
return result
if __name__ == "__main__":
agent = PromptImproveAgent()
user_input = ""
# If called directly from the command line take the first argument as the filename
while user_input != "/exit":
user_input = input("请输入您的prompt: ")
result = agent.process("我的提示: " + user_input)
print(result) |
|
Not sure what is the issue, but I copied the code you shared and it ran with me with Gemini.
Try to move the file to another folder structure or run it on Google Colab, to make sure it is not reading something locally or caching. hopefully, someone else could help. |
|
Thank you, I've tried on Google Colab, but it still called openai |
|
I changed the crew config, it works now. self.crew = Crew(
agents=[self.improver, self.expect],
tasks=[self.improve_prompt_task, self.answer_question_task],
process=Process.sequential,
memory=False,
max_rpm=100,
share_crew=False,
manager_llm=default_llm,
function_calling_llm=default_llm,
) |
|
When you set memory = True, it utilizes its internal module called entity_memory,short_term_memory also when you are using sequential process, you don't have to define manager_llm as it is only needed in hierarchical process. if you want to use the memory then you can use this code: |
|
@punitchauhan771 Thank you very much! The explanation was very clear and detailed, I understand now. |
Just to inform you, Gemini now supports function calling. It would be great if you could update the code to use the latest release version of the Google Gemini library. reference code: |
|
Why isn't the model = "gemini-1.5-pro-latest" ? |
|
I'm running it on Google Colab from crewai import Agent llm=ChatGoogleGenerativeAI(model="gemini-1.5-flash", ERROR: /usr/lib/python3.10/os.py in getitem(self, key) KeyError: 'OPENAI_API_KEY' |
in your agent, you need to add llm argument because by default it used OpenAI model |
|
@punitchauhan771 @Iris-Liu96 Setting up hugging face tokenos.environ["HUGGINGFACE_ACCESS_TOKEN"] = "hf_..."And then accessing embedding model from huggingface in instantiating the crewcrew = Crew(
agents=[..],
...,
...,
embedder=dict(
provider="huggingface", # or openai, ollama, ...
config=dict(
model="mixedbread-ai/mxbai-embed-large-v1",
),
),
)I tried But I'm still not sure tbh if this embeddor/ memory = True actually does work/help in anything. The output with memory=True barely differs from otherwise not including memory. The search/serper calls seem to be same, even after final thought, crew seems to go back to beginning and start again (I'm following the getting started guide) Quick question - I am practically trying to do everything free of cost and using frameworks has been a nightmare because of there default use of OpenAI, should I completely ditch these frameworks and go to basic langchain and langgraph to create agent workflows from ground up or try my hands at different frameworks (if there are any that allow high degree of customizability without having to read multiple docs please educate me). Becauase langchain and langgraph are so low level when it comes to ai agents, I'm a bit hesitant to put in that much work and complexity yk |
|
Hi @AdityaPrasad275 , |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Since Google Gemini Pro API is currently free when using up to 60 API calls per minute. This would be an incredibly helpful integration to add support for Gemini API into the CrewAI code. This will perform better than GPT 3.5 without any API fees. However, CrewAI must be able to simply allow users to define an API call/Functions Limit, so they can set it to 59 API processes per minute to avoid going over the free tier use per minute limits.
The text was updated successfully, but these errors were encountered: