Tavily Search not being called correctly when running pip version

[bveiseh]

Describe the bug
Tavily Search API is being called incorrectly by gpt-4o. Logs show 400 error. According to Tavily docs, 400 means malformed request.

To Reproduce
Used python and pip package to instantiate in a basic test script.
"Error: 400 Client Error: Bad Request for url: https://api.tavily.com/search. Failed fetching sources. Resulting in empty response."

Expected behavior
Researcher correctly calls the Tavily API to return search results on the topic.

Screenshots
N/A

Desktop (please complete the following information):
Linux Ubuntu 22.04
Cursor latest
no front end

[ElishaKay]

Welcome @bveiseh

There are a couple of things you can do to try to debug this:

a) Run the test script at tests/test-your-retriever

cd tests
python -m test-your-retriever

b) Add a variable to your .env file:

SCRAPER=tavily_extract

Sometimes your IP might get blocked from scraping which that might help with

Happy to hear how it goes