-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add fallback options to search #49
Conversation
@spodgorny9 @rolson2 This should be ready for review. @spodgorny9, I added an option to silence those poppler errors. Updated the notebook to run it quietly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested the fallback by running the ords example pipeline and forcing the Google search to fail. The fallback seems to be working. The process initiated DuckDuckGo after Google failed.
Add fallback options to search
Refactor search to allow falling back to other SE options if initial (likely Google) search fails. The fallback list/order of search engines is entirely configurable by the user (as an input to the search function).
Also added an option to blacklist URL parts. This can be useful to further customize the web search piece