Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AlgoExplorer WebScrapper With Selenium #1155

Merged
merged 2 commits into from
Apr 1, 2022
Merged

Conversation

Tee-py
Copy link
Contributor

@Tee-py Tee-py commented Apr 1, 2022

  1. Added Scrapper class to scrape AlgoExplorer Website using selenium✅

@Bhaney44 kindly check this if it solves the problem 👍.

@ChoiceCoin ChoiceCoin merged commit c60a594 into ChoiceCoin:main Apr 1, 2022
@Bhaney44
Copy link
Collaborator

Bhaney44 commented Apr 2, 2022

I'm getting a lot of errors on this.

(base) Brians-MacBook-Pro:teepy_rewards2 brianhaney$ python3 run.py
/opt/anaconda3/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
    creationflags=self.creationflags)
  File "/opt/anaconda3/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/opt/anaconda3/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver': 'chromedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run.py", line 14, in <module>
    web_scrapper = Scrapper(WEB.ALGOEXPLORER)
  File "/Users/brianhaney/Desktop/Programming/Rewards2/Rewards2/teepy_rewards2/scrapper.py", line 113, in __init__
    self.browser = Chrome()
  File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 90, in __init__
    self.service.start()
  File "/opt/anaconda3/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home

@Tee-py
Copy link
Contributor Author

Tee-py commented Apr 2, 2022

I mentioned in the Readme that you need a chrome driver to run the script.

run brew install --cask chromedriver
then xattr -d com.apple.quarantine $(which chromedriver)

This should install chrome driver and also lift the quarantine so the code can access the driver

for more information you can check https://timonweb.com/misc/fixing-error-chromedriver-cannot-be-opened-because-the-developer-cannot-be-verified-unable-to-launch-the-chrome-browser-on-mac-os/ and https://formulae.brew.sh/cask/chromedriver

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants