Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: add script to test changes to trending pages #35

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

javierggt
Copy link
Contributor

@javierggt javierggt commented Sep 20, 2024

Description

This PR adds a script to quickly test changes to the web scraper.

The following produces an html page with the output of one single scraped page, fetching it from the default URL:

twiki_test acq_stat_reports

Typically, when makes changes to the scraper, there will be a test page somewhere to scrape from. For example:

twiki_test kalman_watch3 --url https://icxc.cfa.harvard.edu/aspect/test_review_outputs/kalman_watch/pr-16/no-rad-data

produces a test page with scraped data from the given URL.

Interface impacts

Testing

Unit tests

  • No unit tests
  • Mac
  • Linux
  • Windows

Independent check of unit tests by [REVIEWER NAME]

  • [PLATFORM]:

Functional tests

The output from examples above:

@javierggt javierggt requested a review from jeanconn September 20, 2024 16:22
@jeanconn
Copy link
Contributor

👍 Great idea.

@javierggt
Copy link
Contributor Author

javierggt commented Sep 20, 2024

@jeanconn I fixed a hiccup (it was not working with classes that re-implemented get_url, but I hope it's ok now.

@javierggt
Copy link
Contributor Author

it will still not work in all cases... we can wait until I am testing acq_stats

@jeanconn
Copy link
Contributor

Maybe mark this PR as a draft for now? I think #34 can just get merged and a release installed.

@javierggt javierggt changed the title add script to test changes to trending pages WIP: add script to test changes to trending pages Sep 20, 2024
@jeanconn jeanconn removed their request for review September 25, 2024 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants