Skip to content

Automated Workflow: scrape_all()

Will Norris edited this page Aug 11, 2018 · 2 revisions

If you want to scrape a long time series (1980's-2010's), and you are okay with using SWEpy's optimized selections for file settings (can be seen here) then scrape_all() is the function for you!

1. Instantiate the class

upper_left = [lon_upleft, lat_upleft]
lower_right = [lon_lowright, lat_lowright]

start = datetime.date(startY, startM, startD)
end = datetime.date(endY, endM, endD)

path = os.getcwd()

username = <username>
password = <password>

swepy = swepy(path, start, end, upper_left, lower_right, username, password, high_res = True)

2. Call scrape_all()

swepy.scrape_all()

And watch the magic happen!

Tips:

  1. If you intend to walk away and let SWEpy do its thing for a few hours, then make sure to caffeinate your computer!
    • simply type caffeinate into a fresh terminal window