This repository shows how to scrape Google Maps' places reviews data using SerpApi. Total 3 places' reviews are scraped in this code file, by filtering only Indonesian reviews. Google Maps API has also been used to find out place_id for 3 places.
I've been applying to AI/ML related tasks/jobs on Upwork, and got this scraping job (which paid me 20$ to complete the scraping task). I had done some scraping before, but I got to learn a lot while doing this task. I tried using Selenium to scrape the data, but Google services are very restricted to scrape. Also, I tried Google Maps API but it has a limitation of only 5 reviews to scrape.At last, I tried SerpApi from which I was able to scrape the google maps reviews data successfully. Although SerpApi is a paid service, we can use their Free Tier service in which they provide 100 requests limit per month.
Python web scraping is an automated method used for collecting large amounts of data from websites and storing it in a structured form. If we try to do it manually, it requires a lot of time and manpower, also resulting in human errors. So it's always better to scrape the required data using Scraping.
- Required installs/imports for serpapi, pagination, creating dataframes.
- Setting parameters for serpapi API calls for scraping maps data as well as places' reviews data.
- Make API call and get the required data in dict format. Do this for all the reviews using pagination.
- Extract reviewers' names and their reviews, store it in dataframe, filter only Indonesian reviews (API call scrapes both Indonesian and English reviews, sorting Indonesian reviews first. So locate where Indonesian reviews end and consider only those reviews).
- Create a CSV file to store these Indonesian Google Maps reviews.
Note :- Follow the same above steps for remaining 2 google maps places.
