A simple scraper that scrapes the top X posts from a given subreddit thread, and the top level comments belonging to those posts.
This simple scraper was created for an academic project.
- Set up an app here.
- Fill out the
KEY_sample.pyand rename it toKEY.py.
# Create your virtual environment, f.x.:
python3 -m venv reddit-scrape
# Activate it, f.x.:
. reddit-scrape/bin/activate
# Install dependencies
pip install -r requirements.txt
# Run the scraper
python scraper <subreddit-name> <number-of-posts>The script creates two files: <subreddit-name>_posts.csv and <subreddit-name>_comments.csv