This is an experimental project. There may be problems, bugs and prices might be wrong (they are usually right). Simple Web Scraper with Django and Scrapy, Only Turkish books and Turkish book eCommerce websites (except Amazon)
Installation
for Windows:
pip install scrapy scrapyrt django
for Linux:
pip3 install scrapy scrapyrt django
Start ScrapyRT HTTP API
Open terminal in scrapy_crawlers/book_scraper and then:
scrapyrt
after run the scrapyrt successfully, run django server
open terminal in price_comp/price_comp and then:
python manage.py runserver
for Linux:
python3 manage.py runserver
after that, go to http://127.0.0.1:8000 in your browser and try search a book
I will search "Satranç" which is Turkish translation of Stefan Zweig's The Royal Game
and here is the results:
results are sorting by low to high price
columns are BOOK TITLE, AUTHOR, PUBLISHER, PROVIDER, PRICE and WEBSITE.
You can go to website with GO button.