NOIZZE Crawler

A web page crawler PyPI Package which returns (title(og, head), image(og, meta), description(og, meta)).

Dependency

• BeautifulSoup4

Installation

Run the folowing to install:

pip install noizze-crawler

Usage

import noizze_crawler as nc
import sys


if __name__ == '__main__':
    url = 'https://dvdprime.com/g2/bbs/board.php?bo_table=comm&wr_id=20525678'

    try:
        (title, desc, image_url, html) = nc.crawler(url)

    except nc.HostNotFound as e:
        print("Host Not Found")
        sys.exit(1)
    except nc.HTTPError as e:
        print("HTTP {}".format(e))
        sys.exit(1)

    print(title, desc, image_url)  # html

ChangeLog

• v12: Fixed #13
• v11: Fixed bugs #3 #8
• v10: Fixed bugs
• v9: Added Youtube crawler via Google API #4
• v8: Changed PyPI dependency - bs4
• v7: Changed PEP8
• v6: Added Exceptions HostNotFound, HTTPError