Skip to content
/ qaahl Public

a crawler that can scrap and visualize the path qrawled

Notifications You must be signed in to change notification settings

surbhitt/qaahl

Repository files navigation

Objective

To develop an application capable of scrapping data from webpages and to provide visuals to assist the process through a GUI. Implementing a light weight multithreaded interactive application.

Introduction

Qaahl serves as a minimalist webcrawler providing the basic functionality of a scrapper. A lightweight graphical interface built using pygames.

Libraries utilised

  • Pygame
  • Requests
  • Beautiful Soup (Bs4)

Run

        pip install -r requirements.txt
        python3 main.py $URL $flags
$URL with the link as the base url
$flags with flags e.g. -d 3 performs depth=3 traversal

About

a crawler that can scrap and visualize the path qrawled

Topics

Resources

Stars

Watchers

Forks

Languages