LinkedIn Job Scraper

Overview

This project is a web scraper that extracts job listings for Python developers in the United States from LinkedIn and saves the data to a JSON file. The scraper extracts information such as the job title, company, location, job level, employment type, and job description.

Requirements

This project requires Python 3.11 and the following packages:

• BeautifulSoup4
• requests
• json

You can install these packages using pip by running the following command:

pip install -r requirements.txt

Running the Scraper

To run the scraper, simply run the following command in your terminal:

python main.py

The scraper will fetch the job listings from LinkedIn and save the data to a JSON file named job_listings.json.

Modifying the Scraper

There is already a valid Linkedin URL that fetches Python developer jobs. If you want to directly see the job_listing.json file, I provided that in the directory. And also, if you want to modify the scraper to extract additional data or use a different website, you can edit the main.py file and replace base_url with your url

The file is well-commented and should be easy to follow.

Conclusion

This project demonstrates how to use Python and web scraping to extract data from websites. The code can be modified and used for any purpose. If you have any questions or issues, please feel free to contact me.

Happy scraping!