Home

Welcome to the NewsClassification wiki!

Project overview

NewsClassification is a machine learning project in Python for predicting the popularity of news articles, expressed in number of comments.
The news site used is nu.nl, the most visited news site in the Netherlands.
Scraping is done with the lxml library, text preprocessing with NLTK and machine learning with scikit-learn.

Details

The project consists of three main parts:

• Collecting
• Scrape news articles using lxml
• Get number of comments for scraped articles older than a day
• Preprocessing
• Convert text to lowercase
• Filter punctuation marks and stopwords
• Learning
• Document classification

Results

See Results for an overview of the results achieved so far.
Currently, the multinomial Naive Bayes classifier can classify 50% of the articles correctly.