notebooks/Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "\n",
    "The purpose of this Data Science Experience (DSX) project is to show how recommendations can be done with Apache Spark and integrated into a web application.  This project uses a randomly (but biased) generated dataset with approximately two thousand movies and 500,000 ratings. The ratings have been generated randomly\n",
    "\n",
    "The overall web application architecture can be seen here:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center\" markdown=\"1\">\n",
    "<img src=\"https://raw.githubusercontent.com/snowch/movie-recommender-demo/master/docs/overview_application_architecture.png\" width=\"60%\" style=\"float: center;\" />\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a live demo web application available here: https://movie-recommend-demo.mybluemix.net\n",
    "\n",
    "Below you can see a screenshot from the demo web application where the logged in user has searched for movies with 'harry' in the title and is then rating a movie."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center\" markdown=\"1\">\n",
    "<img src=\"https://raw.githubusercontent.com/snowch/movie-recommender-demo/master/docs/movie_recommender_search_harry.png\" width=\"60%\" style=\"float: center;\" />\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "## Instructions\n",
    "\n",
    "The project is split into a number of different notebooks that focus on specific steps.\n",
    "\n",
    "\n",
    "#### Step 1 - Exploratory analysis\n",
    "\n",
    "In this notebook, we perform some basic exploratory analysis of the ratings dataset before we jump into machine learning.<br>\n",
    "\n",
    "#### Step 2 - Train model\n",
    "\n",
    "Here we use Spark's Machine Learning Library (MLlib) to train a machine learning model on the data.<br>\n",
    "\n",
    "#### Step 3 - Predict ratings\n",
    "\n",
    "In this notebook, we simulate a new user's movie ratings and then use those ratings to predice movies for them.<br>\n",
    "\n",
    "#### Step 4 - Realtime Recommendations \n",
    "\n",
    "The Apache spark trained model is designed to be built as a batch process. This notebook we investigate how we can augment the batch generated model with ratings for new users so that we can provide recommendations without having to wait for the next batch run.<br>\n",
    "\n",
    "#### Step 5 - Setup Movie Recommendation Web App \n",
    "\n",
    "If you haven't setup your own instance of the demo web application with Cloudant and Compose Redis, this step will walk you through that process.<br>\n",
    "\n",
    "#### Step 6 - Install Spark Cloudant\n",
    "\n",
    "In this notebook we install the latest Spark Cloudant library to use Cloudant as a source of rating data and as a destination for the generated recommendations.<br>\n",
    "\n",
    "\n",
    "#### Step 7 - Setup Movie Recommendation Web Application\n",
    "\n",
    "In this notebook, we walk through setting up a web application where users can rate movies and receive recommendations using the Cloudant Datastore Recommender.<br>\n",
    "\n",
    "\n",
    "## Support\n",
    "\n",
    "If you have any questions about this project, please contact me at <chris.snow@uk.ibm.com>\n",
    "\n",
    "\n",
    "## Credits\n",
    "\n",
    "- This site was really useful https://github.com/jadianes/spark-movie-lens\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2 with Spark 1.6",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}