exercises/Breast_cancer_features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/chris-lovejoy/CodingForMedicine/blob/main/exercises/Breast_cancer_features.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Diagnosing Breast Cancer from Histology Features\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**In this exercise, we will train different machine learning models (including a neural network) to predict whether a breast cancer sample is malignant or benign.**\n",
    "\n",
    "Our models will use features extracted from breast samples, from the publicly-available [\"Breast Cancer Wisconsin\" dataset](https://www.kaggle.com/datasets/uciml/breast-cancer-wisconsin-data/download?datasetVersionNumber=2).\n",
    "\n",
    "This is a *classification* problem, because we're asking the algorithm to classify samples into one of two classes: **malignant or benign**. We'll train a neural network for this task. You're then invited to try out other classification algorithms, such as:\n",
    "\n",
    "- Random Forest\n",
    "- K-nearest neighbours\n",
    "- suppor vector machines (SVMs)\n",
    "\n",
    "In this exercise we'll learn how to:\n",
    "\n",
    "- **Download data** and **load it into our Jupyter Notebook**\n",
    "- **Import useful libraries** like pandas and sci-kit learn\n",
    "- **Clean our data**\n",
    "- **Divide our model** for training and testing\n",
    "- **Quantify the performance** of our model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Downloading and importing our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train a machine learning model, the first thing we need is data.\n",
    "\n",
    "There are various open-source datasets available on the internet. Great sources of datasets include [Kaggle](https://www.kaggle.com/), [Papers with Code](https://paperswithcode.com/datasets) and [data.world](https://data.world/datasets/health).\n",
    "\n",
    "For this exercise, we're using a dataset available on Kaggle. You can view information about the dataset and download it [here](https://www.kaggle.com/datasets/uciml/breast-cancer-wisconsin-data). (You may need to create a Kaggle account, which is definitely worth doing - Kaggle is great.)\n",
    "\n",
    "*Note: we want version 2 of the dataset.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downloading and moving our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we've downloaded the data, we're looking for the *.csv* file (it may be within a .zip file, which needs unzipping). CSV stands for 'comma-separated value' and refers to the fact that each row of data is stored with values that are separated by commas. You can open the file in a 'plain text' editor (like Notepad (windows) or TextEdit (Mac)) to see what this looks like. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Where we put that file will depend on whether we're running Jupyter Notebook locally or in Google Colab. These options are discussed in the [Jupyter Notebook setup exercise](https://github.com/chris-lovejoy/CodingForMedicine/blob/main/exercises/Setting_up_Jupyter_Notebook.ipynb).\n",
    "\n",
    "**If you are using Google Colab**, you need to:\n",
    "1. Make sure you are connected to a runtime (click Connect in the top right, if you aren't)\n",
    "2. Select the 'Files' folder on the left hand tab\n",
    "3. Drag our downloaded Breast Cancer 'data.csv' file into the Files tab (it should show the file uploading in the bottom left, and then you'll see 'data.csv' within the Files tab).\n",
    "\n",
    "**If you are running Jupyter Notebook on your local computer**, you can simply:\n",
    "1. Drag our 'data.csv' file into the same folder (aka. directory) as this Jupyter Notebook. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing the 'pandas' library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To load our data, we're going to use a popular library called [\"Pandas\"](https://pandas.pydata.org). A library is a collection of code with ready-made functions that we can use. We can import it with one line of code and then use it for a wide range of functionality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*If you are running on your local computer and get the following error, it means you need to install pandas on your computer. Instructions for doing so are available [here](https://pypi.org/project/pandas/). Drop me a message if any difficulties (message form at the bottom of this document).*\n",
    "\n",
    "\n",
    "> \"ModuleNotFoundError: No module named 'pandas'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing our data into the notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have our data in the correct place and pandas imported, we can load it with the following command. This uses one of the *pandas* functions **read_csv()**, which lets us load csv files and save them as a **'DataFrame'** (which is basically a table)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('./data.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our table should now be saved in the 'df' variable, so we should be able to see it by running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.30010</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.16220</td>\n",
       "      <td>0.66560</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.08690</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.12380</td>\n",
       "      <td>0.18660</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.19740</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.14440</td>\n",
       "      <td>0.42450</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.24140</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.20980</td>\n",
       "      <td>0.86630</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.19800</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.13740</td>\n",
       "      <td>0.20500</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>926424</td>\n",
       "      <td>M</td>\n",
       "      <td>21.56</td>\n",
       "      <td>22.39</td>\n",
       "      <td>142.00</td>\n",
       "      <td>1479.0</td>\n",
       "      <td>0.11100</td>\n",
       "      <td>0.11590</td>\n",
       "      <td>0.24390</td>\n",
       "      <td>0.13890</td>\n",
       "      <td>...</td>\n",
       "      <td>26.40</td>\n",
       "      <td>166.10</td>\n",
       "      <td>2027.0</td>\n",
       "      <td>0.14100</td>\n",
       "      <td>0.21130</td>\n",
       "      <td>0.4107</td>\n",
       "      <td>0.2216</td>\n",
       "      <td>0.2060</td>\n",
       "      <td>0.07115</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565</th>\n",
       "      <td>926682</td>\n",
       "      <td>M</td>\n",
       "      <td>20.13</td>\n",
       "      <td>28.25</td>\n",
       "      <td>131.20</td>\n",
       "      <td>1261.0</td>\n",
       "      <td>0.09780</td>\n",
       "      <td>0.10340</td>\n",
       "      <td>0.14400</td>\n",
       "      <td>0.09791</td>\n",
       "      <td>...</td>\n",
       "      <td>38.25</td>\n",
       "      <td>155.00</td>\n",
       "      <td>1731.0</td>\n",
       "      <td>0.11660</td>\n",
       "      <td>0.19220</td>\n",
       "      <td>0.3215</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>0.2572</td>\n",
       "      <td>0.06637</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566</th>\n",
       "      <td>926954</td>\n",
       "      <td>M</td>\n",
       "      <td>16.60</td>\n",
       "      <td>28.08</td>\n",
       "      <td>108.30</td>\n",
       "      <td>858.1</td>\n",
       "      <td>0.08455</td>\n",
       "      <td>0.10230</td>\n",
       "      <td>0.09251</td>\n",
       "      <td>0.05302</td>\n",
       "      <td>...</td>\n",
       "      <td>34.12</td>\n",
       "      <td>126.70</td>\n",
       "      <td>1124.0</td>\n",
       "      <td>0.11390</td>\n",
       "      <td>0.30940</td>\n",
       "      <td>0.3403</td>\n",
       "      <td>0.1418</td>\n",
       "      <td>0.2218</td>\n",
       "      <td>0.07820</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>927241</td>\n",
       "      <td>M</td>\n",
       "      <td>20.60</td>\n",
       "      <td>29.33</td>\n",
       "      <td>140.10</td>\n",
       "      <td>1265.0</td>\n",
       "      <td>0.11780</td>\n",
       "      <td>0.27700</td>\n",
       "      <td>0.35140</td>\n",
       "      <td>0.15200</td>\n",
       "      <td>...</td>\n",
       "      <td>39.42</td>\n",
       "      <td>184.60</td>\n",
       "      <td>1821.0</td>\n",
       "      <td>0.16500</td>\n",
       "      <td>0.86810</td>\n",
       "      <td>0.9387</td>\n",
       "      <td>0.2650</td>\n",
       "      <td>0.4087</td>\n",
       "      <td>0.12400</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568</th>\n",
       "      <td>92751</td>\n",
       "      <td>B</td>\n",
       "      <td>7.76</td>\n",
       "      <td>24.54</td>\n",
       "      <td>47.92</td>\n",
       "      <td>181.0</td>\n",
       "      <td>0.05263</td>\n",
       "      <td>0.04362</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>...</td>\n",
       "      <td>30.37</td>\n",
       "      <td>59.16</td>\n",
       "      <td>268.6</td>\n",
       "      <td>0.08996</td>\n",
       "      <td>0.06444</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2871</td>\n",
       "      <td>0.07039</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>569 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0      842302         M        17.99         10.38          122.80     1001.0   \n",
       "1      842517         M        20.57         17.77          132.90     1326.0   \n",
       "2    84300903         M        19.69         21.25          130.00     1203.0   \n",
       "3    84348301         M        11.42         20.38           77.58      386.1   \n",
       "4    84358402         M        20.29         14.34          135.10     1297.0   \n",
       "..        ...       ...          ...           ...             ...        ...   \n",
       "564    926424         M        21.56         22.39          142.00     1479.0   \n",
       "565    926682         M        20.13         28.25          131.20     1261.0   \n",
       "566    926954         M        16.60         28.08          108.30      858.1   \n",
       "567    927241         M        20.60         29.33          140.10     1265.0   \n",
       "568     92751         B         7.76         24.54           47.92      181.0   \n",
       "\n",
       "     smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0            0.11840           0.27760         0.30010              0.14710   \n",
       "1            0.08474           0.07864         0.08690              0.07017   \n",
       "2            0.10960           0.15990         0.19740              0.12790   \n",
       "3            0.14250           0.28390         0.24140              0.10520   \n",
       "4            0.10030           0.13280         0.19800              0.10430   \n",
       "..               ...               ...             ...                  ...   \n",
       "564          0.11100           0.11590         0.24390              0.13890   \n",
       "565          0.09780           0.10340         0.14400              0.09791   \n",
       "566          0.08455           0.10230         0.09251              0.05302   \n",
       "567          0.11780           0.27700         0.35140              0.15200   \n",
       "568          0.05263           0.04362         0.00000              0.00000   \n",
       "\n",
       "     ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "0    ...          17.33           184.60      2019.0           0.16220   \n",
       "1    ...          23.41           158.80      1956.0           0.12380   \n",
       "2    ...          25.53           152.50      1709.0           0.14440   \n",
       "3    ...          26.50            98.87       567.7           0.20980   \n",
       "4    ...          16.67           152.20      1575.0           0.13740   \n",
       "..   ...            ...              ...         ...               ...   \n",
       "564  ...          26.40           166.10      2027.0           0.14100   \n",
       "565  ...          38.25           155.00      1731.0           0.11660   \n",
       "566  ...          34.12           126.70      1124.0           0.11390   \n",
       "567  ...          39.42           184.60      1821.0           0.16500   \n",
       "568  ...          30.37            59.16       268.6           0.08996   \n",
       "\n",
       "     compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0              0.66560           0.7119                0.2654          0.4601   \n",
       "1              0.18660           0.2416                0.1860          0.2750   \n",
       "2              0.42450           0.4504                0.2430          0.3613   \n",
       "3              0.86630           0.6869                0.2575          0.6638   \n",
       "4              0.20500           0.4000                0.1625          0.2364   \n",
       "..                 ...              ...                   ...             ...   \n",
       "564            0.21130           0.4107                0.2216          0.2060   \n",
       "565            0.19220           0.3215                0.1628          0.2572   \n",
       "566            0.30940           0.3403                0.1418          0.2218   \n",
       "567            0.86810           0.9387                0.2650          0.4087   \n",
       "568            0.06444           0.0000                0.0000          0.2871   \n",
       "\n",
       "     fractal_dimension_worst  Unnamed: 32  \n",
       "0                    0.11890          NaN  \n",
       "1                    0.08902          NaN  \n",
       "2                    0.08758          NaN  \n",
       "3                    0.17300          NaN  \n",
       "4                    0.07678          NaN  \n",
       "..                       ...          ...  \n",
       "564                  0.07115          NaN  \n",
       "565                  0.06637          NaN  \n",
       "566                  0.07820          NaN  \n",
       "567                  0.12400          NaN  \n",
       "568                  0.07039          NaN  \n",
       "\n",
       "[569 rows x 33 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Visualising our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we start training our model, it's important to understand our dataset. The first step is read any accompanying information we have about our data. For some projects, this may involve asking questions to whoever provided the data. In our case, there is a helpful description on Kaggle [here](https://www.kaggle.com/datasets/uciml/breast-cancer-wisconsin-data).\n",
    "\n",
    "The second step is to explore the data ourselves. There are several functions that can be helpful for this.\n",
    "\n",
    "Lets start by using the **.head()** and **.tail()** functions to look at the top and bottom of our table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.3001</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.1622</td>\n",
       "      <td>0.6656</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.0869</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.1866</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.1974</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.1444</td>\n",
       "      <td>0.4245</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.2414</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.2098</td>\n",
       "      <td>0.8663</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.1980</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.1374</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0    842302         M        17.99         10.38          122.80     1001.0   \n",
       "1    842517         M        20.57         17.77          132.90     1326.0   \n",
       "2  84300903         M        19.69         21.25          130.00     1203.0   \n",
       "3  84348301         M        11.42         20.38           77.58      386.1   \n",
       "4  84358402         M        20.29         14.34          135.10     1297.0   \n",
       "\n",
       "   smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0          0.11840           0.27760          0.3001              0.14710   \n",
       "1          0.08474           0.07864          0.0869              0.07017   \n",
       "2          0.10960           0.15990          0.1974              0.12790   \n",
       "3          0.14250           0.28390          0.2414              0.10520   \n",
       "4          0.10030           0.13280          0.1980              0.10430   \n",
       "\n",
       "   ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "0  ...          17.33           184.60      2019.0            0.1622   \n",
       "1  ...          23.41           158.80      1956.0            0.1238   \n",
       "2  ...          25.53           152.50      1709.0            0.1444   \n",
       "3  ...          26.50            98.87       567.7            0.2098   \n",
       "4  ...          16.67           152.20      1575.0            0.1374   \n",
       "\n",
       "   compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0             0.6656           0.7119                0.2654          0.4601   \n",
       "1             0.1866           0.2416                0.1860          0.2750   \n",
       "2             0.4245           0.4504                0.2430          0.3613   \n",
       "3             0.8663           0.6869                0.2575          0.6638   \n",
       "4             0.2050           0.4000                0.1625          0.2364   \n",
       "\n",
       "   fractal_dimension_worst  Unnamed: 32  \n",
       "0                  0.11890          NaN  \n",
       "1                  0.08902          NaN  \n",
       "2                  0.08758          NaN  \n",
       "3                  0.17300          NaN  \n",
       "4                  0.07678          NaN  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>926424</td>\n",
       "      <td>M</td>\n",
       "      <td>21.56</td>\n",
       "      <td>22.39</td>\n",
       "      <td>142.00</td>\n",
       "      <td>1479.0</td>\n",
       "      <td>0.11100</td>\n",
       "      <td>0.11590</td>\n",
       "      <td>0.24390</td>\n",
       "      <td>0.13890</td>\n",
       "      <td>...</td>\n",
       "      <td>26.40</td>\n",
       "      <td>166.10</td>\n",
       "      <td>2027.0</td>\n",
       "      <td>0.14100</td>\n",
       "      <td>0.21130</td>\n",
       "      <td>0.4107</td>\n",
       "      <td>0.2216</td>\n",
       "      <td>0.2060</td>\n",
       "      <td>0.07115</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565</th>\n",
       "      <td>926682</td>\n",
       "      <td>M</td>\n",
       "      <td>20.13</td>\n",
       "      <td>28.25</td>\n",
       "      <td>131.20</td>\n",
       "      <td>1261.0</td>\n",
       "      <td>0.09780</td>\n",
       "      <td>0.10340</td>\n",
       "      <td>0.14400</td>\n",
       "      <td>0.09791</td>\n",
       "      <td>...</td>\n",
       "      <td>38.25</td>\n",
       "      <td>155.00</td>\n",
       "      <td>1731.0</td>\n",
       "      <td>0.11660</td>\n",
       "      <td>0.19220</td>\n",
       "      <td>0.3215</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>0.2572</td>\n",
       "      <td>0.06637</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566</th>\n",
       "      <td>926954</td>\n",
       "      <td>M</td>\n",
       "      <td>16.60</td>\n",
       "      <td>28.08</td>\n",
       "      <td>108.30</td>\n",
       "      <td>858.1</td>\n",
       "      <td>0.08455</td>\n",
       "      <td>0.10230</td>\n",
       "      <td>0.09251</td>\n",
       "      <td>0.05302</td>\n",
       "      <td>...</td>\n",
       "      <td>34.12</td>\n",
       "      <td>126.70</td>\n",
       "      <td>1124.0</td>\n",
       "      <td>0.11390</td>\n",
       "      <td>0.30940</td>\n",
       "      <td>0.3403</td>\n",
       "      <td>0.1418</td>\n",
       "      <td>0.2218</td>\n",
       "      <td>0.07820</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>927241</td>\n",
       "      <td>M</td>\n",
       "      <td>20.60</td>\n",
       "      <td>29.33</td>\n",
       "      <td>140.10</td>\n",
       "      <td>1265.0</td>\n",
       "      <td>0.11780</td>\n",
       "      <td>0.27700</td>\n",
       "      <td>0.35140</td>\n",
       "      <td>0.15200</td>\n",
       "      <td>...</td>\n",
       "      <td>39.42</td>\n",
       "      <td>184.60</td>\n",
       "      <td>1821.0</td>\n",
       "      <td>0.16500</td>\n",
       "      <td>0.86810</td>\n",
       "      <td>0.9387</td>\n",
       "      <td>0.2650</td>\n",
       "      <td>0.4087</td>\n",
       "      <td>0.12400</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568</th>\n",
       "      <td>92751</td>\n",
       "      <td>B</td>\n",
       "      <td>7.76</td>\n",
       "      <td>24.54</td>\n",
       "      <td>47.92</td>\n",
       "      <td>181.0</td>\n",
       "      <td>0.05263</td>\n",
       "      <td>0.04362</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>...</td>\n",
       "      <td>30.37</td>\n",
       "      <td>59.16</td>\n",
       "      <td>268.6</td>\n",
       "      <td>0.08996</td>\n",
       "      <td>0.06444</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.2871</td>\n",
       "      <td>0.07039</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "564  926424         M        21.56         22.39          142.00     1479.0   \n",
       "565  926682         M        20.13         28.25          131.20     1261.0   \n",
       "566  926954         M        16.60         28.08          108.30      858.1   \n",
       "567  927241         M        20.60         29.33          140.10     1265.0   \n",
       "568   92751         B         7.76         24.54           47.92      181.0   \n",
       "\n",
       "     smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "564          0.11100           0.11590         0.24390              0.13890   \n",
       "565          0.09780           0.10340         0.14400              0.09791   \n",
       "566          0.08455           0.10230         0.09251              0.05302   \n",
       "567          0.11780           0.27700         0.35140              0.15200   \n",
       "568          0.05263           0.04362         0.00000              0.00000   \n",
       "\n",
       "     ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "564  ...          26.40           166.10      2027.0           0.14100   \n",
       "565  ...          38.25           155.00      1731.0           0.11660   \n",
       "566  ...          34.12           126.70      1124.0           0.11390   \n",
       "567  ...          39.42           184.60      1821.0           0.16500   \n",
       "568  ...          30.37            59.16       268.6           0.08996   \n",
       "\n",
       "     compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "564            0.21130           0.4107                0.2216          0.2060   \n",
       "565            0.19220           0.3215                0.1628          0.2572   \n",
       "566            0.30940           0.3403                0.1418          0.2218   \n",
       "567            0.86810           0.9387                0.2650          0.4087   \n",
       "568            0.06444           0.0000                0.0000          0.2871   \n",
       "\n",
       "     fractal_dimension_worst  Unnamed: 32  \n",
       "564                  0.07115          NaN  \n",
       "565                  0.06637          NaN  \n",
       "566                  0.07820          NaN  \n",
       "567                  0.12400          NaN  \n",
       "568                  0.07039          NaN  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A helpful function for understanding each variable is **.describe()**. Let's use this to look at radius_mean. Feel free to modify the code below to understand other variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    569.000000\n",
       "mean      14.127292\n",
       "std        3.524049\n",
       "min        6.981000\n",
       "25%       11.700000\n",
       "50%       13.370000\n",
       "75%       15.780000\n",
       "max       28.110000\n",
       "Name: radius_mean, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.radius_mean.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are similar functions for looking at different averages, in particular median and mode:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.37"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.radius_mean.median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    12.34\n",
       "Name: radius_mean, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.radius_mean.mode()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take some time to look at the different variables and understand them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Cleaning our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may have noticed that there's a column on the far right called \"Unnamed: 32\" and that it's full of \"NaN\" values. \"NaN\" stands for 'not a number' and can be caused by various things. In some cases, an empty column in a spreadsheet file can be picked up as a column, and all values are 'NaN's - that appears to be the case here.\n",
    "\n",
    "We can remove the column using the **drop()** function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop('Unnamed: 32', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're also not fussed about the 'id' column so we can get rid of that too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop('id', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One common problem with real-world datasets is missing data or outliers. We can check the number of missing values with the isna() function. This returns a table which shows whether each cell is a NaN or not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>symmetry_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>radius_worst</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>569 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0        False        False         False           False      False   \n",
       "1        False        False         False           False      False   \n",
       "2        False        False         False           False      False   \n",
       "3        False        False         False           False      False   \n",
       "4        False        False         False           False      False   \n",
       "..         ...          ...           ...             ...        ...   \n",
       "564      False        False         False           False      False   \n",
       "565      False        False         False           False      False   \n",
       "566      False        False         False           False      False   \n",
       "567      False        False         False           False      False   \n",
       "568      False        False         False           False      False   \n",
       "\n",
       "     smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0              False             False           False                False   \n",
       "1              False             False           False                False   \n",
       "2              False             False           False                False   \n",
       "3              False             False           False                False   \n",
       "4              False             False           False                False   \n",
       "..               ...               ...             ...                  ...   \n",
       "564            False             False           False                False   \n",
       "565            False             False           False                False   \n",
       "566            False             False           False                False   \n",
       "567            False             False           False                False   \n",
       "568            False             False           False                False   \n",
       "\n",
       "     symmetry_mean  ...  radius_worst  texture_worst  perimeter_worst  \\\n",
       "0            False  ...         False          False            False   \n",
       "1            False  ...         False          False            False   \n",
       "2            False  ...         False          False            False   \n",
       "3            False  ...         False          False            False   \n",
       "4            False  ...         False          False            False   \n",
       "..             ...  ...           ...            ...              ...   \n",
       "564          False  ...         False          False            False   \n",
       "565          False  ...         False          False            False   \n",
       "566          False  ...         False          False            False   \n",
       "567          False  ...         False          False            False   \n",
       "568          False  ...         False          False            False   \n",
       "\n",
       "     area_worst  smoothness_worst  compactness_worst  concavity_worst  \\\n",
       "0         False             False              False            False   \n",
       "1         False             False              False            False   \n",
       "2         False             False              False            False   \n",
       "3         False             False              False            False   \n",
       "4         False             False              False            False   \n",
       "..          ...               ...                ...              ...   \n",
       "564       False             False              False            False   \n",
       "565       False             False              False            False   \n",
       "566       False             False              False            False   \n",
       "567       False             False              False            False   \n",
       "568       False             False              False            False   \n",
       "\n",
       "     concave points_worst  symmetry_worst  fractal_dimension_worst  \n",
       "0                   False           False                    False  \n",
       "1                   False           False                    False  \n",
       "2                   False           False                    False  \n",
       "3                   False           False                    False  \n",
       "4                   False           False                    False  \n",
       "..                    ...             ...                      ...  \n",
       "564                 False           False                    False  \n",
       "565                 False           False                    False  \n",
       "566                 False           False                    False  \n",
       "567                 False           False                    False  \n",
       "568                 False           False                    False  \n",
       "\n",
       "[569 rows x 31 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isna()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can find the total number of NaNs in each column of our table using the following cell. (See if you can figure out how it works.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(df.isna().values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's great to see that there aren't any missing values in our table! We can say that our data is pretty 'clean', so there's not much else for us to do here. With many real-world datasets there's more to be done here, but we can go onto the next step: preparing our data for training the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Preparing our data for model training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting to binary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Computers like to work with binary values of 0 or 1 much more than with text. So let's convert our 'diagnosis' column values from 'M' for malignant and 'B' for binary into 1 and 0:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['diagnosis'] = df['diagnosis'].map({'M':1,'B':0})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check what the values the 'diagnosis' now has with the **unique()** function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.diagnosis.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalising our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our dataset, different features have different ranges of values. For example, the maximum value of the 'perimeter_mean' feature is 188.5, while for the 'smoothness_mean' feature it is 0.1634.\n",
    "\n",
    "For some machine learning models, this can affect the model's performance because it can care more about changes in the larger feature.\n",
    "\n",
    "Therefore, it's best practice to 'normalise' the data, which involves bringing everything into the same range. One popular way for doing so is called 'max/min' normalisation, and it moves all values into the range 0 to 1.\n",
    "\n",
    "Again, we can use a **library** to do this, so we don't have to write much code ourselves. We'll use the **sci-kit learn** library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "df = pd.DataFrame(scaler.fit_transform(df), columns=df.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we look at our data again, we'll see that all values are now between 0 and 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>symmetry_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>radius_worst</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.521037</td>\n",
       "      <td>0.022658</td>\n",
       "      <td>0.545989</td>\n",
       "      <td>0.363733</td>\n",
       "      <td>0.593753</td>\n",
       "      <td>0.792037</td>\n",
       "      <td>0.703140</td>\n",
       "      <td>0.731113</td>\n",
       "      <td>0.686364</td>\n",
       "      <td>...</td>\n",
       "      <td>0.620776</td>\n",
       "      <td>0.141525</td>\n",
       "      <td>0.668310</td>\n",
       "      <td>0.450698</td>\n",
       "      <td>0.601136</td>\n",
       "      <td>0.619292</td>\n",
       "      <td>0.568610</td>\n",
       "      <td>0.912027</td>\n",
       "      <td>0.598462</td>\n",
       "      <td>0.418864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.643144</td>\n",
       "      <td>0.272574</td>\n",
       "      <td>0.615783</td>\n",
       "      <td>0.501591</td>\n",
       "      <td>0.289880</td>\n",
       "      <td>0.181768</td>\n",
       "      <td>0.203608</td>\n",
       "      <td>0.348757</td>\n",
       "      <td>0.379798</td>\n",
       "      <td>...</td>\n",
       "      <td>0.606901</td>\n",
       "      <td>0.303571</td>\n",
       "      <td>0.539818</td>\n",
       "      <td>0.435214</td>\n",
       "      <td>0.347553</td>\n",
       "      <td>0.154563</td>\n",
       "      <td>0.192971</td>\n",
       "      <td>0.639175</td>\n",
       "      <td>0.233590</td>\n",
       "      <td>0.222878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.601496</td>\n",
       "      <td>0.390260</td>\n",
       "      <td>0.595743</td>\n",
       "      <td>0.449417</td>\n",
       "      <td>0.514309</td>\n",
       "      <td>0.431017</td>\n",
       "      <td>0.462512</td>\n",
       "      <td>0.635686</td>\n",
       "      <td>0.509596</td>\n",
       "      <td>...</td>\n",
       "      <td>0.556386</td>\n",
       "      <td>0.360075</td>\n",
       "      <td>0.508442</td>\n",
       "      <td>0.374508</td>\n",
       "      <td>0.483590</td>\n",
       "      <td>0.385375</td>\n",
       "      <td>0.359744</td>\n",
       "      <td>0.835052</td>\n",
       "      <td>0.403706</td>\n",
       "      <td>0.213433</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.210090</td>\n",
       "      <td>0.360839</td>\n",
       "      <td>0.233501</td>\n",
       "      <td>0.102906</td>\n",
       "      <td>0.811321</td>\n",
       "      <td>0.811361</td>\n",
       "      <td>0.565604</td>\n",
       "      <td>0.522863</td>\n",
       "      <td>0.776263</td>\n",
       "      <td>...</td>\n",
       "      <td>0.248310</td>\n",
       "      <td>0.385928</td>\n",
       "      <td>0.241347</td>\n",
       "      <td>0.094008</td>\n",
       "      <td>0.915472</td>\n",
       "      <td>0.814012</td>\n",
       "      <td>0.548642</td>\n",
       "      <td>0.884880</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.773711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.629893</td>\n",
       "      <td>0.156578</td>\n",
       "      <td>0.630986</td>\n",
       "      <td>0.489290</td>\n",
       "      <td>0.430351</td>\n",
       "      <td>0.347893</td>\n",
       "      <td>0.463918</td>\n",
       "      <td>0.518390</td>\n",
       "      <td>0.378283</td>\n",
       "      <td>...</td>\n",
       "      <td>0.519744</td>\n",
       "      <td>0.123934</td>\n",
       "      <td>0.506948</td>\n",
       "      <td>0.341575</td>\n",
       "      <td>0.437364</td>\n",
       "      <td>0.172415</td>\n",
       "      <td>0.319489</td>\n",
       "      <td>0.558419</td>\n",
       "      <td>0.157500</td>\n",
       "      <td>0.142595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.690000</td>\n",
       "      <td>0.428813</td>\n",
       "      <td>0.678668</td>\n",
       "      <td>0.566490</td>\n",
       "      <td>0.526948</td>\n",
       "      <td>0.296055</td>\n",
       "      <td>0.571462</td>\n",
       "      <td>0.690358</td>\n",
       "      <td>0.336364</td>\n",
       "      <td>...</td>\n",
       "      <td>0.623266</td>\n",
       "      <td>0.383262</td>\n",
       "      <td>0.576174</td>\n",
       "      <td>0.452664</td>\n",
       "      <td>0.461137</td>\n",
       "      <td>0.178527</td>\n",
       "      <td>0.328035</td>\n",
       "      <td>0.761512</td>\n",
       "      <td>0.097575</td>\n",
       "      <td>0.105667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.622320</td>\n",
       "      <td>0.626987</td>\n",
       "      <td>0.604036</td>\n",
       "      <td>0.474019</td>\n",
       "      <td>0.407782</td>\n",
       "      <td>0.257714</td>\n",
       "      <td>0.337395</td>\n",
       "      <td>0.486630</td>\n",
       "      <td>0.349495</td>\n",
       "      <td>...</td>\n",
       "      <td>0.560655</td>\n",
       "      <td>0.699094</td>\n",
       "      <td>0.520892</td>\n",
       "      <td>0.379915</td>\n",
       "      <td>0.300007</td>\n",
       "      <td>0.159997</td>\n",
       "      <td>0.256789</td>\n",
       "      <td>0.559450</td>\n",
       "      <td>0.198502</td>\n",
       "      <td>0.074315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.455251</td>\n",
       "      <td>0.621238</td>\n",
       "      <td>0.445788</td>\n",
       "      <td>0.303118</td>\n",
       "      <td>0.288165</td>\n",
       "      <td>0.254340</td>\n",
       "      <td>0.216753</td>\n",
       "      <td>0.263519</td>\n",
       "      <td>0.267677</td>\n",
       "      <td>...</td>\n",
       "      <td>0.393099</td>\n",
       "      <td>0.589019</td>\n",
       "      <td>0.379949</td>\n",
       "      <td>0.230731</td>\n",
       "      <td>0.282177</td>\n",
       "      <td>0.273705</td>\n",
       "      <td>0.271805</td>\n",
       "      <td>0.487285</td>\n",
       "      <td>0.128721</td>\n",
       "      <td>0.151909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>567</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.644564</td>\n",
       "      <td>0.663510</td>\n",
       "      <td>0.665538</td>\n",
       "      <td>0.475716</td>\n",
       "      <td>0.588336</td>\n",
       "      <td>0.790197</td>\n",
       "      <td>0.823336</td>\n",
       "      <td>0.755467</td>\n",
       "      <td>0.675253</td>\n",
       "      <td>...</td>\n",
       "      <td>0.633582</td>\n",
       "      <td>0.730277</td>\n",
       "      <td>0.668310</td>\n",
       "      <td>0.402035</td>\n",
       "      <td>0.619626</td>\n",
       "      <td>0.815758</td>\n",
       "      <td>0.749760</td>\n",
       "      <td>0.910653</td>\n",
       "      <td>0.497142</td>\n",
       "      <td>0.452315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.036869</td>\n",
       "      <td>0.501522</td>\n",
       "      <td>0.028540</td>\n",
       "      <td>0.015907</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.074351</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.266162</td>\n",
       "      <td>...</td>\n",
       "      <td>0.054287</td>\n",
       "      <td>0.489072</td>\n",
       "      <td>0.043578</td>\n",
       "      <td>0.020497</td>\n",
       "      <td>0.124084</td>\n",
       "      <td>0.036043</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.257441</td>\n",
       "      <td>0.100682</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>569 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0          1.0     0.521037      0.022658        0.545989   0.363733   \n",
       "1          1.0     0.643144      0.272574        0.615783   0.501591   \n",
       "2          1.0     0.601496      0.390260        0.595743   0.449417   \n",
       "3          1.0     0.210090      0.360839        0.233501   0.102906   \n",
       "4          1.0     0.629893      0.156578        0.630986   0.489290   \n",
       "..         ...          ...           ...             ...        ...   \n",
       "564        1.0     0.690000      0.428813        0.678668   0.566490   \n",
       "565        1.0     0.622320      0.626987        0.604036   0.474019   \n",
       "566        1.0     0.455251      0.621238        0.445788   0.303118   \n",
       "567        1.0     0.644564      0.663510        0.665538   0.475716   \n",
       "568        0.0     0.036869      0.501522        0.028540   0.015907   \n",
       "\n",
       "     smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0           0.593753          0.792037        0.703140             0.731113   \n",
       "1           0.289880          0.181768        0.203608             0.348757   \n",
       "2           0.514309          0.431017        0.462512             0.635686   \n",
       "3           0.811321          0.811361        0.565604             0.522863   \n",
       "4           0.430351          0.347893        0.463918             0.518390   \n",
       "..               ...               ...             ...                  ...   \n",
       "564         0.526948          0.296055        0.571462             0.690358   \n",
       "565         0.407782          0.257714        0.337395             0.486630   \n",
       "566         0.288165          0.254340        0.216753             0.263519   \n",
       "567         0.588336          0.790197        0.823336             0.755467   \n",
       "568         0.000000          0.074351        0.000000             0.000000   \n",
       "\n",
       "     symmetry_mean  ...  radius_worst  texture_worst  perimeter_worst  \\\n",
       "0         0.686364  ...      0.620776       0.141525         0.668310   \n",
       "1         0.379798  ...      0.606901       0.303571         0.539818   \n",
       "2         0.509596  ...      0.556386       0.360075         0.508442   \n",
       "3         0.776263  ...      0.248310       0.385928         0.241347   \n",
       "4         0.378283  ...      0.519744       0.123934         0.506948   \n",
       "..             ...  ...           ...            ...              ...   \n",
       "564       0.336364  ...      0.623266       0.383262         0.576174   \n",
       "565       0.349495  ...      0.560655       0.699094         0.520892   \n",
       "566       0.267677  ...      0.393099       0.589019         0.379949   \n",
       "567       0.675253  ...      0.633582       0.730277         0.668310   \n",
       "568       0.266162  ...      0.054287       0.489072         0.043578   \n",
       "\n",
       "     area_worst  smoothness_worst  compactness_worst  concavity_worst  \\\n",
       "0      0.450698          0.601136           0.619292         0.568610   \n",
       "1      0.435214          0.347553           0.154563         0.192971   \n",
       "2      0.374508          0.483590           0.385375         0.359744   \n",
       "3      0.094008          0.915472           0.814012         0.548642   \n",
       "4      0.341575          0.437364           0.172415         0.319489   \n",
       "..          ...               ...                ...              ...   \n",
       "564    0.452664          0.461137           0.178527         0.328035   \n",
       "565    0.379915          0.300007           0.159997         0.256789   \n",
       "566    0.230731          0.282177           0.273705         0.271805   \n",
       "567    0.402035          0.619626           0.815758         0.749760   \n",
       "568    0.020497          0.124084           0.036043         0.000000   \n",
       "\n",
       "     concave points_worst  symmetry_worst  fractal_dimension_worst  \n",
       "0                0.912027        0.598462                 0.418864  \n",
       "1                0.639175        0.233590                 0.222878  \n",
       "2                0.835052        0.403706                 0.213433  \n",
       "3                0.884880        1.000000                 0.773711  \n",
       "4                0.558419        0.157500                 0.142595  \n",
       "..                    ...             ...                      ...  \n",
       "564              0.761512        0.097575                 0.105667  \n",
       "565              0.559450        0.198502                 0.074315  \n",
       "566              0.487285        0.128721                 0.151909  \n",
       "567              0.910653        0.497142                 0.452315  \n",
       "568              0.000000        0.257441                 0.100682  \n",
       "\n",
       "[569 rows x 31 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*(If you're having trouble downloading the library and importing it, you can skip this section and the code will still run - it just won't give as reliable a result.)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a train-test split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we need to divide our data into 'training' and 'test' data. We do this so we can understand whether our model is actually helpful. In the real world, we want to use our model on data it's never seen before. \n",
    "\n",
    "When training it model, we only show it the 'training' data. We can then test it on the 'test' data, and use that to understand how well the model may perform on new (unseen) data.\n",
    "\n",
    "Let's again use a function from the sci-kit learn library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "train, test = train_test_split(df, test_size = 0.15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "test_size determines what ratio of our data is kept back as 'test' data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now for the fun part: training and assessing our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZdRtHgpg5Mg5"
   },
   "source": [
    "## Part 5: Training our model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To keep things simple, we'll start by just using three features for our model. Let's make a list of prediction variables.\n",
    "\n",
    "*(Not sure what a list is? Check out the [Python principles](https://github.com/chris-lovejoy/CodingForMedicine/blob/main/exercises/Python_principles.ipynb) exercise.)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction_var = ['perimeter_mean', 'compactness_mean', 'concavity_mean']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now create our input 'x' variables and our output 'y' variables for the training and test sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sMOSm3EV5M6O"
   },
   "outputs": [],
   "source": [
    "train_x = train[prediction_var]\n",
    "train_y = train.diagnosis\n",
    "\n",
    "test_x = test[prediction_var]\n",
    "test_y = test.diagnosis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now import our model. We're going to use the 'MLPClassifier' from sci-kit learn. 'MLP' stands for 'multi-layer perceptron' and is a basic type of neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can declare our model and it's variables as below. Here, we're using a neural network with 2 hidden layers with 5 neurons. (To understand the different model variants, you can check out the documentation [here](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html).) Feel free to modify these variables and see how the results change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5, 2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, finally, we can train our model and generate predictions on our 'test' set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLPClassifier(alpha=1e-05, hidden_layer_sizes=(5, 2), solver='lbfgs')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(train_x, train_y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(test_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That was it. In two lines of code, we just trained a neural network and used it to make some predictions. We first called **model.fit()** with our training data inputs and outputs, and then we caleld **model.predict()** on our test data inputs.\n",
    "\n",
    "In the next section, we'll look at how good the predictions are."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Quantifying performance of our model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### F1 score, accuracy and AUC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common way to quantify model performance is the F1 score. This tests whether the model is making balanced predictions (and not over-predicting or under-predicting the number of cases of malignant cancer).\n",
    "\n",
    "The score ranges from 0 to 1, with 0 being really bad and 1 being perfect."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "adKPWdrQBT86"
   },
   "source": [
    "The equation for F1 score is:\n",
    "\n",
    "![F1 equation](https://wikimedia.org/api/rest_v1/media/math/render/svg/1bf179c30b00db201ce1895d88fe2915d58e6bfd)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate it using sci-kit learn modules (noticing a pattern?), as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The f1 score is  0.93\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "\n",
    "f1_score = f1_score(test_y, predictions)\n",
    "\n",
    "print(\"The f1 score is \", \"%.2f\" %f1_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other popular metrics are the overall accuracy and the AUC score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fIHWqdQmAkL7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The accuracy is  0.93\n",
      "The AUC is  0.93\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import roc_auc_score \n",
    "\n",
    "accuracy = accuracy_score(test_y, predictions)\n",
    "AUC = roc_auc_score(test_y, predictions)\n",
    "\n",
    "print(\"The accuracy is \", \"%.2f\" %accuracy)\n",
    "print(\"The AUC is \", \"%.2f\" %AUC)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The confusion matrix "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A helpful way to visualise our results, beyond just single-point scores, is the confusion matrix. This involves plotting actual classes vs predicted classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oXhIGmxMDZlj"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[42,  4],\n",
       "       [ 2, 38]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "conf_matrix = confusion_matrix(test_y, predictions)\n",
    "conf_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To better understand this matrix, let's plot it in a more visually-aesthetic way, using the popular visualisation library: [**matplotlib**](https://matplotlib.org)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAHhCAYAAAAFwEUqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAnn0lEQVR4nO3deZgdVZnH8e8rEAgBwhb2JUFAFJB1WFSWkU0EZVhUBhWiICoiiCLCICIioCjighFwwAgIiCiogLIGRPZF9h0SBmQLZGdLgHf+qAo0nT6dvsntVHf4fp7nPtW36lTVezvp/nVVnToVmYkkSZrRu5ouQJKkvsqQlCSpwJCUJKnAkJQkqcCQlCSpwJCUJKnAkJRmUUSsGxFXRsT4iMiI+G4v7Wd4vf0te2P7c5P6+zSy6To09zAk1e9ExIIR8bWIuDYixkXEtIh4NiIuqQNl3jlQw7zAH4HVgCOAzwJ/6u39NiUihtYBlBFxUaHNfBExtm4zZjb29V+99QeH1KpwMAH1JxGxKnAxsDpwBXAZ8DywFLB1/fpRZh7Sy3WsDjwIfCMzf9LL+5oHmA+Ymplv9Oa+uqlhKDAaeKWuZcXMfLpTm12B8+s2z2bm0Fnc10hgr8yMWVh3AeD1zJw2K/uWOuv1v7ildomIgcBFwCrArpnZ+cjthxHxH8B/zIFylqmn43p7R5n5OvB6b++nh/4K7Ex15Hx8p2WfB+4C5gEWmlMF1f8vpmXma5n5ypzar94ZPN2q/mQf4D3ACV0EJACZeUtmjug4rz59d11ETKlf10XETp3XjYgxEXF1RKwRERdHxOSImBgR50fEMh3aXQ1cU7/9TYfTkEO7u35Yb3tMp3kfiIi/RcQzEfFKRPy7Pm28SYc2XW4zIpaMiF9GxBMRMbWe/jIilujUbvr6H46IgyPi0Yh4NSIeioi9uvo+duM54BLgc532sSywHfCbrlaKiI0iYmS9z5fq7+11EbFz5+8RsFf9dXZ4Da/njazfD4mI0yPiWeBFYIUO64zssL396nlHdNrPcvWp4fsjYsEWvwd6B/FIUv3JbvX01J6uEBH7Ab8EHgC+DyQwHLgwIr6YmZ23tTxwNXAB8E1gHeCLwCLAtnWbY4DrgP+pa7m2nj+2lQ8TEe8BLgeeAX4GPEt1hPrBer83drPuYOB6YFXgdOB2YD3gy8CHI2KjzJzcabVjgYHAKcCrdduREfFIZl7XQumnU33/Ns3MG+p5e1Ed7Z5F9cdMZzsDawDnAY8DS9Tr/CkiPp2ZZ9ftjqH6430zqqPV6a7vtL3p37ejgUHAlK4KzcwREbEVcGREjMrMf0bEu+o6Fwa2zsyXev7R9Y6Tmb589YsX8AIwqYX2i1H98nwEWKTD/EWAR4HJwKId5o+hCtFPdtrOL+v5a3SYt2U9b3intsPr+Vt2Uc/VwJgO7w+o2240k88xwzapwiSB/Tq1/Uo9/+gu1v8XMKDD/OWpwvKcHnwvh9bbOInqj+tngFM7LH8AOL/++p6On7OeN6iLbS5IdV33vk7zR1a/mrqsY2Rdx1mF5QmM7OL/wRjg/+qvj6jb7d/0/2lfff/l6Vb1J4sAk1povw3VUcbPM/PN9eqvf0F13WzrTus8lZnndZp3VT1dtbVyZ2piPd2p7nDSip2pjlw7HwmfQtWRaecZ1oARmTl1+pvM/DfwEFUP3R7LzNeAM4FP1T2NP0h1Gvz0btZ5cfrX9TpLUIXkVcB7I2KRVmoAftxCveOBPYBlgb8BRwJ/ycyTWtyn3oEMSfUnk6hOkfXUsHp6bxfL7qmnq3Sa/1gXbV+op0t0sWx2nEvVQ/d/gHERcVVEfCsiVu7BusOAB+vAelP9/kFm/FxQ/myz8rlOp/qjZReqDjtPAZeWGkfEUhFxaodriM9ThfyX6iaLtrj/h1ppnJnXAz8ENq73+/kW96d3KENS/ck9wCIR0VUAdKXlWwjovhdpT7bX3T1Vb+sDkJmvZuY2VL+4j6v3/T3ggc4dWtqk9Nla/j5l5v3ATVSndz8JnJFVL9wZNx4RVLfq7AWcAXwK+AjVkf70a5Et/S7KFq8jRsQAqo5FAIsDK7Wyvt65DEn1J3+sp111DOnKo/V0zS6Wva+ednV0NTum3xKyeBfLhnUxj8y8OTOPrgNzVaojre/PZD+PAe/pPHBC/X512v+5unI6sAnVaesue7XW3k/VEekHmfnNzDwvMy/NzCuobhfprDdu3j4O2BA4hOqMxLkRMagX9qO5jCGp/uR/qU4lHtzVLRwAEbFB3aMVqh6QLwJfjYiFO7RZGPgqVaeey9tc4/TTgG+71hkR/w0s12nekl2s/yTV6cCuQrajC4EhzPgHwxfq+Rf0rNzZci5wFHBgZnZ3+nP6EebbjlgjYi26vnY6pV4+s+9Bj0TE9sBBwG8z80dUHZlWp+qEJHXLW0DUb2TmSxGxI9WIOxdGxGVUIfcCVTD8J9UptePr9hMi4hCq3qk3dbh/bjjVEdsXM3MibZSZD0bEFcAX69OMdwDrUoXBI1Sj1Uz37YjYlmqAhNFUIfIxqlslOt+o39nxwCeAX0bE+lQ9V9cD9qb6Q2Jm68+2ugPUd3vQ9H6q68KH1PckPkgVUl+kOoW+fqf2NwL7AyMi4mJgGnBTZo5utcb6/s3fAg/X2yQzL46InwEHRsSlmXluq9vVO4chqX4lMx+JiPWofsHuChxOdbpvHHAr1XWvszu0HxERT1Pd83hkPftOYOfMvLCXyvwsVe/ZT9dfX0sV4L+iupViugupelx+ElgaeJnql/kXgNO620FmTqx7lR4FfJzq5v5ngZOBI3PGeyQbk5mvR8QOVD1S96LqcXxP/fU6zBiS51AF/u5Ufwi8i+rztRSS9f2QZ1J1MNouMzveS3kIsDlwSkTMUgDrncGxWyVJKvCapCRJBYakJEkFhqQkSQWGpCRJBYakJEkFhqRmWUR8JCIejIhHIuLQpuuR+qr62ZfPRcQ9M2+tvsSQ1CyJiHmobtLfnmqIt/+OiPd1v5b0jjWSarxa9TOGpGbVRsAjmflY/filc4Euh4qT3uky8x+8Na6v+hFDUrNqeeCJDu+frOdJ0lzDkNSs6urxSg7fJGmuYkhqVj0JrNjh/QpUD96VpLmGIalZdQuwWkQMqx9ouzvwl4ZrkqS2MiQ1SzLzNapHD11K9Sik8zLz3markvqmiDgHuIHqQdlPRsTeTdeknvEpIJIkFXgkKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBIanZFhH7Nl2D1B/4s9L/GJJqB3/wpZ7xZ6WfMSQlSSroV4MJDF50sVxqmeWaLkOdTJwwnsGLLtZ0GepkkUEDmy5BnTz//FiWXHJI02Wok7vvvnvS1KmvDu5q2bxzupjZsdQyy/GzU89tugypX9j6A2s2XYLULyy91JLPlZZ5ulWSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkAkNSkqQCQ1KSpAJDUpKkgnmbLkD9x52338T/HPQFAH79u4tYboWVAHj4wfsYddlfufP2m3n26X8z/8CBrDz03Xzi0/uw3oabNFmy1OeMGnUVH9l2GwDuvf9BVl111YYrUnc8klSPTJs2jREnHssCAwfOsOxP545k1OWX8N611mXv/Q5mt90/x4Tx4/j2N/bl4gt/30C1Ut80depUDjzgqwwaNKjpUtRDhqR65ILf/5Ypkyey3Y67zrDsY7vuwW//cBn7f+MItv/4buz8qT058ZSzWX7FlTnjf3/B66+91kDFUt/z0xN/wvhx4/j83vs0XYp6yJDUTD33zFOce+apDN/3awwatNAMy9+31roMmH/+t82bf/4F2GjTLZgyeRLjxz0/p0qV+qzHH3+c4449hu8fcyyDBw9uuhz1UKMhGREfiYgHI+KRiDi0yVpUdvLPf8CwVVZn6+13amm9F55/jnnmmZeFFl6klyqT+o+vH3Qga6/9fvbca3jTpagFjXXciYh5gF8C2wBPArdExF8y876matKMbr7+Gm654R/85OSziYger/d/Yx7l+muvZOMPbsECAxfsxQqlvu/iiy/ikosv5rrrb2zp50jNa/JIciPgkcx8LDOnAucCrR2qqFe9+uornPzzH7DtDruw2nve1+P1XnpxCscdeTDzz78AX9j/kF6sUOr7Xn75Zb5+0Nf43Of3Zv0NNmi6HLWoyZBcHniiw/sn63nqI35/5q95ccpk9tznqz1e59VXX+Gow77KM089yRHH/Iylll62FyuU+r4fHHcsEydM4HtHf7/pUjQLmrxPsqtzDjlDo4h9gX0BhvgLd4554fnn+NPvf8tOu32GF6dM5sUpkwGYPGkSAGOfe5p3zfMulll2hTfXmTZtGt//9td44N47OfzoE1l73Q0bqV3qK5566ilO/MkJfPWAA5kwYQITJkwAYNy4cQA88cT/Mc888zBs2LAGq1R3InOGXJozO47YFPhuZm5Xvz8MIDOPK62z2hpr5s9OPXcOVfjO9ujDD3DAPp/sts2ghRbmvIuvA+D1117j2CO/wc3XX8PB3z6OLbbafk6UqW5s/YE1my7hHe/OO+5go//o/hTr4MGDee75cXOoInVl6aWWfGT8uHGrdbWsySPJW4DVImIY8G9gd2CPButRB8ssuzyHHfXjGeZfO+oy/nn1ZXzpwMMYsvQyALzxxhuccOzh3PjPUXz14O8YkFJt6LBhnHPujANqnH/+H/jj+edz4k9/zoorrdhAZeqpxkIyM1+LiP2BS4F5gNMz896m6tHbDVpoYT605bYzzH989CMAbLDRB98clu60ESdwzZV/Y+11N2TA/Atw1WUXvW2d9TbclMUWX6L3i5b6mMGDB7PLrrvNMP/ee6tfddtut53D0vVxjY7dmpmXAJc0WYNm36MP3w/A3Xfcyt133DrD8uN+epohKalfauya5KzwmqTUc16TlHqmu2uSDksnSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVKBISlJUoEhKUlSgSEpSVLBvLOzckTMC+wELA78NTOfaUtVkiT1AT0+koyI4yPilg7vA7gCOA84Bbg7It7d/hIlSWpGK6dbPwJc2+H9x4DNgR8Be9TzDm1TXZIkNa6V060rAg93eP8xYHRmHgoQEWsCn25jbZIkNaqVI8kBwOsd3v8n1enW6R4Dlm1HUZIk9QWthOQTwCbw5lHjKsA1HZYvBUxpX2mSJDWrldOt5wJHRMRSwJrAJOCSDsvXAx5tY22SJDWqlSPJ44CRwKZAAntm5gSAiBgMfBy4ss31SZLUmB4fSWbmq8De9auzyVTXI19qU12SJDVutgYTmC4z3wAmtmNbkiT1FS0NSxcRK0bE6RHxZERMjYgP1/OH1PP/o3fKlCRpzmtlxJ1hwK3ArsC9wDzTl2XmWGBDYJ92FyhJUlNaOd16DPAGsBbwMvBcp+WXUA0wIEnSXKGV061bAyMy8wmq3q2dPQ6s0JaqJEnqA1oJyUWAp7tZPoA2dQSSJKkvaHXEnTW7Wb4J8MjslSNJUt/RSkj+Cfh8RKzVYV4CRMSuwCeoHpslSdJcoZWQPAZ4ErgJOIsqIA+NiBuowvFO4IS2VyhJUkN6HJKZOYlqSLr/pbrdI4BtgPcAI4D/zMxXeqNISZKa0FJHmzooDwQOjIghVEE5NjO76u0qSVK/Nsu9UesBBCRJmmv1OCQjYvOetMvMf8x6OZIk9R2tHEleTdeDCHQ2z8ybSJLU97USkp8rrP9uYDgwBjhl9kuSJKlvaOV5kr8tLYuIHwG3t6UiSZL6iJYelVWSmeOpbg05pB3bkySpL2hLSNbGA6u0cXuSJDWqLSEZEQsAnwWeacf2JEnqC1q5BeT0wqLFqUbiGQJ8sx1FSZLUF7TSu3V4Yf444CHgoMw8e7YrkiSpj2ild2s7r19KktTnGXySJBUYkpIkFRRPt0bEVbOwvczMrWajHkmS+ozurkmuQs/GapUkaa5UDMnMHDoH65Akqc/xmqQkSQWGpCRJBa0MJkBELAbsDWwMLMaMIWvHHUnSXKOVYelWBq4DlgMmAotQjbYzPSyfB17shRolSWpEK6dbvw8sCmwFrAYE8CmqsDwOmAxs1ub6JElqTCshuRXw68wcxVu3hkRmvpSZhwN3Az9sd4GSJDWllZBcArin/npaPR3YYfnlwDbtKEqSpL6glZAcS/VYLKhOrb4CDO2wfABvD01Jkvq1VkLyXmAdqLqwAjcD+0XEShExFNgXeKDtFUqS1JBWbgH5M/CNiBiYmS8D3wMuBUbXyxPYpc31SZLUmG5DMiI2ycwbATJzBDBi+rLMvCoiNgX2AF4HLsjM63uzWEmS5qSZHUleHxH3AacDZ2bm2I4LM/NW4NbeKk6SpCbN7JrkmcDKwI+BJyPijxGxQ0Q4nJ0kaa7Xbdhl5l7AssAXqI4Ydwb+AjwREcdGxGq9X6IkSc2Y6RFhZk7JzNMy84PAe4DjqTrpHAo8EBHXRMSeEbFgL9cqSdIc1dJp08x8ODMPA1YCdgT+RDXY+W+ApyPi1PaXKElSM2bp2mJmvpGZl2TmJ6gGPD8fWJjqCSGSJM0VWnpUVkcRsRywJzCcasBzgFvaUJMkSX1Cq8+TnA/YCfgcsC0wD9Ujsn4GnJaZ97a9QkmSGtKjkIyIdYHPUw0csBhVx53LqO6f/HNmTiuvLUlS/zSzEXcOoDpqfD/V8yNHAz8FRmbmk71enSRJDZrZkeRPqZ72cQ7V6dRRvV6RJEl9xMxCcn/gd5k5cU4UI0lSX9JtSNaDmvcZgxcayHYfWrvpMqR+4dJbRs+8kSQmTHm1uMwxWCVJKjAkJUkqMCQlSSowJCVJKjAkJUkqMCQlSSoo3gISEVfNwvYyM7eajXokSeozurtPchWqMVolSXpHKoZkZg6dg3VIktTneE1SkqQCQ1KSpIJWH7q8GLA3sDHVcyU7h6wddyRJc40eh2RErAxcBywHTAQWAcbxVlg+D7zYCzVKktSIVk63fh9YFNgKWI3qIcyfogrL44DJwGZtrk+SpMa0EpJbAb+uH7w8/daQyMyXMvNw4G7gh+0uUJKkprQSkksA99RfT6unAzssvxzYph1FSZLUF7QSkmOBxeuvJwOvAEM7LB/A20NTkqR+rZWQvBdYB6ourMDNwH4RsVJEDAX2BR5oe4WSJDWklVtA/gx8IyIGZubLwPeAS4HR9fIEdmlzfZIkNabHIZmZI4ARHd5fFRGbAnsArwMXZOb17S9RkqRmtDSYQGeZeStwa5tqkSSpT3FYOkmSCloZcef0HjTLzNx7NuqRJKnPaOV06/AetEmqsV0lSer3eny6NTPf1fkFzAe8B/g1cCPVOK6SJM0VZuuaZGa+npkPZ+YXgRdwWDpJ0lyknR13/gbs2sbtSZLUqHaG5BLAQm3cniRJjZqt+yQBImJRYGvgIOC22d2eJEl9RSu3gLzBW4/ImmEx1QOYv96OoiRJ6gtaOZI8gxlDMqnC8SHgnMyc3K7CJElqWitjtw7vxTokSepzetxxJyK+ExFrdbN8zYj4TnvKkiSpea30bv0u8P5ulq8FHDlb1UiS1Ie08xaQBYDX2rg9SZIa1e01yYhYBFi0w6wlImKlLpouDnwaeKJ9pUmS1KyZddw5CJh+nTGBn9avrgRwSFuqkiSpD5hZSF5dT4MqLC8A7urUJoEpwI2ZeX1bq5MkqUHdhmRmXgNcAxARKwMnZ+ZNc6IwSZKa1sp9kp/rzUIkSeprWrlP8isRcUU3yy+LiC+2pyxJkprXyi0gw4GHu1n+EPD52apGkqQ+pJWQXA24u5vl99ZtJEmaK7QSkvNRDRhQssBMlkuS1K+0EpIPAdt0s3xb4NHZK0eSpL6jlZA8B9g2Io6OiAHTZ0bEfBFxFFVInt3uAiVJakorz5M8EdgeOBz4ckQ8QDWQwHuphqW7Fjih7RVKktSQHh9JZuY0qqPFQ4EngfWA9anGaz0E2Dozp/ZGkZIkNaGVI8npQXl8/ZpBRMyfma+2ozBJkprWlkdlRcQGETECeKod25MkqS9o6Uiyo4hYHPgMsDfVA5eDqgesJElzhZaPJCNiu4j4PfBvqs48A4CjgLUzc4021ydJUmN6dCQZEcOAzwF7ASsAY4HzgT2AwzPzT71WoSRJDen2SDIi9oiIK6nGbD0EuBXYGVie6ugxer1CSZIaMrMjybOAx4CvAWdn5rjpCyIie7EuSZIaN7NrklOBocBOwPYRMbDXK5IkqY+YWUguQ3UUuQRwJvBsRJwWEZvjqVZJ0lyu25DMzAmZeVJmrg9sSBWU/wWMAv5JNSzd4N4uUpKkJrQyLN3tmfkVYDngs1TPjwT434i4IyK+HRFr9kaRkiQ1oeX7JDPz1cw8OzO3At4NHAMsBnwPuLPN9UmS1JjZGpYuM8dk5neoOvd8FPB+SUnSXGOWh6XrKDMT+Hv9kiRprtCWAc4lSZobGZKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVzNt0AepfbrvtNn531pmMGnUVo0ePZtCgQbxvzTX51rcOY+utt266PKkx/zf6Ec457ec88sDdjHthLBHvYtkVVmKbHXZj+132YL75BrzZ9rln/s3Zp/2cu269gfHjxrLYEkux3kYfYvfPfYUhSy/X4KdQZ4akWnLCj3/EVVddyS677Mp+++3PlBen8NuRv+Ej223DSSeN4Etf/nLTJUqNeP65p5k8aQKbb/MxllxqGV5//XXuv+s2Tv3p0dx52w0ccfwpAEyaOJ6v770L06ZNZYddPsPSy67A4489yN8uPIdbrhvFr875O4MWWqThT6PpDEm1ZP+vHsDpvxnJAgss8Oa8L33py2yw/rocccTh7POFLzDvvP630jvP+htvxvobb/a2eTvu9lkWWmQwF51/Jk8+/hgrrLwK/7j8Isa/MJYjjj+VTTZ/6+zL0sutyKknHs3tN/2Tzbb66JwuXwVek1RLPvCBD7wtIAEGDhzIR3fYkfHjx/PMM880VJnUNy21zPIATJk8CYCXXpoCwBJDlnpbu8WXXBqABQYOnIPVaWYaC8mIOD0inouIe5qqQe3z9FNPMe+887LYYos1XYrUqFdeeZmJE8bx7FNPcs3lf+X8s05l8SWXYtiqawCwzgYfAODkE47ivrtu4/nnnuFfN13LGSf/mDXWWo/1N9qsu81rDmvyvNhI4CTgjAZrUBvcd999XHDBn/jYxz7OoEGDmi5HatQfzzyFs0/7+ZvvV3/fOux/6DHMX5+Bec+a6/Dlg4/ijFNO4Jv7fuLNdht98MMccvTPmMfLFX1KY/8amfmPiBja1P7VHpMmTWL3T32CBRdckBN+cmLT5UiN+/BHd+F962zI5IkTuOv2Gxj98AO8WJ9qnW6JIUuzxprrsd7GH2LZ5VdmzCMP8MffncpRB3+Bo35y+puBqub5J4tm2csvv8xOO32Mxx57jIsv+TsrrbRS0yVJjVt2+ZVYdvnqZ2HzbXbkgnNO44gD9+IXZ17MSsNW5bpRf+eH3z6AX5x5ESuvsjoAm2y+Ne9eY02++/W9ueSC37Hzf+/d5EdQB32+405E7BsRt0bErWPHjm26HNWmTp3KrrvszI033MC5v/8DW2yxRdMlSX3Sltt+nNdem8aov18IwF/OG8lyKw59MyCn23DTLZl/gYHce8ctDVSpkj4fkpl5amZumJkbDhkypOlyBLz22mvs/qlPcsUVlzNy5BnsuOOOTZck9VnTpk4FYMrkiQCMf2Esb7zx+gztMpPMN3jttWlztD51r8+HpPqWN954g+F77clf/vJnRow4mU/tvnvTJUl9woRxz3c5/5ILfgdUHXgAVlj53Tz1xBgeuOeOt7W79sqLmfrqq6y2xtq9Wqda09g1yYg4B9gSWDIingSOzMzTmqpHPfPNbx7Mueeew+ZbbMHAgQP53VlnvW351ttsw9JLL91QdVJzTvrht5k0cTxrr78JQ5ZalhenTOL2m67ljluu471rb8B/fmQnAHb77Be57YZr+PYBe7LDrp9mmeVXYswjD/D3C89l8SWXYoddP9PwJ1FHkZlN19BjG264Yd50861Nl/GO9uEPb8k/rrmmuPyKK0ex5ZZbzrmCVHTpLaObLuEd5R+XX8QVF/+RMY8+wMTx45hvwACWX2kYm221Ax//5HAGzD//m21HP3w/55z+Cx66/y7GPz+WhQcvyvobb8Zn9j3ozcEHNOfssNlaj+TUF1frapkhKc2lDEmpZ7oLSa9JSpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVGBISpJUYEhKklRgSEqSVBCZ2XQNPRYRY4HHm65DM1gSeL7pIqR+wJ+VvmnlzBzS1YJ+FZLqmyLi1szcsOk6pL7On5X+x9OtkiQVGJKSJBUYkmqHU5suYG4WEUMjIiPiu93N6619qa38WelnDEnNtsycK3/wI2LLOjA6vqZExG0RcWBEzNN0jbOiDsLvRsS6TdfyTjO3/qzMzeZtugCpHzgHuAQIYDlgOPBTYE1g34ZqehwYCLw2C+sOBY4ExgB3tHG70lzHkJRm7vbMPGv6m4j4FXA/sE9EHJGZz3ZeISIWzszJvVVQVt3SX+kv25X6K0+3Si3KzEnADVRHlqtExJiIuDoi1ouISyNiInDX9PYRsVpEnBkRT0fE1Lr9jyJiUOdtR8SHIuK6iHg5Ip6NiJOAhbpoV7x2GBG7RsSoiJgQES9FxIMR8fOIGBARw4FRddPfdDiNfHV3242IeSPiWxFxX0S8EhEvRMQFEbF2qa6I2DEibqnbP11/5nk7tV8zIv4QEf+OiFcj4pm69h168E8h9TqPJKUWRUQAq9Zvp98YvhJwFfAH4I/UwRYRG9TzJwCnAP8G1gEOAD4YEVtk5rS67cbAFcBk4If1OrsDZ7RQ2zHA/wD3AScCTwPvBnYFvgP8Azi2bnMqcG296gxHw538DvgkcDnwK2AZ4CvADRGxWWb+q1P7jwL7AScDpwM7AQcD4+v9ExFLUH1vqNs9TnWz/YbAxsDFPf3cUq/JTF++fHXxArYEkipclgSGAO8Hfl3Pv6FuN6Z+v08X27gTeABYuNP8net1hneYdz0wFVi9w7wBwM112+92mD+0i3kb1fOuAhbotL/grcFDtuy875lsd5t63u+nb6Oe/36qa5fXdrH+i8DQTvu/B3i6w7yP120/2fS/tS9fpZenW6WZOwoYCzxHFXqfB/4C/FeHNuOA33RcqT4V+X7gbGD+iFhy+gv4J1WQbFu3XQrYFPhzZj40fRuZOZXqiLAnPl1PD8vMt11XzFoPt9PZzvX0mI7byMy7gIuAD0VE5yG9LszMMR33T3Wad5mImH76eGI93T4iFpnF2qReZUhKM3cq1dHU1lRBNiQzd8q3d9h5NDNf77Tee+vp9JDt+HoOGAQsXbdZpZ4+0MX+7+thnatRHZnd2cP2PTUMeIOqs1Jn93Ro09FjXbR9oZ4uAZCZ11CdSh4OPF9fiz0qIt432xVLbeI1SWnmHs7MK2bS5qUu5kU9PQH4e2G98Z3adnW0F13M60oU1p9dPd1/R53/YOhye5m5V0T8iOoa5oeAbwCHR8TXMvOkWdiv1FaGpNR7Hq6nr/cgZB+tp+/tYllX87ryIPARqlO8N3fTrtUgfRTYrq7jrk7Lph/1jW5xm28Vk3kP1RHp8RGxKHAT8IOI+OVsnCKW2sLTrVLv+RfVL/8vRcQqnRfWt1UsDpCZzwE3AjtFxOod2gwADurh/s6up8dGxPxd7G/6EdyUerp4D7d7YT09rMM2iIi1qDrf/DMzx/ZwWx3rWTwi3vY7KDMnUAXugsACrW5TajePJKVekpkZEZ+l6m16V0ScDtxLFQCrArsAhwEj61W+DlwNXBcRv+StW0B69HOamTdHxA+BbwG3RcTvgWeorhfuRtX7dQLVNc7JwH4R8VI977nMvKqw3csj4ry6lsUi4iLeugXkFarbWWbFnsBBEXEB8AgwDdiC6qj1vMx8eRa3K7WNISn1osy8IyLWowrDjwNfogqoMVTheGWHtjdExDbAD4BDgUlU913+Cri7h/s7NCLuBPYHDqE6W/QE1bB6L9VtXo6I3YHvUw2vNz9wDW/ds9iVTwO3U3WyOYGqZ+41wBGZ2aPaunA1sB6wI7As1XXM0VT3U3o9Un2CD12WJKnAa5KSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBUYkpIkFRiSkiQVGJKSJBX8Pz3UyExRMlcAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 540x540 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7.5, 7.5))\n",
    "ax.matshow(conf_matrix, cmap=plt.cm.Blues, alpha=0.3)\n",
    "for i in range(conf_matrix.shape[0]):\n",
    "    for j in range(conf_matrix.shape[1]):\n",
    "        ax.text(x=j, y=i,s=conf_matrix[i, j], va='center', ha='center', size='xx-large')\n",
    "\n",
    "plt.xlabel('Predictions', fontsize=18)\n",
    "plt.ylabel('Actual Values', fontsize=18)\n",
    "plt.title('Confusion Matrix', fontsize=18)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hopefully, the confusion matrix is now self-explanatory!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have a go at modifying our pipeline, including:\n",
    "\n",
    "- Different features (using the 'prediction_var' list)\n",
    "- Different parameters (such as different learning rates and neural network layers, or different train/test splits)\n",
    "- Different models (see below)\n",
    "\n",
    "Popular classifiers to try as alternative models include:\n",
    "\n",
    "- [Random Forest](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)\n",
    "- [K-nearest neighbours](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)\n",
    "- [Support vector machines (SVMs)](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)\n",
    "\n",
    "These links will take you to the documentation. See if you can work out how to import them and then train the model based on the description and examples that they provide.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fill out the form below and we'll provide feedback on your code.\n",
    "\n",
    "**Any feedback on the exercise? Any questions? Want feedback on your code? Please fill out the form [here](https://docs.google.com/forms/d/e/1FAIpQLSdoOjVom8YKf11LxJ_bWN40afFMsWcoJ-xOrKhMbfBzgxTS9A/viewform).**"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "ML medics exercise.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "data_science",
   "language": "python",
   "name": "data_science"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}