notebooks/12-feature_scaling_model_importance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Scaling and Model Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Learn and implement different approaches to feature scaling, and when to use it\n",
    "* Building some grasp of when to use what machine learning algorithm\n",
    "* Apply model evaluation skills to a difficut data set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Objectives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll be working with a dataset that can be difficult to accurately predict! We'll learn why it's important to go beyond just using accuracy when working in classification problems, hypothesize some approaches to working with the features we have, and have a discussion on what are we gaining versus losing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Class Notes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Data\n",
    "\n",
    "Included in the repository today is a used car sale data set. Many features relate to cost, though they are all defined in the data dictionary here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Field Name                          Definition\n",
      "RefID                               Unique (sequential) number assigned to vehicles\n",
      "IsBadBuy                            Identifies if the kicked vehicle was an avoidable purchase\n",
      "PurchDate                           The Date the vehicle was Purchased at Auction\n",
      "Auction                             Auction provider at which the  vehicle was purchased\n",
      "VehYear                             The manufacturer's year of the vehicle\n",
      "VehicleAge                          The Years elapsed since the manufacturer's year\n",
      "Make                                Vehicle Manufacturer\n",
      "Model                               Vehicle Model\n",
      "Trim                                Vehicle Trim Level\n",
      "Submodel                            Vehicle Submodel\n",
      "Color                               Vehicle Color\n",
      "Transmission                        Vehicles transmission type (Automatic, Manual)\n",
      "WheelTypeID                         The type id of the vehicle wheel\n",
      "WheelType                           The vehicle wheel type description (Alloy, Covers)\n",
      "VehOdo                              The vehicles odometer reading\n",
      "Nationality                         The Manufacturer's country\n",
      "Size                                The size category of the vehicle (Compact, SUV, etc.)\n",
      "TopThreeAmericanName                Identifies if the manufacturer is one of the top three American manufacturers\n",
      "MMRAcquisitionAuctionAveragePrice   Acquisition price for this vehicle in average condition at time of purchase\n",
      "MMRAcquisitionAuctionCleanPrice     Acquisition price for this vehicle in the above Average condition at time of purchase\n",
      "MMRAcquisitionRetailAveragePrice    Acquisition price for this vehicle in the retail market in average condition at time of purchase\n",
      "MMRAcquisitonRetailCleanPrice       Acquisition price for this vehicle in the retail market in above average condition at time of purchase\n",
      "MMRCurrentAuctionAveragePrice       Acquisition price for this vehicle in average condition as of current day\n",
      "MMRCurrentAuctionCleanPrice         Acquisition price for this vehicle in the above condition as of current day\n",
      "MMRCurrentRetailAveragePrice        Acquisition price for this vehicle in the retail market in average condition as of current day\n",
      "MMRCurrentRetailCleanPrice          Acquisition price for this vehicle in the retail market in above average condition as of current day\n",
      "PRIMEUNIT                           Identifies if the vehicle would have a higher demand than a standard purchase\n",
      "AcquisitionType                     Identifies how the vehicle was aquired (Auction buy, trade in, etc)\n",
      "AUCGUART                            The level guarntee provided by auction for the vehicle (Green light - Guaranteed/arbitratable, Yellow Light - caution/issue, red light - sold as is)\n",
      "KickDate                            Date the vehicle was kicked back to the auction\n",
      "BYRNO                               Unique number assigned to the buyer that purchased the vehicle\n",
      "VNZIP                               Zipcode where the car was purchased\n",
      "VNST                                State where the the car was purchased\n",
      "VehBCost                            Acquisition cost paid for the vehicle at time of purchase\n",
      "IsOnlineSale                        Identifies if the vehicle was originally purchased online\n",
      "WarrantyCost                        Warranty price (term=36month  and millage=36K)\n"
     ]
    }
   ],
   "source": [
    "lines = open('../data/lemons_description.txt')\n",
    "for line in lines:\n",
    "    print line.strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RefId                                  int64\n",
      "IsBadBuy                               int64\n",
      "PurchDate                             object\n",
      "Auction                               object\n",
      "VehYear                                int64\n",
      "VehicleAge                             int64\n",
      "Make                                  object\n",
      "Model                                 object\n",
      "Trim                                  object\n",
      "SubModel                              object\n",
      "Color                                 object\n",
      "Transmission                          object\n",
      "WheelTypeID                          float64\n",
      "WheelType                             object\n",
      "VehOdo                                 int64\n",
      "Nationality                           object\n",
      "Size                                  object\n",
      "TopThreeAmericanName                  object\n",
      "MMRAcquisitionAuctionAveragePrice    float64\n",
      "MMRAcquisitionAuctionCleanPrice      float64\n",
      "MMRAcquisitionRetailAveragePrice     float64\n",
      "MMRAcquisitonRetailCleanPrice        float64\n",
      "MMRCurrentAuctionAveragePrice        float64\n",
      "MMRCurrentAuctionCleanPrice          float64\n",
      "MMRCurrentRetailAveragePrice         float64\n",
      "MMRCurrentRetailCleanPrice           float64\n",
      "PRIMEUNIT                             object\n",
      "AUCGUART                              object\n",
      "BYRNO                                  int64\n",
      "VNZIP1                                 int64\n",
      "VNST                                  object\n",
      "VehBCost                             float64\n",
      "IsOnlineSale                           int64\n",
      "WarrantyCost                           int64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn import tree\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "# Load in data and create sets. dropping all na columns on the live data set.\n",
    "lemons = pd.read_csv('../data/lemons.csv')\n",
    "lemons_oos = pd.read_csv('../data/lemons_oos.csv')\n",
    "print lemons.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below includes a very simple \"benchmark\" script. One common test we'd want to consider for model evaluation is if we can do better than random, which we can use the sklearn DummyClassifier to do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "did better than stratified? True\n",
      "did better than most_frequent? True\n",
      "did better than uniform? True\n"
     ]
    }
   ],
   "source": [
    "lemons = lemons.dropna(axis=1)\n",
    "# Generating a list of continuous data features from the describe dataframe. \n",
    "# Then, removing the two non-features (RefId is an index, IsBadBuy is the prediction value)\n",
    "features = list(lemons.describe().columns)\n",
    "features.remove('RefId')\n",
    "features.remove('IsBadBuy')\n",
    "\n",
    "best_score = -1\n",
    "for depth in range(1, 10):\n",
    "    scores = cross_val_score(tree.DecisionTreeClassifier(max_depth=depth, random_state=1234),\n",
    "                                lemons[features],\n",
    "                                lemons.IsBadBuy,\n",
    "                                scoring='roc_auc',\n",
    "                                cv=5)\n",
    "    if scores.mean() > best_score:\n",
    "        best_depth = depth\n",
    "        best_score = scores.mean()\n",
    "\n",
    "# Is the best score we have better than each DummyClassifier type?\n",
    "from sklearn import dummy, metrics\n",
    "for strat in ['stratified', 'most_frequent', 'uniform']:\n",
    "    dummyclf = dummy.DummyClassifier(strategy=strat).fit(lemons[features], lemons.IsBadBuy)\n",
    "    print 'did better than %s?' % strat, metrics.roc_auc_score(lemons.IsBadBuy, dummyclf.predict(lemons[features])) < best_score\n",
    "\n",
    "# seems so!\n",
    "\n",
    "# Create a classifier and prediction.\n",
    "clf = tree.DecisionTreeClassifier(max_depth=depth, random_state=1234).fit(lemons[features], lemons.IsBadBuy)\n",
    "#\n",
    "y_pred = clf.predict(lemons_oos[features])\n",
    "#\n",
    "## Create a submission\n",
    "submission = pd.DataFrame({ 'RefId' : lemons_oos.RefId, 'prediction' : y_pred })\n",
    "submission.head()\n",
    "submission.to_csv('submission.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Unnamed: 0  RefId  prediction\n",
      "0           0      3           0\n",
      "1           1      4           0\n",
      "2           2      8           0\n",
      "3           3      9           0\n",
      "4           4     10           0\n"
     ]
    }
   ],
   "source": [
    "csv = pd.read_csv('submission.csv')\n",
    "print csv.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a good start for us!\n",
    "\n",
    "In order for us to work on improving this model, we'll have to continue exploring the data set available, impute missing values, create new features, and scale numerical values. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Scaling\n",
    "\n",
    "Feature scaling can play a significant role in the performance of our model. Many of the techniques that follow are typically more applicable to algorithms were the algorithm is less dependent on learning weights; in particular, for classifiers. \n",
    "\n",
    "### Zeroing on the mean / Centering a feature\n",
    "\n",
    "One common technique is to subtract the mean away from all values in a feature. This effectively \"zeroes\" the feature, and is easier for the model to assert the normal centers of the data as being the same.\n",
    "\n",
    "$x^` = x - mean(x)$\n",
    "\n",
    "\n",
    "### Normalizing the feature\n",
    "\n",
    "Another common technique takes the above one step further: not only do we center the data on 0, but then provide a scope for the data to reside in (either -1 to 1, or 0 to 1, are typical). Normalizing the fe\n",
    "\n",
    "**Normalizing to 0 and 1 (where 0 remains 0)**\n",
    "\n",
    "$x^` = \\dfrac{x_0}{max(x)}$\n",
    "\n",
    "\n",
    "**Normalizing to 0 and 1 (where min == 0)**\n",
    "\n",
    "$x^` = \\dfrac{x_0 - min(x)}{max(x) - min(x))}$\n",
    "\n",
    "**Normalizing to -1 and 1 (where mean == 0)**\n",
    "\n",
    "$x^` = \\dfrac{x_0 - mean(x)}{max(x) - mean(x)}$\n",
    "\n",
    "**Standardization using mean and standard deviation (where mean = 0)**\n",
    "\n",
    "Standardization is a slightly different process for normalizing where our data splits are represented using standard deviations instead.\n",
    "\n",
    "$x^` = \\dfrac{x_0 - mean(x)}{std(x))}$\n",
    "\n",
    "### Practice: Writing python functions to handle these transformations\n",
    "\n",
    "Assuming the input of an array, how would we end up writing code to handle each of these transformations?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ True  True  True  True  True]\n",
      "[ True  True  True  True  True]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from __future__ import division\n",
    "class Transformations(object):\n",
    "\n",
    "    def mean_at_zero(self, arr):\n",
    "        return np.array([i - np.mean(arr) for i in arr])\n",
    "\n",
    "    def norm_to_min_zero(self, arr):\n",
    "        return np.array([i / max(arr) for i in arr])\n",
    "    \n",
    "    def norm_to_absolute_min_zero(self, arr):\n",
    "        \"\"\"should be a range of 0 to 1, where 0 maintains its 0 value\"\"\"\n",
    "    \n",
    "    def norm_to_neg_pos(self, arr):\n",
    "        \"\"\"should be a range of -1 to 1, where 0 represents the mean\"\"\"\n",
    "    \n",
    "    def norm_by_std(self, arr):\n",
    "        \"\"\"should be a range where 0 represents the mean\"\"\"\n",
    "\n",
    "## tests to make sure we built this correctly:\n",
    "transformer = Transformations()\n",
    "a = np.array([1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "print transformer.mean_at_zero(a) == np.array([-2, -1, 0, 1, 2])\n",
    "print transformer.norm_to_min_zero(a) == np.array([0.2, 0.4, 0.6, 0.8, 1.0])\n",
    "#print transformer.norm_to_absolute_min_zero(a) == np.array([0.0, 0.25, 0.5, 0.75, 1.0])\n",
    "#print transformer.norm_to_neg_pos(a) == np.array([-1.0, -0.5, 0.0, 0.5, 1.0])\n",
    "#print transformer.norm_by_std(a) == np.array([-1.414213562373095, -0.7071067811865475, 0.0, 0.7071067811865475, 1.414213562373095])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scikit learn has functions to also handle some of this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  2.  3.  4.  5.]\n",
      "[-2. -1.  0.  1.  2.]\n",
      "[-1.41421356 -0.70710678  0.          0.70710678  1.41421356]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "print a\n",
    "print preprocessing.scale(a, with_mean=True, with_std=False)\n",
    "print preprocessing.scale(a, with_mean=True, with_std=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solving Power Laws"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In class, we previously learned about linear transformations, particularly using log when we're looking at a log-log distribution.\n",
    "\n",
    "<img src='img/powerlaw.jpg'>\n",
    "\n",
    "However, transforming to log isn't always going to land us a perfect linear fit. Instead, we'd ideally like to solve for the power law, which is identifying a curve's amplitude and index. We can do this with one extra step: fitting a linear model to the log(10)-transformed data, optimizing against the error. scipy has a handy function to solve this for us.\n",
    "\n",
    "Another option could be to experiment with the [plfit](https://github.com/keflavich/plfit) library (not included in anaconda)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import scipy as sp\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "class PowerLaw(object):\n",
    "    def fit(self, x, y, transform=True):\n",
    "        \"\"\"\n",
    "        returns back the amplitude and index of a powerlaw relationship.\n",
    "        assumes the data is not already log10 transformed.\n",
    "        return: [index, amp], also stored on the instance\n",
    "        \"\"\"\n",
    "        if transform:\n",
    "            x = np.log10(x)\n",
    "            y = np.log10(y)\n",
    "        # define our (line) fitting function and error function to optimize on\n",
    "        fitfunc = lambda p, x: p[0] + p[1] * x\n",
    "        errfunc = lambda p, x, y: (y - fitfunc(p, x))\n",
    "        # defines a starting point to optimize from.\n",
    "        p_init = [1.0, -1.0] \n",
    "        out = sp.optimize.leastsq(errfunc, p_init, args=(x, y), full_output=1)\n",
    "        result = out[0]\n",
    "        self.index = result[1]\n",
    "        self.amp = 10.0**result[0]\n",
    "        return np.array([self.amp, self.index])\n",
    "    \n",
    "    def transform(self, x):\n",
    "        \"\"\"returns the x-transformed data\"\"\"\n",
    "        return self.amp * (x**self.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "amp: 0.895599738306 index -0.409432655356\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x117bbc5d0>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgIAAAFpCAYAAADunkDIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8XNWd9/HPnd7URtW2LMsFHxsXwAWDqbYJJBuedCCJ\nIQkkkGQJ2TQSlhQ2hM2ySwhJHlIWAkuWkk3gIWwghYQWB2zcMG7Yx0W2JdvqXaM+M88fd2Y0qpZs\naWak+b1f6KWZuXfunKuxmK/OOfd3jHA4jBBCCCHSkyXZDRBCCCFE8kgQEEIIIdKYBAEhhBAijUkQ\nEEIIIdKYBAEhhBAijUkQEEIIIdKYLVkvrJQqALYD64AQ8Fjk+x7gVq21XNcohBBCTLCk9AgopezA\nfwIBwAB+CNyptb40cv/9yWiXEEIIkW6SNTRwH/BzoDJyf5nWekPk9p+AK5LSKiGEECLNJHxoQCn1\nKaBWa/0XpdQ/Y/YAGHG7tAFZpziGE1iJGSSCE9RUIYQQIlVYgWnAVq1113geOBlzBG4EwkqpK4Bz\ngV8B+XHbM4CmUxxjJfD3iWmeEEIIkbIuAV4fzwMmPAhorS+L3lZKvQp8DrhPKXWZ1vpvwHuAl09x\nmEqAJ598kqKioglrqxBCCJEKqqqqWL9+PfQNqY+bpF01ECcMfBV4WCnlAN4BnjnFc4IARUVFFBcX\nT3DzhBBCiJQx7sPhSQ0CWus1cXcvT1Y7hBBCiHQlBYWEEEKINCZBQAghhEhjEgSEEEKINCZBQAgh\nhEhjaRMEgqEwf9x4hJrG9mQ3RQghhEgZaRMEGpo7+fn/28Ujv9+T7KYIIYQQKSNtgkBetotZRRls\n2VtNa3t3spsjhBBCpIS0CQKGYbB2RQm9wRAbdpxIdnOEEEKIlJA2QQDg8uXFWCwGL28tT3ZThBBC\niJSQVkHAn+limSrgYEUT5VUtyW6OEEIIkXRpFQQA1q2cCcAr2yqS3BIhhBAi+dIuCJx/dhE+t51X\nt1cQDIaS3RwhhBAiqRK+6JBSygo8DMzHXHnwc4ATeB44ENnt51rr307E6zvsVi49bwZ/3HiUHQdq\nWbGwcCJeRgghhJgUktEjcDUQ0lpfDHwL+FfgPOB+rfWayNeEhICodStLABkeEEIIIRIeBLTW/wt8\nNnK3FGgClgPvVUr9TSn1S6WUbyLbcNbMbGYW+nhzTyVtUlNACCFEGkvKHAGtdVAp9Svgx8CTwBbg\na1rry4Ay4K6JfH3DMFi3ooSe3hB/f1tqCgghhEhfSZssqLX+JKAw5wv8RWu9I7LpOcyhggl1+fJi\nLAa8LMMDQggh0ljCg4BS6nql1B2Rux1ACHhWKbUy8tg6YNtEtyM3y825qgB9rJGK6taJfjkhhBAi\nJSWjR+BZ4Dyl1N+APwP/BHwe+JFS6lXgQuCeRDRk3QqpKSCEECK9JfzyQa11O3DdEJsuSnRbVi2e\nhtdl49XtFVz/noVYLUaimyCEEEIkVdoVFIrntFu55Lxi6ps72XmwNtnNEUIIIRIurYMA9A0PyEJE\nQggh0lHaBwE1K4cZ+V7e3F1JoKMn2c0RQgghEirtg4BhGKxbWUJ3b4jXd0pNASGEEOkl7YMAwJrl\nMzEMeHmrXD0ghBAivUgQAPKy3ZxzVj77jjZwsrYt2c0RQgghEkaCQER0ISKpNCiEECKdSBCIuGBx\nEW6njVe2VRAKhZPdHCGEECIhJAhEuBw2Ljl3BnVNHew+VJfs5gghhBAJIUEgztpITYGXtklNASGE\nEOlBgkCcs2f7mZbrZeOuSto7paaAEKnuoYce4uKLL6a7u/uMjrN27Vq6u7t56KGH2LVr1xkd68EH\nH+Saa67hox/96JDHeu6553jf+97H+vXreeaZZwDo7OzktttuY/369dxyyy00NDScURuEGAsJAnHM\nmgIz6e4J8sbOk8lujhDiFJ5//nmuvvpq/vCHP4zL8W655RaWLl162s/fu3cvW7du5emnn+aBBx7g\n7rvv7re9oaGBn/zkJzzxxBM88cQTPP/885w4cYJf//rXKKV48skn+cAHPsDPf/7zMz0VIUYt4YsO\nASilrMDDwHwgDHwO6AIew1yWeA9wq9Y64bP21iyfyRN/3s/L2yp416pZiX55IcQobd68mdLSUq67\n7jpuv/12PvjBD3LDDTewYMECDh48iMfjYcWKFbz++uu0tLTw6KOP8tJLL/Hyyy8TCARobGzk1ltv\n5corr4wd84477uC9730vl1xySeyxH/3oR2zfvj123zAMHnnkEex2+6A2bd++nYsvvhiAadOmEQwG\naWhowO/3A3D8+HEWLFhAZmYmAEuWLGHnzp289dZb3HzzzQBccskl/OxnPxv/H5gQw0hKEACuBkJa\n64uVUpcB3488fqfWeoNS6ufA+4HnEt2wAr+HpfPy2HWojsq6ANPyvIlughBiFJ5++mk+8pGPMHv2\nbBwOR6wb/pxzzuGb3/wmn/nMZ3C73Tz66KPccccdbNmyBcMw6Ojo4LHHHqO+vp5rrrmGdevWxY5p\nGINXIP3Sl7406jYFAgGys7Nj971eL4FAIBYEZs2axaFDh6ivr8fj8bBp0yZKS0tpa2vD6/XGntPa\n2npaPxMhTkdSgoDW+n+VUi9E7pYCjcAVWusNkcf+BFxJEoIAmDUFdh2q45VtFax/94JkNEEIMYLm\n5mb+/ve/09jYyOOPP05bWxtPPPEEAGeffTYAmZmZzJs3L3Y7Oo9g5cqVAOTm5pKZmXnK8fgHHniA\nt956q99jjz766JA9Aj6fj0AgELsfCATIyMiI3c/KyuKf//mfue2228jOzmbRokXk5OT0e14gEIj1\nGAiRCMnqEUBrHVRK/QrzL/9rgHfFbW4DshLZnnA4TFewm7auANNn9uLObeTF/Rvxz6kiGA5ycckK\nst0JbZIQYhi///3v+chHPsLtt98OmJPt1q5di9/vH/Kv+nh79+4FoK6ujkAgQG5u7oj7f/nLXx51\nu5YtW8Z9993Hpz/9aSorKwmFQv16CILBIHv37uWpp56iu7ubm266iS9/+cuUl5ezYcMGli5dyoYN\nG1ixYsWoX1OIM5W0IACgtf6kUqoQ2AK44jZlAE3j+VrhcJi/H9vCiZYqWrsDtHUFaO1uozXyva0r\nQE+ot+8Jc6ET+K8dWwH4f+/8kU+e+xEuK73glP+jEUJMrGeeeYb77rsvdt/lcnHVVVfFZuGPpK6u\njk996lO0trZy1113YbFY+v1On8nv96JFi1ixYgXXXXcdoVCIu+66C4AXXniB9vZ2rr32WgA++MEP\n4nQ6uemmm8jJyeFjH/sY3/jGN/j4xz+Ow+Hg/vvvP+02CDFWRjic+Cp6SqnrgWKt9b1KqUzgbeAg\n8H2t9d+UUr8AXtZaPz3M80uBIy+//DLFxcWjes3GjmY++/s7Bj3utrvIdPjwOb1kOLz4nD4yHF46\nAgZ/2VjFopIiLjgvh6f3vkBnbxdLCxdyy4qPU+DLO82zF0Iky+9+9zvKysr46le/muymCDEmx48f\nj85nma21Pjqex05Wj8CzwH8ppf4G2IF/AvYDDyulHMA7wKmj/RjkuLO491130BXsJiPywe9zeLFZ\nrEPuHwqFeWvDSxzY1cW3PnQ5F85cxsPbn2JH5V6++ufvcd2S9/EPZ63BYpErMIWYTKRHT4j+ktIj\ncKZOp0fgdPz6xf089RfNP113HlecX0I4HOb1Y1t5bMdvae0OcJa/lM+uvJ6S7BkT1gYhhBBiInsE\n5M/ZEayJlBx+JbIioWEYXFJ6Pg+85y4uLlnJwYajfOOv/8Zv9zxPT1AqEQohhJh8JAiMoCjXy+K5\nuew+XEdVfd8lQZmuDL544U3ccck/ku3M5Jm9f+Qbf/k3DtSVJbG1QgghxNhJEDiFd51vVhf8t19t\npbW9fz3zZdOXcP97vs2V8y7leEsl3375B/zXW7+ls6czGU0VQgghxkyCwClcvqyYK1fNouxEM9/5\nz420DQgDHrubzyz/GN9d+xWKMvL508FX+eqfv8fOqneS1GIhhBBi9CQInILFYnDrR87hXeeXcOh4\nM99+aBNtHYPnAyzMP4v7rvoWH1z4buo7mvjXv/1fHtz8GK1dbUlotRBCCDE6EgRGwWIx+MI153LF\nyhIOVTSZPQNDhAGH1c7Hlr6fe991B7NzZrLh6Ga+8qe72Vi+ncl4dYYQQoipT4LAKFksBrddey7r\nVs7kYEUTdz20kcAQYQCgNGcm37/iG1x/zgdp7+3kR5t+yX2v/4KG9nEtliiEEEKcMQkCY2CGgfNY\nu2ImB8qbuOuhTbR3Dh0GrBYr71twJfdf9S0WFcxn28ldfPnP3+Wvh/5OKBxKcMuFEEKIoUkQGCOr\nxeCL153H5cuL0eWNI4YBgKKMAr5z+Ze4ZcV6AB7e/hR3v/ojKltrEtVkIYQQYlgSBE6D1WLwpY8u\n47Lzitl/rJF/efjNEcOAYRhcMfdiHnjPXayccQ7v1B7kay/ew3P7XiQYCiaw5UJMfsePH+e6664b\n9f7XXnstJ0+enMAW9ffggw9yzTXX8NGPfpRdu3YNu9+3v/3tQYsL7dy5kxtuuGGimyhEPxIETpPV\nYvDlj53HpefNYN/RhlOGAQC/O5uvXfRZvrL6Zjw2F0/teo47//rvHGmsSFCrhUg/iVxbYO/evWzd\nupWnn36aBx54gLvvvnvI/f7nf/6HgwcP9mvbww8/zLe+9S16eqRKqUgsCQJnwGq18JWPLeOSc80w\ncPcjm+no6h3xOYZhcMHMZTzwnru4vPRCjjRV8M9/vZendj1Hd2/3iM8VQvR3ww038P3vf58bb7yR\na665JvaX/wMPPMCHPvQhbr31VhobGwFobW3li1/8Ip/4xCf4xCc+wYEDB6isrOSqq66itraWTZs2\nsX79ekKhvjk827dv54Ybbuj39eqrrw7bnu3bt3PxxRcDMG3aNILBYOz1o3bs2MHOnTu57rrr+l1N\nNGvWLB588EG5wkgkXMJXH1RK2YFHgVmAE7gHOAE8DxyI7PZzrfVvE92202G1Wvjqx5cRCod5Y+dJ\nvvvLN/mXz1yAyznyj9bn9PKPqz7BxbNW8p/bnuS5fS+yuWIHn115PWcXnJWg1gsx+Z1zzjnceeed\nPPDAA7zwwgtceOGFbNu2jWeffZa2tjauuuoqwuEwv/jFL7jwwgv52Mc+xtGjR7nzzjt56qmnuP32\n2/nGN75BfX09Dz30UL8VRZcvX87jjz8+6rYEAgGys7Nj971eL21tbeTk5ABQU1PDgw8+yE9/+lP+\n+Mc/9nvulVdeyfHjx8/wpyHE2CVjGeL1QK3W+gallB94G/gucL/W+odJaM8Zs1otfG39cgjDG7tO\ncvcjm/nOp1edMgwALC1ayP3v/jb/s/v3/OnAq/zLqz9k7ezVvGf+GmZlT9zKikJMFQsXLgTMv8Dr\n6uo4duwYixcvBsDn8zF//nwADhw4wObNm2MfwK2trQCsW7eOBx54gNWrV1NYWNjv2Nu3b+dHP/pR\nv8duvPFG1q5dG7v/uc99jkAggFKKWbNmEQj0rUsSCATIyMiI3X/xxRdpbGzk5ptvpq6ujs7OTubO\nncsHPvCB8fpxCDFmyQgCTwPPRG4bQA+wHFBKqfcDB4Evaa0nVUk+m9XC165fTviJMBt3VfK9Rzfz\n7U+vwuU49Y/YZXPyqfOu4aKSFfxiy+O8cmQjrxzZyNycWayZs5qLS1bicbgTcBZCTD4D5wDMmzeP\nJ598knA4TEdHB4cOHQJg7ty5LF68mKuvvpr6+nqeecb839Cjjz7KRRddxK5du9i5cyfnnHNO7Fij\n6RH4xS9+Ebu9d+9e7rvvPj796U9TWVlJKBTq10MQHV4A+N3vfkdZWZmEAJF0CQ8CWusAgFIqAzMU\nfBNwAQ9rrXcope4E7gJuT3TbzpTNauH261fwH49vY9PuSr73yOjDAMBZubP596u+yY7KPbxc9gY7\nKvdwePsx/vvtZ7igeBlr56xmYf5ZCZ38JEQqGul3YMGCBVx66aV8+MMfpqCggLy8PAzD4HOf+xzf\n/OY3+c1vfkMgEOC2225j9+7d/OEPf+A3v/kN5eXlfPGLX+Q3v/kNPp/vtNq1aNEiVqxYwXXXXUco\nFOKuu+4C4IUXXqC9vZ1rr732lOchv98i0YxkTExRSs0EngV+qrV+TCmVpbVujmw7G/iJ1vqKEZ5f\nChx5+eWXKS5Ove7znt4Q//H4Vt7cU8W5Z+XzrU+vwmm3jvk4DR1NbDi6mVfK3qCqrRaAIl8+a2av\n5rLZF+B3Z5/iCEIIIaaC48ePs27dOoDZWuuj43nshF81oJQqBP4CfF1r/Vjk4ReVUisjt9cB2xLd\nrvFkt1n4+g0rWbWoiLcP1nLPo5vp6hl7vQC/O5sPLLyKH//Dd/mXNV/h0tJVNHQ08evd/8vnn7+T\nezf8lC3H36ZXahEIIYQ4TQnvEVBK/Ri4BtBxD98J/ADoBiqBW0aaI5DqPQJRPb0h7v3VVra8U8V5\n8/P51k2rcJxGz0C89u4O3ijfxitH3uBwwzEAspwZXFq6irVzLmJGZtF4NF0IIUQKmcgegaQMDZyp\nyRIEAHp6g3z/sa1s21fNsgUFfPNT559xGIgqbzrBK2VvsOHYFtq6zZnKKncOa+ZcxOqZy3DZXePy\nOkIIIZJLgsAAkykIQP8wsHxBAd+88XzstvEJAwA9wR62ntjFq0feYFfVfsKEcdmcrJ65nLVzLuKs\n3NkyAUkIISYxCQIDTLYgANDdE+RfH9vCW/trWLGwkDs/tXJcw0BUbaCe145s4rUjm6htbwBgRmYR\na2dfxKWl55Plyhz31xRCCDGxJAgMMBmDAJhh4J5HN7PjQC0rzy7knz85MWEAIBQOsada80rZG2w5\nsZPeUC9Ww8LyGUtZO/sizi06u18FNSGEEKlrIoPAmOsIKKW80VoAYmwcdivfvGkV9zy6ma3vVPPv\n/72Nb3xiJXbb+H8gWwwLS4sWsrRoIa1dbfz92BZeLdvIluNvs+X42/jd2Vw++wLWzF5NoS9/3F9f\nCCHE5DDmHgGl1GHgRq31holp0qjaUMok7BGI6uoJcs8jm3n7YC2rFhVNWBgYKBwOc6SxnFfKNvJ6\n+VbaezoAWFQwn7WzL2JV8bk4bI4Jb4cQQoixSamhAaXUlcDPgP8F7tRad41ng0bZhlImcRAA6Ozu\n5Z5HN7PzYB0XLpnG129Ygc2auK76rt5uNh/fwatHNrK3xlzryWN3c3HJStbOWc3snBKZYCiEECki\npYIAmMMDwN3AFcBtQKxRWuvy8WrcCK9fyiQPAmCGge89spldh+pYvXQat1+f2DAQVdVaw6tHNvHa\n0U00djQDMCu7mLWzV3PJrPPxOb0Jb5MQQog+KRcEAJRSHuAx4F1AU/RxrfXscWnZyK9dyhQIAgCd\nXb3c/chmdh+u46Kl0/na9cuTEgYAgqEgO6ve4ZWyjWw/uYtgOITdYmNl8bmsnb2axYUKiyETDIUQ\nItFSarIggFLq/wAPAi8CJVrr1vFsVDpxOW1859Or+Jdfvskbu05iPAlfW78caxLCgNViZdn0JSyb\nvoSmzhY2HN3Mq2Ub2Vi+jY3l28j35rJm9oVcXnoheV5/wtsnhBBi/J3OHIGnMZcNvllr/fKEtOrU\nbShlivQIRHV09fLdX77J3rJ6Vi0q4vr3LKR0WvKv+Q+HwxyoL+OVso1srNhOV28XBgZLixayds5q\nVkxfit1qT3YzhRBiSku1HoFqYIlcQji+3E4bd33mAr77yzfZvLeKzXurmFecxRUrS7h0WTEZnuTM\n5jcMA5U3F5U3l0+ddw2bKrbzStlGdla9w86qd8hweLmkdBVrZ6+mJHtGUtoohBDi9ElBoRTTGwyx\nZW8VL20tZ/v+GkKhMDarhQsWF3HF+SWcO78AqyX5s/mPt1TyStlGNhx9k5Yuc32oef5S1s5ZzeqS\nFXjs7iS3UAghpo6UnCx4upRSduBRYBbgBO4B9mFOPAwBe4BbtdbDNmwqB4F4DS2dvLa9gpe2llNR\nbX7Y5ma5WLtiJlesLGF6vi/JLYTeYC/bK3fzStlG3q7aSzgcxmG1c8HMZaydfREL8+fJZYhCCHGG\nploQ+BSwVGv9FaWUH3gb2AHcr7XeoJT6OfCi1vq5EY5RShoEgahwOMyB8kZe2lrBhh3Hae/sBWBh\nqZ93nV/CRedMx+NK/jh9Q3sTrx3dxKtlG6kO1AEwzVfAmjmruaz0AnLcWUluoRBCTE5TLQh4AUNr\n3aaUygW2AA6t9czI9vcBV2qtvzDCMUpJoyAQr7O7lzd3V/LS1nJ2HaojHAanw8pFS6dzxfklLJ6T\nm/S/wEPhEPtqD/FK2Ru8eXwHPcEeLIaF86YtYu2cizincKFUMBRCiDFItcmCZyQ6yVAplQE8DXwL\n+EHcLm2A/Ok4DJfDxuXLZ3L58pnUNLTz8rYKXt5azivbKnhlWwVFuR7WrSxh7YqZFOR4ktJGi2Fh\nUcF8FhXM56bu63ijfGukNsFutp/cjd1iQ+XNZUnhApYULmBOToksgCSEEEmSlMmCSqmZwLPAT7XW\njymlKuJ6BN4PXKG1vm2E55eSpj0CQwmFwuwtq+elreW8seskXd1BDAPOOSufK1aWcMGSaTjtE7PK\n4VgcbTzO6+Vb2FW1j6NNx2OPe+1uFhUoMxgULWCaryDpvRpCCJFKplSPgFKqEPgL8I9a61cjD+9Q\nSl2mtf4b8B4gKfUJJiuLxWDJvDyWzMvjsx9cwus7T/LSlnLePlDL2wdq8bpsXHpeMVecX8JZM7OT\n9iFbmlNMaU4xnAMtna3sqdHsqt7P7ur9bDnxNltOvA1ArieHJYULWFq4gMUFimyZWyCEEBMmGXME\nfgxcA+i4h/8J+AngAN7BLFaU9lcNnKnjNa28vNUcMmho6QRgZmEGV6wsYc2KYnIyXEluYZ/qtlp2\nV+9nV/V+9lRr2rr7ylTMzJoeCwYL88/CbU+ddgshRCJMqcmC40GCwNgEQ2F26Bpe2lrO5j1V9AZD\nWCwGKxYUcsX5M1mxsCghyyCPVigc4ljTCXZX72N39X7eqT1ET7AHAKthYV7u7FgwmJc7G5sl+cMe\nQggxkSQIDCBB4PS1BLrZsOM4L20t5/Bxc6XBLJ+Dy5fN5IrzS1KirPFA3cEeDtSVsbt6P3uq93Oo\n8RjRf7cum5Oz889icSQYzMyaLvMLhBBTjgSBASQIjI8jJ5t5aWs5r20/TkugGyAlyhqfSqC7nb01\nB9gdmV9wsrU6ti3LlcmS6MTDwgWyOJIQYkqQIDCABIHx1dMbYtu+Kv66ZXBZ43edP4tz5uenRFnj\n4dS3N8ZCwe7q/TR1tsS2TfMVxK5GWFQwH5/Dm8SWCiHE6ZEgMIAEgYnT2NLJqwPKGudluViTQmWN\nRxIOhzneUhkLBe/UHKSj15woaRgGc3JKYr0FKm8uDlk5UQgxCUgQGECCwMQbrqzx2bP9XLEydcoa\nn0pvKMjhhqPsrtbsrt7PgfoygqEgAHarnQWRwkZLCxdQmj1TChsJIVKSBIEBJAgkVldPkE27K3l5\nSzk7D9USDoPLYWV1CpU1Hq3Onk721R1id9V+dtdojsUXNnJ4WByZX7C0cAGFvvxJc15CiKlNgsAA\nEgSSJ76scXVDO0BKlDU+Xc2dLeyp0eyuMmsY1LU3xLblefyxYYTFhYpsV+pdUSGESA8SBAaQIJB8\nk6Ws8ViEw+FIYSNzGGFPTf/CRrOyZkRCwQLOzp+HSwobCSESRILAABIEUkt7Z0+srPG+o+Zf1KlS\n1vhMhEIhjjZVxKod7qvrX9hoft6cWI/BXH+pFDYSQkwYCQIDSBBIXUOVNS4pMssaX748tcoaj5VZ\n2OhwbH2Essby/oWNCuazNBIMijOnTcrwI4RITRIEBpAgkPqGK2u8fEEBqiSHaXlepuf5mJ7vnRRX\nHwylrTvAOzUH2RUphVzZWhPblu3KjPUWLClcQK4nJ4ktFUJMdhIEBpAgMLkMVdY4XpbPwfQ8XyQc\neGMhYVqeF6978oSEukCDWb+gxpxj0BxX2Gh6RmEsFCwqmI/XMbkmVQohkmtKBgGl1CrgXq31GqXU\nMuD3wMHI5p9rrX87wnNLkSAwKVXVBzhe08bJujYq6wKcrAtQWReguqGdUGjwv8Usn4NpuV6m5/sm\nVUgIh8NUNJ/sK2xUe5DO3i7ALGw0N2dWLBjMz5sjhY2EECOayCBgG8+DjZZS6uvA9UBb5KFlwA+1\n1j9MRntE4hTleinK9QKF/R7vDYaoaWznZG0gEhDaYiHhQEUT+481DjpWfEiI9iakSkgwDIOS7BmU\nZM/gvWodvaEgh+qPxlZUPFh/hEMNR/ndvj/jsNpZkDcvFgxKc4qxGFLYSAiRGEkJAsAh4EPA45H7\ny4H5Sqn3Y/YKfElr3Tbck8XUY7NazDkDeYNLGA8VEqK9CWMJCdHehGSEBJvFyoL8uSzIn8s1i6+m\no6eTfbWHYj0Gu6r3sat6HwAZDi+LClVs4mGhLz/h7RVCpI+kBAGt9bOR7v2ozcBDWusdSqk7gbuA\n25PRNpF6pmJIcNtdLJu+mGXTFwPQ1NFsFjaq1uyq3sebFW/xZsVbAOR7c2PVDhcXKDJdGQlpoxAi\nPSSrR2Cg32mto7PIngN+kszGiMljNCGhsi7AydrRhYRMr8McYkhwSMh2Z3HxrPO5eNb5hMNhqtpq\nI8MImj01mlfK3uCVsjcAmJVdHAsGC/Ln4bI5J6xdQoipL1WCwItKqdu01luBdcC2ZDdITH7xIWH5\ngv7bBoaEyvoAJ2vbRhUSpuUNnrw4niHBMAymZRQwLaOAK+ddRigU4khTRWQYYR/7aw9zrOk4L+iX\nsFqsqNw5LI4Eg7n+WVilsJEQYgySHQSi08Q/DzyolOoGKoFbktckkQ5OJyRU1gU4OMqQYA49jE9I\nsFgszPXPYq5/Fh9YeBXdvd3o+jIzGFTtZ1/tId6pPchv9zyP2+5iUf58c+Jh0QJmZBRJYSMhxIik\njoAQY9AbDFHb2GFe1TAgJFQ3tBMc4hLI4ULCtDwfvnHoSWjrCrCnRrMnskZCZVtfYaMcV1a/wkZ+\nT/YZv57rXzz+AAAgAElEQVQQIvGm3OWDQkxWNquFaZEP9aF6EqIhIToXYTQ9CbFLH08zJPicXi6Y\nuYwLZi4DoDZQH1k4aR97qjUbjm1mw7HNAMzILGJJgdlbsCh/Ph6H+8x+IEKISU+CgBDjJD4kDBQM\nhqgZEBIqI0HhUEUT+hQhYVqeLxIWTh0S8r25rJ2zmrVzVscKG+2KK2z050Ov8edDr2EYBvP8pbGJ\nh2flzsYuhY2ESDsSBIRIAGsCQkJ06CE+JMQXNrparaM32MvBhiOxpZYP1h/hYP0Rnn3nTzisdhbm\nnxULBiXZM6SwkRBpQIKAEEk2mpAwsEbCSCEhw+OITFQcOiQszD+Lhflnce3iq2nv6ehX2Ghn1Tvs\nrHrHPI7Tx+ICFQsGBb68Cf9ZCCEST4KAECksPiQso6DftuFCQmXdWENCMR9ZsIBPnWensaM5Nulw\nd/V+NlVsZ1PFdgBy3FnkuXPIcWeT7c4kx5VFjjvy5coi251FptMnvQhCTDISBISYpE4VEmqbOgYV\nUhpNSDCHHFZx3Yw1OHyd1AUrONBwkMONxyhrqiDYcHT4NhkWslyZkWCQSY47mxxXZiQwmLez3Vlk\nOTOk3oEQKUKCgBBTkNVqiS3wNFJIqIws7nTqkFBCoV8x22XD6QpidXVj2LvA3kXQ0kGv0UE3ATpC\nAdqDbRxrOsHhxmPDts8wDLKcGbGehGivQo47k2xXFv5Ir0O2KwubBAYhJpQEASHSTHxIYAwhoby6\nje6e4ICj2YCMyFe8MFh7MRydGPYuLI5uHO4ebK5uLA4zQASCnbR0VHKkqWLE9nptXrJcmfjdWfg9\n2eR6sslxZ5Ed19OQ7cqUpZyFOE0SBIQQMSOFBDBrJXR09dLR2Ut7Vy/tnT20d/bS0dUb+d7/fntn\nD+2R/Tu6emmvM7e3d/USCoWBMFiCscBgOLpiPQ3R2632Ltq6ajjZVjly28NOnIYHt8WH1+Yjw5FB\nljOTbFcmfk82+d4c8jNyyPF6cDttuBw2LBapuiiEBAEhxKjZrBYyPA4yPI4zOk44HKa7N0R7Z48Z\nKmLhoScSMPruR0NHW0cHrd2tBHpaaQ8F6A4H6DHa6bWYISJk76LX0Ua7rZH6bqAbGGIx83CvjXCP\nk3CPE2vQhS3swYEHt8WLx+bDZ88gw5FBhtOD12XH47LhdtlxO214XDY8Thue+PsuG3abDF+IyUuC\ngBAi4QzDwGm34rRbyTnDVZWDoTCdcT0SzR0d1LY2UR9oorGziabOFlqiASLYRqc1QLelnaA7AEBv\n5KsdqI87brjDSrjFSbjbDA309N0O97j6Hg/asFmt/YJBLCg4bbhdA4JDNEi44rY7zcDhctqwSi+F\nSDAJAkKISc1qMfC67ZHFndxAJlB4yuf1Bntp6myhsbOZxg7zq769ibpAEw3tTTR2NtPS1Uqgd/Dk\nyXhG2Iol6IJeF4EeJy1dDno67WaI6HES7naZgaHXDpz6Q97lsJq9EM6+sOAZECbckTDRd7svTLgj\n4cJpt8qCU2JUkhYElFKrgHu11muUUvOAx4AQsAe4VWs9+VZDEkJMGjarjTyvnzyvf8T9gqEgzV2t\nsbDQ1NlMQ0czTR3NNHY209TRQkNnE82djYRcIciAoaYtWgwrXpsPt8WL0/BgD3uwhdwYQRf0uAh1\nO+jpcNDVaaWzM0igs4fapo4hJmiOjsVixPU49PVCRHsqhgoSbtfQQcNmldoQU1lSgoBS6uvA9fSN\n4P0QuFNrvUEp9XPg/cBzyWibEELEs1qs+N3Z+N0jr9wYCoVo6W6jqSMSFOJ6GszA0ExjZwv1ndUE\nQwM+3K2YnRlusBgWslwZFLvMSyiznJn47Bl4rD5cFq8ZIMJuLEEXnV2hU0/Q7OyhvrmTipq2yATN\nsXPYLH3DGbEwEZ0/MXiIIz5MxPdiyATN1JSsHoFDwIeAxyP3l2mtN0Ru/wm4EgkCQohJxGKxkO0y\nr1IozZk57H6hcIi27vbBgSHyPdrTUN5ycuRaDBhkujIiBZuyycnJJD9WjyGu4qMrE5vVdloTNDvi\nrgyJbm9q7aSj6/R6KQyD/nMnhuqFiAsO7kivhEzQnFhJCQJa62eVUqVxD8VHxDYgK7EtEkKIxLAY\nFjKdPjKdPkqyZwy7XzgcJtDTTlNH/3kMfb0L5v3KtlqONh0f8TUznL5YwaYc14AS0dlZzI4Udhpt\nLYboBM1BQSLuEtKB4SJ62wwZPTS3dVNZ105vMDSmn1+UzWoZ+wTNaMiQCZr9pMpkwfh/CRlAU7Ia\nIoQQqcAwDHwOLz6Hl+KsaSPu29HTSWNHE42dLcMGhtr2esqbT4x4HK/D06/CY3yJaLPioxkYXDbn\ngAmap6+nNzigFkVcL0RXLx1xQx3tkRDRER80OnuoaWinvauX8GnOLBvLBM1+4WOKTNBMlSCwQyl1\nmdb6b8B7gJeT3SAhhJgs3HYXbnsR0zOLRtyvs7crLhy0DJr4GA0Qx1tGLt7ktrliww/ZrmjvQnZc\ngDC/3DbXKT8Y7TYrWT4rWT7nmM87XjgcprM7aA5txPVQtA83h2KIcJGuEzSTHQSi+e2rwMNKKQfw\nDvBM8pokhBBTk8vmpCijgKKMwVUj43UHe8xLKzuaIldKDO5paOhs5mRr9YjHcVodZLsjPQmurNii\nU/5IWehoYPDaPWf8l7RhGOZf5s4z/1gLRipo9g1x9MQFiVFU0BynCZo+j51//PA5rFo8co/QmUpa\nENBaHwVWR24fBC5PVluEEEL0cVjtFHhzKfDmjrhfb7CXpq4W8xLKjqbIxMeWQRMf99ceJszwH4h2\ni81cfMoVv6x1/56GHFcWPqc3IctcW60WfB4HvvGuoBnrhRjdBM2e3hDWBPQMJLtHQAghxCRls9rI\n8/jJ84yuFkP0EspoT0NjZ4v5WCQwHGo4Sig8/ORBq8UaG4rIdmfhj65eOWCp60xnBhZL8msfjGcF\nzYkkQUAIIcSEGnUthnCIlq62fvMYzEmQZuGm6ITII00VBBuODnucaC2G2DLXrv5LXZuBIYssVwZW\nWeZagoAQQojUYDHiajEwfC2GcDhMW3egX/2FvvkL0aGJJipaKilrLB/2OAYGmU5f3MTHvkssoxMh\no/MZbNap+3E5dc9MCCHElGQYBhlOHxlOHyWMXIuhvaejX1jof6WEOTQxqloMDm//iY/9gkJWbB6D\nw3Zm8wqSQYKAEEKIKckwDLwOD16Hh+LMUdRiiF4RMWSJ6Bbq2xupaD454nG8dnf/4YghehpyXJm4\n7K7xPNUzIkFACCFE2jNrMbiYnjHyypVdvd2DijUNLhHdwomWqpFfz+aKXRnRf+Jj3DwGdxYe+5kV\nbBoNCQJCCCHEKDltDop8+RT58kfcL1qLYajAEH+lRGVrzYjH+fzKG1gzZ/V4nsIgEgSEEEKIcTbq\nWgyhIM3Rgk0DAkNbd+CU5aXHgwQBIYQQIklsFiu5nhxyPTlJa0PyKy4IIYQQImkkCAghhBBpTIKA\nEEIIkcZSao6AUmoH0BS5W6a1/nQy2yOEEEJMdSkTBJRSLiCstV6T7LYIIYQQ6SJlggBwDuBRSr2I\n2a47tdabh9nXClBVNXLBBiGEEGIqiPu8G/dVklIpCASA+7TWjyilzgL+pJSar7Ueak3KaQDr169P\naAOFEEKIJJsGHB7PA6ZSEDgAHALQWh9UStVjnvCJIfbdClwCVALBhLVQCCGESA4r5mfi1vE+cCoF\ngZuAJcCtSqnpQCbmB/0gWusu4PUEtk0IIYRItnHtCYgywuHwRBx3zJRSduAxoAQIA1/XWr+Z1EYJ\nIYQQU1zKBAEhhBBCJJ4UFBJCCCHSmAQBIYQQIo1JEBBCCCHSWFKvGlBKWYCfAUuBLuAzWuvDcdtv\nBm4BeoF7tNZ/UErlAU8BLuAkcKPWumMs+ybuDEeWgPP3Y16WuTtyyN9prX+SoNM7pfE8/8j++cBG\nYJHWulsp5QaeAPKBVuCTWuu6hJ3gCBJw7gZwHPP9B9iktb4zMWd3auP8b//LwHWRp/5Ra313Kr/3\nkJDzT6f3/1bgk5iTzH+gtX46ld//BJz7mN/7ZPcIfABwaK1XA3cA90c3KKWKgNuA1cBVwL8ppRzA\nd4AntNaXAjuAz45l34Sd2ehM9PkvA57SWq+JfKVMCIgYl/OP7H8V8BfMX/yozwM7I/v+N/CtCT+j\n0Zvoc58LbI9771PmQyBivP7tzwY+DlwY+bpSKbWE1H7vYeLPP13e/1zM9/pCYF3ccVL5/Z/ocx/z\ne5/sIHAR8GeASDnhFXHbzgfe0Fr3aK1bMIsNLY1/DvAn4Apg5Rj2TSUTff7LgGVKqdeUUr+N/CNL\nJeN1/mAWlloHNA51/Mj3VHr/J/rclwMzlFKvKKX+oJSaP2FncnrG6/zLgXdrrcNa6zBgBzpJ7fce\nJvb8O0iT919rXQ+co7UOAtMxzx1S+/2f6HMf83uf7CCQCbTE3Q9Guk0AMoDmuG2tQFbkOdHH24Z4\n7FT7ppKJPv/9wHe01pcDzwH/d5zbf6bG6/zRWr+ktW4Y4vjNA56fKib63E8C39darwW+j9lNmkrG\n5fy11kGtdb1SylBK/QB4S2t9kNR+72Fiz/8QafL+A2itg0qpL2AOjUXPM5Xf/4k+9zG/98kOAi2Y\nJx5liVtbYOC2DMwlilswfygDHxvtvqlkIs+/EXgFeC3y2HPAeePY9vEwXuc/0vFT9f2f6HPfBvwe\nQGv9BjBjHNo8nsbt/CMrlz4JeIF/jDtGqr73MPHnnzbvP4DW+kHM8ruXKaUuH2nfFDDR5z7m9z7Z\nQeAN4B8AlFIXALvitm0BLlFKOZVSWcBCYE/8c4D3ABvGuG8qmcjz3wv8EvhwZN91mP9AUsl4nf8p\njz+KfRNtos/9LuBLkeOfAxwb19afufE8//8F3tZafz7SPc4I+6aKiT7/tHj/lVLzlVL/L/JYL+bk\nu9BQ+07guYzVRJ/7mN/7pFYWjMxujM6eBLgReC9wSGv9vFLqM5izJy3Av2qtf6eUKgB+hZmKaoGP\nR2ZPjnrfBJ7iiBJw/qXAo4CB2Z30Ga11deLOcGTjef5xxywDFui+qwZ+hZmWuyL71iTo9EaUgHPP\nxuwS9AE9wK1a6+gs4qQbr/MH3o05m3oT5r9zMCdg7SJF33tIyPlr0uD9j/y/7zuYH45hzKsm7kmX\n3/1hzn3Mv/tSYlgIIYRIY8keGhBCCCFEEkkQEEIIIdKYBAEhhBAijUkQEEIIIdKYBAEhhBAijUkQ\nEEIIIdJYUlcfjKeUWoe5gpYH+A+t9a5TPEUIkQCRamVPYdY1r408djuwSmv9kWS2TQhx5lKpR8Ct\ntb4F+AFwZbIbI4Qwaa1fwyxQ8jCAUupC4GbgpiQ2SwgxTlKqoJBSyou5MM7XdYqsHS2EAKWUHbP8\n6X8BXwBuiKycJoSY5BISBJRSq4B7tdZrIqssRcsrdmGWvT2slMoD/gNztbzjE94oIcSYKKXOxizd\n+69a67uS3R4hxPiY8KEBpdTXMbsUnZGHPgA4tNarMWti3x95/H6gEPg3pdSHBx1ICJFsF2PWOX+X\nUsqa7MYIIcZHIiYLHgI+BDweuX8x8GcArfVmpdSKyO1PjvaASiknsBKoBILj2lohxFDmAd/DXM3y\nPuAHSqkfJ7dJQqQVK+YiSlu11l3jeeAJDwJa62cjq+BFZWCurRwVVErFr8c8GiuBv49H+4QQYxL9\nvbuAyFKnQoiEugR4fTwPmIzLB1sww0DUWEMAmD0BPPnkkxQVFY1bw4QQQohUVFVVxfr16yHy+Tee\nkhEE3gD+D/C0UuoCzMlHYxUEKCoqori4eDzbJoQQQqSycR8OT2QQiF6e8DvMyUZvRO7fmMA2CCGE\nECJOQoKA1voosDpyOwx8PhGvK4QQQoiRpVJlQSGEEEIkmAQBIYQQIo1JEBBCCCHSmAQBIYQQIo1J\nEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQBIYQQ\nIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHSmAQB\nIYQQIo1JEBBCCCHSmAQBIYQQIo1JEBBCCCHS2JQMAuFwmHA4nOxmCCGEECnPluwGjLeuniCfu/dl\nAh09FPo9g74KIt89LnuymyqEEEIk3ZQLAg6bheULCtDHGqluCHC0smXI/TI8DgpzPRTmREJCbiQo\nRO477NYEt1wIIYRIvCkXBAzD4AvXnAuYQwSt7T1UNwSobminur7d/N5o3j5W2cKhiqYhj+PPdFLo\n95rBILd/r0JethubdUqOqgghhEgzUy4IxDMMg0yvg0yvg7Nm5gzaHgqFaWztpKahoy8sxH3p8kb2\nHW0Y9DyLxSAvy2UGBb+bQr+3X1DwZ7qwWIxEnKIQQghxRqZ0EDgVi8UgN8tNbpabhbP9g7YHgyHq\nmjupaWinuiFAVSQg1ES+7ymrI3x48HFtVgsFOe5+cxKK/F4Kc82hhyyfA8OQoCCEECL50joInIrV\naon9lb+EvEHbe3qD1DR2mEMOje1U1/f1KtQ0trPjQO2Qx3U5rLGAUNhv6MFLgd+Dzy0TGYUQQiSG\nBIEzYLdZmZHvY0a+b8jtHV29sd6Dvq++sFBe1Trk87xu+/BXPOR4cDlH97bFX0IpPRBCCCGGMuWC\nQCgc4qFtT1HdVmvWEwAg3Hc78j1MeMBt83v/26e7/1DPhbARJpwbBj9kESYjHCYYChMKhQiFwoTC\nka9QiJPhMCcBWoG2MJT3naNhmF/m7bh6CUbfh3/0zAHsFhs+p5cMhw+fwxO7neH04nN4yXB4I4/F\nfXd4sVrkygkhhJjqplwQ6A0F2V21j9r2/pP8DAzM/wwMAMMYcDu6T/zt092//3MNw2J+eI9y/+g+\noRD0BkP0BsP09obpDYbo6Q3R2xOmJxgi+lkfDhv9ztNmteC0W7HbLDhsVizWEL2hTmoD9ZQ3nxj1\nz9Jtd5HhiASIuIAQvW0GiWig8JDh8OG2u6T3QQghJpEpFwQcVjsPXn0PYcLmB+sU/VAKhsI0NHcO\nebVDdUM79c0dDFVc0WoJk5trI9dvITPLwJsRxuUO43AFMWw99NBJW3fA/OoK0NodoLzlJD3BnlG1\ny2pYYmFhYG9DhtNnbnN4yHD6+m2zW2VehBBCJMOUCwJgjodH/raesqwWg/wcN/k5bhbPHby9pzdE\nXdPQl0VWN7SzT3cNcVQbDlsmBf4iCv0e5vj7JjHmZNnwZoQJWboJdJsBobWrf2Bojd1uo6WzlZOt\n1aMu9ey0Ogb1NJiBIdLrEHu8Lzx47R4sFqnnIIQQZ2JKBgEBdpuFaXlepuV5h9ze2d1LbWNHpNCS\neWlkTWN7rPDS8Zq2IZ/ndtr6JjDm5lCYMwPl91BYZNZScMdNZAyFQ7T3dPQFhWhwiNxu7W6jrStA\nW3d77HZVWy1Hm46P6hwNDLwOT2TIwoNvQC+DLy48RHsgMp0+HDbH2H+gQggxRY0qCCillgJnAUHg\nkNZ6z4S2Skw4l8PGzMIMZhZmDLk90NEz5NUONQ3tVNWPonSz30NRXB2FQn8es/JLRlW6uSfYQ1t3\neywwmN/bzB6H7nbaYrf7tte01xMMBUd17j6HF787m1xPNjnubHLd2fjd2fg92eS6c/C7s/E6PFN2\nWEkIIeINGwSUUhbgs8CXgDbgGNALlCqlsoAfAf+ptQ4loqEisbxuO3NmZDFnRtagbeFwmJZA9+Ah\nh/oANY3tHD156tLNAxeBKvR7yM92Y7VasFvt5LizyHEPfu3hhMNhOnu7IsMTbbFehr4gYX5v6myh\noaOJ2vaRJ046rPZIWMjpFxZyPTmx0JDtzJShCSHEpDdSj8DTwEvAKq11v/+rK6WygU8CzwHvm7jm\niVRkGAZZPidZPifzS4Yv3VxV33+4IbrOw4ilm7PdQy4EVZTrISdj+NLNhmHgtrtw210UeHNHdR4d\nPZ00dDTR0NFEfXujebs9cr+jkYaOZvbWHBj2+RbDQo4rC787i5y43oRcT7SHwbzvkImQQogUNlIQ\n+KTWesiB4kgw+LFS6pGJaZaYzOJLNy9i8IdybzA6kbF9iIJL7ew+XMfuIUo3221m6WZzISjvoIJL\nmd6xlW52213MsBcxI7No2H16g700djZTHwkIDR2NNLQ3UR8JEA3tjZQ1VRBsODrsMTIiQxHRYBAL\nCu4c/O4scj05eOxuGYoQQiTFsEEgPgQopdYDZwPfBz6stf7vgfsIMVo2q4WiXC9FuUNPZOzuCcZ6\nEqJBIX6dhxO1tcDg8s0uh3XAcEP/sOA9jdLNNquNfG8u+SP0MoTCIVq62iK9CY2RHob44NBMdaCO\nYyMMRTitDvyxgBA3BBF3P8uZIUMRQohxd8rJgkqpfweKgWXAfwA3KaXO1Vp/ZaIbJ9KTw26luCCD\n4oKhJzK2d/ZE1ngImGs8xA89NLRzbJjSzT63PbbwU3QyY2Gu1+xl8HtwOU7vIhqLYSHblUm2K5M5\nlAy7X3tPR2zoIX44or6jicZ2cziisrVmxNfJcWeZwSDSmzCwlyFHhiKEEGM0mv/zXYUZArZrrZuV\nUlcAuwEJAiIpPC47pdPslE7LHLQtHA7T1tETFwz611GoqGrl8PHmIY+bneEcYiEos4chP9uD3XZm\nf4177G48WW6Ks6YNu09PsIfGjua+eQrtzZH5Cn3zF8oajnEwfGTYY2Q4fZGwkD3kkESuO0cqQAoh\nYkYTBAZek+Uc4jEhUoJhGGR4HGR4HMybmT1oezgcpqm1KzbcUDPg8shDFU3oY42DnmcxwJ/lHnIx\nqEK/F3+WC+swExnHwm61U+DLo8A3eLXLqFA4REtna2yewlATHavaajk2Qj0Gp81JgcfP7JwS5vpn\nMdc/i9KcmdKbIEQaGk0QeBr4H8CvlPoycAPw6wltlRATxDAMcjJd5GS6WFDqH7Q9GApT39wRG26I\nXfUQuTzynSP17C2rH/Q8m9UgP7vvaoeCAVc+ZPuc4/YXuMWwkO3OItudxVxmDblPOBw2hyJiYaFv\ncmN0OKK6rZaKlko2HNsMmOWhS7JmxILBXH8pM7OmyeJTQkxxpwwCWut7lVLvxlz/bibwHa31CxPe\nMiGSwGoxKMgxP8iXDFO6ubapb05CTWP/+QlvH6yFg4Of57BbKfS7YxMYC+KGH4oiExnHs6veMMyq\ni16Hh5lZ04fcJxQOcbK1msP1xzjccIzDDUc52nScI00VvFT2utluq53Z2TNjwWBebilFvnwZVhBi\nChmpoNBlxNa3owN4Pm7bpVrrDRPcNiFSjt1mYXqej+l5viG3d3b3DnlJZGyOQvXQF9p4XLZ+Qw0F\nfjdF0dAwoHTzeLEYFoozp1GcOY3LZl8AmJdLljefjAWDww3HONhwFF1fFnue1+5mTqTXYJ6/lLn+\nWfjd2RIOhJikRvq/y3eBkVaMWTPObRFi0nM5bJQUZVJSNHgiI0BbR08kKAxeDOpkXYAjJ4cu3Zzl\nc/QNN0SudohOaizIcWO3jU/3vc1qY46/hDn+Et7FJQB09XZzpLEiFgwONxxjd/V+dlfvjz0v25XZ\n12sQCQkZzqHDkhAitYxUR+DyBLZDiLTgc9vxjaZ0c307VQ2BvsskG9o5crKFg0OUbjYM8Ge6Bg03\nFER6F/KyXFitp3/Fg9PmYEH+XBbk942VtHUHKGsojwWDww3H2H5yN9tP7o7tU+DNZW6kx2Cefxaz\nc0pw212n3Q4hxMQYTR2BS4DbAS9gAaxAida6dGKbJkR6GUvp5mgvQvxVD/pYw7Clm/OzB1/xEC28\nNFLp5uH4HF6WFi1kadHC2GONHc1xweAohxqOsaliO5sqtpvnh8GMzKLYZMR5/lJmZc/ALlcqCJFU\noxl4/CXw75hrC/wE+AfgrYlslBBisH6lm+eMXLp5cPnmALsO1Q15XLN084BLIuOufBht6eYcdxYr\nZixlxYylgNnDUROo43DDMQ5FAkJZYznHWyr529E3AbBarMyKXalgDisUZ06TCopCJNBogkCH1vpR\npVQp0AjcDGyf0FYJIcZsLKWbY9UY4yoznqgdeiKj22k1JzAOKLQU/fK4hv6L3jAMCn35FPryWV2y\nAoBQKMSJ1qpIODDnHBxrOkFZYzl/Pfx3wCy3PDtnZr9hhUK5UkGICTOqIKCU8gMauAB4FfBMaKuE\nEONuTKWbh7ji4Wjl0BMZMzz2uKEG76DhB6e9byKjxWJhZtZ0ZmZN5/LZFwJmNUXzSoWjsZ4DXV/G\n/rq+lae8Dg9zc2b1G1bwewYXjBJCjN1ogsAPgd8CHwS2AtcjPQJCTDmnKt3c2t7Tb05CfGXG8qpW\nDg1TujknWro5cllkod8bWefBQ162G7vVHvuAvzLynM6eTo40VfQbVthVvY9d1fv6juvK6lf8aJ5/\nFj7n0L0hQojhjaag0NNKqee01j1KqfOBc4HXJ75pQohUYRgGmV4Hmd6hSzeHQmGa2rqoia0UGehX\nmfFgRRP7hyndnBs/kTE2/OCl0F/M/LPmxUo3t3UFONx4LC4cHGXbyV1sO7krdrxCX35sOGFu5EoF\nl805cT8YIaaA0Vw1cC3wHWAxUAA8BXwBeG5imyaEmCwsFgN/pgv/cKWbgyHqmzvNOQlxC0JFhyL2\nltWz5/AwpZsHTmT0L2HRzFUUnuMlaG2nrLE8VuPgUMMxNpZvY2P5NsAMMMWZ0+LCQSmzsmZgs45/\ngSYhJqvR/DZ8G1gHoLU+pJRaBvwVCQJCiFGyWi0UROYMDF26OUhtY8egeQnRoYe3D9QOeVynwxq5\nuqGQIv9szva7cfq66LDW0dhbTXlLBUcaK6hoPslrRzYBYLPYmJU9o19lxBkZRXKlgkhbowkCdq11\ndfSO1rpGKTWBTRJCpBu7zcr0fB/T84cp3dzVS3XjwEsi40s3tw7xLBde99kU5CwjK68bW0YLPY4G\nmuj8SlsAABCiSURBVEM1HG08zuGGY/wFs1K6y+ZkQd5cFhcqFhcoSrNnSjAQaWM0QeANpdSvgScj\n968DNk1ck4QQoj+X08asokxmjVC6OXq1Q3QhqKpY6eZ2jpwMYq6gPs38MkIYnhZ8/gCO7FZCrkbe\nrnqHt6veAcBtc7Oo4CyWFC5gcaGiOHOaXL4opqzRBIFbgduAzwI9wAbgZxPVIKXUWuBjWuubJ+o1\nhBBTi89tx1eczdziwRMZw+EwzW3dQ67vUN3QTu3JdnqDYbB3Ys1swJJZTyCzod9ERHvYRYGjhDmZ\nc1lapFgwrZi8HE9sIqMQk9lorhroVEo9qbW+Tyl1KbAUsAPd490YpdRc4DxACpILIcaFYRhkZzjJ\nznCiZg2eyBgKhWlo6YxNYKxu6KC6IcDxxhqqu8tpt1URzmjghHGAE/UH+Hv9nwi95SLcmosvWESh\no4TinHzzaoecyBUPuR5yMpzSiyAmhdFcNfALIKSU+inm8MBfMFce/PB4N0ZrfRi4Xyn1+HgfWwgh\nhmKxGORlu8nLHr50c21jO++cLGdXteZoy2FqwscJ5p2ggxMcZTtlnR5C+3IJtfgJtvih14nDZjGv\neBi0EJQZFjI8dgkKIiWMZmjgfGA5cBfwqNb6LqXUtrG+kFJqFXCv1nqNUsqCObywFOgCPhMJAUII\nkVJsVgvT8nxMyzubdfz/9u49yM66vuP4e3eTTcgmQEKyISQQFki+NIkGA1FwqKARqg6MSC9Oxcto\nvU4RtFXHYhUvbaVUvFSmM+joeHcqXqigVSrUWtCCVmgMxm9uXAxC7hAuhiS72z+eZzdr2N2cjXvO\n2d3n/ZrZSc6zz3nO9+zkyfns7/k9398SAHp6e3jgkd+wZmuy+uFfsXbbep6a+mvo/DUAU3tm0vbk\nbB7ZcTQPrj8Sup/ehvmIKZOGXAhquNbN0mirJQi0ll8vBd4cER2MsMVwRLyLoiNhXzPzi4D2zHxu\nGRCuKbdJ0pjX2tLKiTMXcOLMBVwQq+ju6WbTrgdYsyVZszXJ7RvZM30XTIdpC1uYP30+x045gRm9\n8+DxWezYta//UsTQrZvbmVt2Yjx4Iai5s6bRPqB1s/T7qCUIfAF4CPhxZt4REb8EPjXC19kAXAz0\nDfmfDXwPoDzmGQN3zsxXjfD4ktQ0ba1tLDqmi0XHdPGyJS9iX/c+1u+4jzVbk3u2Jut23MvmxzcX\n+7a0csrCLs5eGSztXMyxU+eza/e+AwtB7Xyyv/HS/cO0bp515JQyGHQcWAxq5oHWzZPavP1Rtall\nsuBHI+ITmdldbjonMwfv7jH0Mb5Zrl7YZwYwMAZ3R0RrZvaM5LiSNBZNbpvMks5FLOlcBFzAnv1P\nkds39o8YrNuxidy+kW/88rtMbpvMqbNPYmln8IxnBhfNWkZba/Hbfk9PL7se2zPIstLF17paWzfP\n6vid0YVZR06l1TseVKqpz+aAEMBIQ8AQdlOEgT6GAEkT1tRJU1h+7BKWH1vMMXhi75Os3baeNVvX\ncc+W5BflF8ARk6Zy6pxTWNYZLJsbLDx6PsccdQRLup4+kbG7u4ftj+4p2jXvPNA7oW+dh6FbN7cy\nZ+YRT5ujMHfWNI49poOjprs+Q5U0q+H27cCFwPURcSaw+hD7S9KE0dE+jTPmL+eM+csB2L3nMe7Z\ntq5/xOCuh9Zw10NrAJje3sGSzkX9wWD+jGP77zZoa2vt/wAfzL793eV6Dn2XG363l8JQrZsv/dPl\n/NGZJ47+G9eY1Ogg0Fv++S3gvIi4vXz82gbXIUljxpFTZ3DW8adz1vGnA7DzyUdYs7UIBWu2JHdu\nvps7N98NwNFTj2RZZzG/YNncoLNj9pC3IU6e1Mb8OdOZP0Tr5t8+tf9plxx27d5D13FH1eeNakxq\n6e3tHXaHiPhPig/wvn9pvcBvgV8C/5CZT79AVWflfIN7b7nlFhYsWNDol5ekhunt7WXrE9v7RwvW\nbF3Ho3sOTLGaM20WS8s1EpZ1BrOmPb27osa/zZs3s2rVKoCuzLxvNI9dy4jAWoougp+lCAOvABZQ\n3EnwGYq7ASRJddDS0sLc6XOYO30Oq04+m97eXh7c/XD/iME9W9fxw3t/0r+64nEz5vZfRljSuZgj\npww+GiD1qSUInJmZKwY8/r+I+FlmXhIR3uYnSQ3U0tLCgqPmseCoebxo0bn09PZw/yMP9o8YrN22\nnps3/oibNxYrKy48an7/iMGSOYuY1n5Ek9+BxppagsCkiFiWmWsAImIZ0BoR04D2ulYnSRpWa0sr\nXTOPp2vm8Vx46gvZ39PNpp339/cw+NX2Tdz/6IN8d92ttLS0cPLMhSybW8wxiNknM3WSdwhUXS1B\n4DLg3yNiC0WHwZkUXQKvpGg2JEkaIya1trF49kksnn0SFy95MXu797F+x739IwYbdtzLhp33ccPa\n79PW2sbiY7pYWs4vWHTMiUxus7Vx1RxysiBAREwCngF0A2szc19EtGTmoZ9cB04WlKTDs2ffHn61\nfWN/D4NNux6gt7yhq71tMqfOPoVXnXYxC4/2/9axpKmTBSOiE7gWWFXuf2tEvDkzt4xmIZKk+ps6\neSqnzVvKafOWAvD43idYu21D/4jB6i1rWbttg0GgQmq5NHAdRQOg11NcGngjxd0CF9SxLklSA0xv\n72Dl/OWsLJsb7d2/18sDFVNLEDgpM1824PHVEfHqehUkSWqe9knOAa+aWpan6omIE/oeRMRCir4C\nkiRpnKtlROC9wI8j4s7y8ZkUlwckSdI4V8syxDdFxApgJcUIwpszc2vdK5MkSXU3ZBCIiCuH+NaK\niOjNzA/WqSZJktQgw80RGHw5q0N/T5IkjRPDXRq4LjMfGu7JETHvUPtIkqSxa7ggcFVEbAY+n5nr\nBn4jIv4AeB1wHHBJHeuTJEl1NGQQyMzXRMQFwKcjYjHwG2AfcDywEfinzLyxMWVKkqR6GPaugcy8\nCbgpImYBJwO9wKbM3NmI4iRJUn3V0keA8oPfD39JkiaYWjoLSpKkCWrIIBARHY0sRJIkNd5wIwKr\nI+J5DatEkiQ13HBB4C3AZyPimoiY0qiCJElS4wwZBDLzZmB5+fDOiHheRJzQ99WY8iRJUj0d6vbB\nJyLivRS9A/4NeGTAt7vqWZgkSaq/YYNARFwIXAt8HzghMx9rSFWSJKkhhlt98HrgdOB1mXlL40qS\nJEmNMtyIwBbgGZn5RKOKkSRJjTXcWgOXNrIQSZLUeHYWlCSpwgwCkiRVmEFAkqQKMwhIklRhBgFJ\nkirMICBJUoUZBCRJqjCDgCRJFWYQkCSpwgwCkiRVmEFAkqQKMwhIklRhBgFJkirMICBJUoUZBCRJ\nqjCDgCRJFWYQkCSpwgwCkiRVmEFAkqQKMwhIklRhBgFJkirMICBJUoUZBCRJqjCDgCRJFWYQkCSp\nwgwCkiRVmEFAkqQKMwhIklRhBgFJkirMICBJUoUZBCRJqrBJzS6gT0Q8F3hj+fDyzHy0mfVIklQF\nY2lE4A0UQeAzwMubXIskSZUwloJAW2buBR4C5jW7GEmSqqAhlwYi4jnAVZn5/IhoBf4FeCbwFPD6\nzNwIPBkR7cBxwMONqEuSpKqr+4hARLwL+DQwpdx0EdCemc8F3g1cU27/FHAdxSWCL9a7LkmS1JgR\ngQ3AxRz4cD8b+B5AZt4REWeUf/858Noaj9kG8PDDDhxIkia+AZ93baN97LoHgcz8ZkScOGDTDGD3\ngMfdEdGamT0jOOw8gEsuuWQUKpQkadyYB2wczQM24/bB3RRhoM9IQwDAT4E/pJhY2D1ahUmSNEa1\nUYSAn472gZsRBG4HLgSuj4gzgdUjPUBmPgXcNtqFSZI0ho3qSECfRgaB3vLPbwHnRcTt5eNa5wVI\nkqRR1tLb23vovSRJ0oQ0lhoKSZKkBjMISJJUYQYBSZIqbMysPjgaIqIT+E5mrmx2LVJVRMTpwKVA\nC/CuzNza5JKkyoiIVRQL9U0Drs7MEd+JN9FGBN4J3NfsIqSKmQK8DfgOcFaTa5Gq5ojMfCPwEeD8\nwznAhAkCEfEW4MvAnmbXIlVJZv4YWAq8A7i7yeVIlZKZN0VEB3AZ8LnDOcaYvn2wllULI+KDwCKg\nE1gHvAC4IjO/0ay6pfFuhOfex4C7KDqGXpmZlzerbmkiqPH8+xBwCnA5cBXwvszcfDivN2bnCJSr\nFr4SeLzc1L9qYflDuga4KDPfd9DzvmAIkA7fSM+9iHgB8FlgL8UKopIO0wjOv/eW+38emA18OCJu\nOJzPvzEbBKhx1cKDZearG1OeNGGN6NzLzFuBWxtaoTRxjfT8e83v+4Jjdo5AZn4T2D9g06CrFja2\nKmni89yTmqcZ5994OplHY9VCSSPnuSc1T93Pv/EUBG4HXgJwuKsWSjosnntS89T9/BvLcwT6uGqh\n1Byee1LzNOz8G9O3D0qSpPoaT5cGJEnSKDMISJJUYQYBSZIqzCAgSVKFGQQkSaowg4AkSRVmEJAk\nqcIMApIkVdh46CwoaRARcS7wFWB5Zm4rt70TeE5m/skg+18G3As8BlyZmc9vYLmDioj3A72Z+YFh\n9vk88DeZ+ZuGFSZViCMC0jiVmT8EvgR8GiAizgLeALzu4H0jYi5wYWbe2Mgaa1BLa9OrgI/VuxCp\nqhwRkMa39wB3lr/tXwq8KjN3D7LfXwLXH7wxIhYDnwJmAk8Al2XmzyJiAfBl4GjgF8A5mXn8Qc9d\nBfwjxYf5LuDPM3NHRLwdeBPQDdyYme+OiGXAPwPTgU7gmsz85EHHexHwAWAyxcjFGzJzZ2aujYgT\nI+KkzNx0OD8kSUNzREAaxzJzH3AJ8FHgq5l5xxC7Xgj8aJDtXwI+npnLgbcDX4+IduAT5fGWA18H\n5g/y3PcAb8rMlcCNwIqIWAm8BVgJPBM4PSJWAH8BfCgznw28APj78hgtABExB/gwcH5mrgBupggZ\nfW4DLjjUz0PSyBkEpPHvbGAbxQplbUPsswjYPHBDRHQAJ2fmDQBliNgJBPBC4Ivl9huARwY55reB\nGyLik8DazPwP4Bzg25n5WGZ2Z+Z5mflz4K+BaRHxbuDvgI7yGH2XBp4NnAD8MCLuohjBOGXAa91f\nvgdJo8wgII1jEbEEeD9wFvAU8LdD7NoD7Dto22DnfwvFJcNuYKhQAUBmfhw4F9gAXB0RVwB7D6rv\nuIg4muKyxEuBe4ArytcZqA24LTOflZnPohhRGDjhcX/5HiSNMoOANE5FxFTgX4F3ZOZ9wGuAt0bE\ncwbZfSPQNXBDZj4GbIqIl5XHOxOYC6wBfgC8otz+Yoq5Age//v8AMzLzE8DHgWcB/w28JCI6ImIS\nxV0Np1OMMFxZTlY8t3x+KwcCwR3AWRHR91v/+4CPDHi5LmB9TT8YSSNiEJDGr48CqzPzKwCZ+QDw\nNuBLETHtoH1vBPpuF+zlwJD8K4HLImI1xWS+i8t5B5cDfxwRPwf+jMEvDVwBfC4ifga8nuKD/i7g\nWuAnwN3Af2XmLRSjFrdFxP8C51NMBuzqqyUzt1Dc7fC1spbTgL8a8FrPK9+DpFHW0ttby907ksaz\n8vbBr2XmOTXu/1bgB+WM/RXAdeWkwIaLiOXAFZn58ma8vjTROSIgVUD5G/e3IuKlNT5lPfDVckTg\nWor+BM3yTorJhpLqwBEBSZIqzBEBSZIqzCAgSVKFGQQkSaowg4AkSRVmEJAkqcL+HywUUKMjEy3l\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x117a9c710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "xdata=np.array([ 0.00010851,  0.00021701,  0.00043403,  0.00086806,  0.00173611, 0.00347222])\n",
    "ydata=np.array([ 29.56241016,  29.82245508,  25.33930469,  19.97075977,  12.61276074, 7.12695312])\n",
    "\n",
    "powerlaw = PowerLaw()\n",
    "powerlaw.fit(xdata, ydata)\n",
    "print 'amp:',powerlaw.amp, 'index', powerlaw.index\n",
    "\n",
    "sns.set_style('white')\n",
    "plt.figure()\n",
    "plt.subplot(2, 1, 1)\n",
    "plt.plot(xdata, powerlaw.transform(xdata))\n",
    "plt.plot(xdata, ydata)\n",
    "plt.text(0.0020, 30, 'Ampli = %5.2f' % powerlaw.amp)\n",
    "plt.text(0.0020, 25, 'Index = %5.2f' % powerlaw.index)\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('Y')\n",
    "plt.subplot(2, 1, 2)\n",
    "plt.loglog(xdata, powerlaw.transform(xdata))\n",
    "plt.plot(xdata, ydata)\n",
    "plt.xlabel('X (log scale)')\n",
    "plt.ylabel('Y (log scale)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reviewing Model performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have introduce several different concepts in class to understanding model performance throughout the last few weeks; here is one primary location that we can refer to for regressions and classification problems.\n",
    "\n",
    "### Regressions\n",
    "\n",
    "**R-Squared**\n",
    "\n",
    "Definition: On (technically) a scale of 0 to 1, how well does this regression explain the variance in our data?  \n",
    "_note: can be negative if the regression model is inversely related, though this rarely occurs_\n",
    "\n",
    "math: $R^2=\\dfrac{SS_{res}}{SS_{tot}}$\n",
    "\n",
    "* we like this because: It's easily explainable on a scale, independent of what your y represents. very universal\n",
    "* we don't like this because: it doesn't actually represent any \"error\" amount\n",
    "\n",
    "**Root Mean Squared Error (RMSE)**\n",
    "\n",
    "Definition: The square root of the mean of the squared errors, where squared error = $(y_{true} - y_{pred})^{2}$\n",
    "\n",
    "math: $\\sqrt{\\dfrac{1}{n}\\sum(y_{true} - y_{pred})^{2}}$\n",
    "\n",
    "* we like this because: It represents error against the y units, unlike $R^2$, and punishes larger error\n",
    "* we don't like this because: if the scale of y changes dependent on the model, it's tough to compare results\n",
    "\n",
    "### Classification\n",
    "\n",
    "**Confusion Matrix**\n",
    "\n",
    "Definition: Given class labels, a true vs predicted label for all observations.\n",
    "\n",
    "\n",
    "<img src='img/confusion_matrix_metrics.png'/>\n",
    "\n",
    "* we like this because: It's the core object for solving for a variety of classification metrics\n",
    "* we don't like this because: we actually just like this\n",
    "* **business prop**: in a business decision, what's the value to error? How much is a true positive worth to a business vs a false positive or false negative? Consider weighting against some business goal.\n",
    "\n",
    "**Accuracy**\n",
    "\n",
    "Definition and math: $\\dfrac{TP + TN}{TP + TN + FP + FN}$\n",
    "\n",
    "* we like this because: It is a nice \"overall\" number to look at.\n",
    "\n",
    "**Misclassification Rate**\n",
    "\n",
    "Definition and math: $\\dfrac{FP + FN}{TP + TN + FP + FN}$\n",
    "\n",
    "* we like this because: it provides the 1 - accuracy value, a number we want to drive down\n",
    "\n",
    "\n",
    "**false positive rate (fpr)**\n",
    "\n",
    "Definition: The percent of the negatives were predicted as positive (how often is the predictor wrong on negatives?)\n",
    "\n",
    "Math: $\\dfrac{FP}{FP + FN}$\n",
    "\n",
    "**true positive rate/recall (tpr)**\n",
    "\n",
    "Definition: The percent of the positives were accurately measured as positives\n",
    "\n",
    "Math: $\\dfrac{TP}{TP + FN}$\n",
    "\n",
    "** ROC Curve and AUC**\n",
    "\n",
    "Definition: The area of a [1, 1] plot given the a line drawn from [0,0], [fpr, tpr], and [1, 1]\n",
    "\n",
    "Note there are two routes to go with using AUC in python:\n",
    "\n",
    "* **plotting classes**: calculate the roc using the binary class labels to gain a single point on the plot and visualize\n",
    "* **plotting scores/probalities**: calcuate the roc using the probalities for class $y_1$. Helpful to visualize when the data may not be weighted normally (a priori is not around .5)\n",
    "\n",
    "All of sklearn's model metrics are in the [sklearn metrics page](http://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics). Keep this around as a reference point so you know how to run the metrics you need to use!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Guided Practice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Back to the data problem at hand!\n",
    "\n",
    "We'll start by working through an explain on each column in the dataset together as a class and come up with some ideas on how to handle each column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Auction\n",
      "ADESA      10128\n",
      "MANHEIM    28645\n",
      "OTHER      12315\n",
      "Name: Auction, dtype: int64\n",
      "Auction\n",
      "ADESA      0.153732\n",
      "MANHEIM    0.114400\n",
      "OTHER      0.118149\n",
      "Name: IsBadBuy, dtype: float64\n",
      "auct_adesa\n",
      "0             0.115527\n",
      "1             0.153732\n",
      "Name: IsBadBuy, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print lemons.groupby('Auction').Auction.count()\n",
    "print lemons.groupby('Auction').IsBadBuy.mean()\n",
    "\n",
    "# seems like the ADESA auction is particularly worse for bad buys (about 36% more)\n",
    "# it may help to create a new column that specically refers to \"is_adesa\"\n",
    "\n",
    "lemons['auct_adesa'] = lemons.Auction.apply(lambda x: 1 if x == 'ADESA' else 0)\n",
    "\n",
    "print lemons.groupby('auct_adesa').IsBadBuy.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([  1.00000000e+00,   2.15900000e+03,   5.93600000e+03,\n",
      "         1.11580000e+04,   1.19690000e+04,   9.06400000e+03,\n",
      "         5.57700000e+03,   3.23400000e+03,   1.53200000e+03,\n",
      "         4.58000000e+02]), array([ 0. ,  0.9,  1.8,  2.7,  3.6,  4.5,  5.4,  6.3,  7.2,  8.1,  9. ]), <a list of 10 Patch objects>)\n",
      "VehicleAge\n",
      "0             0.000000\n",
      "1             0.044465\n",
      "2             0.062163\n",
      "3             0.083259\n",
      "4             0.110118\n",
      "5             0.146293\n",
      "6             0.180384\n",
      "7             0.219852\n",
      "8             0.253916\n",
      "9             0.316594\n",
      "Name: IsBadBuy, dtype: float64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAFVCAYAAAAUiG2GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE7hJREFUeJzt3X+M33V9wPHnQTn0xvXCgIIW8CSkr/CPI0Vl0tiWyGRt\nZljMEgmIkUTNYpMJI8HRLSU1OIw/MHNDIxjCj8BMZMxFDaSLzhWbZnQGhwR8ieRuQ8fPE/qDUmp7\n3R/fT+Pt/Pau970en3vdPR+J6ffe9/7C65vgPe/z+X6+n/YdOnQISZJU03FtDyBJknpnyCVJKsyQ\nS5JUmCGXJKkwQy5JUmGGXJKkwpZMtyEiLgQ+l5kXR8T5wFeAg8DrwEcy84WI+DjwCeAAcFNmfi8i\nTgXuA94E/C9wdWa+1m3vnLwySZIWgb6pPkceEdcDHwb2ZOZFEfFD4C8y87GI+AQQwOeBfwUuAN4M\n/Ah4J/BF4D8z8+6I+DSd8H8T2DJ5b2bun6PXJ0nSgjbdqfVfAB8E+pqvL8/Mx5rHJwCvAe8GtmXm\nbzJzV/OcdwCrgIeavQ8ClwDvOsJeSZLUgylPrWfmAxExPOHr5wAi4iJgA/Be4I+BnROethsYApZO\nWN/TZW3i3q4i4kQ68X+Wzul8SZIWuuOBtwA7MvP16TZP+x75ZBHxIWAjsD4zxyJiFzA4Ycsg8Aqw\ni064X5y0Nnnvy1P8694FPDzTGSVJWgDeS+ct6CnNKOQR8WE6F6qtzczDAX4E+Gxz9Pwm4DzgcWAb\nsB64C1gHbJ1i75E8C3DvvfdyxhlnzGRU6ZgYGRnh2s//C28ePLXtUbp6bfdLfPn6y3j729/e9iiS\njpHnnnuOK6+8EpoGTudoQ34oIo4D/g74b+CBiAD4YWZujoiv0DlyPg7YmJmvR8RNwF3NVeovAlc0\nV61P3jvVhW4HAc444wzOPPPMoxxVOnb27t3L4GnncNLJy9sepaslb/oVp59+uv//kBamo3pLedqQ\nZ+YocFHz5SlH2PMN4BuT1l6gcyQ+7V5JktQbbwgjSVJhhlySpMIMuSRJhRlySZIKM+SSJBVmyCVJ\nKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmFGXJJkgoz5JIkFWbIJUkqzJBLklSYIZckqbAlbQ+g\nxW3//v2Mjo62PcYRjYyMtD2CJE3JkKtVo6OjXHXDfQwMLWt7lK7Gfvkkp5x5XttjSNIRGXK1bmBo\nGSedvLztMbrau/P5tkeQpCn5HrkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySpMEMuSVJhhlyS\npMIMuSRJhRlySZIKM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmFGXJJkgoz5JIk\nFWbIJUkqzJBLklSYIZckqTBDLklSYYZckqTClky3ISIuBD6XmRdHxLnAncA48DiwITMPRcSNwHrg\nAHBNZu6Yyd45eF2SJC0KUx6RR8T1wO3Aic3SLcDGzFwN9AGXRcRKYHVmXghcDtzaw15JktSD6U6t\n/wL4IJ0QA6zMzK3N4weBS4BVwBaAzHwGWBIRp85g7ynH6LVIkrToTBnyzHyAzinww/omPN4NDAFL\ngZ1d1meyV5Ik9WCmF7uNT3i8FHgF2AUMTlgfbNZnsleSJPVgpiF/NCLWNI/XAVuBbcClEdEXEWcD\nfZk5dpR7j8vMX8/+ZUiStDhNe9V641Dz53XA7RHRDzwB3N9cif4wsJ3OLwYbZrD3k8fodUiStChN\nG/LMHAUuah4/BaztsmczsHnS2lHvlSRJvfGGMJIkFWbIJUkqzJBLklSYIZckqTBDLklSYYZckqTC\nDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySpMEMuSVJhhlySpMIMuSRJhRlySZIKM+SSJBVm\nyCVJKsyQS5JU2JK2B5DUu/GDBxgZGWl7jCkNDw/T39/f9hjSgmXIpcL27Rlj023bGRh6uu1Rutq7\n8wXuufkKVqxY0fYo0oJlyKXiBoaWcdLJy9seQ1JLfI9ckqTCDLkkSYUZckmSCjPkkiQVZsglSSrM\nkEuSVJghlySpMEMuSVJhhlySpMIMuSRJhRlySZIKM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGG\nXJKkwgy5JEmFGXJJkgoz5JIkFWbIJUkqbMlMnxARJwB3AW8DDgIfb/68ExgHHgc2ZOahiLgRWA8c\nAK7JzB0RcW63vbN/KZIkLT69HJGvB47PzFXAZ4C/Bb4EbMzM1UAfcFlErARWZ+aFwOXArc3zb5m8\nd5avQZKkRauXkCewJCL6gCFgP3BBZm5tvv8gcAmwCtgCkJnPNM85FVjZZa8kSerBjE+tA68Cw8DP\ngFOADwCrJ3x/N53ALwXGuqz3TVjb06xJkqQe9HJEfi3wUGYGcD5wN3DChO8vBV4BdgGDE9YHm/Xx\nLmuSJKkHvYT813QiDfAynaP6RyNiTbO2DtgKbAMujYi+iDgb6MvMsSPslSRJPejl1PqXgTsiYivQ\nD9wA/Bi4PSL6gSeA+5ur1h8GttP5hWFD8/zrJu+d5WuQJGnRmnHIM/NV4ENdvrW2y97NwOZJa091\n2ytJkmbOG8JIklSYIZckqTBDLklSYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySp\nMEMuSVJhhlySpMIMuSRJhRlySZIKM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmF\nGXJJkgoz5JIkFWbIJUkqzJBLklSYIZckqTBDLklSYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrM\nkEuSVJghlySpMEMuSVJhhlySpMIMuSRJhRlySZIKM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGG\nXJKkwpb08qSIuAH4ANAPfBXYCtwJjAOPAxsy81BE3AisBw4A12Tmjog4t9veWb4OSZIWpRkfkUfE\nWuA9mXkRsAY4C/gSsDEzVwN9wGURsRJYnZkXApcDtzb/iFsm7531q5AkaZHq5dT6+4GfRsS3ge8A\n3wUuyMytzfcfBC4BVgFbADLzGWBJRJwKrOyyV5Ik9aCXU+un0TkK/xPgHDox75vw/d3AELAUGOuy\nPnHvnmZNkiT1oJeQvwQ8mZkHgJ9HxD5g+YTvLwVeAXYBgxPWB5v18S5rmiP79+9ndHS07TGOaGRk\npO0RJKm0XkL+I+BTwC0R8VZgAPh+RKzJzH8H1gHfB54GPh8RX6RzBN+XmWMR8WiXvZojo6OjXHXD\nfQwMLWt7lK7Gfvkkp5x5XttjSFJZMw55Zn4vIlZHxCN03mP/JDAK3B4R/cATwP3NVesPA9ubfRua\nf8R1k/fO/mVoKgNDyzjp5OXTb2zB3p3Ptz2CJJXW08fPMvPTXZbXdtm3Gdg8ae2pbnslSdLMeUMY\nSZIKM+SSJBVmyCVJKsyQS5JUmCGXJKmwnq5al6SjMX7wQImb/gwPD9Pf39/2GFJPDLmkObNvzxib\nbtvOwNDTbY9yRHt3vsA9N1/BihUr2h5F6okhlzSn5vMNiaSFwPfIJUkqzJBLklSYIZckqTBDLklS\nYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySpMEMuSVJhhlySpMIMuSRJhRlySZIK\nM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmFGXJJkgoz5JIkFWbIJUkqzJBLklSY\nIZckqTBDLklSYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySpsCW9PjEilgE/Bt4H\njAN3Nn8+DmzIzEMRcSOwHjgAXJOZOyLi3G57Z/MiJElarHo6Io+IE4CvA68CfcAtwMbMXN18fVlE\nrARWZ+aFwOXArc3Tf2fv7F6CJEmLV6+n1r8AfA14tvl6ZWZubR4/CFwCrAK2AGTmM8CSiDj1CHsl\nSVIPZhzyiPgo8GJmbmmW+pr/HbYbGAKWAju7rE/cu6dZkyRJPejlPfKrgUMRcQlwPnAXcNqE7y8F\nXgF2AYMT1geb9fEua5IkqQczPiLPzDWZuTYzLwZ+AnwEeCgi1jRb1gFbgW3ApRHRFxFnA32ZOQY8\n2mWvJEnqQc9XrU9wCLgOuD0i+oEngPubq9YfBrbT+YVhQ7P/d/YegxkkSVqUZhXy5qj8sLVdvr8Z\n2Dxp7alueyVJ0sx5QxhJkgoz5JIkFWbIJUkqzJBLklSYIZckqTBDLklSYYZckqTCDLkkSYUZckmS\nCjPkkiQVZsglSSrMkEuSVJghlySpMEMuSVJhx+LvI5ekssYPHmBkZKTtMaY0PDxMf39/22NonjLk\nkha1fXvG2HTbdgaGnm57lK727nyBe26+ghUrVrQ9iuYpQy5p0RsYWsZJJy9vewypJ75HLklSYYZc\nkqTCDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJghlySpMEMuSVJhhlySpMIMuSRJhRlySZIKM+SS\nJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmFGXJJkgoz5JIkFWbIJUkqzJBLklSYIZck\nqTBDLklSYUvaHqC6/fv3Mzo62vYYRzQyMtL2CJKkOTTjkEfECcAdwNuAE4GbgCeBO4Fx4HFgQ2Ye\niogbgfXAAeCazNwREed22zv7l9KO0dFRrrrhPgaGlrU9Sldjv3ySU848r+0xJElzpJcj8iuBFzPz\nqoj4feAnwKPAxszcGhFfAy6LiP8BVmfmhRFxFvBPwLuBWybvBb59TF5NSwaGlnHSycvbHqOrvTuf\nb3sESdIc6uU98m8Bm5rHfcBvgJWZubVZexC4BFgFbAHIzGeAJRFx6hH2SpKkHsz4iDwzXwWIiEE6\nUf8b4IsTtuwGhoClwFiX9b4Ja3uaNUmS1IOerlpvTpX/ALg7M/+Rzvvdhy0FXgF2AYMT1geb9fEu\na5IkqQczDnlEnE7nlPn1mXlns/xoRKxpHq8DtgLbgEsjoi8izgb6MnPsCHslSVIPernYbSOd0+Gb\nIuLwe+WfAr4SEf3AE8D9zVXrDwPb6fzCsKHZex1w+8S9s3kBkiQtZr28R/4pOuGebG2XvZuBzZPW\nnuq2V5IkzZx3dpMkqTBDLklSYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrMv49ckuax8YMHGBkZ\naXuMKQ0PD9Pf39/2GIuWIZekeWzfnjE23badgaGn2x6lq707X+Cem69gxYoVbY+yaBlySZrnBoaW\ncdLJy9seQ/OU75FLklSYIZckqTBDLklSYYZckqTCDLkkSYUZckmSCjPkkiQVZsglSSrMkEuSVJgh\nlySpMEMuSVJhhlySpMIMuSRJhRlySZIKM+SSJBVmyCVJKsyQS5JUmCGXJKkwQy5JUmGGXJKkwgy5\nJEmFGXJJkgoz5JIkFWbIJUkqbEnbA0iS6ho/eICRkZG2x5jS8PAw/f39bY8xZwy5JKln+/aMsem2\n7QwMPd32KF3t3fkC99x8BStWrGh7lDljyCVJszIwtIyTTl7e9hiLlu+RS5JUmCGXJKkwQy5JUmGG\nXJKkwgy5JEmFGXJJkgpr5eNnEXEc8FXgHcDrwMcyc35+CFGSpHmsrSPyPwX6M/Mi4K+AL7U0hyRJ\npbUV8lXAQwCZ+R/AO1uaQ5Kk0tq6s9tSYNeErw9GxHGZOT5p3/EA1177l5x44olv2HAzcc4557D7\nxVc5sG/X9JtbsPflX3Fwv/P1yvlmZ77PB/N/Ruebndd2v8Tzzz/PwMBA26Mcteeee+7ww+OPZn9b\nId8FDE74ulvEAd4C8Nhj//WGDNWLHTseAWB3y3NMZT/ONxvONzvzfT6Y/zM63+x87GPfaXuEXr0F\nmPb6sbZCvg34APCtiPhD4LEj7NsBvBd4Fjj4Bs0mSVKbjqcT8R1Hs7nv0KFDcztOFxHRx2+vWge4\nOjN//oYPIklSca2EXJIkHRveEEaSpMIMuSRJhRlySZIKM+SSJBXW1sfPpuX92OdeRJwA3AG8DTgR\nuCkzy37gcj6LiGXAj4H3+QmNuRERN9D5WGs/8NXMvKPlkRaU5ufFXXR+XhwEPp6Z2e5UC0dEXAh8\nLjMvjohzgTuBceBxYENmHvHK9Pl8RO792OfelcCLmbkaWAf8Q8vzLEjND8CvA6+2PctCFRFrgfc0\nPy/WAGe1O9GCtB44PjNXAZ8BPtvyPAtGRFwP3E7ngArgFmBj87O5D7hsqufP55B7P/a59y1gU/O4\nDzjQ4iwL2ReAr9G5sZHmxvuBn0bEt4HvAN9teZ6FKIElzX1Ahujc0E3Hxi+AD9L5OQywMjO3No8f\nBC6Z6snzOeRd78fe1jALUWa+mpl7ImKQTtT/uu2ZFpqI+Cidsx5bmqW+Kbard6cBFwB/Bvw5cG+7\n4yxIrwLDwM+A24C/b3WaBSQzH+D/H0hN/Dmxh84vTkc0n8N4tPdj1yxExFnAD4C7M/Obbc+zAF0N\n/FFE/BtwPnBXRJze8kwL0UvAlsw80FyDsC8iTm17qAXmWuChzAzgD+j8t9zf8kwL1cTWDQKvTLV5\nPod8G533ZJjmfuzqUROULcD1mXlny+MsSJm5JjPXZubFwE+Aj2Tm823PtQD9CPhjgIh4K/B7wFir\nEy08v+a3Z0lfBk7gKP92Ls3YoxGxpnm8Dtg61eZ5e9U68M90jmS2NV9f3eYwC9RGOqdsNkXE4ffK\n12XmvhZnkmYsM78XEasj4hE6ByifnOoqX/Xky8AdEbGVzicDbsjM11qeaaE5/N/sdcDtzRmPJ4D7\np3qS91qXJKmw+XxqXZIkTcOQS5JUmCGXJKkwQy5JUmGGXJKkwgy5JEmFGXJJkgr7PxPMHOTNcd1u\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11520fa50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print plt.hist(lemons.VehicleAge)\n",
    "\n",
    "print lemons.groupby('VehicleAge').IsBadBuy.mean()\n",
    "\n",
    "# there seems to be a stronger relationship with bad buys as vehicles are older.\n",
    "# is there anything we should do here?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Group Work\n",
    "\n",
    "1. Continue to parse through each column and determine relationships against IsBadBuy.\n",
    "    1. Is there missing data? How should it be imputed? What does the missing data mean?\n",
    "    2. Are there any clear relationships?\n",
    "    3. Do we need to be concerned with feature scaling?\n",
    "\n",
    "2. Generate a model in your group. The goal should be a cross validated model that, on average, performs better than the benchmark on the training data.\n",
    "3. Once you've created a model your team is comfortable with, generate a \"submission\" csv file on the out of sample data. Ed, Julia, and Pooja will post \"scores\" against the actual values for the out of sample data.\n",
    "\n",
    "Once you've done so, use the rest of the class today to work on your projects individually. Use this time to practice everything we learned today and to improve your project 2 for Monday."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}