volunteer_verification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Volunteer Verification\n",
    "\n",
    "- In this notebook, I create a dataset with trees for which the health status determined by volunteers does not align with that predicted by my final model (created in the [main notebook](eda_modeling_evaluation.ipynb)).\n",
    "\n",
    "## Import libraries and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataframe packages\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# visualizations packages\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import seaborn as sns; sns.set_style('ticks')\n",
    "\n",
    "# custom functions\n",
    "from functions import *\n",
    "\n",
    "# metrics functions\n",
    "from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, confusion_matrix, classification_report\n",
    "\n",
    "# saving/loading packages\n",
    "import pickle\n",
    "import gzip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load dataframe\n",
    "with gzip.open('data/nyc_trees_clean_features_pro_vol.pkl', 'rb') as hello:\n",
    "    trees = pickle.load(hello)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(207776, 24)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create dataframe of only volunteer entries and drop user_type column\n",
    "trees = trees[trees.user_type == 'Volunteer'].drop(columns='user_type')\n",
    "\n",
    "# confirm\n",
    "trees.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prep data for prediction\n",
    "\n",
    "- I can borrow a lot of the code from the [Reproducibility](eda_modeling_evaluation.ipynb#Reproducibility) section in the [main notebook](eda_modeling_evaluation.ipynb) to get this set of data in the proper format.\n",
    "- I'll have to do a bit of maneuvering when it comes to tree species (see below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# rein in some outliers' values to 4 standard deviations from the mean\n",
    "rein_extremes(trees, ['tree_diameter', 'log_block_count', 'neighbor_dist'], 4)\n",
    "\n",
    "# rename columns that I have to manually dummy, so the new 0 and 1 values make sense\n",
    "trees.rename(columns={'curb_loc': 'on_curb', 'sidewalk': 'sidewalk_damage'}, inplace=True)\n",
    "# trees on curb --> 1 | trees offset from curb --> 0\n",
    "trees['on_curb'] = np.where(trees.on_curb == 'OnCurb', 1, 0)\n",
    "# damage to sidewalk near tree --> 1 | no sidewalk damage --> 0\n",
    "trees['sidewalk_damage'] = np.where(trees.sidewalk_damage == 'Damage', 1, 0)\n",
    "\n",
    "# create list of columns to convert nos and yeses\n",
    "yes_no_cols = ['root_stone', 'root_grate', 'root_other', 'trunk_wire', 'trunk_light', 'trunk_other',\n",
    "               'branch_light', 'branch_shoe', 'branch_other']\n",
    "# custom conversion function\n",
    "# 'No' --> 0 | 'Yes' --> 1\n",
    "yes_to_one(trees, yes_no_cols)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- One difference from just running everything in the [Reproducibility](eda_modeling_evaluation.ipynb#Reproducibility) section is that I need to use only the tree species from the professional data and convert species that appear only in the volunteer data to *Other*.\n",
    "    - First, I'll bring over and assign a list of species from the professional data.\n",
    "    - Next, I'll change any species from the volunteer data that don't appear in the list to *Other*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# species list from professional data\n",
    "species_list = [\n",
    "    'red maple', 'pin oak', 'honeylocust', 'London planetree',\n",
    "    'ginkgo', 'willow oak', 'Amur maple', 'hedge maple',\n",
    "    'American elm', 'American linden', 'crab apple', 'silver maple',\n",
    "    'eastern redcedar', 'tulip-poplar', 'Norway maple',\n",
    "    'swamp white oak', 'Sophora', 'Callery pear', 'scarlet oak',\n",
    "    'Japanese zelkova', 'sweetgum', 'white oak', 'northern red oak',\n",
    "    'silver linden', 'Other', 'Chinese elm', 'horse chestnut',\n",
    "    'littleleaf linden', 'sawtooth oak', 'sycamore maple', 'black oak',\n",
    "    'Japanese tree lilac', 'hardy rubber tree', 'green ash',\n",
    "    'English oak', 'white ash', 'Siberian elm', 'golden raintree',\n",
    "    'black locust', 'purple-leaf plum', 'cherry', 'crimson king maple',\n",
    "    'Kentucky coffeetree', 'sugar maple', 'tree of heaven',\n",
    "    'American hornbeam', 'Cornelian cherry', \"Schumard's oak\",\n",
    "    'hawthorn', \"'Schubert' chokecherry\", 'mulberry', 'Japanese maple',\n",
    "    'shingle oak', 'magnolia', 'American hophornbeam', 'ash',\n",
    "    'dawn redwood', 'common hackberry', 'eastern redbud',\n",
    "    'katsura tree', 'bald cypress', 'serviceberry', 'maple',\n",
    "    'flowering dogwood', 'Amur maackia', 'European hornbeam'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert species not in the species_list to 'Other'\n",
    "trees['species'] = np.where(trees.species.isin(species_list), trees.species, 'Other')\n",
    "\n",
    "# confirm if lists match\n",
    "# converting each to a set ensures the order doesn't matter\n",
    "set(trees.species.unique()) == set(species_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Now that that's taken care of, I can return to using my [Reproducibility](eda_modeling_evaluation.ipynb#Reproducibility) code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# multivariate columns to dummy\n",
    "dummy_cols = ['steward', 'guards', 'boroname', 'cb_num', 'species']\n",
    "# convert community board column to string so I can easily dummy it\n",
    "trees['cb_num'] = trees.cb_num.astype(str)\n",
    "\n",
    "# dummy dataframe\n",
    "tree_dummies = pd.get_dummies(trees[dummy_cols])\n",
    "# reformat column names\n",
    "tree_dummies.columns = [col.replace(' ', '_').replace(\"'\", \"\") for col in list(tree_dummies.columns)]\n",
    "\n",
    "# columns to drop (including dummy columns to prevent multicollinearity)\n",
    "dummy_cols.extend(['steward_None', 'guards_None', 'boroname_Staten_Island', 'cb_num_101', 'species_Other',\n",
    "                  'council_dist', 'st_assembly', 'st_senate', 'nta'])\n",
    "\n",
    "# combine dataframe and dummy columns; drop columns\n",
    "trees_full = pd.concat([trees, tree_dummies], axis=1).drop(columns=dummy_cols)\n",
    "\n",
    "# create features and target\n",
    "X = trees_full.drop(columns='health')\n",
    "y = trees_full['health']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import scaler and model\n",
    "\n",
    "- Unpickle the final versions of my MinMaxScaler and Random Forest Classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load scaler\n",
    "with gzip.open('data/final_scaler.pickle', 'rb') as hello:\n",
    "    scaler = pickle.load(hello)\n",
    "\n",
    "# load model\n",
    "with gzip.open('data/final_model.pickle', 'rb') as hello:\n",
    "    model = pickle.load(hello)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scale and predict\n",
    "\n",
    "- Use the scaler to transform features dataframe.\n",
    "- Use the model to predict health status.\n",
    "- Look at some statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# scale data using scaler fitted to professional data\n",
    "X_scaled = scaler.transform(X)\n",
    "\n",
    "# use final model to predict on volunteer data\n",
    "y_preds = model.predict(X_scaled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.6214914138302787\n",
      "F1 score: 0.661731851883151\n",
      "Recall: 0.6214914138302787\n",
      "Precision: 0.7231549873396961\n",
      "\"Good\" Precision score: 0.8573883161512027\n",
      "\n",
      "----------------------------------------------------------------------------------------------------\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "        Good       0.22      0.35      0.27     33456\n",
      "        Poor       0.86      0.69      0.77    165254\n",
      "        Fair       0.13      0.30      0.18      9066\n",
      "\n",
      "    accuracy                           0.62    207776\n",
      "   macro avg       0.40      0.45      0.41    207776\n",
      "weighted avg       0.72      0.62      0.66    207776\n",
      "\n",
      "\n",
      "----------------------------------------------------------------------------------------------------\n",
      "\n",
      "CONFUSION MATRIX:\n",
      "\n",
      "[[ 11600  15931   5925]\n",
      " [ 37680 114770  12804]\n",
      " [  3146   3159   2761]]\n"
     ]
    }
   ],
   "source": [
    "# print statistics for predictions\n",
    "print(f'Accuracy: {accuracy_score(y, y_preds)}')\n",
    "print(f'F1 score: {f1_score(y, y_preds, average=\"weighted\")}')\n",
    "print(f'Recall: {recall_score(y, y_preds, average=\"weighted\")}')\n",
    "print(f'Precision: {precision_score(y, y_preds, average=\"weighted\")}')\n",
    "# check custom good precision\n",
    "print(f'\"Good\" Precision score: {good_precision(y, y_preds)}')\n",
    "\n",
    "\n",
    "print('\\n' + '-' * 100 + '\\n')\n",
    "\n",
    "\n",
    "# print out report for predictions\n",
    "print(classification_report(y, y_preds, target_names=y.unique()))\n",
    "\n",
    "\n",
    "print('\\n' + '-' * 100 + '\\n')\n",
    "\n",
    "\n",
    "# print out confusion matrix for predictions\n",
    "print(\"CONFUSION MATRIX:\\n\")\n",
    "print(confusion_matrix(y, y_preds))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUQAAAEYCAYAAAAkpo9KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzddZxU1fvA8c/MJgsLS8PSEg8lKCFKCXY3Kj8sxBbEQGwBA1ExwMBAJWy+XzG+dqCIRSoI8hDSufSyXb8/7t1la2YXdmdnYJ83r3mxc86ZO+dOPHPq3uvJycnBGGMMeINdAWOMCRUWEI0xxmUB0RhjXBYQjTHGZQHRGGNcFhCNMcZVaQOiiMwWkc985NUXkUwRObeEbfQTkRwRqVZOdeokIv3LY1tlqMN4EfnxIMpfKCJNAlilQyYil4jIJhFJFpGzy7ittSIytLzqFkgiUk1EhpRQ5rDZn4pUaQMiMB04TURqFJN3KbAL+LJiq8QnQIcKfs5DJiLNgI+A4l7DUPAE8BXQDvihjNvqDrxZ5hpVjLuAm0soczjtT4UJD3YFgmgG8AJwATC1UN5A4D1VzazgOnkq+PnKKtTrGwfMUdV1Zd2QqiaUQ30qSonvy2G2PxXGU5mPVBGR/wBVVPXsfGnNgTVAN1VdICKtgPHAiUA2TovoTlVNFJF+wCwgFqjjPu5oVf3b3dY1wHhVrePezwEGA8NwWoIK3KSqv7nd1BPdavykqv1EpAEwETgT2A98Dtylqnvd7fnMz7cfDwF3Ar+r6lnFvAZnAE8DrYBvgW1Aa1Xt5+YPAu4BBEhz9/cGVd3m7k+uMao62l95H+9BP5yWXGdgC/C4qr7p5vl77XP3bwDwGNAcmA8MUVUtVLd1QL9SvD83ACOBJm7Zsao6zc1b65Z9UUQ8wK0472MzYDlwv6p+4ZadAiQDMcDFQArwgqo+6uM1mILTI4kDLgMSgFtwPlOPAdWBD4EbVTVHRKrivGcXAHWBrcAkVR3r7tNb+TbfApgCLAP6A7Xd1/NL97V9z817X1WHu/V5Cecz1VlVE4ur85GqMneZwek2nyoicfnSLgeWusGwJjAHyAD6ABcBvSlbV+MR4EGcALAPeMVNvwjYCNzv/g1OAAA4ATgXaAm8n29bJeUDnAX0AO4uXBERaQN8CvzHrc8vwJB8+T1xvlxPA21wvoDHunUEOM79vx8wvhTlCz+/AF8DPwPHAA8Dr4jIyQfx2j8MXI/TBawDPOWmN8QJMre7eX6JyLHAJOBet+4TgCki0rqY4vcCjwKjgE7Ax8CnItI5X5nrcALxMcAzwCPuc/hyK/CPu725wLvANcDZwE3AtTjvMcBzOO/5BTg/PBOBx93tf+A+31/ua7DBfcwQ4DbgHFX9J/dJVXUnMBS4VUSOEZETgRuBaypbMITK3WUG+AInKF3IgV/VgTiBEmAQEAZcpaopkNeq+N3HF6U0XlLVL91tPQ18IiKRqrpLRLKARPfv/jhfjn6qmu6WHwRsEpEOQL0S8pPc55ugqit91GUI8JeqjnHvP+k+b7R7PxW4XlVzX491IvIJ0NG9n9vt2qmq+0WkpPKFXYfz4zPSvb/CDYReSn7tM9zHPK6qP7v5L+MEK1R1q9tK3KuqCW6ryp/mQA6wwe1iTxKRVfn2Efc5PDgt7sdVNffHZ7SI9MBpXQ5y01ap6ij37ydFZCROYF7k4/lVVZ92n+MtnJbvnaq6BFgiIg/h9Co+xfnhekVVF7qPfVpEHgY6quoiEdkPZKrqVnd7AD+o6rc+nniGiFwOvIzzuZqgqrNLeL2OSJU6IKpqhoh8gDOJ8paItMf58r7jFmkP/Jn7hXTNA9JxPpx7DuFp8wenfe7/Ee428+uA0+Xa5X6g82uL8+vvL3+B+/e/furSEVhYKG0u0BdAVReKSKL7ZWyPMznREaflVsTBlnfLzCu0jRfBmb3G/2v/p5tW+PWM8Lm3/n0F/IETcJfjDD9MUdXC73FdnJbob4XS5+AEsVyrCuUnllC3/O9Tio+0KPfvacA5InIFTgvxGKAazg9IabZfnFtxhnB2AQ+UUPaIVdm7zABvAyeLSC2c1uEsVd3o5qX6eIyHoh++4gZji/vBKRz4crdX3GNzu1z5b62Bb0qRnyt/QCksp5jnzqufiJwCLMEZX5yN03V72dfGDra8+1y+JgBK+9oXfj19bc/v++MG3t5AL5yhiNOA+SJyainr5S2hXv7qBlDcBF62j7Jv4nTvU3F6Mz0p+cfZ3+cAnPcsBmiEE2QrpUofEFX1N5zAcjbOL/y0fNn/AMeISJV8ad1xfun/oaDcL0BsvrSjDrI6+b+0/wDxOF3oVaq6Cqeb+BxOt6ak/NJYwoFxwFxd8v09HPhQVa9W1Umq+jvOFyf3i104yJRUvrAVhZ4PEXldRJ7h4F770vD7/ojICcBoVf1VVR9Q1U44reyL8pVHVfcBm3GCUH49D7FeB0VEauOMLV6lqve73fY0nKVPvt6XkrYZDbwBvIgzBv2miFTK3mOl3OlivA3cATQG/psv/R2cQftpIjIGqIXzy/ydqi4TkfyBZxvOAPZdInIPzvje4IOsx36gnbvdb4GlwPsiMgLIAl7C+UKvxZkF9ZdfmsXSrwK3ichTwGScVtE5wK9u/iagn4h0cet2Nc7s4x/56gtO4NpQivKFTQKGi8ijOD9ExwFX4kwE/Y7/1755KfYvv5LenyTgfhHZiTNO1wanSz+5mG2Nw5kkWY8z5DAQ57Xrd5B1OhT7cLrfF4nIGpwfxadxgmFul3o/0EBEjgLWl2KbY4CqOK93FZxZ8xE4+1mpVPoWouttnNnQj1Q1dzICVU0GTsf59Z2HEyxnU6jV4JbNxvmCtcNZxnAHPmZX/ZgAXAF87W7vPJwxnVnAjzjLK85S1ayS8kvzZKq61t2/k4HFOEtEJuUrMgpYjbPPv+CMB44A2otItDtDOdm9jSmpvI/nPxendb4E5ws5RFV/OJjXvpT76vf9UdXFOBMiN+AEhDeAZ1T1raJb40Wc2eyn3XqfizN7+/Oh1O1gqGoG8H/AKTj7MQ1niOQLoKtb7D84QXMZzufaJxHphrOQe7iqJqrqduA+YJQUMzh9pKvU6xCNMSY/ayEaY4zLAqIxxrgsIBpjjMtmmQsRkbo4M4Zr8b3mzJgjRTTOUTrflNcJH9yzINUpodiO8jjpRnmzgFjUaTizzsZUJldw4AitQyYizbLxrvX6XFOeJ1lE2odaULSAWNRagN7X3EVcg5A872lA/aI7g12FoHnwjEq3yoQN69bw/NiHwP3cl4M6XrLZEdmRDG/xh49HZCdRJ/3vGJxWpAXEEJcKENegCbWbHur5Gw5fEbtiSy50hGol7YNdhWAq1+GhjLBYMsJ8nDc4K3SnLiwgGmPKn8fr3HzlhSgLiMaY8ufxODdfeSHKAqIxpvx5vOD1cTaybGshGmMqE+syG2OMy4OfLnOF1uSgWEA0xpQ/b5jvLrOv9BBgAdEYEwAeP13j0G0iWkA0xpQ/ayEaY4zL46eFaMtujDGViscDXluHaIwx1mU2xpg81mU2xhiXHbpnjDEuf4fu2ZEqxphKxQ7dM8YYl3WZjTHGZV1mY4xxWZfZGGNcXj8tRK8FRGNMZWItRGOMyeVnUsXOdmOMqVSsy2yMMQ6Px4PHRwvRV3oosIAYQNlZWXz71nMs/OYj0pOTaN29D+fdNppqNesUW37+lzOYM2Myu7dspGbDJvS59Dq6nnFJXv7vn7zDZy+MLvAYrzeMR79ZHsjdOCReD1zfqxlndqhPlcgw5q7dzbPfr2Z3ckaJj33ygvZUiQzjtg+X5KXFVQlnaL+j6NG8Jnhg4fo9vPjjGhL2pwdyNw5JVlYWE558hJkfvk3y/v307n8KD419ljp16/t93Pq1/3LhKSfw+eyFNIhvlJe+a2cC40bfx5xZ35KTk8PxvU/knlHjCpQJNYEMiCLSA3gFaAP8CVyjqiuLKXcbcDdQHfgJGKyqO/1tO3TbrkeA76dNZNG3Mxlwz1Nc99y77EvYyrujhxZb9u/ZX/HpxFH0vewGhr/5Fb0uuZaPn32Qf379Pq/MtjVK2xNO5t4Pf8273fPBnIranYMy+ISmnNGhHo99tYJhHyymbrVIHjuvbYmPO69TA3q2rFUkfdTZbWlYI4o7//s3d8z4mzrVInn8/HaBqHqZvfjM43w84x3GTXidaR99zdYtmxl+3RV+H7Nm9UquG3g+yclJRfLuunkwm9avY/J7n/LmB/9j+9atDBsyMFDVLxfOuR08xd/KEA9FJBqYCTwLxAFfAzOKKXcxcCdwElAfSAKeKmn7IRUQRSRHRJJEZH+hW9USHne/iEyuqHqWRmZGOr/NnMpp195Jq669adS6A5c9+Dzrli5g3dKFRcon79vNyVcNp8vpF1OrYRO6n3Up9Vu0YfWiX/PKbFu7koat2hFbq27ezVdrM5jCvR4GdInntTnrmL9uDyu2JzH6c6VToxp0jI/1+bhGcdHc0LsZSzbvK5BeJSKMLk1r8M7cTazcnsSqhCSm/7GRdg1iiY0OrU5Oeno60ydP4o57R9PrxJPo0OkYnp00hYXzfmPRvN+Lfcy0yS8x4My+xFavUSQvaX8if/zyE9fdegftj+5Mu46duOG2Efz910L27N4V6N05ZLktRF+3MugP7FPV6aqaATwONBORToXK3Qg8oqorVTUVuAV4sqSNh9anydFZVVcdzANUdWygKnOotqz+h7TkJFp07pGXVrNBY2o2aMy6JfNo1qFLgfLHnXPgFz8rK5Nlc74lYf1qThl8e1769nUrOf78QYGvfBm1rleVqlHhLNqwNy9t6740Nu9NpVOj6vy9ObHIY7weeODMNrw7byNNalahUVyVvLz0rGxS0rM4s0M9/ty4l5ycHM5oX48Nu1PYn5pZIftUWsuXLiZpfyLH9eyTl9aoSTMaNWnG/D9+5djuxxd5zA9ff86YpyZSt159rr7krAJ5kVHRxFStxscz3qH7Cb3xeL18MuNdmrZoSfUacQHfn0PmL/AdSJ8vIoVzx6jqaD9bbgvkjRGpapaIrHbTF+crdyzwqYgsAuKBr4BhJVU7FANisdxW4kQONIGX4YwJLBGR0UArVb1CRKYAUUBfYJmqnhqM+u5L2ApA9ToFx41ia9djj5tXnI26hFeHDSA7O4uuZw5AevQHYO+OraQk7mXF3Nl8P+0FMlKTad7pOM64fmSR5wi2utWiAIqM7+3cn0692KhiH3PFcU0gB96bt4mRp7UqkJeVncPYr1Zy92mt+HLo8eTkwO7kdIZ+sIScwOzCIdu6ZRMA9RrEF0ivV78hWzdvLPYxU2Z8AcDcX2cXyYuIiGDs868w6u5hHNe2ER6Ph9p16zH9o6/xhvBsrcfjdI995bm6qeqCg9x0VSC5UFoyEFMorSYwBLgQ2A28DTwPXOtv46H7ihZ1N9AA6Iizswo87KNsT6AzcHHFVK2ojLRUPF4vYeERBdLDIyLJTE/z+bhaDRpz88sfcdGIJ/j7py/49q3nANi+1hkz9oaHc9kDz3HRiHHs2LiWN0deRUZaauB25BBER3jJys4hK7tguErPyiYqvOhHrk29qlzerRGPf7XCZ4BrVqsK/yYkMfzDJQz7cAkbdqfy+HntqBIRWmdfTk1Jwev1EhFR8H2PjIokLc33++7PmlUraNOuA1P+8wXT/vsVzY9qxbAhA0naX7SlHSoC2GVOBqoUSosB9hdKSwNeUNW1qroXeAw4r6SNh2ILcaGIZOe7f4uqvgu8AOQA6UALYA9QeNwg13equqOkJ3JblqPKVt3ihUdGkZOdTVZWJmFhB17mzIx0IqMLv58HxNSoSUyNmsS3ak/Snp38MP1FTrl6OK279eH+//5B1RoHJhzqNW/NU5f3Rv/4kY59zwjEbhyStMxswrwewjyQlS/CRYZ5ScnILlA2MszDQ2cJr/+yjk17ig/snRpVZ0ivZlz82jx2Jjmtzvs/WcaM67tzVsd6/HfRloDty8GKjo4mOzubzMxMwsMPvO/paenExBRuxJRs/h+/MPGpR5k1X6nXoCEAL775Picf156ZH7zNFUNuLre6lyfnZDe+Wohl2vRynJYfACISBrTCaSDltwJndjlXGKVYER6KAbGLjzHEmjhT7ccC/+D8UvjaQd990nzcsYrR+dNEpCswv5R19alGPefDm7gzgTj3b+f+9mK7uGv++oOoqrHEt2qfl1a/hZCRlkpK4h6qxtUuEAwBqteuR0yNmuxNCJ2AALA90WkJ1a4WyfbEA93m2tUi2bG6YCupfcNYmteO4aY+zbmpT3PACZIej4evh53AlVMW0qFhLDuT0vOCIcD+tCw27E4pMNYYChrENwYgYdtWGjZqnJe+fdsW6jU4+6C399eCedSt3yAvGAJUrxFH86NasX7tv2WvcIB4vF6fXXpP2br6s4DaIjIYeAe4F1gH/F2o3FTgFhH5GNiF05v8sKSNH05d5peBeUBdVe0NfOGnbNCHlhoe1ZaomKqsXTw3L2331o3s3rqR5kd3L1J+9gev853bPc61cfliqsbVJqZGLX6dOZVxl/UiK/PAOr7d2zaRtGcX9Zu3DtyOHIJVCUkkpWVyTOMDs6YNqkcRXyOaPzcWnEFetnU/l78xn2unL8q7zV61E922n2unL2Ln/jS270+jVkwEcVUOdEOjwr3E14hm4+6UCtuv0mjb/miqVotl3u8HlkNt2rCOTRvW0e34Xge9vQYN49mZsJ2dO7bnpaUkJ7Nh/VqatWhZLnUOGI+PWxmoagpwNs6s8U7gNOASVc0RkaUikjvrOBF4AyeAbgASgJElbT8UW4i+xAHJqpotIu2BoTgvSEgKj4yix7mD+PLVccRUr0nVmrX5bMIoWnQ6jqbtjyUzI52UxL1Uia1BeEQkPS+6mqn3DeHnDyfTvteprFk8l58/fJ0zb7oPj8eD9OjHt28+x0fj76ff/91E8r49/O/lx2jWsRutuvYO9u4WkJGVw8y/tnDLiS3Yk5LBnuQM7jy5JYs27GXZlkTCvR6qR4ezLzWT9MzsIl3l5PQs0vKl/7p6F9sT0xlzjvDST2vIzM5hSM9mpGVm89Wy7cVVIWgio6IYePX1PP3I/dSsVZtatevyyP130P2E3hzT9TjS09PZu2c3NeJqEhkZWeL2+p12Fg3iG3PnTVcz8uGxREREMPHpx4iOjub8Af9XAXt0aLxeLzk+WoJlnQxyJ2KKtCpUtUO+v7NxltmUuNSmQN3KVLOKdRcwSEQScRZiTgFaikiE30cF0SnX3kHnk89jxrgRvHHXFcTVb8TAUS8AsH7pIsZd2pP1SxcB0LpbHwY+/AJ/fvcJE68/m9nvv8Y5Qx+mx7nOh752fDMGP/kWexO2MGnoxUx/6CYatBCufPSVoO2fP5PnrOPbfxJ46CxhwqVHszUxjYc++weAjvHV+eTmHnSMr17CVhwpGdnc9uES9qRkMP7iDkwYcDQeDwx9fzHJ6VmB3I1DMvyehznnossYOXQI1ww4i/hGTZjw2tsA/Dn/d/oe05I/5xe/JrGwqlWrMeU/X1Crdh1uGHQh1ww4G3Lg7ZnfUC22dK9fMOSOIRZ/C3btfPPk5AS9dxlScscQz7n3eWo3Da2uaEX48s9SDb8ekV7/v2ODXYUKt0qXcdeNV8ChLYEpIvf7s++Ym8iqVvyhhWH7N1H9z1fK7TnL0+HUZTbGHCY8Hj+TKnY+RGNMZWJnuzHGGJcFRGOMceWe7cZXXqiygGiMKXfWQjTGmFylO9tNyLGAaIwpdx6vxzmnm6+8EGUB0RhT7jx+rrrnsavuGWMqE6/X4/PQPWshGmMqF38ncgjdeGgB0RgTADapYowxDq/H96SKBURjTKXi8fieVLGAaIypVDw2hmiMMQ5/h+7lWEA0xlQmXq/vy5DmeD1kF5sTfBYQjTHlzt8QonWZjTGVisfjweujhZhtkyrGmMokkGe7EZEeOJckbgP8CVyjqisLlfEAeyl43agpqjrU37YtIBpjyl2gAqKIRAMzgXuA94H7cC46d0yhoi2BFFUtehF0P0L34gbGmMOW1+tMrBR/K9Om+wP7VHW6qmYAjwPNRKRToXLHAH8ddL3LVDVjjPEhd2Kl8K2M2gLLc++oahaw2k3P7xignogsEZGtIvKmiJR43VbrMhtjyl1ua7A4OQfS54tI4ewxqjraz6arAsmF0pKBmEJpacAvwCj3/lTgReAqf/W2gGiMKXelHEM8lOsyJwNVCqXFAPvzJ6jqo/nvi8jDwA8lbdy6zMaYcueru1wO3eblOLPLAIhIGNAK0PyFRGSkiHTOlxSF02r0y1qIPnRrEEeLZnWCXY0K9+iI54NdhaBpPrx3sKtQ4ZK3F25slY8ArkOcBdQWkcHAO8C9wDrg70LlBDhVRAYAEcBYYHpJG7cWojGm3OV2mX3dDpWqpgBnA7cAO4HTgEtUNUdElorIILfoHcAWYBVO63EZcH9J2/fZQhSRWw6iki+Xtqwx5sgXyLN/ueOO3YtJ75Dv732UMIFSHH9d5rtLuY0cwAKiMSaPx4PPLnMIH7nnOyCqaouKrIgx5shxxF+oXkRigStwBisfA3oAf6vqugDVzRhzmDpcA2KpJlVEpA2wAhiOM5hZHRgELBaR4wNXPWPM4cj3YXu+Z59DQWlnmScAH6hqWw6s5RkEvAeMD0TFjDGHtwAduhdQpQ2IJ1Bo4kRVc4CnKXqWCWNMJXektxBTgLrFpLcG9pVfdYwxRwKvx7kUafG3YNfOt9IGxKnAi/nGC+uKyHnAJJzV4sYYc4C/w/ZCOCCWdpb5QZz1hrNwjgn8DcjE6UaXuPrbGFO5hHk8hPloCoaF8EBiqQKiqmYC94nIGJwz0YYDq1Q1KZCVM8Ycno7oZTcAIlIV+D9gCDAQuEhEIgJVMWPMYeww7TKXdh3i8ThnlHgM6AB0BZ4BVohI4TPVGmMquTCPx+8tVJW2hfgC8BHQTFVPV9VTgWY4Y4mvBqpyxpjDkwc/Z7sJ4SZiaQNiR2C8e1EXIO80PI8AxwWiYsaYw9iR3GUGFgJ9iknvjnOeMWOMyRPmhTCvx8ct2LXzrbTnQ5yLsw6xK/AHkAV0Bm4GxgW0hsaYw87hOst8MOdD3Aqc6d5yJQDX4ky2GGNMnhCOez7Z+RCNMeXO32xyKM8yH8z5EMOB+kCYm+TBOWqlq6q+F4C6GWMOU0dilzmPiJwNvAkUdxm63TinATPGGMA5242vQ/eOhLPdjAO+w1lisw84Cedola3A0MBUzRhzuPLg57rMwa6cH6XtMrfGudSfishCoKqqfigiGTgnfng/YDU0xhx2nOAXmItMiUgP4BWcC9b/CVyjqiv9lJ8OeFT1ipK2fTDnQ8xy/14BdHL/XuBWyhQjKyuLl8eP4ZyebTmpc2PuH3o1u3Zs91n+u88/4qpz+9C/UyMuObkL0159jqysrLz8vxb8zo2Xn0H/To04t3d7XnpqFBnp6RWxKwfN6/XwyLDz+Pebx0n45RnefXoI9WrF+izfqF4c7z49hO1zxrPu+yd4/r5LqRJ94FD5+rVjmTZuMOt/eIK1343l2XsGEBMdWRG7ctCysrJ46IH7aNGkIXXiqjHwskvYtm2bz/IL5s+nf99e1KoeQ8d2rXln+rQC+QkJCVx79ZU0aViXxg3qMGjgpWzcuDHQu1EmXp9rEMt2glgRiQZmAs8CccDXwAw/5S/AOQdD6epdynKzgYdFJA6YD1zontihH3aCWJ8mTxzHlzPf5+GnJjHp3c/ZvnUz9w0t/lKxv/30LaPvuoFzL72S6Z/N4ZYRo3j7tQlMnfQsAFs2refOIQNo36kr0z+bw0NPvsxXn3zIy+PHVOQuldqDN53FFef24LqHpnPqkOdoVC+O98ZfV2zZyIhw/jdpKDWrx3DS4Ge58p43OatPR8YOvwCA8HAv/5s0DGlRn8vueJ0Lhr7MMW0bM+P5Gypyl0rtsUdG8/b0qUx+axrf/jCbTRs3MvDSi4stm5CQwHlnn84xx3bht7kLueXW27jphiF89+03eWWuvmIga9eu4bMvvuHzr75jy+bNXHbJhRW1O4fEU8KtDPoD+1R1unvk3ONAMxHpVLigiNQBngTeKu3GS9tlHgF8inOmm5dxLja1D4gEHijtk1UmGenpfDj1Ve58aBzH9e4PwKPPv8FF/TuzeOEfdOrSo0D5me+9Rb/Tz2PAlc6XvHGzFqxdvYLP//sO1w69my0bN3Diaecw/P7H8/JPPutC5v/2U8XuWClEhIdx68B+3PXUf/jhj+UAXHnvW+gXj3B85xb8/teaAuUvO7MbDepWp/81z7AnMQWAR1/5ghsG9AbgzN4d6dg6no7nj2H1+gRne/e8xYovH6F311bMWbCqAvfOv/T0dF56YQLPPDeRk085FYDp77xP29Yt+O3XXzmhZ88C5ae8OZnqNWrwzHMT8Hq9SNu2/LloIc8/O55TTj2NxMREfpz1AzM++oRjjj0WgJH33s+F553Nrl27qFWrVoXvY2l43LNj+8pzzReRwtljVHW0n023BZbn3lHVLBFZ7aYvLlT2ZeApoAnQqjT1Lu35EFcC7UQkRlVTROQE4HRgk6r+UZpt5BKRU4CRQBcgGlgDvKaqLxzMdkp4juuAK1S1X3lt82Ct+GcJyUmJdOnROy+tYeOmNGzclL/m/VYkIF5zywiqVIkpkOb1ekjctweALj160aVHr7w8XfoXs7/7nP6nnxfAvTg0naUx1atVYfb8A8M667fsYu2mHfQ6tmWRgHhqz3b88PvyvGAIMP3T35n+6e8AtGpaly0Je/OCIcCm7XvYsSeJPiEWEP/6808SExPpe2K/vLRmzZvTrHlzfvnl5yIB8Zc5P9O7d1+83gOdtb4n9mP4sFvIyckhOjqaatWq8fb0qfTpeyJer5d33p5Gy1atiIuLq6jdOmhej++ucb5A2U1VFxzkpqsCyYXSkoECXx4RuQyIVdU3RGR0aTfu79C99j7Sc/9cnltOVUt1PLOIDMS57MBdwHk445JdgKkiEq2qT5e24qEuYetmAOrWb1ggvU69BmzbuqlI+faduhS4n5S4j4/efYvj+5xcpOypXUNsvlQAACAASURBVJqxP3Efbdp3YvCtI8qx1uWjUX3ni7o5YU+B9C0Je2lcv2aR8q2a1uOneSt4+JazGXhWd3Jy4JMf/mL0S5+Rlp7JloS91KoRQ0x0JMmpzphptZgoalWPoV5N3+OSwbBpkzO2F9+oUYH0hg3j2bhhQ9HyGzfS+ZhjC5aNjyc5OZmdO3dSp04dXntjCkNvvoEGdeLweDzUq1+f736YXSCIhhp/V9gr46RKMlClUFoMsD/3jojUB8biDOkdFH8txL9xLhvgq/q5eTkcWKztkzvmOBG4XVWn5Mv6Q0SuwAmMiMjdOEt5quJcsmCoqm7zl+cuGn8GuBrYAXxfUn0CLTU1Ga/XS3hEwXPoRkZGkZ6W5uNR7mNTkrnnlitIS0vh5rtHFcjLzs5mwpSZ7Nu7m+ceu4+7rr+MV977MqQWu1aJjiQrK5vMzOwC6WnpmURFFT2ncPWq0Vx9wQl888syBo18k/h6NXjunkupW6sa1z00na9/Wca+pFReemggtz/xITnkMOG+y8jJgYiIUh9bUCFSkp33PaLQ+x4VFUVaamqR8skpyURHRxcpC5Dqll+hy+nQ8WgefHg0Xm8YY0Y9yGWXXMisn38lNja0fhBylbLLfCiW4wzdASAiYTjdYc1X5lSgAbDEbcBFA14R6aSqRcYa8/P3aSrvQ/d6ArEUs4hbVefjjCfcDNyEs0MbcALoe8BJ/vKAYe7/bXF2/gdgfTnX/6BERVUhOzubzMxMwsMPvMzp6WlFusb57dm1k7tvGsjaVcqEKTNp2KhpgXyv15vXmnz4yZe5bsCpLFk0t0gXPJhS0zIIC/MSFuYlK+tAUIyKDCc5peiPQUZmFrv3JnHtg1PJzs5h4TJnHPLdp69j5PiP2LU3iQG3v8brj1zJ5p+eJCUtg0nv/8TiFRvZtz+lyPaCKbpK8e97WloaMVWrFilfJboKaYV+IHPvV61alTlzfmbMqIdYuWYD8fHxAHz434+Rls2YPnUKtwwdFsC9OXRej+8F2GVclz0LqC0ig3EucHcvzsmr/84toKpvA2/n3ne7zK1Ks+zG37HM6w69zsVqCOxU1bx3X0R+Bo7mwGGA84FnVHWFm38HsFdEGgOD/ORdBExU1a1u3os4XXK/3BdqVEnlDkW9hk6XaWfCVuo3bJyXvmP71iLd6FxbNq5n+OCLSE7az6R3P6dV2455eWtWLidh25a8CRqAlu6oRsK2LYHYhUO2cetuABrWqc7GbQe6zQ3r1mDz9r1Fym9O2ENqWibZ2Tl5af/8uxWAZvG12LU3iT8Wr6HTBY9Qt2Y1EpPTSE3LYOOsJ5ny8W8B3puD07hxEwC2bNlCkyZN8tK3bNlcpBsN0LhJE7ZuKfj+bdm8mWrVqlGjRg3m/vE7DRo2zAuGAHFxcbRq3YbVq0Nn7LQwL76XsJSlo+/OYZyNsw5xIvAXzhrpHBFZCoxV1UO+EmhFDkJsx4nseX0JVe2jqnE4lySIwrn287p8+ftxDg1sXEJefSD/wFypgrmqjlZVT/4b0O0Q96+A1m07ElM1lkVzf8lL27JxPVs2rueY7j2LlN+1M4FbrzyX7OxsXvvg6wLBEOCXWV/z8J3XkZZ2oNu1bPFCAFq0KjJTF1SLV2xi3/4UendtnZfWtGEtmjeqw5yFRb/EvyxcTWdpRHj4gY9jh5YNyczMYt3mXbRsWpfv37yDmtVjSNi9n9S0DHp1aUlcbJW8WexQ0alzZ2JjY5kz+8Ds/7q1a1m3di29e/ctUr5nr97MmTObnJwDPwY//TiLE3r2wuv10qhRY7Zv28b27QfWryYnJ7N2zb+0atW6yPZCRaDWIQKo6gJV7a6qsaraO3dRtqp2KC4Yut/zEluHULEB8VecAdHL/JTZiHNpAgBEpDpQEyeY+svbCuTvWxbfBKtAkVFRXDzoWl4Y9zC/zf4OXfoXD90+hGOP60XHY7uTkZ7OzoRteQurx4++m727d/HIc5OJio5mZ8I2diZsy1vIfeaFlwPw+L1DWbt6BXPnzGLsfcM45ewLOap1u6DtZ3HSMzJ5bcbPPHHHhZzasx3HtG3M9HGDmT1/JXOXrCUiPIz6tWOJCHeGnif/Zw5RkRG88ehVtGlen/49hLF3XMg7/5vLrr1JrNu8k/h6NXj2ngEc1aQOfbu1ZurYa5jy8W/8u2FHkPe2oKioKG646Rbuu2cE33z9FYsWLuTKQZfTp++J9Dj+eNLT09m6dSvp7vt+9eAh7EhIYNgtN7H8n394+cUX+OD9d7lzxEgAzj7nXBo3acKV/3cZCxcs4O8lS7jmqkFEV6nCoCuLX9MaCpwL1fu+haoKG5FW1VT3pLOT3NXm7+MEyB7AE8A2nH7/gyLyHc444TPAAlX9V0T85b0L3CEin+NM8gzFCZJBdcMdD5KZmcmYETeSmZHB8X1PYcQoZyJ9yaK53HrFubz09me079yVn775jOzsbIZcXHBWOSwsjDnLd1C7bn1enPYJE554kCEXn0x0lRhOP/9SbrrjwWDsWolGv/Q/IsLDePOxq4kID+PbX5dx+7gPATi+81F8M3k4p103gZ8XrGT7rkROve55nrrrYn579x72p6Tx/hfzeGjipwBkZmZz0W2v8Ow9A/jj/fvYsy+Z6Z/+wWOvfhHMXfRp9COPkZGRwbVXX0FGRgannn4Gz098CYDff/uV00/pz9ffzaLvif2oX78+n/zvK+664zaO734sTZs1Y/Jb0+jX/yQAqlWrxlffzuL+e+/mgnPPJCcnh+N79uL7WT9TvXr1YO6mf37OdhPKJ0r05G+ql0RE2uFMXHwD1APWqmrpN+BsoxfOOsSeONPnG3AuYPUssAtnkPRGoBbO5MitqrpJRDx+8rw4K9ZvxJl+/wQ4+lDWIbpnBZ8/+vkptAixlldF6D8gNANsRdg978VgV6HCLVv6d+5RNIeyJrCI3O/PiXc8S1yT4tdC79mwip+eu7PcnrM8lfb0X9VxWnRnANk4xy8/DzQVkbNUdXNpn1BVfwHO91PkCfdW+HE5fvKygfvcmzEmyAK47CagSjuG+AzOpEdjnBM9gLPUZT9OYDTGmDzeEm6hqrR1Oxu4O39L0F2WMxQ4JRAVM8YcvgI5yxxIpZ1UqcaBlmF+oR7wjTFBkHuCWF95oaq0wewrnNN/5QbQHBGpC4wHvg1IzYwxh63DddlNaQPiMJx1frtwDqT+DufQuOo4pwIzxpg8Xj8Xqg/hc1KU+vRf24BeItIfaO8+7h/g24NddmOMOfL5awmGcgvxoBZmq+osnIOrjTHGJ4/7z1deqCrtOsQEnCNAiqWq9cqtRsaYw16YB8J9dI3DQjceHtQlBAo/riVwDbYY2hhTyBF9oXpVnVpcuojMw5lUKTbfGFM5OZMqvvNCVVlP7vAXzskZjDEmjxffh+55j4AxxOKurxKLc22U0D1LpTEmKDx+ZplDuMdc6hair+urbMAZRzTGmDxhHg9hPiKfr/RQUNqA2AXn7NS5coB0YJutQzTGFOHnqnsh3GMudUD8GLhIVRcGsjLGmCPDkb4wO/dyo8YYUyKvxzl0z1deqCptQHwf+FZEPgD+pdCZb1T15fKumDHm8OX1c4JYX+mlJSI9cK661wb4E7gm90JT+crUAl7DuTxxJvAuzikMM/xtu7QB8VIgETirmLwcwAKiMSaPx88YYlnioXs9ppnAPTgNtfuAGcAxhYpOxIlZ8UBVnEuOXAu86m/7PgOiiDwMjFfVZFUt74vWG2OOYF4/s8xlbCH2B/ap6nQAEXkc5wJznVR1cb5ygwGPqqaLSDzOGf93lVhvP3mjcE4Ma4wxB8VTwq0M2gJ5F+NW1SxgtZtOvvQMNxjOdPM34EwO++WvyxzCQ5/GmFBWyiNV5otI4ewxqjraz6ar4ly+OL9knPO0FmcgzvXbPwXGAnf72XaJY4iN3T67X6q6vqQyxpjKo5RHqhzKZUiTcS5fnF8MzgXvilDVVGCLiIwDHqOMAXFeCfm5y3HCSihnjKlEAni2m+XAkNw7IhIGtAI0fyER+RZ4UlW/c5OigD0lbbykgNgf2HkwtTXGGK/H9yF6ZVyHOAuoLSKDgXeAe4F1OIcX57cIeNA9I1d1t1yJq2H8BcQcYLmqbj+UWhtjKi9/kydliYeqmiIiZ+OsQ5yIc8atS1Q1R0SWAmNV9R2cSeHncE4+kwxMooQlN2CTKj7N37qHNZE7gl2NCvfA07cHuwpB88GiyjcUvmXN1oBsN5AniHXHHbsXk94h398pwE3urdT8BcSpFH8tZmOM8cuLn3WIIdzW8hkQVXVwRVbEGHPkCFSXOdDKesZsY4wpIlCH7gWaBURjTLkL4KF7AWUB0RhT7o7o6zIbY8zBsC6zMca4ArgwO6AsIBpjyp0HPy3ECq3JwbGAaIwpd0f6VfeMMeYg+J5UCeU2ogVEY0y5s0kVY4xxWZfZGGPyWJfZGGMA6zIbY0yeMHx3jUP59PoWEI0xARHCDUGfLCAaY8rfYXr+LwuIxphy5/X4uQxpCA8iWkA0xpS7w7SBaAHRGBMAh2lEtIBojCl3Hj9d5rJeZCqQLCAaY8pdIBuIItID5zKkbYA/gWtUdWWhMlVwLkN6AU6c+wK4TVX9XqzeW8a6GWNMER48eZciLXIrQ0gUkWhgJvAsEAd8DcwopuhooAXQDjgKqAE8U9L2LSAaY8pd7pEqvm5l0B/Yp6rTVTUDeBxoJiKdCpWLBh5V1d2qug+YDPQsaePWZQ6g7Kwsvn3rORZ+8xHpyUm07t6H824bTbWadYotP//LGcyZMZndWzZSs2ET+lx6HV3PuCQv//dP3uGzF0YXeIzXG8aj3ywP5G4cEg/Qr2VtOsXHEhXmZfXOZL7SBJLSs4ot3zk+lhOa1SQuOpzdKZn8vm43f21JzMvv2rg6Z7atV+Ax2dk5jP1hdSB345BkZ2Xx31fGM+d/M0hNTuLoE07kyrsfpUbtusWWn/3pB3z59qskbN5IvUZNOPOKG+lz7qV5+ft27WD60w/z9x8/Ex4RQZ9zBnDxzXcTFh66X99Sdpnni0jh7DGqOtrPptsCeR94Vc0SkdVu+uJ86cMLPe48YEkJ1baAGEjfT5vIom9nMuCep6hSvSafTRjFu6OHcsOE94uU/Xv2V3w6cRQX3P4ozTsdx+pFv/Hxsw8SU70m7XqeDMC2NUrbE07mgjsezXtcqA5Q9z2qFp3iY/l06TZSMrI5o21dLunUgKnzNxUp27ZeVc5sW48v/tnO+t0pNK8Vw9nt6pGckcXKHckA1KsWhSbs54t/Eip6Vw7azNef45fP/8P1o5+jWo04pj31EC/eexMPvP7fImXn/fAF0558kKvvG0vbY3uwbN4vvDX2XqrVqMmxfU8F4MV7bgKPh/te+YDdCVuZ/MgIvGFhXHLLyIretdJzu8e+8lzdVHXBQW65KpBcKC0ZiPH1ABEZDlwMHF/Sxq3LHCCZGen8NnMqp117J6269qZR6w5c9uDzrFu6gHVLFxYpn7xvNydfNZwup19MrYZN6H7WpdRv0YbVi37NK7Nt7UoatmpHbK26eTdfrc1g8nrguKZxzFq1kzW7UtiamMbMJVtpEleFxjWii5SPiQhj9uqdLN6SyJ7UTP7cvI/tSem0qHXgM163aiTbEtNJSs8qcAs1mRnpfPvBW1x8y0g69uhD87ZHc/NjL7Dyr/msXDy/SPn9e3ZzwQ130OecAdRt1JQTLxhI45bCsnm/ALBq8QJW/DWP60c9Q9M27enc6yQuG3Y/382YSkZ6WkXvXqkFsMucDFQplBYD7C9cUEQ8IvIUcB9wiqquKGnjQWshikgOzs7luLcM4Evg1pJmgg4HW1b/Q1pyEi0698hLq9mgMTUbNGbdknk069ClQPnjzhmY93dWVibL5nxLwvrVnDL49rz07etWcvz5gwJf+TJqEBtFVLiXdbtT8tL2pmayJyWDJnHRbNybWqD8wk378v72eKBt3arUiYngx9U789LrVotk/sa9ga98Ga1fsYzUpP207XKgMVI3vgl1GjZmxaJ5tO7UrUD5/hcdeD+zMjNZ8OPXbF67motuHAHAij/nUbthY+o2appXrm3X40lN2s/6Fcto2fHYAO/RoQngLPNyYEjuHREJA1oBmr+QiIQDbwNHA8er6trSbDzYXebOqroKQESqAx8Dk4CBfh91GNiXsBWA6nXqF0iPrV2PPW5ecTbqEl4dNoDs7Cy6njkA6dEfgL07tpKSuJcVc2fz/bQXyEhNpnmn4zjj+pFFniPYYqOcj1ViWmaB9MS0TKpH+/7INYyNYnD3xni9HhZt2ssqt7scGxVGlYgwWtWOoe9RtYgM87Budyrfr9zB/hBrJe7avgWAmvUaFEiPq1ufXds3+3zcmmWLeXTIBWRnZdH3vMvo3PukvO3VrFvw/Y1z7+/atjl0A6LH93BOGVuIs4DaIjIYeAe4F1gH/F2o3BNAe6DXwTSwgh0Q86jqPhH5L3ALgIjEAxNxZ5WAF4FnVTWnhLwfgbXAWcCnqnpdRe8LQEZaKh6vl7DwiALp4RGRZPrp6tRq0JibX/6ILauW8fnLj1GtZh1Ou/ZOtq91lll5w8O57IHnSN67m2/efJY3R17FrZM+ISKqaFc0WCLCPGTn5JCdUzA9KzuHcD/XoNyTmsEbczfQIDaK06QuSelZ/Lh6F3WrRjqPz8lh5pKtxESG0a9lba7o2ojJf2wgs/ATBVF6agoer5fwQu97REQkGWm+3/e68U0YNfUz1utS3nl2DNVr1eGSW0aSnppCRGRUgbLh4RF4PJ7Q7jITmKvuqWqKiJyNsw5xIvAXcIn73V8KjMVZhnOb+5CN+SZuNqlqkVmc/EImIIpIU+D/gJ/cpJk4O9sYaAJ8BewAppaQBwfWHvkdIxWR0cCoctyNPOGRUeRkZ5OVlUlY2IGXOTMjncjowkMgB8TUqElMjZrEt2pP0p6d/DD9RU65ejitu/Xh/v/+QdUatfLK1mvemqcu743+8SMd+54RiN04JJnZOXg9HjweyMkXq8K8HjKyfAevlIxsUjLS2bY/nZjIMPoeVYufVu/i310pPPPTv6RkZOeVTdi/heF9mtOqTgzLtycFcncOSkRUtPO+Z2YWmAXOyEgnqorPcX+qxdWkWlxNmrXpwL5dO/lk8vNcdONdRERFk5mRXqBsZmYGOTk5REX73l4oCNR0nzsR072Y9A757kYVzi+NYE+qLBSRPSKyF/gVZxzgXhFpBXQD7lLVFHcwdDxwpb+8fNv9n6rud9cf+aSqo1XVk//mbrvMatRrCEDizoKzook7txfbxV3z1x9sXrWsQFr9FkJGWiopiU6LP38wBKheux4xNWqyN2FLeVS53OxLdbrKsZEFf29jo8KLdKMBmsZFU79aZIG07fvTiQjzUiXC+YjmD4YA+9OzSM7IonpUyPymA1C7fjwAe3ZuL5C+J2EbNes2KFJ++cLfWbdiaYG0xq2E9LRU9u/bQ+368ezZUXRbULRbHkp8Lsr2N/scAoIdELuoapyq1lDVxqp6rRvE6gK7VDUxX9n1OC1Cf3m5fA/SVZCGR7UlKqYqaxfPzUvbvXUju7dupPnRRX7cmP3B63z31nMF0jYuX0zVuNrE1KjFrzOnMu6yXmRlZhzY3rZNJO3ZRf3mrQO3I4dgW2IaaZnZNK15oBtfIzqcuCoRrM830ZKrZ/Oa9GtZu0BaoxrR7E/LJDkjm+5NajC8T3Py97ZrRIdTNTKchKR0QkmT1u2IrloNXfh7XlrC5g3s2LKRNsceV6T8F9Mm8dGk8QXS/l36F9Vr1SE2rhatj+lGwqb17Nx2YPzxnwW/EV21Gk3btA/cjpRRAGeZAyrYAdGXjUAtEYnNl9Yc2F5CXq6gDyqFR0bR49xBfPnqOFbMnc2mlUv54LHbadHpOJq2P5bMjHQSdyXkdYd6XnQ1K+b+xM8fTmbnpnXM/3IGP3/4OidffRsejwfp0Y+05CQ+Gn8/CetXs+7vBbw7ZijNOnajVdfeQd7bgrJyYP7GvZzSug5H1Y6hQWwUFx7dgHW7U9i0Lw2vB6pGhuUFuLnr99CqTgzHN42jZpUIjomP5YRmccz+dxcAq3YkERnm5Zz29agdE0HjGtFc3KkB63ensGZX0QAbTBGRUZx08ZW8P2Esi3/7kbXLlzDpwWFIl+NpdXQXMjPS2bNje977ftrlQ1j86yy+fPtVtm1Yy0+fvM+X01/hguvvwOPx0OrorrTseCyT7r+VtcuXsPjXWXz4whOcPvA6wiMiS6hN8ByuATG0+hsuVd0gInOAZ9xFlU2AO4Hx/vKCV+PinXLtHWRlZTJj3AiyMjNo070v597mDFmuX7qIN0ZcwZDxb3PUMT1o3a0PAx9+gR+mv8h3U56nRt2GnDP0YbqdOQCA2vHNGPzkW3zzxjNMGnox3rAI2vU8mbNuui+Yu+jTj6t3EuaBCzrUx+uF1TucI1UAmsRV4cqujZi+YBPrdqfw764U/rN4K32PqsWJLWuxLzWTrzWBPzc7nYDdKZm8u3AT/VvV5trjmpCVncOKHUl8t2JHMHfRp4tvct7v1x6+nazMTOdIlZHOYvqVixfw5M2Xc8+k92nX9QQ6Ht+XW8dN4pPXJ/DRq89Qq148g0aM4cTzLwecruewp15j2pMPMvaGAUTHVOPE8y/n/OsKH4gRWjx+rrpXlmOZA82TkxOcxpS7DrF17rKbYvLjgZeAvkAq8DIwNt8ss6+8H4G3VXXyIdarKzD/nHufp3bT0OqKVoQQmrCtcE1qhG6LK1C2rFnBWw/dCod21EgRud+f5ye/S2spvku/Updx+3X/V27PWZ6C1kJ0JzD85W8GLjyEvH5lrpwxpkwCtewm0EKyy2yMObwdrl1mC4jGmHJnF6o3xhiXBURjjMnj78zYoRsRLSAaY8qdtRCNMcbl9YCv83j4Ob9H0FlANMYESAhHPh8sIBpjyp2tQzTGGJfHT5fZxhCNMZWKLcw2xhiXzTIbY4zLAqIxxuSxhdnGGANYC9EYY/IEctmNiPTAuepeG+BP4BpVXemjbDQwG7hfVb8raduhegkBY8xhzFPCv0PlBriZwLNAHPA1zmVHiyvb2M0vehEjHywgGmPKXQCvqdIf2Keq01U1A3gcaCYinfIXEpFawALgM5yL0JWKBURjTPnzFwzLFhDbAstz76hqFrDaTc9vP9BOVcdzEBedszFEY0y5K+XC7PkiUjh7jKqO9rPpqkByobRkICZ/gqqmA7tKWd08FhCNMeWulGe7OZSLTCUDVQqlxeC0CMvMuszGmPLnKeF26JbjzC4DICJhQCtAy7RVl7UQjTHlzgN4fcyelHHZzSygtogMBt4B7gXWAX+XbbMOayEaY8pdoBqIqpoCnA3cAuwETgMuca/JvlREBpWl3tZCNMaUP3+Rr4xNRHfcscjaQlXt4KN889Ju2wJiUdEAe7ZuCHY9giK71AsUjjzh1SKCXYUKt2NT3hK96PLc7to1//rsMq9d8295PlW5soBYVHOAOVOeCXI1jKlQzYFfymE7O4DkB+4dGVNCuWS3bEixgFjUN8AVwFogNQjPPx/oFoTnDQW27xUvGicYflMeG1PVdSLSHqhTQtEdqrquPJ6zPHlycipxHykEiUiOqobw+UACx/a9cu57KLFZZmOMcVlANMYYlwVEY4xxWUAMPWOCXYEgsn03QWWTKsYY47IWojHGuCwgGmOMywKiMca4LCAaY4zLAqIxxrgsIJrDgojYYW0m4CwghjARqfTvj4icCOCeANSCogmoSv+FC1UiEqaq2SLiEZGWwa5PMIjIacCrInIDVJ6g6F4nxASBBcQQJCLhqprlfvl/B66tDIGgGL8Dk4D/E5Gr4MgPiiLidd97r4iME5Gmwa5TZWIBMcSISD1VzXS/9COA3ar6wJEeCPLLbSGp6j6gHRAFPCwiQ9z0I/a1UNVs9883gTqqut5feVO+LCCGEBG5Gxjo3u0IXAR0EpFe4ASCYNWtIuVrIf2Ocw3e54FPgStF5Ga3zBEbFEXkReBY4JNg16WysYAYWn5S1QkicguwBbgBWIjTZTw6uFWrcAJEAjeq6gfA48AbwBUiMhCOnB+IYibPZgPZwEWVdfw4WCwghoB8XcS5ItIduBh4DNjo/h8PXCciHYNXy8AqJijsxAkKZ7rjajtxrskbBowXkasruo6B4I4X506etRORpqr6ITAUaAsMEpFmQa5mpWEBMchyJ1Dcvz2qOg/nVFA1gSeB5e7/8cAd7vUqjiiFgkJjEWmjqtuBJcAFQBv3tdkILAKmAD8Gr8blw92nTPfH4EecCaSvRWQmkADcBZyJ0yo+Kng1rTzs9F9B5C6tyXK/EB/hXNRqPXAf0AsYBuwGRgJH43ShR6jqtiBVudy5QSHHfQ2+xukmNwZ+A54DJuAMH+wEtuO8Jl1UdU2QqlzuROQjYJ+qXiMiJwGnAOcDZwGtcV6Dt4DnVTUzeDU98lkLMUjyL68A5rrJy4CTgRdxLgk5EYjFaTksBq4/koIhFBgH/BjYAAwA+uEEgnuAq4EfgAigIXDiERYMI3GufPcygKr+ALyA8/7fqqrfAQ8A/7FgGHh2GdIgEJEGqrrVvTsM2KyqF4hIXaADzuUoX8VpEUYAVwIxqro3KBUOMBGphRP4R7hdZYAeIrIcJyjc6ZYLP9yDgvtDmJ0vqTrO0qKWwFy317BFRDbjXpZUVT8OQlUrJQuIFUxEfsTpBl/oJnlwxosAxgMpwFM4S00+Ay4FflPVlIqtaeAUExRScdYa9haRlYDHzX8JZ+ggV1YFVrPc5QZ0d7lQJyANWAGMBV4UkdWqmttbSAF2uS3IjCNlRj3UWUCsQCLyMRCmqhfmS/4TZyD9eqCHqrYVkRjgZpwhjfqq+m8QqhsQhYJCK6CKqi521xyeh7PMaCnODHM8kJ4bQA/noFBoAuVnIB1nrHQlBqYNiQAACNBJREFUMA1nIm2OiEzF+ZG8ADhZVdODVefKyCZVKog7cN5QVU/IlxYOZLmTCrcCzVR1pIjcBFwOnH8kdZMLTaB8j/OD3AD4F2fi6AkgE8gAVuMMGfRR1SVBqnK5KLTf7wFpqnqViBwP9AGGANcB1YC+OC3HD1R1edAqXUlZC7ECuMHwaFVtnS+tBfA0zsTBapxWwQARaY0zqXDSkRQMocDRJR8Ca3HGT8NxgsQzOEGhG9ADp8t4gqr+E5zalg83CEbh7E8OEAO8AqCqv4vIapwfhUGqejPwVbDqamyWuaJMBmqLSAeA/2/v/GO2rMo4/hFzZmKtIGpkQaR+TSFILdQWgxwKzELRZjMEVzNMi3KpG5rFj4bg8AeIcygppDkdaAgGqWS5fhEjSZDhVykwJWGCw4laGNgf13ns5uV9fV+Nl4f38fr8w/3c55z7Ps94933Oua5zzldST2La9KTtvwHYnkmI4zxi6ryyTn3d60jqLqln2af9BrHG8mbb221vsz2UEIpJtufbvgwY3wBieCORPV8g6fzy3Y8kEme1ZVcvENn1I1t+UrKvyCnzPkLSMCJW9E1ifd1ttn9Syg6y/Xo9+9deSLoF+AQRL5tHLCGaCywp1/8pI8fRRIhgRN06uxcp8eIPE0toTgS+RCypGk4kz75m+1el7iXAycBI2/+uT48TyCnzPsP2YknnAb8EZlfEsFMDi+ECYjp4LtDd9u8rS2z6Ap8nkkqvAL2AnY3w41BCJN1sn1w+PwkMIhJksyV9EHhA0nwiuTIEODXFsP7kCHEfI2kwcBcw1PaKevenvZB0FTDY9oDKvV7AUqAnkUX+KfBJwMBIYtH1qn3f271HC/Hio4BfECPAleXeKcSo8FXgAduuR3+T3ckY4j7G9sPE7ovFkk5qrX4Hphdl9wWApMOJU1zmEFvTIJIN9wMbgRM7uhgWmosXPwL8vCKGB9j+te1Jtq9NMdx/SEGsA7YXE0tK7pL03nr3Z28j6RCgH3Bo5fZngWm2J5btacuBHbZvAqY2iiiU/9uRwKOShhNieIvtyfC/JTj17GPSMimIdaJsx+pj+1/17ks7sAvYRpxpCIDtRbZvgDcF8wXgofp0r30pongeMU1eansivBkvTjHcj0lBrCO2t9e7D+1BSQ5cB1xUO8y1CRcQy0z+XOo3nEjYXgKcBpwpqbYneddbt0rqTSZVknah7MG9kjjo9GoiVvh+YvnJOGJbWsOstWyJstxqDrGk6E917k7SCimISbtR9mSfS5z6vQ3YQpxtOLGjb8d7O0g6g1h7+ukGDZE0DCmISbtT1t0dBrxE7ON914mCpM6NGiJpJFIQkyRJCplUSZIkKaQgJkmSFFIQkyRJCimISZIkhRTEJEmSQgpikiRJIQUxSZKkkAfENhiSNgA9Krd2ApuAe4Af783FwZK2EF7KcyTNATrbPrsN7c4EVth+9h2+dxpwgu2BzZSdT5yq07UNz+kJrCcO2XjiHfTj/2qf7H+kIDYmVwC3l+sDCSP0O4EPEEZO7cH3CKOst0RSD+A+oA/hJZIk+w0piI3Jy7Y3VT5vlDSdMLFqF0F8Gw6BrYpmktSLFMR3D68Tfr9IGk9YfXYiDJDG2p4r6VLCGrQL4XVyqe1lpc2BwGTgG6XdxOrDm06ZJZ0NXAUcRdisjrO9iJhiAqyWNMH2eEn9iePCjidGjbcS095d5VlDCMvWI4CHgc1t/dKSTiNM4PsSNqDLgAttP1WpNkzSQsL/5SHg27afL+0/CswAhgLbCU+cHzSaRWwSZFKlwZHUSdLnCKFbUCkaAvyWEMTFksaUOhcSp1svBh4pcTIIcRtNHHw6iHCP69LCOwcRMcufEVPj24D5ko4mjKUgvKenSeoGPEgITW9gLHAxYVxf8yNZCMwnRO0PhHNhW757z9J2HnAMcfTYhwhxrfJdYAzwBUIU51bK7iv/ngR8GfgUcHdb3p90PHKE2JhcK2lKuT6YGBktJKbMNV4FptQOZ5V0BTGKW1LKJ0saCFws6XJCKCdVrDNHEWbzzXERcL/ta8vn6yQdCnQmTsoG2Gp7u6TLgOW1I/aBdZLGAdOBKYT4PW57QimfWgS3LdYLBxGj3BvL5/WS7ij9q3J58bpB0gXAKklHAB8HPgMMtL2jlH+dCEEcS7gFJg1ECmJjcjWRRIGwudzcjMXlhooYdia8k2cXH+UaBxPT7K7AR4DHagW2n5P0zxbef0zl/bX6k8q7ejapeywwSFI1+90JOERSF2LU+FiTNsuBAbSC7aclvVIEvTdhadCPOJOxyrLK9RPEd+5NeEm/D3hRUpMmHA38pbU+JB2LFMTGZIvtda3Uea1yXfs7GA00PcX6NWKECXsmRHa08OwdzdRtifcA9wI/bKbspfLutr53NyT1IabYS4FHCdvT/uw5QtxZuT6AEOQdpW/PAIObefxmWggZJB2XFMQE29skbQI+Znte7b6kmcTo7HZiVNUf+GMp60pMKZvjKeC46g1JDxJxyQVN6q4FhlQFvLjVfRUYBawmEhpVjqNtfAv4q+0RlWefw54C2xf4R7k+nphqryWEsjuRtd9c2vcAZgLfZ3chTRqAFMSkxjXAjyQ9D6wgrDTHEObxb0i6HrhS0t+BdcBUWv77uQH4naTvAEuIZMQAQkRqU+N+kp4FbgLGSppRrnsAs4B7be+SNKuUX0N4Hp8KnE4R5lbYCJwj6YvlejhhcLW1Sb3pkl4m4qq3AvfYXi/pGWANcHfJwO8sfTyMiJ+29IOQdFAyy5zUmA5MI4RxDTACOMt2TXimEUI3ixCjlcDTzT2omCmNIgym1hBT8TNsr7W9lRC22cAE288R7nQnAI8Thkx3ApeUZ20o5acAq4CzgJvb+J1mENPlRUS873QiOdRN0uGVelOBOwgP5dXED0HNJe8rwIvAb4is/CZgmO0cHTYgaSGQJElSyBFikiRJIQUxSZKkkIKYJElSSEFMkiQppCAmSZIUUhCTJEkKKYhJkiSFFMQkSZLCfwGKPwFx96P8hwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# confusion matrix - normalized\n",
    "cm = confusion_matrix(y, y_preds)\n",
    "classes = ['Fair', 'Good', 'Poor']\n",
    "\n",
    "plot_confusion_matrix(cm, classes,\n",
    "                      normalize=True,\n",
    "                      title='Volunteer data confusion matrix',\n",
    "                      cmap=plt.cm.Blues)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a reference dataframe\n",
    "\n",
    "- Using my model's predictions, I can create a dataframe that professional staff could refer to in order to double-check the health status determined by a volunteer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    Good\n",
       "3    Good\n",
       "4    Good\n",
       "5    Good\n",
       "6    Good\n",
       "Name: health, dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# look at a sample of volunteer entries\n",
    "y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    Fair\n",
       "3    Good\n",
       "4    Good\n",
       "5    Good\n",
       "6    Good\n",
       "dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert predictions to pandas series\n",
    "y_preds_series = pd.Series(y_preds, index=y.index)\n",
    "\n",
    "# look at a sample of model predictions\n",
    "y_preds_series.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "78645"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# build list of indices for trees with differing volunteer entries and model predictions\n",
    "mismatch_indices = [i for i,v in y.iteritems() if y[i] != y_preds_series[i]]\n",
    "\n",
    "# number of trees\n",
    "len(mismatch_indices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load original dataframe\n",
    "trees_initial = pd.read_csv('data/2015StreetTreesCensus_TREES.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>created_at</th>\n",
       "      <th>tree_id</th>\n",
       "      <th>block_id</th>\n",
       "      <th>the_geom</th>\n",
       "      <th>tree_dbh</th>\n",
       "      <th>stump_diam</th>\n",
       "      <th>curb_loc</th>\n",
       "      <th>status</th>\n",
       "      <th>health</th>\n",
       "      <th>spc_latin</th>\n",
       "      <th>...</th>\n",
       "      <th>st_assem</th>\n",
       "      <th>st_senate</th>\n",
       "      <th>nta</th>\n",
       "      <th>nta_name</th>\n",
       "      <th>boro_ct</th>\n",
       "      <th>state</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>x_sp</th>\n",
       "      <th>y_sp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>09/05/2015</td>\n",
       "      <td>204026</td>\n",
       "      <td>218365</td>\n",
       "      <td>POINT (-73.93660770459083 40.717580740099116)</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>OnCurb</td>\n",
       "      <td>Alive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Gleditsia triacanthos var. inermis</td>\n",
       "      <td>...</td>\n",
       "      <td>50</td>\n",
       "      <td>18</td>\n",
       "      <td>BK90</td>\n",
       "      <td>East Williamsburg</td>\n",
       "      <td>3044900</td>\n",
       "      <td>New York</td>\n",
       "      <td>40.717581</td>\n",
       "      <td>-73.936608</td>\n",
       "      <td>1.001823e+06</td>\n",
       "      <td>200716.891267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>09/07/2015</td>\n",
       "      <td>208649</td>\n",
       "      <td>103940</td>\n",
       "      <td>POINT (-73.98729652382876 40.7627238542921)</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>OnCurb</td>\n",
       "      <td>Alive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Tilia americana</td>\n",
       "      <td>...</td>\n",
       "      <td>75</td>\n",
       "      <td>27</td>\n",
       "      <td>MN15</td>\n",
       "      <td>Clinton</td>\n",
       "      <td>1012700</td>\n",
       "      <td>New York</td>\n",
       "      <td>40.762724</td>\n",
       "      <td>-73.987297</td>\n",
       "      <td>9.877691e+05</td>\n",
       "      <td>217157.856088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>09/05/2015</td>\n",
       "      <td>203719</td>\n",
       "      <td>302371</td>\n",
       "      <td>POINT (-73.91117076849402 40.78242822973097)</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>OnCurb</td>\n",
       "      <td>Alive</td>\n",
       "      <td>Good</td>\n",
       "      <td>Platanus x acerifolia</td>\n",
       "      <td>...</td>\n",
       "      <td>36</td>\n",
       "      <td>13</td>\n",
       "      <td>QN72</td>\n",
       "      <td>Steinway</td>\n",
       "      <td>4010500</td>\n",
       "      <td>New York</td>\n",
       "      <td>40.782428</td>\n",
       "      <td>-73.911171</td>\n",
       "      <td>1.008850e+06</td>\n",
       "      <td>224349.036588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>09/05/2015</td>\n",
       "      <td>203726</td>\n",
       "      <td>302371</td>\n",
       "      <td>POINT (-73.91201956608866 40.78173511421239)</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>OnCurb</td>\n",
       "      <td>Alive</td>\n",
       "      <td>Poor</td>\n",
       "      <td>Platanus x acerifolia</td>\n",
       "      <td>...</td>\n",
       "      <td>36</td>\n",
       "      <td>13</td>\n",
       "      <td>QN72</td>\n",
       "      <td>Steinway</td>\n",
       "      <td>4010500</td>\n",
       "      <td>New York</td>\n",
       "      <td>40.781735</td>\n",
       "      <td>-73.912020</td>\n",
       "      <td>1.008615e+06</td>\n",
       "      <td>224096.273970</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>08/31/2015</td>\n",
       "      <td>192998</td>\n",
       "      <td>211160</td>\n",
       "      <td>POINT (-73.92113023015189 40.664317398984245)</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>OnCurb</td>\n",
       "      <td>Alive</td>\n",
       "      <td>Fair</td>\n",
       "      <td>Platanus x acerifolia</td>\n",
       "      <td>...</td>\n",
       "      <td>55</td>\n",
       "      <td>20</td>\n",
       "      <td>BK81</td>\n",
       "      <td>Brownsville</td>\n",
       "      <td>3090000</td>\n",
       "      <td>New York</td>\n",
       "      <td>40.664317</td>\n",
       "      <td>-73.921130</td>\n",
       "      <td>1.006131e+06</td>\n",
       "      <td>181314.985517</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 42 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    created_at  tree_id  block_id  \\\n",
       "2   09/05/2015   204026    218365   \n",
       "7   09/07/2015   208649    103940   \n",
       "10  09/05/2015   203719    302371   \n",
       "11  09/05/2015   203726    302371   \n",
       "14  08/31/2015   192998    211160   \n",
       "\n",
       "                                         the_geom  tree_dbh  stump_diam  \\\n",
       "2   POINT (-73.93660770459083 40.717580740099116)         3           0   \n",
       "7     POINT (-73.98729652382876 40.7627238542921)         9           0   \n",
       "10   POINT (-73.91117076849402 40.78242822973097)        11           0   \n",
       "11   POINT (-73.91201956608866 40.78173511421239)         8           0   \n",
       "14  POINT (-73.92113023015189 40.664317398984245)        30           0   \n",
       "\n",
       "   curb_loc status health                           spc_latin  ... st_assem  \\\n",
       "2    OnCurb  Alive   Good  Gleditsia triacanthos var. inermis  ...       50   \n",
       "7    OnCurb  Alive   Good                     Tilia americana  ...       75   \n",
       "10   OnCurb  Alive   Good               Platanus x acerifolia  ...       36   \n",
       "11   OnCurb  Alive   Poor               Platanus x acerifolia  ...       36   \n",
       "14   OnCurb  Alive   Fair               Platanus x acerifolia  ...       55   \n",
       "\n",
       "   st_senate   nta           nta_name  boro_ct     state   Latitude  \\\n",
       "2         18  BK90  East Williamsburg  3044900  New York  40.717581   \n",
       "7         27  MN15            Clinton  1012700  New York  40.762724   \n",
       "10        13  QN72           Steinway  4010500  New York  40.782428   \n",
       "11        13  QN72           Steinway  4010500  New York  40.781735   \n",
       "14        20  BK81        Brownsville  3090000  New York  40.664317   \n",
       "\n",
       "    longitude          x_sp           y_sp  \n",
       "2  -73.936608  1.001823e+06  200716.891267  \n",
       "7  -73.987297  9.877691e+05  217157.856088  \n",
       "10 -73.911171  1.008850e+06  224349.036588  \n",
       "11 -73.912020  1.008615e+06  224096.273970  \n",
       "14 -73.921130  1.006131e+06  181314.985517  \n",
       "\n",
       "[5 rows x 42 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create dataframe of trees with differing differing volunteer entries and model predictions\n",
    "trees_to_check = trees_initial[trees_initial.index.isin(mismatch_indices)]\n",
    "\n",
    "# look at the first five rows\n",
    "trees_to_check.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(78645, 42)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# confirm number of trees\n",
    "trees_to_check.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save/Load dataframe of trees to check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # uncomment to save\n",
    "# with gzip.open('data/nyc_trees_to_check.pkl', 'wb') as goodbye:\n",
    "#     pickle.dump(trees_to_check, goodbye, protocol=pickle.HIGHEST_PROTOCOL)\n",
    "\n",
    "# # uncomment to load\n",
    "# with gzip.open('data/nyc_trees_to_check.pkl', 'rb') as hello:\n",
    "#     trees_to_check = pickle.load(hello)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:learn-env] *",
   "language": "python",
   "name": "conda-env-learn-env-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}