asl_recognizer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Artificial Intelligence Engineer Nanodegree - Probabilistic Models\n",
    "## Project: Sign Language Recognition System\n",
    "- [Introduction](#intro)\n",
    "- [Part 1 Feature Selection](#part1_tutorial)\n",
    "    - [Tutorial](#part1_tutorial)\n",
    "    - [Features Submission](#part1_submission)\n",
    "    - [Features Unittest](#part1_test)\n",
    "- [Part 2 Train the models](#part2_tutorial)\n",
    "    - [Tutorial](#part2_tutorial)\n",
    "    - [Model Selection Score Submission](#part2_submission)\n",
    "    - [Model Score Unittest](#part2_test)\n",
    "- [Part 3 Build a Recognizer](#part3_tutorial)\n",
    "    - [Tutorial](#part3_tutorial)\n",
    "    - [Recognizer Submission](#part3_submission)\n",
    "    - [Recognizer Unittest](#part3_test)\n",
    "- [Part 4 (OPTIONAL) Improve the WER with Language Models](#part4_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='intro'></a>\n",
    "## Introduction\n",
    "The overall goal of this project is to build a word recognizer for American Sign Language video sequences, demonstrating the power of probabalistic models.  In particular, this project employs  [hidden Markov models (HMM's)](https://en.wikipedia.org/wiki/Hidden_Markov_model) to analyze a series of measurements taken from videos of American Sign Language (ASL) collected for research (see the [RWTH-BOSTON-104 Database](http://www-i6.informatik.rwth-aachen.de/~dreuw/database-rwth-boston-104.php)).  In this video, the right-hand x and y locations are plotted as the speaker signs the sentence.\n",
    "[![ASLR demo](http://www-i6.informatik.rwth-aachen.de/~dreuw/images/demosample.png)](https://drive.google.com/open?id=0B_5qGuFe-wbhUXRuVnNZVnMtam8)\n",
    "\n",
    "The raw data, train, and test sets are pre-defined.  You will derive a variety of feature sets (explored in Part 1), as well as implement three different model selection criterion to determine the optimal number of hidden states for each word model (explored in Part 2). Finally, in Part 3 you will implement the recognizer and compare the effects the different combinations of feature sets and model selection criteria.  \n",
    "\n",
    "At the end of each Part, complete the submission cells with implementations, answer all questions, and pass the unit tests.  Then submit the completed notebook for review!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part1_tutorial'></a>\n",
    "## PART 1: Data\n",
    "\n",
    "### Features Tutorial\n",
    "##### Load the initial database\n",
    "A data handler designed for this database is provided in the student codebase as the `AslDb` class in the `asl_data` module.  This handler creates the initial [pandas](http://pandas.pydata.org/pandas-docs/stable/) dataframe from the corpus of data included in the `data` directory as well as dictionaries suitable for extracting data in a format friendly to the [hmmlearn](https://hmmlearn.readthedocs.io/en/latest/) library.  We'll use those to create models in Part 2.\n",
    "\n",
    "To start, let's set up the initial database and select an example set of features for the training set.  At the end of Part 1, you will create additional feature sets for experimentation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker\n",
       "video frame                                                           \n",
       "98    0         149     181      170      175     161      62  woman-1\n",
       "      1         149     181      170      175     161      62  woman-1\n",
       "      2         149     181      170      175     161      62  woman-1\n",
       "      3         149     181      170      175     161      62  woman-1\n",
       "      4         149     181      170      175     161      62  woman-1"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from asl_data import AslDb\n",
    "\n",
    "\n",
    "asl = AslDb() # initializes the database\n",
    "asl.df.head() # displays the first five rows of the asl database, indexed by video and frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "left-x         149\n",
       "left-y         181\n",
       "right-x        170\n",
       "right-y        175\n",
       "nose-x         161\n",
       "nose-y          62\n",
       "speaker    woman-1\n",
       "Name: (98, 1), dtype: object"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "asl.df.ix[98,1]  # look at the data available for an individual frame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The frame represented by video 98, frame 1 is shown here:\n",
    "![Video 98](http://www-i6.informatik.rwth-aachen.de/~dreuw/database/rwth-boston-104/overview/images/orig/098-start.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Feature selection for training the model\n",
    "The objective of feature selection when training a model is to choose the most relevant variables while keeping the model as simple as possible, thus reducing training time.  We can use the raw features already provided or derive our own and add columns to the pandas dataframe `asl.df` for selection. As an example, in the next cell a feature named `'grnd-ry'` is added. This feature is the difference between the right-hand y value and the nose y value, which serves as the \"ground\" right y value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  \n",
       "video frame           \n",
       "98    0          113  \n",
       "      1          113  \n",
       "      2          113  \n",
       "      3          113  \n",
       "      4          113  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "asl.df['grnd-ry'] = asl.df['right-y'] - asl.df['nose-y']\n",
    "asl.df.head()  # the new feature 'grnd-ry' is now in the frames dictionary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "asl.df sample\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>grnd-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx  grnd-ly  \n",
       "video frame                                      \n",
       "98    0          113        9      -12      119  \n",
       "      1          113        9      -12      119  \n",
       "      2          113        9      -12      119  \n",
       "      3          113        9      -12      119  \n",
       "      4          113        9      -12      119  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<font color=green>Correct!</font><br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from asl_utils import test_features_tryit\n",
    "# TODO add df columns for 'grnd-rx', 'grnd-ly', 'grnd-lx' representing differences between hand and nose locations\n",
    "asl.df['grnd-rx'] = asl.df['right-x'] - asl.df['nose-x']\n",
    "asl.df['grnd-lx'] = asl.df['left-x'] - asl.df['nose-x']\n",
    "asl.df['grnd-ly'] = asl.df['left-y'] - asl.df['nose-y']\n",
    "# test the code\n",
    "test_features_tryit(asl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 113, -12, 119]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# collect the features into a list\n",
    "features_ground = ['grnd-rx','grnd-ry','grnd-lx','grnd-ly']\n",
    "# show a single set of features for a given (video, frame) tuple\n",
    "[asl.df.ix[98,1][v] for v in features_ground]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Build the training set\n",
    "Now that we have a feature list defined, we can pass that list to the `build_training` method to collect the features for all the words in the training set.  Each word in the training set has multiple examples from various videos.  Below we can see the unique words that have been loaded into the training set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training words: ['JOHN', 'WRITE', 'HOMEWORK', 'IX-1P', 'SEE', 'YESTERDAY', 'IX', 'LOVE', 'MARY', 'CAN', 'GO', 'GO1', 'FUTURE', 'GO2', 'PARTY', 'FUTURE1', 'HIT', 'BLAME', 'FRED', 'FISH', 'WONT', 'EAT', 'BUT', 'CHICKEN', 'VEGETABLE', 'CHINA', 'PEOPLE', 'PREFER', 'BROCCOLI', 'LIKE', 'LEAVE', 'SAY', 'BUY', 'HOUSE', 'KNOW', 'CORN', 'CORN1', 'THINK', 'NOT', 'PAST', 'LIVE', 'CHICAGO', 'CAR', 'SHOULD', 'DECIDE', 'VISIT', 'MOVIE', 'WANT', 'SELL', 'TOMORROW', 'NEXT-WEEK', 'NEW-YORK', 'LAST-WEEK', 'WILL', 'FINISH', 'ANN', 'READ', 'BOOK', 'CHOCOLATE', 'FIND', 'SOMETHING-ONE', 'POSS', 'BROTHER', 'ARRIVE', 'HERE', 'GIVE', 'MAN', 'NEW', 'COAT', 'WOMAN', 'GIVE1', 'HAVE', 'FRANK', 'BREAK-DOWN', 'SEARCH-FOR', 'WHO', 'WHAT', 'LEG', 'FRIEND', 'CANDY', 'BLUE', 'SUE', 'BUY1', 'STOLEN', 'OLD', 'STUDENT', 'VIDEOTAPE', 'BORROW', 'MOTHER', 'POTATO', 'TELL', 'BILL', 'THROW', 'APPLE', 'NAME', 'SHOOT', 'SAY-1P', 'SELF', 'GROUP', 'JANA', 'TOY1', 'MANY', 'TOY', 'ALL', 'BOY', 'TEACHER', 'GIRL', 'BOX', 'GIVE2', 'GIVE3', 'GET', 'PUTASIDE']\n"
     ]
    }
   ],
   "source": [
    "training = asl.build_training(features_ground)\n",
    "print(\"Training words: {}\".format(training.words))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The training data in `training` is an object of class `WordsData` defined in the `asl_data` module.  in addition to the `words` list, data can be accessed with the `get_all_sequences`, `get_all_Xlengths`, `get_word_sequences`, and `get_word_Xlengths` methods. We need the `get_word_Xlengths` method to train multiple sequences with the `hmmlearn` library.  In the following example, notice that there are two lists; the first is a concatenation of all the sequences(the X portion) and the second is a list of the sequence lengths(the Lengths portion)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[-11,  48,   7, 120],\n",
       "        [-11,  48,   8, 109],\n",
       "        [ -8,  49,  11,  98],\n",
       "        [ -7,  50,   7,  87],\n",
       "        [ -4,  54,   7,  77],\n",
       "        [ -4,  54,   6,  69],\n",
       "        [ -4,  54,   6,  69],\n",
       "        [-13,  52,   6,  69],\n",
       "        [-13,  52,   6,  69],\n",
       "        [ -8,  51,   6,  69],\n",
       "        [ -8,  51,   6,  69],\n",
       "        [ -8,  51,   6,  69],\n",
       "        [ -8,  51,   6,  69],\n",
       "        [ -8,  51,   6,  69],\n",
       "        [-10,  59,   7,  71],\n",
       "        [-15,  64,   9,  77],\n",
       "        [-17,  75,  13,  81],\n",
       "        [ -4,  48,  -4, 113],\n",
       "        [ -2,  53,  -4, 113],\n",
       "        [ -4,  55,   2,  98],\n",
       "        [ -4,  58,   2,  98],\n",
       "        [ -1,  59,   2,  89],\n",
       "        [ -1,  59,  -1,  84],\n",
       "        [ -1,  59,  -1,  84],\n",
       "        [ -7,  63,  -1,  84],\n",
       "        [ -7,  63,  -1,  84],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -7,  63,   3,  83],\n",
       "        [ -4,  70,   3,  83],\n",
       "        [ -4,  70,   3,  83],\n",
       "        [ -2,  73,   5,  90],\n",
       "        [ -3,  79,  -4,  96],\n",
       "        [-15,  98,  13, 135],\n",
       "        [ -6,  93,  12, 128],\n",
       "        [ -2,  89,  14, 118],\n",
       "        [  5,  90,  10, 108],\n",
       "        [  4,  86,   7, 105],\n",
       "        [  4,  86,   7, 105],\n",
       "        [  4,  86,  13, 100],\n",
       "        [ -3,  82,  14,  96],\n",
       "        [ -3,  82,  14,  96],\n",
       "        [  6,  89,  16, 100],\n",
       "        [  6,  89,  16, 100],\n",
       "        [  7,  85,  17, 111]]), [17, 20, 12])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training.get_word_Xlengths('CHOCOLATE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "###### More feature sets\n",
    "So far we have a simple feature set that is enough to get started modeling.  However, we might get better results if we manipulate the raw values a bit more, so we will go ahead and set up some other options now for experimentation later.  For example, we could normalize each speaker's range of motion with grouped statistics using [Pandas stats](http://pandas.pydata.org/pandas-docs/stable/api.html#api-dataframe-stats) functions and [pandas groupby](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html).  Below is an example for finding the means of all speaker subgroups."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>grnd-ly</th>\n",
       "      <th>...</th>\n",
       "      <th>delta-lx</th>\n",
       "      <th>delta-ly</th>\n",
       "      <th>norm-grnd-rx</th>\n",
       "      <th>norm-grnd-ry</th>\n",
       "      <th>norm-grnd-lx</th>\n",
       "      <th>norm-grnd-ly</th>\n",
       "      <th>delta-norm-grnd-rx</th>\n",
       "      <th>delta-norm-grnd-ry</th>\n",
       "      <th>delta-norm-grnd-lx</th>\n",
       "      <th>delta-norm-grnd-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>speaker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>man-1</th>\n",
       "      <td>206.248203</td>\n",
       "      <td>218.679449</td>\n",
       "      <td>155.464350</td>\n",
       "      <td>150.371031</td>\n",
       "      <td>175.031756</td>\n",
       "      <td>61.642600</td>\n",
       "      <td>88.728430</td>\n",
       "      <td>-19.567406</td>\n",
       "      <td>31.216447</td>\n",
       "      <td>157.036848</td>\n",
       "      <td>...</td>\n",
       "      <td>0.207010</td>\n",
       "      <td>0.272768</td>\n",
       "      <td>1.383538e-16</td>\n",
       "      <td>6.130673e-17</td>\n",
       "      <td>6.855577e-16</td>\n",
       "      <td>2.108858e-15</td>\n",
       "      <td>-0.002346</td>\n",
       "      <td>-0.000487</td>\n",
       "      <td>0.001420</td>\n",
       "      <td>-0.001815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>woman-1</th>\n",
       "      <td>164.661438</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>162.655120</td>\n",
       "      <td>57.245098</td>\n",
       "      <td>60.087364</td>\n",
       "      <td>-11.637255</td>\n",
       "      <td>2.006318</td>\n",
       "      <td>104.026144</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.286492</td>\n",
       "      <td>-0.237255</td>\n",
       "      <td>-3.192798e-18</td>\n",
       "      <td>1.558666e-16</td>\n",
       "      <td>-1.600753e-16</td>\n",
       "      <td>6.753252e-17</td>\n",
       "      <td>0.004865</td>\n",
       "      <td>-0.000157</td>\n",
       "      <td>-0.003470</td>\n",
       "      <td>0.002501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>woman-2</th>\n",
       "      <td>183.214509</td>\n",
       "      <td>176.527232</td>\n",
       "      <td>156.866295</td>\n",
       "      <td>119.835714</td>\n",
       "      <td>170.318973</td>\n",
       "      <td>58.022098</td>\n",
       "      <td>61.813616</td>\n",
       "      <td>-13.452679</td>\n",
       "      <td>12.895536</td>\n",
       "      <td>118.505134</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.015179</td>\n",
       "      <td>-0.163393</td>\n",
       "      <td>-2.149570e-16</td>\n",
       "      <td>-1.394718e-16</td>\n",
       "      <td>-5.530972e-16</td>\n",
       "      <td>1.729086e-16</td>\n",
       "      <td>-0.001435</td>\n",
       "      <td>0.000886</td>\n",
       "      <td>0.001413</td>\n",
       "      <td>0.000159</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x      left-y     right-x     right-y      nose-x  \\\n",
       "speaker                                                               \n",
       "man-1    206.248203  218.679449  155.464350  150.371031  175.031756   \n",
       "woman-1  164.661438  161.271242  151.017865  117.332462  162.655120   \n",
       "woman-2  183.214509  176.527232  156.866295  119.835714  170.318973   \n",
       "\n",
       "            nose-y    grnd-ry    grnd-rx    grnd-lx     grnd-ly  \\\n",
       "speaker                                                           \n",
       "man-1    61.642600  88.728430 -19.567406  31.216447  157.036848   \n",
       "woman-1  57.245098  60.087364 -11.637255   2.006318  104.026144   \n",
       "woman-2  58.022098  61.813616 -13.452679  12.895536  118.505134   \n",
       "\n",
       "                ...          delta-lx  delta-ly  norm-grnd-rx  norm-grnd-ry  \\\n",
       "speaker         ...                                                           \n",
       "man-1           ...          0.207010  0.272768  1.383538e-16  6.130673e-17   \n",
       "woman-1         ...         -0.286492 -0.237255 -3.192798e-18  1.558666e-16   \n",
       "woman-2         ...         -0.015179 -0.163393 -2.149570e-16 -1.394718e-16   \n",
       "\n",
       "         norm-grnd-lx  norm-grnd-ly  delta-norm-grnd-rx  delta-norm-grnd-ry  \\\n",
       "speaker                                                                       \n",
       "man-1    6.855577e-16  2.108858e-15           -0.002346           -0.000487   \n",
       "woman-1 -1.600753e-16  6.753252e-17            0.004865           -0.000157   \n",
       "woman-2 -5.530972e-16  1.729086e-16           -0.001435            0.000886   \n",
       "\n",
       "         delta-norm-grnd-lx  delta-norm-grnd-ly  \n",
       "speaker                                          \n",
       "man-1              0.001420           -0.001815  \n",
       "woman-1           -0.003470            0.002501  \n",
       "woman-2            0.001413            0.000159  \n",
       "\n",
       "[3 rows x 34 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_means = asl.df.groupby('speaker').mean()\n",
    "df_means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "To select a mean that matches by speaker, use the pandas [map](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.map.html) method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>grnd-ly</th>\n",
       "      <th>left-x-mean</th>\n",
       "      <th>right-x-mean</th>\n",
       "      <th>left-y-mean</th>\n",
       "      <th>right-y-mean</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx  grnd-ly  left-x-mean  right-x-mean  \\\n",
       "video frame                                                                  \n",
       "98    0          113        9      -12      119   164.661438    151.017865   \n",
       "      1          113        9      -12      119   164.661438    151.017865   \n",
       "      2          113        9      -12      119   164.661438    151.017865   \n",
       "      3          113        9      -12      119   164.661438    151.017865   \n",
       "      4          113        9      -12      119   164.661438    151.017865   \n",
       "\n",
       "             left-y-mean  right-y-mean  \n",
       "video frame                             \n",
       "98    0       161.271242    117.332462  \n",
       "      1       161.271242    117.332462  \n",
       "      2       161.271242    117.332462  \n",
       "      3       161.271242    117.332462  \n",
       "      4       161.271242    117.332462  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "asl.df['left-x-mean'] = asl.df['speaker'].map(df_means['left-x'])\n",
    "asl.df['right-x-mean'] = asl.df['speaker'].map(df_means['right-x'])\n",
    "asl.df['left-y-mean'] = asl.df['speaker'].map(df_means['left-y'])\n",
    "asl.df['right-y-mean'] = asl.df['speaker'].map(df_means['right-y'])\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "df_std\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>grnd-ly</th>\n",
       "      <th>left-x-mean</th>\n",
       "      <th>right-x-mean</th>\n",
       "      <th>left-y-mean</th>\n",
       "      <th>right-y-mean</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>speaker</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>man-1</th>\n",
       "      <td>15.154425</td>\n",
       "      <td>36.328485</td>\n",
       "      <td>18.901917</td>\n",
       "      <td>54.902340</td>\n",
       "      <td>6.654573</td>\n",
       "      <td>5.520045</td>\n",
       "      <td>53.487999</td>\n",
       "      <td>20.269032</td>\n",
       "      <td>15.080360</td>\n",
       "      <td>36.572749</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>woman-1</th>\n",
       "      <td>17.573442</td>\n",
       "      <td>26.594521</td>\n",
       "      <td>16.459943</td>\n",
       "      <td>34.667787</td>\n",
       "      <td>3.549392</td>\n",
       "      <td>3.538330</td>\n",
       "      <td>33.972660</td>\n",
       "      <td>16.764706</td>\n",
       "      <td>17.328941</td>\n",
       "      <td>27.117393</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>woman-2</th>\n",
       "      <td>15.388711</td>\n",
       "      <td>28.825025</td>\n",
       "      <td>14.890288</td>\n",
       "      <td>39.649111</td>\n",
       "      <td>4.099760</td>\n",
       "      <td>3.416167</td>\n",
       "      <td>39.128572</td>\n",
       "      <td>16.191324</td>\n",
       "      <td>15.050938</td>\n",
       "      <td>29.320655</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            left-x     left-y    right-x    right-y    nose-x    nose-y  \\\n",
       "speaker                                                                   \n",
       "man-1    15.154425  36.328485  18.901917  54.902340  6.654573  5.520045   \n",
       "woman-1  17.573442  26.594521  16.459943  34.667787  3.549392  3.538330   \n",
       "woman-2  15.388711  28.825025  14.890288  39.649111  4.099760  3.416167   \n",
       "\n",
       "           grnd-ry    grnd-rx    grnd-lx    grnd-ly  left-x-mean  \\\n",
       "speaker                                                            \n",
       "man-1    53.487999  20.269032  15.080360  36.572749          0.0   \n",
       "woman-1  33.972660  16.764706  17.328941  27.117393          0.0   \n",
       "woman-2  39.128572  16.191324  15.050938  29.320655          0.0   \n",
       "\n",
       "         right-x-mean  left-y-mean  right-y-mean  \n",
       "speaker                                           \n",
       "man-1             0.0          0.0           0.0  \n",
       "woman-1           0.0          0.0           0.0  \n",
       "woman-2           0.0          0.0           0.0  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<font color=green>Correct!</font><br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from asl_utils import test_std_tryit\n",
    "# TODO Create a dataframe named `df_std` with standard deviations grouped by speaker\n",
    "df_std = asl.df.groupby('speaker').std()\n",
    "# test the code\n",
    "test_std_tryit(df_std)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part1_submission'></a>\n",
    "### Features Implementation Submission\n",
    "Implement four feature sets and answer the question that follows.\n",
    "- normalized Cartesian coordinates\n",
    "    - use *mean* and *standard deviation* statistics and the [standard score](https://en.wikipedia.org/wiki/Standard_score) equation to account for speakers with different heights and arm length\n",
    "    \n",
    "- polar coordinates\n",
    "    - calculate polar coordinates with [Cartesian to polar equations](https://en.wikipedia.org/wiki/Polar_coordinate_system#Converting_between_polar_and_Cartesian_coordinates)\n",
    "    - use the [np.arctan2](https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.arctan2.html) function and *swap the x and y axes* to move the $0$ to $2\\pi$ discontinuity to 12 o'clock instead of 3 o'clock;  in other words, the normal break in radians value from $0$ to $2\\pi$ occurs directly to the left of the speaker's nose, which may be in the signing area and interfere with results.  By swapping the x and y axes, that discontinuity move to directly above the speaker's head, an area not generally used in signing.\n",
    "\n",
    "- delta difference\n",
    "    - as described in Thad's lecture, use the difference in values between one frame and the next frames as features\n",
    "    - pandas [diff method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.diff.html) and [fillna method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html) will be helpful for this one\n",
    "\n",
    "- custom features\n",
    "    - These are your own design; combine techniques used above or come up with something else entirely. We look forward to seeing what you come up with! \n",
    "    Some ideas to get you started:\n",
    "        - normalize using a [feature scaling equation](https://en.wikipedia.org/wiki/Feature_scaling)\n",
    "        - normalize the polar coordinates\n",
    "        - adding additional deltas\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>grnd-ly</th>\n",
       "      <th>left-x-mean</th>\n",
       "      <th>right-x-mean</th>\n",
       "      <th>left-y-mean</th>\n",
       "      <th>right-y-mean</th>\n",
       "      <th>norm-rx</th>\n",
       "      <th>norm-ry</th>\n",
       "      <th>norm-lx</th>\n",
       "      <th>norm-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>1.153232</td>\n",
       "      <td>1.663433</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>1.153232</td>\n",
       "      <td>1.663433</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>1.153232</td>\n",
       "      <td>1.663433</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>1.153232</td>\n",
       "      <td>1.663433</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>119</td>\n",
       "      <td>164.661438</td>\n",
       "      <td>151.017865</td>\n",
       "      <td>161.271242</td>\n",
       "      <td>117.332462</td>\n",
       "      <td>1.153232</td>\n",
       "      <td>1.663433</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx  grnd-ly  left-x-mean  right-x-mean  \\\n",
       "video frame                                                                  \n",
       "98    0          113        9      -12      119   164.661438    151.017865   \n",
       "      1          113        9      -12      119   164.661438    151.017865   \n",
       "      2          113        9      -12      119   164.661438    151.017865   \n",
       "      3          113        9      -12      119   164.661438    151.017865   \n",
       "      4          113        9      -12      119   164.661438    151.017865   \n",
       "\n",
       "             left-y-mean  right-y-mean   norm-rx   norm-ry   norm-lx   norm-ly  \n",
       "video frame                                                                     \n",
       "98    0       161.271242    117.332462  1.153232  1.663433 -0.891199  0.741835  \n",
       "      1       161.271242    117.332462  1.153232  1.663433 -0.891199  0.741835  \n",
       "      2       161.271242    117.332462  1.153232  1.663433 -0.891199  0.741835  \n",
       "      3       161.271242    117.332462  1.153232  1.663433 -0.891199  0.741835  \n",
       "      4       161.271242    117.332462  1.153232  1.663433 -0.891199  0.741835  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO add features for normalized by speaker values of left, right, x, y\n",
    "# Name these 'norm-rx', 'norm-ry', 'norm-lx', and 'norm-ly'\n",
    "# using Z-score scaling (X-Xmean)/Xstd\n",
    "\n",
    "features_norm = ['norm-rx', 'norm-ry', 'norm-lx','norm-ly']\n",
    "features_l_r = ['right-x', 'right-y', 'left-x', 'left-y']\n",
    "\n",
    "def norm_score(x, m, std):\n",
    "    return (x - m) / std\n",
    "\n",
    "for i, val in enumerate(features_norm):\n",
    "    mean = asl.df['speaker'].map(df_means[features_l_r[i]])\n",
    "    std = asl.df['speaker'].map(df_std[features_l_r[i]])\n",
    "    asl.df[val] = norm_score(asl.df[features_l_r[i]], mean, std)\n",
    "\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>...</th>\n",
       "      <th>delta-lx</th>\n",
       "      <th>delta-ly</th>\n",
       "      <th>norm-grnd-rx</th>\n",
       "      <th>norm-grnd-ry</th>\n",
       "      <th>norm-grnd-lx</th>\n",
       "      <th>norm-grnd-ly</th>\n",
       "      <th>delta-norm-grnd-rx</th>\n",
       "      <th>delta-norm-grnd-ry</th>\n",
       "      <th>delta-norm-grnd-lx</th>\n",
       "      <th>delta-norm-grnd-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx         ...          delta-lx  \\\n",
       "video frame                                    ...                     \n",
       "98    0          113        9      -12         ...               0.0   \n",
       "      1          113        9      -12         ...               0.0   \n",
       "      2          113        9      -12         ...               0.0   \n",
       "      3          113        9      -12         ...               0.0   \n",
       "      4          113        9      -12         ...               0.0   \n",
       "\n",
       "             delta-ly  norm-grnd-rx  norm-grnd-ry  norm-grnd-lx  norm-grnd-ly  \\\n",
       "video frame                                                                     \n",
       "98    0           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      1           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      2           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      3           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      4           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "\n",
       "             delta-norm-grnd-rx  delta-norm-grnd-ry  delta-norm-grnd-lx  \\\n",
       "video frame                                                               \n",
       "98    0                     0.0                 0.0                 0.0   \n",
       "      1                     0.0                 0.0                 0.0   \n",
       "      2                     0.0                 0.0                 0.0   \n",
       "      3                     0.0                 0.0                 0.0   \n",
       "      4                     0.0                 0.0                 0.0   \n",
       "\n",
       "             delta-norm-grnd-ly  \n",
       "video frame                      \n",
       "98    0                     0.0  \n",
       "      1                     0.0  \n",
       "      2                     0.0  \n",
       "      3                     0.0  \n",
       "      4                     0.0  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO add features for polar coordinate values where the nose is the origin\n",
    "# Name these 'polar-rr', 'polar-rtheta', 'polar-lr', and 'polar-ltheta'\n",
    "# Note that 'polar-rr' and 'polar-rtheta' refer to the radius and angle\n",
    "\n",
    "asl.df['polar-rr'] = np.sqrt( np.power(asl.df['grnd-rx'], 2) + np.power(asl.df['grnd-ry'], 2))\n",
    "asl.df['polar-rtheta'] = np.arctan2(asl.df['grnd-rx'], asl.df['grnd-ry'])\n",
    "asl.df['polar-lr'] = np.sqrt( np.power(asl.df['grnd-lx'], 2) + np.power(asl.df['grnd-ly'], 2))\n",
    "asl.df['polar-ltheta'] = np.arctan2(asl.df['grnd-lx'], asl.df['grnd-ly'])\n",
    "\n",
    "features_polar = ['polar-rr', 'polar-rtheta', 'polar-lr', 'polar-ltheta']\n",
    "\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>...</th>\n",
       "      <th>norm-lx</th>\n",
       "      <th>norm-ly</th>\n",
       "      <th>polar-rr</th>\n",
       "      <th>polar-rtheta</th>\n",
       "      <th>polar-lr</th>\n",
       "      <th>polar-ltheta</th>\n",
       "      <th>delta-rx</th>\n",
       "      <th>delta-ry</th>\n",
       "      <th>delta-lx</th>\n",
       "      <th>delta-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "      <td>113.35784</td>\n",
       "      <td>0.079478</td>\n",
       "      <td>119.603512</td>\n",
       "      <td>-0.100501</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "      <td>113.35784</td>\n",
       "      <td>0.079478</td>\n",
       "      <td>119.603512</td>\n",
       "      <td>-0.100501</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "      <td>113.35784</td>\n",
       "      <td>0.079478</td>\n",
       "      <td>119.603512</td>\n",
       "      <td>-0.100501</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "      <td>113.35784</td>\n",
       "      <td>0.079478</td>\n",
       "      <td>119.603512</td>\n",
       "      <td>-0.100501</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.891199</td>\n",
       "      <td>0.741835</td>\n",
       "      <td>113.35784</td>\n",
       "      <td>0.079478</td>\n",
       "      <td>119.603512</td>\n",
       "      <td>-0.100501</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx    ...      norm-lx   norm-ly  \\\n",
       "video frame                               ...                          \n",
       "98    0          113        9      -12    ...    -0.891199  0.741835   \n",
       "      1          113        9      -12    ...    -0.891199  0.741835   \n",
       "      2          113        9      -12    ...    -0.891199  0.741835   \n",
       "      3          113        9      -12    ...    -0.891199  0.741835   \n",
       "      4          113        9      -12    ...    -0.891199  0.741835   \n",
       "\n",
       "              polar-rr  polar-rtheta    polar-lr  polar-ltheta  delta-rx  \\\n",
       "video frame                                                                \n",
       "98    0      113.35784      0.079478  119.603512     -0.100501       0.0   \n",
       "      1      113.35784      0.079478  119.603512     -0.100501       0.0   \n",
       "      2      113.35784      0.079478  119.603512     -0.100501       0.0   \n",
       "      3      113.35784      0.079478  119.603512     -0.100501       0.0   \n",
       "      4      113.35784      0.079478  119.603512     -0.100501       0.0   \n",
       "\n",
       "             delta-ry  delta-lx  delta-ly  \n",
       "video frame                                \n",
       "98    0           0.0       0.0       0.0  \n",
       "      1           0.0       0.0       0.0  \n",
       "      2           0.0       0.0       0.0  \n",
       "      3           0.0       0.0       0.0  \n",
       "      4           0.0       0.0       0.0  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO add features for left, right, x, y differences by one time step, i.e. the \"delta\" values discussed in the lecture\n",
    "# Name these 'delta-rx', 'delta-ry', 'delta-lx', and 'delta-ly'\n",
    "\n",
    "asl.df['delta-rx'] = asl.df['right-x'].diff().fillna(0)\n",
    "asl.df['delta-ry'] = asl.df['right-y'].diff().fillna(0)\n",
    "asl.df['delta-lx'] = asl.df['left-x'].diff().fillna(0)\n",
    "asl.df['delta-ly'] = asl.df['left-y'].diff().fillna(0)\n",
    "\n",
    "features_delta = ['delta-rx', 'delta-ry', 'delta-lx', 'delta-ly']\n",
    "\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>...</th>\n",
       "      <th>delta-lx</th>\n",
       "      <th>delta-ly</th>\n",
       "      <th>norm-grnd-rx</th>\n",
       "      <th>norm-grnd-ry</th>\n",
       "      <th>norm-grnd-lx</th>\n",
       "      <th>norm-grnd-ly</th>\n",
       "      <th>delta-norm-grnd-rx</th>\n",
       "      <th>delta-norm-grnd-ry</th>\n",
       "      <th>delta-norm-grnd-lx</th>\n",
       "      <th>delta-norm-grnd-ly</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.230994</td>\n",
       "      <td>1.557506</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx         ...          delta-lx  \\\n",
       "video frame                                    ...                     \n",
       "98    0          113        9      -12         ...               0.0   \n",
       "      1          113        9      -12         ...               0.0   \n",
       "      2          113        9      -12         ...               0.0   \n",
       "      3          113        9      -12         ...               0.0   \n",
       "      4          113        9      -12         ...               0.0   \n",
       "\n",
       "             delta-ly  norm-grnd-rx  norm-grnd-ry  norm-grnd-lx  norm-grnd-ly  \\\n",
       "video frame                                                                     \n",
       "98    0           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      1           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      2           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      3           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "      4           0.0      1.230994      1.557506     -0.808262      0.552186   \n",
       "\n",
       "             delta-norm-grnd-rx  delta-norm-grnd-ry  delta-norm-grnd-lx  \\\n",
       "video frame                                                               \n",
       "98    0                     0.0                 0.0                 0.0   \n",
       "      1                     0.0                 0.0                 0.0   \n",
       "      2                     0.0                 0.0                 0.0   \n",
       "      3                     0.0                 0.0                 0.0   \n",
       "      4                     0.0                 0.0                 0.0   \n",
       "\n",
       "             delta-norm-grnd-ly  \n",
       "video frame                      \n",
       "98    0                     0.0  \n",
       "      1                     0.0  \n",
       "      2                     0.0  \n",
       "      3                     0.0  \n",
       "      4                     0.0  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO add features of your own design, which may be a combination of the above or something else\n",
    "# Name these whatever you would like\n",
    "\n",
    "# TODO define a list named 'features_custom' for building the training set\n",
    "\n",
    "# Normalize Grounded Features\n",
    "features_norm_grnd = ['norm-grnd-rx', 'norm-grnd-ry', 'norm-grnd-lx','norm-grnd-ly']\n",
    "features_grnd = ['grnd-rx', 'grnd-ry', 'grnd-lx','grnd-ly']\n",
    "\n",
    "df_std = asl.df.groupby('speaker').std()\n",
    "df_mean = asl.df.groupby('speaker').mean()\n",
    "\n",
    "for feature, root_feat in zip(features_norm_grnd, features_grnd):\n",
    "    asl.df[feature] = (asl.df[root_feat] - asl.df['speaker'].map(df_means[root_feat])) / asl.df['speaker'].map(df_std[root_feat])\n",
    "    \n",
    "# Delta Values for Normalized Grounded Features\n",
    "features_delta_norm_grnd = ['delta-norm-grnd-rx', 'delta-norm-grnd-ry', 'delta-norm-grnd-lx', 'delta-norm-grnd-ly']\n",
    "asl.df[features_delta_norm_grnd] = asl.df[features_norm_grnd].fillna(0).diff().fillna(0)\n",
    "\n",
    "# custom feature for building the training set\n",
    "features_custom = features_norm_grnd + features_delta_norm_grnd + features_polar\n",
    "\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>left-x</th>\n",
       "      <th>left-y</th>\n",
       "      <th>right-x</th>\n",
       "      <th>right-y</th>\n",
       "      <th>nose-x</th>\n",
       "      <th>nose-y</th>\n",
       "      <th>speaker</th>\n",
       "      <th>grnd-ry</th>\n",
       "      <th>grnd-rx</th>\n",
       "      <th>grnd-lx</th>\n",
       "      <th>...</th>\n",
       "      <th>norm-grnd-lx</th>\n",
       "      <th>norm-grnd-ly</th>\n",
       "      <th>delta-norm-grnd-rx</th>\n",
       "      <th>delta-norm-grnd-ry</th>\n",
       "      <th>delta-norm-grnd-lx</th>\n",
       "      <th>delta-norm-grnd-ly</th>\n",
       "      <th>norm-polar-rr</th>\n",
       "      <th>norm-polar-rtheta</th>\n",
       "      <th>norm-polar-lr</th>\n",
       "      <th>norm-polar-ltheta</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>video</th>\n",
       "      <th>frame</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">98</th>\n",
       "      <th>0</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.573439</td>\n",
       "      <td>0.959419</td>\n",
       "      <td>0.542499</td>\n",
       "      <td>-0.735219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.573439</td>\n",
       "      <td>0.959419</td>\n",
       "      <td>0.542499</td>\n",
       "      <td>-0.735219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.573439</td>\n",
       "      <td>0.959419</td>\n",
       "      <td>0.542499</td>\n",
       "      <td>-0.735219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.573439</td>\n",
       "      <td>0.959419</td>\n",
       "      <td>0.542499</td>\n",
       "      <td>-0.735219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>149</td>\n",
       "      <td>181</td>\n",
       "      <td>170</td>\n",
       "      <td>175</td>\n",
       "      <td>161</td>\n",
       "      <td>62</td>\n",
       "      <td>woman-1</td>\n",
       "      <td>113</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.808262</td>\n",
       "      <td>0.552186</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.573439</td>\n",
       "      <td>0.959419</td>\n",
       "      <td>0.542499</td>\n",
       "      <td>-0.735219</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             left-x  left-y  right-x  right-y  nose-x  nose-y  speaker  \\\n",
       "video frame                                                              \n",
       "98    0         149     181      170      175     161      62  woman-1   \n",
       "      1         149     181      170      175     161      62  woman-1   \n",
       "      2         149     181      170      175     161      62  woman-1   \n",
       "      3         149     181      170      175     161      62  woman-1   \n",
       "      4         149     181      170      175     161      62  woman-1   \n",
       "\n",
       "             grnd-ry  grnd-rx  grnd-lx        ...          norm-grnd-lx  \\\n",
       "video frame                                   ...                         \n",
       "98    0          113        9      -12        ...             -0.808262   \n",
       "      1          113        9      -12        ...             -0.808262   \n",
       "      2          113        9      -12        ...             -0.808262   \n",
       "      3          113        9      -12        ...             -0.808262   \n",
       "      4          113        9      -12        ...             -0.808262   \n",
       "\n",
       "             norm-grnd-ly  delta-norm-grnd-rx  delta-norm-grnd-ry  \\\n",
       "video frame                                                         \n",
       "98    0          0.552186                 0.0                 0.0   \n",
       "      1          0.552186                 0.0                 0.0   \n",
       "      2          0.552186                 0.0                 0.0   \n",
       "      3          0.552186                 0.0                 0.0   \n",
       "      4          0.552186                 0.0                 0.0   \n",
       "\n",
       "             delta-norm-grnd-lx  delta-norm-grnd-ly  norm-polar-rr  \\\n",
       "video frame                                                          \n",
       "98    0                     0.0                 0.0       1.573439   \n",
       "      1                     0.0                 0.0       1.573439   \n",
       "      2                     0.0                 0.0       1.573439   \n",
       "      3                     0.0                 0.0       1.573439   \n",
       "      4                     0.0                 0.0       1.573439   \n",
       "\n",
       "             norm-polar-rtheta  norm-polar-lr  norm-polar-ltheta  \n",
       "video frame                                                       \n",
       "98    0               0.959419       0.542499          -0.735219  \n",
       "      1               0.959419       0.542499          -0.735219  \n",
       "      2               0.959419       0.542499          -0.735219  \n",
       "      3               0.959419       0.542499          -0.735219  \n",
       "      4               0.959419       0.542499          -0.735219  \n",
       "\n",
       "[5 rows x 39 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO add features of your own design, which may be a combination of the above or something else\n",
    "# Name these whatever you would like\n",
    "\n",
    "# TODO define a list named 'features_custom' for building the training set\n",
    "\n",
    "# Normalize Polar Features\n",
    "features_norm_polar = ['norm-polar-rr', 'norm-polar-rtheta', 'norm-polar-lr','norm-polar-ltheta']\n",
    "features_polar = ['polar-rr', 'polar-rtheta', 'polar-lr', 'polar-ltheta']\n",
    "\n",
    "df_std = asl.df.groupby('speaker').std()\n",
    "df_mean = asl.df.groupby('speaker').mean()\n",
    "\n",
    "for feature, root_feat in zip(features_norm_polar, features_polar):\n",
    "    asl.df[feature] = (asl.df[root_feat] - asl.df['speaker'].map(df_means[root_feat])) / asl.df['speaker'].map(df_std[root_feat])\n",
    "\n",
    "# custom feature for building the training set\n",
    "features_custom2 = features_norm_polar\n",
    "\n",
    "asl.df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Question 1:**  What custom features did you choose for the features_custom set and why?\n",
    "\n",
    "**Answer 1:**\n",
    "\n",
    "I created two custom feature sets, the first set (features_custom) normalizes the ground features with repect to the nose co-ordinates (features_norm_grnd), takes the delta between these normalised ground features (features_delta_norm_grnd) and finally adds the polar feature (features_polar). This normalisation around the nose co-ordinates helps to reduce errors due to speaker height. Adding the delta of this normalisation was shown to improve accuracy in the lecture material. Finally through experimentation below I discovered that the polar features yielded the most accuracy of the 4 feature sets so I combined them in this custom feature.\n",
    "\n",
    "The second custom feature set (features_custom_2) was the normalisation of the polar features (features_norm_polar) which would hopefully improve on the results seen from the polar features set due to standardising the data. Findings to be discussed below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part1_test'></a>\n",
    "### Features Unit Testing\n",
    "Run the following unit tests as a sanity check on the defined \"ground\", \"norm\", \"polar\", and 'delta\"\n",
    "feature sets.  The test simply looks for some valid values but is not exhaustive.  However, the project should not be submitted if these tests don't pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9, 113, -12, 119]\n",
      "[1.1532321114002382, 1.6634329223668574, -0.89119923044101379, 0.74183544610811614]\n",
      "[113.35784048754634, 0.079478244608206572, 119.60351165413162, -0.10050059905462982]\n",
      "[0.0, 0.0, 0.0, 0.0]\n",
      "[1.2309941677338545, 1.5575064175922233, -0.80826165004287043, 0.55218642740668555, 0.0, 0.0, 0.0, 0.0, 113.35784048754634, 0.079478244608206572, 119.60351165413162, -0.10050059905462982]\n",
      "[1.5734394584886571, 0.95941868201635139, 0.54249851337954813, -0.73521895146904148]\n"
     ]
    }
   ],
   "source": [
    "# Verify values of features before tests\n",
    "\n",
    "# Collect the features into a list\n",
    "features_ground = ['grnd-rx','grnd-ry','grnd-lx','grnd-ly']\n",
    "features_norm = ['norm-rx', 'norm-ry', 'norm-lx', 'norm-ly']\n",
    "features_polar = ['polar-rr', 'polar-rtheta', 'polar-lr', 'polar-ltheta']\n",
    "features_delta = ['delta-rx', 'delta-ry', 'delta-lx', 'delta-ly']\n",
    "\n",
    "# Show a single set of features for a given (video, frame) tuple\n",
    "print([asl.df.ix[98,1][v] for v in features_ground])\n",
    "print([asl.df.ix[98,1][v] for v in features_norm])\n",
    "print([asl.df.ix[98,1][v] for v in features_polar])\n",
    "print([asl.df.ix[98,1][v] for v in features_delta])\n",
    "print([asl.df.ix[98,1][v] for v in features_custom])\n",
    "print([asl.df.ix[98,1][v] for v in features_custom2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "....\n",
      "----------------------------------------------------------------------\n",
      "Ran 4 tests in 0.016s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.runner.TextTestResult run=4 errors=0 failures=0>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import unittest\n",
    "# import numpy as np\n",
    "\n",
    "class TestFeatures(unittest.TestCase):\n",
    "\n",
    "    def test_features_ground(self):\n",
    "        sample = (asl.df.ix[98, 1][features_ground]).tolist()\n",
    "        self.assertEqual(sample, [9, 113, -12, 119])\n",
    "\n",
    "    def test_features_norm(self):\n",
    "        sample = (asl.df.ix[98, 1][features_norm]).tolist()\n",
    "        np.testing.assert_almost_equal(sample, [ 1.153,  1.663, -0.891,  0.742], 3)\n",
    "\n",
    "    def test_features_polar(self):\n",
    "        sample = (asl.df.ix[98,1][features_polar]).tolist()\n",
    "        np.testing.assert_almost_equal(sample, [113.3578, 0.0794, 119.603, -0.1005], 3)\n",
    "\n",
    "    def test_features_delta(self):\n",
    "        sample = (asl.df.ix[98, 0][features_delta]).tolist()\n",
    "        self.assertEqual(sample, [0, 0, 0, 0])\n",
    "        sample = (asl.df.ix[98, 18][features_delta]).tolist()\n",
    "        self.assertTrue(sample in [[-16, -5, -2, 4], [-14, -9, 0, 0]], \"Sample value found was {}\".format(sample))\n",
    "                         \n",
    "suite = unittest.TestLoader().loadTestsFromModule(TestFeatures())\n",
    "unittest.TextTestRunner().run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part2_tutorial'></a>\n",
    "## PART 2: Model Selection\n",
    "### Model Selection Tutorial\n",
    "The objective of Model Selection is to tune the number of states for each word HMM prior to testing on unseen data.  In this section you will explore three methods: \n",
    "- Log likelihood using cross-validation folds (CV)\n",
    "- Bayesian Information Criterion (BIC)\n",
    "- Discriminative Information Criterion (DIC) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Train a single word\n",
    "Now that we have built a training set with sequence data, we can \"train\" models for each word.  As a simple starting example, we train a single word using Gaussian hidden Markov models (HMM).   By using the `fit` method during training, the [Baum-Welch Expectation-Maximization](https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm) (EM) algorithm is invoked iteratively to find the best estimate for the model *for the number of hidden states specified* from a group of sample seequences. For this example, we *assume* the correct number of hidden states is 3, but that is just a guess.  How do we know what the \"best\" number of states for training is?  We will need to find some model selection technique to choose the best parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for BOOK is 3\n",
      "logL = -2331.1138127433196\n"
     ]
    }
   ],
   "source": [
    "import warnings\n",
    "from hmmlearn.hmm import GaussianHMM\n",
    "\n",
    "def train_a_word(word, num_hidden_states, features):\n",
    "    \n",
    "    warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n",
    "    training = asl.build_training(features)  \n",
    "    X, lengths = training.get_word_Xlengths(word)\n",
    "    model = GaussianHMM(n_components=num_hidden_states, n_iter=1000).fit(X, lengths)\n",
    "    logL = model.score(X, lengths)\n",
    "    return model, logL\n",
    "\n",
    "demoword = 'BOOK'\n",
    "model, logL = train_a_word(demoword, 3, features_ground)\n",
    "print(\"Number of states trained in model for {} is {}\".format(demoword, model.n_components))\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The HMM model has been trained and information can be pulled from the model, including means and variances for each feature and hidden state.  The [log likelihood](http://math.stackexchange.com/questions/892832/why-we-consider-log-likelihood-instead-of-likelihood-in-gaussian-distribution) for any individual sample or group of samples can also be calculated with the `score` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for BOOK is 3\n",
      "hidden state #0\n",
      "mean =  [ -3.46504869  50.66686933  14.02391587  52.04731066]\n",
      "variance =  [ 49.12346305  43.04799144  39.35109609  47.24195772]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [ -1.12415027  69.44164191  17.02866283  77.7231196 ]\n",
      "variance =  [ 19.70434594  16.83041492  30.51552305  11.03678246]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [ -11.45300909   94.109178     19.03512475  102.2030162 ]\n",
      "variance =  [  77.403668    203.35441965   26.68898447  156.12444034]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def show_model_stats(word, model):\n",
    "    print(\"Number of states trained in model for {} is {}\".format(word, model.n_components))    \n",
    "    variance=np.array([np.diag(model.covars_[i]) for i in range(model.n_components)])    \n",
    "    for i in range(model.n_components):  # for each hidden state\n",
    "        print(\"hidden state #{}\".format(i))\n",
    "        print(\"mean = \", model.means_[i])\n",
    "        print(\"variance = \", variance[i])\n",
    "        print()\n",
    "    \n",
    "show_model_stats(demoword, model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Try it!\n",
    "Experiment by changing the feature set, word, and/or num_hidden_states values in the next cell to see changes in values.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [ -9.30211403  55.32333876   6.92259936  71.24057775]\n",
      "variance =  [ 16.16920957  46.50917372   3.81388185  15.79446427]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [ -5.40587658  60.1652424    2.32479599  91.3095432 ]\n",
      "variance =  [   7.95073876   64.13103127   13.68077479  129.5912395 ]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [   0.58333333   87.91666667   12.75        108.5       ]\n",
      "variance =  [  39.41055556   18.74388889    9.855       144.4175    ]\n",
      "\n",
      "logL = -601.3291470028622\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_ground) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [ 0.53813142 -0.04704542  0.04545    -0.61902479]\n",
      "variance =  [ 0.02432711  0.04933945  0.03044391  0.13395751]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [ 1.0625066  -0.1740204  -1.31812255 -1.61791893]\n",
      "variance =  [ 0.05296979  0.00373254  0.03820247  0.05531266]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [ 0.38934967 -0.18383814 -0.34623283 -1.59201968]\n",
      "variance =  [ 0.0697732   0.02960059  0.07727065  0.06911305]\n",
      "\n",
      "logL = 22.651977224475196\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_norm) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [  6.05010053e+01  -9.31757873e-02   9.14166393e+01   2.57297227e-02]\n",
      "variance =  [  6.15523307e+01   3.56540651e-03   1.30626433e+02   1.81995354e-03]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [ 56.19235087  -0.16348589  71.60640024   0.0958256 ]\n",
      "variance =  [  5.16348282e+01   4.22849414e-03   1.74535632e+01   1.18341913e-03]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [  8.81313612e+01   8.31771967e-03   1.09304329e+02   1.18434797e-01]\n",
      "variance =  [  2.06973439e+01   5.41190376e-03   1.41647879e+02   1.92144403e-03]\n",
      "\n",
      "logL = -179.2152206555347\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_polar) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [-0.54151701 -0.13549492  0.          0.        ]\n",
      "variance =  [  7.40033314e+00   2.07610444e-01   4.54757788e-04   4.54757788e-04]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [ 0.07434035  4.32590509  1.08475329  2.81908645]\n",
      "variance =  [ 14.12290048  15.39826602  12.40214192  17.46963045]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [ 2.26955318 -0.2690072  -0.67005544 -9.1341518 ]\n",
      "variance =  [  9.26031519   7.66293533   7.67530134  15.28076516]\n",
      "\n",
      "logL = -256.56151832675334\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_delta) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [  2.75816298e-01   2.75261089e-02  -1.71938988e-01  -1.10828925e+00\n",
      "  -3.89974692e-02   4.75904502e-02   1.23360647e-02   3.60082273e-02\n",
      "   6.20416282e+01  -1.30880607e-01   7.89409010e+01   5.59242377e-02]\n",
      "variance =  [  3.47272237e-02   5.60949860e-02   5.59031342e-02   2.19433003e-01\n",
      "   2.88464625e-02   8.12671797e-03   1.84156158e-02   6.82866144e-03\n",
      "   6.36702682e+01   4.13408222e-03   5.87051196e+01   2.39820200e-03]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [  9.94163854e-01  -1.51765543e-02  -1.22453620e+00  -1.32713153e+00\n",
      "   1.15118143e-01  -2.80436736e-02   9.00308735e-47  -8.65854150e-02\n",
      "   8.81313612e+01   8.31771967e-03   1.09304329e+02   1.18434797e-01]\n",
      "variance =  [  9.67595591e-02   7.38463628e-03   4.41641100e-02   1.08803085e-01\n",
      "   5.12727796e-02   4.82382686e-03   3.30794927e-02   3.27113038e-02\n",
      "   2.06973439e+01   5.41190376e-03   1.41647879e+02   1.92144403e-03]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [  4.67429439e-01  -2.04275026e-01  -2.52636528e-01  -5.87687117e-01\n",
      "   5.04059246e-02   3.76477596e-02  -1.94380276e-03  -2.43545085e-01\n",
      "   5.36266745e+01  -9.40519613e-02   9.44501098e+01   3.97008337e-02]\n",
      "variance =  [  2.65635645e-02   1.53851676e-02   2.83439123e-02   4.06051297e-01\n",
      "   8.45173420e-03   2.79172139e-03   2.22138665e-02   3.57740514e-02\n",
      "   1.46615679e+01   5.43439934e-03   2.41866614e+02   3.10125315e-03]\n",
      "\n",
      "logL = -46.81668662159864\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_custom) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of states trained in model for CHOCOLATE is 3\n",
      "hidden state #0\n",
      "mean =  [-0.43567333  0.37225216 -0.32965362 -0.29811062]\n",
      "variance =  [ 0.01187406  0.05490274  0.25105733  0.01524059]\n",
      "\n",
      "hidden state #1\n",
      "mean =  [-0.26057161  0.48991141 -1.754827   -0.33867755]\n",
      "variance =  [ 0.02935861  0.13162366  0.02613229  0.05371838]\n",
      "\n",
      "hidden state #2\n",
      "mean =  [-0.00892579  0.57001662 -0.85620942 -0.25997993]\n",
      "variance =  [ 0.02685728  0.0121504   0.02703956  0.04334482]\n",
      "\n",
      "logL = 38.85569423161961\n"
     ]
    }
   ],
   "source": [
    "my_testword = 'CHOCOLATE'\n",
    "model, logL = train_a_word(my_testword, 3, features_custom2) # Experiment here with different parameters\n",
    "show_model_stats(my_testword, model)\n",
    "print(\"logL = {}\".format(logL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Visualize the hidden states\n",
    "We can plot the means and variances for each state and feature.  Try varying the number of states trained for the HMM model and examine the variances.  Are there some models that are \"better\" than others?  How can you tell?  We would like to hear what you think in the classroom online."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXmcXFWV+L+net/XpNPpbKwiKEETCNtIGBDZBIQQEARB\nkYkjyAgO6OhPwBFERpBVAiKbrAEEA6IgaCeABAiLQAhIgIR09t639Frn98e9jS9FV3d1dXeqqvt8\nP5/3qffe3c4979U9d3v3iqpiGIZhGMlGKNECGIZhGEZ/mIEyDMMwkhIzUIZhGEZSYgbKMAzDSErM\nQBmGYRhJiRkowzAMIykxA2WMG0SkQkSWikiLiFyVaHlGExG5RETuHsB9hYjMjeI2V0RqBgh7h4j8\nbATENIwBMQOVoojIKSKyXERaRWSDiPxJRA70bv0WTiKiIrJz4Hp3EVksIk2+0P6biOwfESbTx/ee\niLSJyGoRuU1EZgT8HC0iL3n3OhG5R0SmBNzPEJHnBsnPHSLSIyKVgXsLff5aRaRLRLoD138SkRk+\nT60Rx0lRkjkbqAUKVfWCARU8CIkopH1+/yYi7SLyjogcGm9cqrqHqlaPoHijSuS7G4P/ahE5a4TS\nXiQih4lIlohsjHDL8v+HZhHZKCLnj0SahsMMVAri/wTXAJcDFcA04EbgmCHEsRPwPPAmsAMwGXgE\neEpE9gt4fcjHewpQBMwElgOH+HjmAfd6ecqBPYBO4DkRKYlRljzgBKAJ+FrffVVdoKr5qprv8/pA\n37WqHhGIojhwP19VH4iS1HTgbU2Cr9NFJD2OYPcBrwFlwI+Ah0RkwogKZvTHLNw7vyfwVoTbJcAu\nuHfrYOBCETl8u0o3llFVO1LowBmJVuDEAfxcAtzdz30FdvbnvwOe6MfPTcBSf34osBWYGiUdAdYA\nF0bcD+H+yD/112cAzw0g7+nAWuA84K1Y8wTM8HlKj0FvdwDdQJfX36Fezh8A7wN1wCKgNBDmQWAj\nznAuBfbw98+OiOuxSP0G0vyZP58L1AAX+Th/5+8fDbwONAJ/B/aMIv+uOMNfELi3FFgwgL4WAXcB\nLcAKYHbAfTVwqD/P8bI2AG8D/w3UBPx+DnjVx/MAcH9fvgbLg0/n+8AbXo8PANlRZN4ZWOL91eIq\nJH35VKDN6/skoAR4HNji5X4cmOL9Xwb0Ah3e/w3+/m7AX4B64F1gfgzvTQnwgT//NnBlhPt64LDA\n9U+B+xNVPoy1I+EC2DHEBwaHAz0MUCgTm4HaCJzZj5+D/Z87B7gCWDJAOrv5OHfox+1S4AV/fgYD\nG6hngCtxrcEeYFYseWIIBsr7vyOiYD0PWAZMAbKAm4H7Au7fAAq82zXA69HiitRvpB+cgeoBfuHj\ny8EV/JuBOUAa8HVcgZ7Vj+xfAVZG3LseuH6Ad6ADONLH/XNgWcB9Nf8yUFcAzwKlwFRc5aLGu2Xi\nKiHfAzKAeTjj3JevAfPgz1/CtdBLgZVEN6r34VqGISAbOHAA3ZbhWt25/hk9CDwacK8Gzgpc5+Eq\nQWcC6V7uWmD3KLIcgjO4rbiKSKP/bfPnB+GMlwIVgXAnAG8mqnwYa4d18aUeZUCtqvYM4m++iDQG\njwj3cmBDP+E24AqIUp9Wf36CcRDFz4aAe1REZBrOKN6rqptwxur0wcJFUBuR10/HGG4B8CNVrVHV\nTlyhPq+v+01Vb1PVloDbTBEpGqJsQcLAxaraqapbcS2xm1X1RVXtVdU7ca2kffsJm49rWQRpxhXO\n0XhOVZ9Q1V5ci3lmFH/zgctUtV5V1wLXBdz2xRmma1S1W1UfAl4OuMeSh+tUdb2q1gOPAXtFkaMb\n11U2WVU7VDXquKWq1qnqw6rarqotuFbTQdH841p5q1X1dlXtUdXXgIeBE6PE/4yqFgOPej9VOGNb\nrqrFqroE90xg2+cy2DMxhoAZqNSjDiiPYQxjkf8jfXxEuNcClf2Eq8QVpA0+rf78BOMgip/KgPtA\nnIZrGbzur+8BThGRjBjC9lEekdeVMYabDjwSMOArca3HChFJE5ErROR9EWnGFU4Qg9EdgC2q2hGR\n/gURlYipuNZGJK1AYcS9Ily3WzSCA/rtQHaU92YyrnXRx5oIt3WqqlHcY8lDpBz59M+FuG7jl/ws\nw29E8YeI5IrIzSKyxj+fpUCxiKRFCTIdmBMh56nApCjx13g/XwXuxLUSpwMbRORq763V/wafy2DP\nxBgCZqBSjxdwNdTjhhnP0/Rfe5yP65pr9372Cc7Ii+Bd3LjKNvGISAjX1fFMDHKcDuzoZ0BtBK7G\nGYEjY8rF8FgLHBFh3LJVdR1uUsixuLGqIlx3IrgCFFzXTiTtuC6nPiILv8gwa3Etl2D6uap6Xz9x\nr8DpKVg7n+nvD5cNOKPSx7QItyoRkSjuQ8nDgKjqRlX9lqpOBv4D+PUAM/cuAD4FzFHVQuAL/n60\n57MW110dlDNfVb8dRZYpuO70p33l7hbgOz7c+d5PA04/wZbpSD0TAzNQKYeqNgE/AW4UkeN8TTJD\nRI4QkSuHENWlwP4icpmIlIpIgYicizMYF/m0nsYNKj8iIrNEJN37WyAi3/C16u8DPxY37T1bRCYB\nt+Jqlb8KpCfePXjsB+wE7IPr9tkL+AxuVuBQu/niYSFwmYhM9wJOEJFjvVsBriJQhzM6l0eE3QTs\nGHHvdVzrL83P5BqoywngN8ACEZkjjjwROSrCCAGgqv/08V/sdXc88FlcN9VwWQT8UERKfGXk3IDb\nC7ixs+/69+x43PMach4GQ0RODFSGGnBGJuyvI/VdgJvA0ygipcDFEdFF+n8c2FVETvP5yBCRvQfp\nDp6FmxwC8HncTL5I7sK9/yU+rm/hxh6NEcAMVAqiqlcB5wM/xs1iWgucg+svjzWO94ADcTW+1bia\n4AnAl1T1+YDXecATuNlXTbgB9Nm41hXqpnSfhhtEr8PNAssBDlDVukA8++MKlODxTeAPqvqmrz1v\nVNWNwLXA0b7giYVG2fY7qFi/RbkWWIybWt+CmzAxx7vdhevKWufztCwi7G+B3X13UZ/ezwO+jBtE\nP5VBnoeqLscVaDfgCuRVuAkl0TgZp/sG3KSHeaq6ZdBcDs6luLx+CDyFG6/qk7ELON7LVY+bQff7\nYeRhIPYGXhSRVtxzOU9VP/BulwB3en3Px01aycF1Iy8D/hwR17W48cQGEbnOj1MdhtPhely3Y9+E\nlWjMAl71rcfd6L9ldDFuFuga3MSMK1U1UhYjTmTbrmXDMAzDSA6sBWUYhmEkJWagDMMwjKRkWAZK\nRA4XkXdFZJWI/KAf991E5AUR6RSR7w8nLcMwDGN8Ec96YAD47w1uBL6Im2r8sogsVtW3A97qge8y\n/CnRhmEYxjgjbgOFm2q6qm+WjYjcj/tu5GMDpaqbgc0ictRQIi4vL9cZM2YMQ7TE0tbWRl5eXqLF\nSElMd/Fjuosf0138xKO7V155pVZVB13oeDgGqoptvz6v4V9TdIfFjBkzWL68v08OUoPq6mrmzp2b\naDFSkmTWXUcTrFkCHzwNjR9C2adg4mfcMelzEIq2hsF2Ipl1l+yY7uInHt2JyJrBfQ3PQI0oInI2\nbl0vKioqqK6uTqxAw6C1tTWl5U8kyai79o9yWXX9zjS8WgJhIZTVS3ZlB+89mYN2u2HcvB1b2fnc\nVRTvFbnk4fYjGXWXKpju4mc0dTccA7WObZdHmeLvxYWq3oJbToTZs2drKtdmrDYWP8mku3APvHA1\nvPoTyMiFf/sh7PhFmLJvGulZeYR7oeF9+Oh5WHJJPv/43l7sPg+++H9QPGP7y5tMuks1THfxM5q6\nG46BehnYRUR2wBmmk3HrlxlGytPwATx0Mqx/GXb7Chz1a8iPWFkvlAZlu7rjMyfD338Jz/0cVj0J\nJ/8Bdjg4MbIbxlgh7mnmfruHc4AncatAL1LVFX6dtgUAIjJJRGrwy/L4FYIjV2Q2jKSi9l24/QtQ\nvwrmPQDzH/6kcYokIwcO+n/wnZVQNA3uORxW/n7gMIZhDMywxqBU9QncOm3BewsD5xtxXX+GkRJs\nXgF3HQIonLEEKj47tPDF0+HMpXDv0fDgiXDUTTDr7FER1TDGPLaShGF4Nr4Od84FCcVnnPrIKYXT\nn4adD4fH/wNeumFExTSMcYMZKMPAdefddSik57gWUPluw4svIxdOehR2Ow7+9F14eyQ2xTCMcYYZ\nKGPc014H9/jtEb/+VyiNtkXeEEnLgOPvhan7we9PhY+ibmBuGEZ/mIEyxjU9HfDAcdD0kZt5N1LG\nqY+MHDh5sRubuu8YqH1nZOM3jLGMGShj3KJh+MM3XMvmuDth2gGjk05uGZz6Z0jLhHuOgLaR2GLQ\nMMYBZqCMccuS/4W37oN/vxw+c9LoplWyA3z1MWjdCA/Og96u0U3PMMYCZqCMccnbD8OSS2Dm6XDg\nJzaKGR2q9oZjfgtrlsIT54JtZm0YA5M0a/EZxvZi4z/g0dNhyr5w9M0gsv3S/uwpsOlNeP4KqNgT\n9vnO9kvbMFINa0EZ44q2LXD/sZBdAvN/D+nZ21+GQy6DXb8Mfz4PPnhm+6dvGKmCGShj3NDTCYtO\ngLZNcPKjUFCZGDkkBMff7b61enCeW1rJMIxPYgbKGBeowmNnwUfPwrG3w+TZiZUnqxBOeRxCGXDf\n0e5bLMMwtsUMlDEuWPJTeONuOPhnbuXxZKB4hmvJNa11LTub2WcY22IGyhjzvHG3n7H3dfi3/0m0\nNNsydX8/s28JPHa2zewzjCA2i88Y03zwNCz+JsyYC1++ZfvO2IuVPU91awEuucQtNHvYVckpp2Fs\nb8xAGWOWNUvd8kJln3Iz9tIyEy1RdA76CWytg2W/guxid20Y4x0zUMaYZO0LcO9Rbg2805+GnJJE\nSzQwInD4NdDZDNUXQ1YR7HteoqUyjMRiBsoYc6xf7na0zZ8Epz8DeRMTLVFsSAiOuRW6WuDJ/3Jr\nBe73vURLZRiJwyZJGGOKf/4R7jzYbxr4VyiYnGiJhkYo3W3R8enj4anz4ckLnKEyjPGIGShjTKAK\ny66F+4+Bsl3hzOegaGqipYqP9CyYtwj2PgeWXe32kurpTLRUhrH9sS4+I+Xp7YI//xcsv8ntYPuV\nuyEzL9FSDY9QGhxxnTOyT18EjWvc6hMlOyZaMsPYflgLykhp1r0Mt8xyxmm/78P8h1PfOPUhAgdc\nCCfcD1tWwMKZ8Opv7VspY/xgBspISbq3wl8uhN/uC1sb4KuPw2H/5yYajDU+cxJ8+023PNNjZ7kd\ngBs+SLRUhjH6jMG/szGW6WqFF66G63eGv/8ffO6b8J8rYNejEi3Z6FI0zc1IPOwqeP8puOFT8Nh/\nuGWSDGOsYmNQRkpQ9x68eQ+8dD1srYcZB7vZbjMOSrRk2w8JwX7nu7UEn70cXrkF/nGH22Nqz9Os\n688YewzLQInI4cC1QBpwq6peEeEu3v1IoB04Q1VfHU6axvigpxM2vu5aCysfgk1vuPu7fhkO/CFM\n3S+x8iWSgslw5A1ufOrZy+HNe+H1OyCzfD86vwY7/Ltb4y+3LNGSGsbwiNtAiUgacCPwRaAGeFlE\nFqvq2wFvRwC7+GMOcJP/NQx6u9w2E60b3ZhK44fwz2d34b0LnXEKdzt/Uw+AL/3KfRtUNC2xMicT\nRdPg6IVON/98DP52bQvLf53Fi9c49/JPw6S93LT70l2gdCfIq3AfLo+ViSTG2GY4Lah9gFWq+gGA\niNwPHAsEDdSxwF2qqsAyESkWkUpV3TCMdAck3At//M/Rij02NqzflZb7EivDiBHRbfRxN5L6c39o\n2F1rrzsP9zoD09vtDFFvJ3S3Q1cbdLe5iQ1dLZ9MLj1/IlNmu66sqn1gyn6J21gwVcjIgT3mw5aJ\nb3HAnLmsXw4fPQdrn4OaF+Ct+/nEc0zPcWv+Zea7IyPXfX+VluXWLAyl+yMNJM3NKJQQIO4Q/wsR\nC9um6CK3Y+o/O8pMmgl7b6cydjgGqgoIDtHW8MnWUX9+qoBPGCgRORs4G6CiooLq6uq4hNJe4a2H\n9o0r7EgR1lLqZAx9WRlZ6ASuRfTjqTZ955KmiCiS7o5QRhhJV9Kye0mb2Et2dpj8gm4yCnvIKOwm\no7iL7Mkd5FR20EEj+fn5AGwCNr0L2I6zMdHa2srzL1a7i/1g8n4wGQh3CVs35NCxIZvuxky6GjLo\nbsykpy2N3q1pdG1No6M9jXBTCO0Wwt0htFfQsKC9AmFQ9Q89eA7bGr4UHgMbc//ZUWTdmgbadn/n\n4+vW1ta4y+vBSJpJEqp6C3ALwOzZs3Xu3Llxx3Vwgncnra6uZjjyj2dMd/Fjuosf091QmOQPx2jq\nbjgGah0QXExmir83VD+f4JVXXqkVkTXDkC3RlAO1iRYiRTHdxY/pLn5Md/ETj+6mx+JpOAbqZWAX\nEdkBZ3ROBk6J8LMYOMePT80BmmIZf1LVCcOQK+GIyHJVnZ1oOVIR0138mO7ix3QXP6Opu7gNlKr2\niMg5wJO4aea3qeoKEVng3RcCT+CmmK/CTTM/c/giG4ZhGOOBYY1BqeoTOCMUvLcwcK7Ad4aThmEY\nhjE+saWORodbEi1ACmO6ix/TXfyY7uJn1HQnauujGIZhGEmItaAMwzCMpMQMlGEYhpGUmIEyDMMw\nkhIzUIZhGEZSYgbKMAzDSErMQBmGYRhJiRkowzAMIykxA2UYhmEkJWagDMMwjKTEDJQxbhCRChFZ\nKiItInJVouUZTUTkEhG5ewD3FSIyN4rbXBGpGSDsHSLysxEQ0zAGxAxUiiIip4jIchFpFZENIvIn\nETnQu/VbOImIisjOgevdRWSxiDT5QvtvIrJ/RJhMH997ItImIqtF5DYRmRHwc7SIvOTd60TkHhGZ\nEnA/Q0SeGyQ/d4hIj4hUBu4t9PlrFZEuEekOXP9JRGb4PLVGHCdFSeZs3L41hap6wYAKHoREFNI+\nv38TkXYReUdEDo03LlXdQ1WrR1C8USXy3Y3Bf7WInDVCaS8SkcNEJEtENka4zReRv/tnUj0S6Rn/\nwgxUCiIi5wPXAJcDFcA04EbgmCHEsRPwPPAmsANud/BHgKdEZL+A14d8vKcARcBMYDlwiI9nHnCv\nl6cc2APoBJ4TkZIYZckDTgCagK/13VfVBaqar6r5Pq8P9F2r6hGBKIoD9/NV9YEoSU0H3tYkWIBS\nROLZSeA+4DWgDPgR8JCIpPTeaSnCLNw7vyfwVoRbPe7dv2J7CzUuUFU7UujAGYlW4MQB/FwC3N3P\nfQV29ue/A57ox89NwFJ/fiiwFZgaJR0B1gAXRtwP4f7IP/XXZwDPDSDv6cBa4DzgrVjzBMzweUqP\nQW93AN1Al9ffoV7OHwDvA3XAIqA0EOZBYCPOcC4F9vD3z46I67FI/QbS/Jk/nwvUABf5OH/n7x8N\nvA40An8H9owi/644w18QuLcUWDCAvhYBdwEtwApgdsB9NXCoP8/xsjYAbwP/DdQE/H4OeNXH8wBw\nf1++BsuDT+f7wBtejw8A2VFk3hlY4v3V4iokfflUoM3r+ySgBHgc2OLlfhyY4v1fBvQCHd7/Df7+\nbsBfcEblXWB+DO9NCfCBP/82cGUUf2cB1YkqF8bqYS2o1GM/IBvX2hkOX8QVwJEsAg4QkRxcIf6S\nqq6NEsencK23beJR1TDwsE8jFr6Oax3cD+wmIrNiDBczqnoGcA+ugMlX1aeBc4HjgINwLcgGXEu0\njz8BuwATcQX0PT6uWyLi+nKMYkwCSnEtubNF5HPAbcB/4FpFNwOLRSSrn7B74ArKlsC9f/j70TgG\np9Ni3O7WN0TxdzGwkz++hHsegOviBR7FVWhKcc/6hIB7LHmYDxyOa6nviauw9Mf/Ak/hjMIU4HoA\nVf2Cd5+p/2ohh4DbcbqchqtI3eD9/wh4FjjH+z/Ht9L/gmvtT8TtAP5rEdm9P0FE5BARacRVnKb4\n82uB74hIo4gcFCUPxghiBir1KANqVbVnEH/z/R/p4yPCvRzY0E+4Dbj3otSn1Z+fYBxE8bMh4B4V\nEZkGHAzcq6qbgGdwLaqhUBuR10/HGG4B8CNVrVHVTlyrY15f95uq3qaqLQG3mSJSNETZgoSBi1W1\nU1W34lpiN6vqi6raq6p34lpJ+/YTNh/XsgjSDBQMkN5zqvqEqvbiDMzMKP7mA5epar2vjFwXcNsX\nyACuUdVuVX0IeDngHkserlPV9apaDzwG7BVFjm6cwZmsqh2qGnXcUlXrVPVhVW33RvsyXEUjGkcD\nq1X1dlXtUdXXcJWoE6PE/4yqFuOM84lAFa41WK6qxaq6ZIC0jBHCDFTqUQeUxzCGscj/kT4+Itxr\ngcp+wlXiCtIGn1Z/foJxEMVPZcB9IE4DVqrq6/76HuAUEcmIIWwf5RF5XRljuOnAIwEDvhLXNVQh\nImkicoWIvC8izbjCCWIwugOwRVU7ItK/IKISMRXXmoukFSiMuFeE63aLRnBAvx3IjvLeTMa1FPpY\nE+G2TlU1insseYiUIz+KvBfiuo1f8rMMvxHFHyKSKyI3i8ga/3yWAsUikhYlyHRgToScp+Jatf3F\nX+P9fBW4E9js49ggIldHk8sYWcxApR4v4Gqoxw0znqfpv/Y4H3hBVdu9n32CM/IieBc3rrJNPCIS\nwnUDPRODHKcDO4rIRj9D6mqcETgyplwMj7XAERHGLVtV1+EmhRyL6+Yswo13gStAwY2JRNIO5Aau\nIwu/yDBrcS2XYPq5qnpfP3GvwOkp2GKa6e8Plw04o9LHtAi3KhGRKO5DycOAqOpGVf2Wqk7GdRn+\neoCZexfgupjnqGoh0NcNGO35rAWWRMiZr6rfjiLLFFy35NO+cncL8B0f7vyh5s2IDzNQKYaqNgE/\nAW4UkeN8TTJDRI4QkSuHENWlwP4icpmIlIpIgYicizMYF/m0nsb12z8iIrNEJN37WyAi3/C16u8D\nPxY37T1bRCYBt+Jq+78KpCfePXjshxv32AfX7bMX8BncOMFQu/niYSFwmYhM9wJOEJFjvVsBriJQ\nhzM6l0eE3QTsGHHvdVzrL01EDmfgLieA3wALRGSOOPJE5KgIIwSAqv7Tx3+x193xwGdx3VTDZRHw\nQxEp8ZWRcwNuLwA9wHf9e3Y87nkNOQ+DISInBipDDTgjE/bXkfouwI07NYpIKW4cLUik/8eBXUXk\nNJ+PDBHZe5Du4Fm4sUeAz+Nm8kXKnCYi2UA6EPLPZiitf2MAzEClIKp6FXA+8GPcLKa1wDm4/vJY\n43gPOBBXC1+NqymfAHxJVZ8PeJ0HPIGbfdWEm503G9e6wg9YnwZ8D1eYv42bFXaAqtYF4tkfV6AE\nj28Cf1DVN33teaOqbsQNRh/tC55YaJRtv4OKtYZ7LW7ywFMi0gIsA+Z4t7twXVnrfJ6WRYT9LbC7\n7y7q0/t5wJdxs9lOZZDnoarLgW/hBvcbgFVEn0AAbmB/tvf7c2Ceqm4ZNJeDcykurx/iJin8LiBj\nF3C8l6seN4Pu98PIw0DsDbwoIq2453Keqn7g3S4B7vT6no+b2p2D60ZeBvw5Iq5rceOJDSJynR+n\nOgynw/W4bsdfAP1NSOljFvCqbz3uRv+t1dNw7/JNwL/5898MKddGVGTbrmXDMAzDSA6sBWUYhmEk\nJWagDMMwjKTEDJRhGIaRlJiBMgzDMJKSeBasHHXKy8t1xowZiRYjbtra2sjLy0u0GCmJ6S5+THfx\nY7qLn3h098orr9Sq6qALHSelgZoxYwbLl3/ik4OUobq6mrlz5yZajJTEdDc01nZ383ZnJx/19PDi\nRx8RmjCBTBGyRcgJhZiSns7M7Gw+k5VFfsg6TKJh7138xKM7EVkzuK8kNVCGYfSPqrKyq4un29r4\na1sb73V3f+xWkJPDtHCYHlU6VGkPh9nS2wu45RU+lZnJ0fn5fDk/n/J0++sbyY+9pYaRAoRVqW5v\n59bGRv7R2UkImJWdzQ8KC5mdnc20jAxeXrr0EzXZpt5e3uzs5I3OTp5rb+eX9fVcU1/PF3JzOb2o\niL1zchKSH8OIBTNQhpHEqCp/amvjpoYGPujuZkp6Oj8qK+OI/HxK0qKti/ovitLSODA3lwNzc/nP\nkhLe7+rikZYWFre2csaGDRyQk8N/lZaye9ZACyoYRmIwA2UYSco/u7q4rLaW5R0d7JqZyZUTJ/Kl\nvDzSt1m3dWjslJnJ98vKOKekhPuam/lNYyMnrlvHEXl5XFhWxkTr+jOSiJhGTUXkcBF5V0RWicgP\n+nEXEbnOu78hIp8PuK0WkTdF5HURSd2ZD4axnWgLh/l5bS3zampY1dXFJeXlPFRVxVH5+cMyTkGy\nQyHOLC7myWnTWFBczF/b2zmmpoYHm5sJ2/JnRpIwqIHy+6vcCBwB7A58VT65C+URuJ1Hd8FtYHZT\nhPvBqrqXqs4evsiGMXZ5raODE2pquKe5mXkFBfxx6lROLCwkbYQMUyQFoRDnlpbyyJQp7JaZySW1\ntXxjwwY+Cky+MIxEEUsLah9glap+4Fc2vh+3T06QY4G71LEMt3HYQBvdGYYRoEuVa+vrOX39esLA\nHZWV/GTCBIpjGGcaCaZnZHB7ZSWXlpfzTlcX82pqeLxloL0QDWP0GXQ1cxGZBxyuqmf569Nwm4Sd\nE/DzOHBF3xbNIvIMcJGqLheRD3HbNPTitoa+JUo6Z+NaX1RUVMy6//77h525RNHa2kp+frRNQ42B\nGI+6q01L47fl5XyUlcW+ra3Ma2ggJ45utpHSXX1aGneUlfF+djZzWluZ39BA9hjv9huP791IEY/u\nDj744Fdi6VHbHiOiB6rqOhGZCPxFRN5R1aWRnrzhugVg9uzZmsofzdlHf/Ez3nT317Y2rtqyBQV+\nVV7OYTtG7oEYOyOpu2NUuaWxkZuADcXFXFNRwafG8Ey/8fbejSSjqbtYuvjWse120FP8vZj8+O2z\nUdXNwCNsuxunYYxLulW5sq6OczdtYkp6Og9WVXFYEtXg00X4z5IS7qispEOVU9av5w/W5WdsZ2Ix\nUC8Du4jIDiKSiduRcnGEn8XA6X42375Ak6pu8Ns/FwCISB5uR8u3RlB+w0g5tvT08M0NG7izqYmT\nCgu5e/IUw8xKAAAgAElEQVRkpmYk5y7hs3JyeLCqis9mZfE/W7ZwyZYtdIbDgwc0jBFg0C4+Ve0R\nkXOAJ4E04DZVXSEiC7z7QtyW4EfitntuB870wSuAR9yOyaQD96pq5NbMhjFueK2jg+9t2kRLOMwv\nJkzg6IKCRIs0KOXp6dxaWcl19fX8tqmJlV1dXFNRQaV9M2WMMjG9Yar6BM4IBe8tDJwr8J1+wn0A\nzBymjIaR8qgq9zU384u6OirT07l58uSUGtNJF+H8sjJmZmfzw82bObGmhl9WVLCvLZVkjCK2vLFh\njDLt4TA/3LKFy+rqOCA3lweqqlLKOAU5JC+PB6qqKE1L41sbNnBbYyODzQQ2jHgxA2UYo8ia7m5O\nXb+ex1tbObekhBsqKijaTt82jRY7ZGZyX1UVX8zL46r6es7zXZaGMdKYgTKMUeIvbW2ctG4dm3p6\nWDhpEgtKSgiN0ooQ25u8UIirJk7korIylrS3c2JNDe90diZaLGOMYQbKMEaYLlUur63lvzZtYkZG\nBg9WVXFgbm6ixRpxRITTi4q4ffJkOv1U9EXNzdblZ4wYZqAMYwT5qLubr61bxz3NzZxeVMRdkydT\nlaRTyEeKz2dn89CUKczKzubS2lou2LyZZr9RomEMBzNQhjECqCoPNzdzQk0Na3t6uK6igovKysgc\nI116g1GWlsbNkyZxfmkpz7S1ccK6dbza0ZFosYwUxwyUYQyTut5ezt20iZ/U1vLZrCx+P2UKh+Tl\nJVqs7U5IhG8WF/O7yZMJAV9fv55f1tXRYRMojDgxA2UYcaKqPNbSwrFr1/L81q1cVFbGrZWV4/4D\n1j2zs3l4yhROKCjg9qYm5q1bxz+sNWXEgRkow4iDNd3dfGvjRn6wZQtT/USI04uKxswsveGSHwpx\nyYQJ/GbSJLaGw3xt/Xr+t7aWRhubMobA+K7qGcYQaQuHua2xkdubmsgAflxWxvxR3FAw1dk/N5c/\nTJ3KdfX13NfczJOtrfxXaSnHFxSYMTcGxVpQhhED3arc39zMEWvXsrCxkX/PzeWxqVP5alGRGadB\nyA+F+J/ych6sqmLHzEwurq3lxHXrWNLeblPSjQGxFpRhDECXH2f6bVMTa7q7mZWdzQ0VFeyZnZ1o\n0VKO3bKyuLOykj+2tXFDfT3/uXEjM7OyOLekhH1zchAz9EYEZqAMox+ae3v5fUsLdzY1sbm3l90z\nM7m+ooKDc3OtIB0GIsLR+fl8KS+PR1taWNjQwFkbN/LpzEy+VlTEkfn542ZqvjE4ZqAMw6OqLO/o\n4PctLTzZ1kanKnOys7lswgT2sxr+iJIhwomFhRyTn88fWlu5u6mJH23ZwtX19XyloIAv5+ezc2Zm\nosU0EowZKGNc06vKPzo7eaatjWfa2ljb00O+CMfl5zOvsJDdU3TV8VQhKxRifmEhJxYUsGzrVu5u\nbub2xkZubWzkU5mZHJWfz8G5ueyQkWEVhHGIGShjXKGqfNTTw0tbt/JyRwfLtm6lrreXdGC/nBy+\nXVLCYXl55IRs/tD2RETYLzeX/XJzqevt5c+trTzW2srV9fVcXV9PVXo6B+bmsm92NntlZzNxnH9r\nNl6I6SmLyOHAtbgddW9V1Ssi3MW7H4nbUfcMVX01lrCGMVp0hMN81NPDh11drOzq4u3OTt7p6qLO\nf4tTnpbGPtnZ/HteHl/IzSXfjFJSUJaWxqlFRZxaVMT67m6e3bqVZ9vbWdzSwgPNzQBMTk9nz6ws\nds3MZJfMTHbOzKQqPd1mVI4xBjVQIpIG3Ah8EagBXhaRxar6dsDbEcAu/pgD3ATMiTGsYcRMjyqt\n4TBt4TDN4TANvb00+N9NPT1s6u1lc08Pa7u72Rj4KDQd2Ckzky/k5PDZ7Gz2yc5mhnUbJT2TMzI4\nKSODkwoL6VLlnc5OXu/s5PWODt7o7OTPbW0f+00HKtPTmZKRweT0dMrT0j4+itPSKAyFKPS/uSL2\nHVYKEEsLah9gld++HRG5HzgWCBqZY4G7/Nbvy0SkWEQqgRkxhB1RelW5vK5utKKPifUlJTxbW5tQ\nGYZKzF+jBL5b0Rh+VdX9+iPc96tKb+C3V5UeoHbiRG5dt44u3LdHXap0hMNsVaVDlc4BvpvJACrS\n06lIT2fvnBymZ2QwPSODGRkZ7JSRQZa1kFKaTBH2zM5mz+xsTi8qAtyH0+93dfFeVxdre3qo6e5m\nXU8PS9rbqe/tZaBVAHNEyA2FyBKht7KS62pqyBQhQ4R03Db3GSKEcN0/af485H8l8hChz+QFfyPN\n4DbXAxjJZDWfu2ZmclJh4XZJKxYDVQWsDVzX4FpJg/mpijEsACJyNnA2QEVFBdXV1TGI9kl6gcer\nquIKO1JoTg6vNjQkVIbRpL8/zsd/yIABifyzhgLuob5DFQHSVEkDNBxGmppIUyVHlQJVMlTJ9Ed2\nOOwOVXLCYfLDYfJ7e8kLh8kLh/v98nyzP8Y6ra2tcf9vUp0yf+wVuBcG2kIhmtLSaA+Ftjk6Rej0\nv90ibO3tRbu76RGhU4ReoEcEBXpFCANhf63+vC8N+GTlDEC9/2ik6ifKn+7ooCLQCBjN9y5pRhpV\n9RbgFoDZs2fr3Llz447rxRGSKV6qq6sZjvzjGdNd/Jju4sd0Fz+jqbtYDNQ6YGrgeoq/F4ufjBjC\nfoJXXnmlVkTWxCBbslIOpFYfX/Jguosf0138mO7iJx7dTY/FUywG6mVgFxHZAWdcTgZOifCzGDjH\njzHNAZpUdYOIbIkh7CdQ1QmxCJ+siMhyVZ2daDlSEdNd/Jju4sd0Fz+jqbtBDZSq9ojIOcCTuLHC\n21R1hYgs8O4LgSdwU8xX4aaZnzlQ2NHIiGEYhjG2iGkMSlWfwBmh4L2FgXMFvhNrWMMwDMMYDJt3\nOzrckmgBUhjTXfyY7uLHdBc/o6Y7sf1YDMMwjGTEWlCGYRhGUmIGyjAMw0hKzEAZhmEYSYkZKMMw\nDCMpMQNlGIZhJCVmoAzDMIykxAyUYRiGkZSYgTIMwzCSEjNQhmEYRlJiBsoYN4hIhYgsFZEWEbkq\n0fKMJiJyiYjcPYD7ChGZG8VtrojUDBD2DhH52QiIaRgDYgYqRRGRU0RkuYi0isgGEfmTiBzo3fot\nnERERWTnwPXuIrJYRJp8of03Edk/Ikymj+89EWkTkdUicpuIzAj4OVpEXvLudSJyj4hMCbifISLP\nDZKfO0SkR0QqA/cW+vy1ikiXiHQHrv8kIjN8nlojjpOiJHM2bt+aQlW9YEAFD0IiCmmf37+JSLuI\nvCMih8Ybl6ruoarVIyjeqBL57sbgv1pEzhqhtBeJyGEikiUiGyPcfun/Gy3+mZw+EmkaDjNQKYiI\nnA9cA1wOVADTgBuBY4YQx07A88CbwA7AZOAR4CkR2S/g9SEf7ylAETATWA4c4uOZB9zr5SkH9gA6\ngedEpCRGWfKAE4Am4Gt991V1garmq2q+z+sDfdeqekQgiuLA/XxVfSBKUtOBtzUJFqAUkXh2s74P\neA23u/mPgIdEJKX3TksRZuHe+T2BtyLc2oAv4/4bXweujazkGcNAVe1IoQP3R2gFThzAzyXA3f3c\nV2Bnf/474Il+/NwELPXnhwJbgalR0hFgDXBhxP0Q7o/8U399BvDcAPKeDqwFzgPeijVPwAyfp/QY\n9HYH0A10ef0d6uX8AfA+UAcsAkoDYR4ENuIM51JgD3//7Ii4HovUbyDNn/nzuUANcJGP83f+/tHA\n60Aj8Hdgzyjy74oz/AWBe0uBBQPoaxFwF9ACrABmB9xXA4f68xwvawPwNvDfQE3A7+eAV308DwD3\n9+VrsDz4dL4PvOH1+ACQHUXmnYEl3l8trkLSl0/FGYNW4CSgBHgc2OLlfhyY4v1fBvQCHd7/Df7+\nbsBfgHrgXWB+DO9NCfCBP/82cOUg/hcDF2zvcmGsHgkXwI4hPjA4HOhhgEKZ2AzURuDMfvwc7P/c\nOcAVwJIB0tnNx7lDP26XAi/48zMY2EA9A1yJaw32ALNiyRNDMFDe/x0RBet5wDJgCpAF3AzcF3D/\nBlDg3a4BXo8WV6R+I/3gDFQP8AsfXw6u4N+M24U6DVcDXw1k9SP7V4CVEfeuB64f4B3owG0kmgb8\nHFgWcF/NvwzUFcCzQCkwFVe5qPFumbhKyPeADGAezjj35WvAPPjzl3At9FJgJdGN6n24lmEIyAYO\nHEC3ZbhWd65/Rg8Cjwbcq4GzAtd5uErQmbh98D6HM4K7R5HlEJzBbcVVRBr9b5s/P6ifMDnABuDw\n7VkmjOXDuvhSjzKgVlV7BvE3X0Qag0eEeznuzxTJBlwBUerT6s9PMA6i+NkQcI+KiEzDGcV7VXUT\nzlgNtR+/NiKvn44x3ALgR6pao6qduEJ9Xl/3m6repqotAbeZIlI0RNmChIGLVbVTVbfiWmI3q+qL\nqtqrqnfiWkn79hM2H9eyCNKMK5yj8ZyqPqGqvbgW88wo/uYDl6lqvaquBa4LuO2LM0zXqGq3qj4E\nvBxwjyUP16nqelWtBx4D9ooiRzeuG3ayqnaoatRxS1WtU9WHVbVdVVtwraaDovnHtfJWq+rtqtqj\nqq8BDwMnRon/GVUtBh71fqpwxrZcVYtVdUk/wRYC/8DtIG6MAGagUo86oDyGMYxF/o/08RHhXgtU\n9hOuEleQNvi0+vMTjIMofioD7gNxGq5l8Lq/vgc4RUQyYgjbR3lEXlfGGG468EjAgK/EtR4rRCRN\nRK4QkfdFpBlXOEEMRncAtqhqR0T6F0RUIqbiWhuRtAKFEfeKcN1u0QgO6LcD2VHem8m41kUfayLc\n1qmqRnGPJQ+RcuRHkfdCXLfxS36W4Tei+ENEckXkZhFZ45/PUqBYRNKiBJkOzImQ81RgUpT4a7yf\nrwJ34lqJ04ENInJ1P/7/D/gMrtsw4WOcYwUzUKnHC7ga6nHDjOdp+q89zsd1zbV7P/sEZ+RF8C5u\nXGWbeEQkhOt+eSYGOU4HdhSRjX6G1NU4I3BkTLkYHmuBIyKMW7aqrsNNCjkWN1ZVhOtOBFeAguty\niqQd1+XUR2ThFxlmLa7lEkw/V1Xv6yfuFTg9BVtMM/394bIBZ1T6mBbhViUiEsV9KHkYEFXdqKrf\nUtXJwH8Avx5g5t4FwKeAOapaCHzB34/2fNbiuquDcuar6rejyDIF153+tK/c3QJ8x4c7P+hXRC4F\njgAOU9XmoeXaGAgzUCmGqjYBPwFuFJHjfE0yQ0SOEJErhxDVpcD+InKZiJSKSIGInIszGBf5tJ7G\nDSo/IiKzRCTd+1sgIt/wNcXvAz8WN+09W0QmAbfiavu/CqQn3j147AfsBOyD6/bZC1cLvZehd/PF\nw0LgMhGZ7gWcICLHercCXEWgDmd0Lo8IuwnYMeLe67jWX5qIHM7AXU4AvwEWiMgcceSJyFERRggA\nVf2nj/9ir7vjgc/iuqmGyyLghyJS4isj5wbcXsCNnX3Xv2fH457XkPMwGCJyYqAy1IAzMmF/Hanv\nAtwEnkYRKQUujogu0v/jwK4icprPR4aI7D1Id/As3OQQgM/jZvJFyvxDXGXmUFWtGzSTxpAwA5WC\nqOpVwPnAj3GzmNYC5+D6y2ON4z3gQFwtfDWupnwC8CVVfT7gdR7wBG72VRNuAH02rnWFuindp+EG\n0etws8BygAMi/rD74wqU4PFN4A+q+qavPW9U1Y3AtcDRvuCJhUbZ9juo8wcPAj6dxbip9S24CRNz\nvNtduK6sdT5PyyLC/hbY3XcX9en9PNyU477uowGfh6ouB74F3IArkFfhJpRE42Sc7htwkx7mqeqW\nQXM5OJfi8voh8BRuvKpPxi7geC9XPW4G3e+HkYeB2Bt4UURacc/lPFX9wLtdAtzp9T0fN2klB9eN\nvAz4c0Rc1+LGExtE5Do/TnUYTofrcd2OfRNWojELeNW3Hnej/9bq5bgW5arA+/c/Q8240T9i3aWG\nYRhGMmItKMMwDCMpMQNlGIZhJCVmoAzDMIykxAyUYRiGkZTEs2DlqFNeXq4zZsxItBhx09bWRl5e\nXqLFSElMd/Fjuosf0138xKO7V155pVZVB13oOCkN1IwZM1i+/BOfHKQM1dXVzJ07N9FipCTjRXft\nHTU0tLxKfs7OFOZ9mm2/g42P8aK70cB0Fz/x6E5E1gzuK0kNlGGMVbp7mthU/wxNrW8CIRpbXiM3\nayoVZV8iN7sq0eIZRlJhBsowthONLf9gfe3jgFJefCDlRQfQ1LaCzQ1/48P1t1JaOIfK8sMTLaZh\nJA1moAxjO9DZXcf62sfJyZpM1YSvkJnh1u4tLZxFUf5n2FT3FPXNL5KbPY2i/N0TLK1hJAc2i88w\nRhnVMOs3/wGRdKZMnPexceojLZRFZfmRZGdWsqH2j/T0tiVIUsNILsxAGcYoU9f0Iu2da6ksO5yM\n9P7XUBVJo2ricYTDnWyo/SO2BJlhmIEyjFGls6uWzQ1/pSB3V4ry9xzQb3bmRCaUzqW5bSXNbSOx\ni4ZhpDZmoAxjlFBV1m1ZTEjSqSw/Oqap5OVF+5OTVcWG2iesq88Y95iBMoxRorltBVs71zKp7EtR\nu/YiEQkxecKx9IY72NLw7ChLaBjJjRkowxgFVMNsbqgmK2MiRfkzhxQ2O3MCxQV70dC8nO6eplGS\n0DCSn2EZKBE5XETeFZFVIvKDftx3E5EXRKRTRL4/nLQMI5Voan2Dru46JpbOjWuViAnFXwDUWlHG\nuCZuAyUiacCNwBHA7sBXRSTyA4564LvAL+OW0DBSjLD2srlhCdmZlRTk7hZXHJkZxZQUzqKh5TW6\nuhtGWELDSA2G04LaB1ilqh/4baHvB44NelDVzar6MtA9jHQMI6VobHmN7p5GJpYePKw19sqL/w2R\nEFsaloygdIaROgxnJYkqYG3gugaYE29kInI2cDZARUUF1dXVwxAtsbS2tqa0/Ikk9XUXpnTyS4R7\nC3nlpRpg3bBiyyueRDj8Dz74Zya9PbkD+k193SUO0138jKbukmapI1W9BbgFYPbs2ZrKKwvbysjx\nk+q6q2taxsa6TnaaehJ5n95h2PH19O7Nex9dy4xd2placeSAflNdd4nEdBc/o6m74XTxrQOmBq6n\nMNzqomGkML3hTrY0PEte9g7k5QzfOAGkp+VRWrSvn7K+YUTiNIxUYTgG6mVgFxHZQUQygZOBxSMj\nlmGkHnWNL9Abbqei9JARjbe8eH/SQjlsqn9mROM1jGQnbgOlqj3AOcCTwEpgkaquEJEFIrIAQEQm\niUgNcD7wYxGpEZHCkRDcMJKJnt426ppeoDBvd3JGeF+ntFA25cX/RtvW92nd+uGIxm0YycywxqBU\n9QngiYh7CwPnG3Fdf4YxptnSsJSwdjOx5OBRib+0cG/qm5axuf4Z8iZ/c0R24DWMZMdWkjCMYdLV\n3UBD83JKCj5HVmb5qKQRCqUzoXQuWzvX0dK2clTSMIxkwwyUYQyTzQ3VICEmlBw0qukU588kK6Oc\nTQ1/RTU8qmkZRjJgBsowhkHb1tU0tb5BWeEcMtJHd3hVJMTE0kPo6q6jtunvo5qWYSQDZqAMI056\nw52s2/IomemlTCj5wnZJsyD3UxTm7c6W+mo6ujZtlzQNI1GYgTKMONlY9yTdPc1UTTyOUChzu6Qp\nIlSWH0UoLZt1mx8lrL3bJV3DSARmoAwjDlra/0ljy2uUF+1PbvbUwQOMIOlpuUwu/zIdXRttnT5j\nTGMGyjCGSE/vVtZveYysDLdFeyIozPsUxfkzqW18jvYOW8DFGJuYgTKMIdDb28FHG++lp7fdde1J\n4paznFR2OBlpBazddD8dXVsSJodhjBZmoAwjRnp621m94S46OtcztWIeOVmVCZUnLS2baZVfA2D1\n+jtIy2hNqDyGMdKYgTKMGOjpaWX1+jvp7N7M1EknU5j36USLBLjt4WdMPgORdIon/oOt1t1njCHM\nQBnGAKgqTa0r+GD9b+nqaWDapFMoyN0l0WJtQ1ZGGTtMPgPVdFZvuIvaxucJh3sSLZZhDJthGSgR\nOVxE3hWRVSLyg37cRUSu8+5viMjnh5OeYWxP2rZ+yIfrb6Vm80OEJIMZlaeRn7NjosXql8yMEho3\n7UVu9jQ21T/NqrXX09Dyuq04YaQ0cY/wikgacCPwRdxuui+LyGJVfTvg7QhgF3/MAW5iGLvuGsZo\nodpLd08LWztraN36AW1bP6S7p5H0tEImTziW4vw9EUnuDodwbxbTK0+lbeuHbKx/mvVb/sCmuqfI\ny3H7U+VmTyczvZhQKCPRohpGTAxnCtI+wCpV/QBARO4HjgWCBupY4C5VVWCZiBSLSKWqjtrOa6ph\nNtY9OVrRx0ReyTo21G5NqAypSvy6037u+Huq/iqMqqLag2o34XA3vdpJT08zPb3/mmAQCmWRlz2D\n8uIDKM6fmXIFel7ODuw4+Sxa2t+hpe1dWrd+SHPbv/6WaaEcMtILCYVyCEkGoVAGIukIIRD5+LcP\nIXLl9LG3krr9Z2MnK3MipYWztktawzFQVcDawHUNn2wd9eenCviEgRKRs4GzASoqKoaxx71SVvVq\nnGFHhqxcpbbBlqGJh5HXnStMVSVwHkI1DcLut7c3n3BvGeGeLHq68+jpKvB+W4HnR1CW0aW1tbWf\n/00xsBdp6VtJz2omLa2TUFoXobROJNSASC8SCiMSxhl5RSRo7D9p+Mci9p+Nna6OUt6oa/n4uv/3\nbmRI3EccEajqLcAtALNnz9bh7XE/OnvyxEp1dTXDk3/8YrqLH9Nd/Jju4mc0dTccA7UOCK7xMsXf\nG6qfT/DKK6/UisiaYciWaMqB2kQLkaKY7uLHdBc/prv4iUd302PxNBwD9TKwi4jsgDM6JwOnRPhZ\nDJzjx6fmAE2xjD+p6oRhyJVwRGS5qs5OtBypiOkufkx38WO6i5/R1F3cBkpVe0TkHOBJIA24TVVX\niMgC774Qtx38kcAqoB04c/giG4ZhGOOBYY1BqeoTOCMUvLcwcK7Ad4aThmEYhjE+Se4PO1KXWxIt\nQApjuosf0138mO7iZ9R0J66RYxiGYRjJhbWgDMMwjKTEDJRhGIaRlJiBMgzDMJISM1CGYRhGUmIG\nyjAMw0hKzEAZhmEYSYkZKMMwDCMpMQNlGIZhJCVmoAzDMIykxAyUMW4QkQoRWSoiLSJyVaLlGU1E\n5BIRuXsA9xUiMjeK21wRqRkg7B0i8rMRENMwBsQMVIoiIqeIyHIRaRWRDSLyJxE50Lv1WziJiIrI\nzoHr3UVksYg0+UL7byKyf0SYTB/feyLSJiKrReQ2EZkR8HO0iLzk3etE5B4RmRJwP0NEnhskP3eI\nSI+IVAbuLfT5axWRLhHpDlz/SURm+Dy1RhwnRUnmbNy+NYWqesGACh6ERBTSPr9/E5F2EXlHRA6N\nNy5V3UNVq0dQvFEl8t2NwX+1iJw1QmkvEpHDRCRLRDZGuF0pImtFpFlE1ojI/4xEmobDDFQKIiLn\nA9cAlwMVwDTgRuCYIcSxE24/8zeBHYDJwCPAUyKyX8DrQz7eU4AiYCawHDjExzMPuNfLUw7sAXQC\nz4lISYyy5AEnAE3A1/ruq+oCVc1X1Xyf1wf6rlX1iEAUxYH7+ar6QJSkpgNvaxIsQCki8ewkcB/w\nGlAG/Ah4SERSeu+0FGEW7p3fE3grwu02YA9VLQT2B04VkeO3s3xjF1W1I4UOnJFoBU4cwM8lwN39\n3FdgZ3/+O+CJfvzcBCz154cCW4GpUdIRYA1wYcT9EO6P/FN/fQbw3ADyng6sBc4D3oo1T8AMn6f0\nGPR2B9ANdHn9Herl/AHwPlAHLAJKA2EeBDbiDOdSXEEEriUWjOuxSP0G0vyZP58L1AAX+Th/5+8f\nDbwONAJ/B/aMIv+uOMNfELi3FFgwgL4WAXcBLcAKYHbAfTVwqD/P8bI2AG8D/w3UBPx+DnjVx/MA\ncH9fvgbLg0/n+8AbXo8PANlRZN4ZWOL91eIqJH35VKDN6/skoAR4HNji5X4cmOL9Xwb0Ah3e/w3+\n/m7AX4B64F1gfgzvTQnwgT//NnDlAH6rcBW+CweL147YjoQLYMcQHxgcDvQwQKFMbAZqI3BmP34O\n9n/uHOAKYMkA6ezm49yhH7dLgRf8+RkMbKCeAa7EtQZ7gFmx5IkhGCjv/46IgvU8YBkwBcgCbgbu\nC7h/AyjwbtcAr0eLK1K/kX5wBqoH+IWPLwdX8G/G7TadBnwdV6Bn9SP7V4CVEfeuB64f4B3owG0Y\nmgb8HFgWcF/NvwzUFcCzQCkwFVe5qPFumbhKyPeADGAezjj35WvAPPjzl3At9FJgJdGN6n24lmEI\nyAYOHEC3ZbhWd65/Rg8Cjwbcq4GzAtd5uErQmbh98D6HM4K7R5HlEJzBbcVVRBr9b5s/Pyjg9wfe\nnwIf4A2lHcM/rIsv9SgDalW1ZxB/80WkMXhEuJcDG/oJtwFXQJT6tPrzE4yDKH42BNyjIiLTcEbx\nXlXdhDNWpw8WLoLaiLx+OsZwC4AfqWqNqnbiCvV5fd1vqnqbqrYE3GaKSNEQZQsSBi5W1U5V3Ypr\nid2sqi+qaq+q3olrJe3bT9h8XMsiSDOucI7Gc6r6hKr24lrMM6P4mw9cpqr1qroWuC7gti/OMF2j\nqt2q+hDwcsA9ljxcp6rrVbUeeAzYK4oc3bhu2Mmq2qGqUcctVbVOVR9W1XZVbcG1mg6K5h/Xylut\nqrerao+qvgY8DJwYJf5nVLUYeNT7qcIZ23JVLVbVJQG/V+Cew+dxeo58TkacmIFKPeqA8hjGMBb5\nP9LHR4R7LVDZT7hKXEHa4NPqz08wDqL4qQy4D8RpuJbB6/76HuAUEcmIIWwf5RF5XRljuOnAIwED\nvhLXeqwQkbT/3969B0lRnnsc//52Abl4A/FAwl3lmKDxEhCO4cRQ0RPBG144xhteAAkGCFakokTL\nYFka4l0LEVG8heQYqoyRolADCqmUBcolUYNEQREBRRAFXEDWXZ7zx9Mrw7DL7s7sMDO7z6dqane6\ne3MatHsAAA9aSURBVLuffnZ3nu63335b0iRJ70vahn84QR2K7n5sMrOv0rZ/Q9pBRBf8bCNdGXBo\n2rTD8Ga3mqRe0N8BtKzh7+bb+NlFlTVp89abmdUwvy77kB7HwTXE+yu82fiNpJfhsBqWQ1JrSY8m\nHRO24c2Ah0sqreFHugH90uK8HOhYw/rXJctcCjyNnyV2Az6RdF/68ub+gTeJ31ZT3KF+okAVn4X4\nEer5Wa5nHtUfPV6MN83tSJbpm9ojL827+HWVvdYjqQRvfnmlDnFcCRwlaUPSQ+o+vAicVae9yM5a\nYFBacWtpZuvxTiGD8WtVh+HNieAfoODNOel24E1OVdI//NJ/Zi1+5pK6/dZm9n/VrHs5nqfUM6YT\nk+nZ+gQvKlW6ps3rJEk1zK/PPuyXmW0ws2vN7NvAz4Ap++m5dwNwLNDPvIPCacn0mn4/a/Hm6tQ4\nDzaz62qIpTPenD4vObibBoxOfu6X+9mNZsDRte1rqJsoUEXGzLYCtwIPSzo/OZJsLmmQpLvqsarb\ngB9IukNSO0mHSBqLF4wbk23Nwy8qPy+pt6RmyXKjJA1LjqrHA7fIu723lNQReBw/2r8/ZXtK5qe+\nTsX/mfvizT4nAcfjvQLr28yXianAHZK6JQEeKWlwMu8Q/EBgM1507kz72U+Bo9Km/RM/+yuVNJD9\nNzkBPAaMktRPro2ks9OKEABm9l6y/t8kubsQ+B7eTJWtmcAESW2Tg5GxKfMW4tfOfpH8nV2I/77q\nvQ+1kfS/KQdDX+BFZnfyPj3fh+BnK1sktQN+k7a69OVnA/8paWiyH80lnVJLc3BvvHMIePPdkrR4\nSyT9LMmbJPUFRlO3A7NQB1GgipCZ3Qv8ErgF78W0FhiDt5fXdR0rgf/Gj8I/xI+ULwLONLPXUhYd\nAszBe19txS+g98HPrjDv0j0Uv4i+Ge8F1grob2abU9bzA/wDJfU1HHjBzN5Ojp43mNkG4EHgnOSD\npy62aO/7oPZ3hJvqQWAW3rX+S7zDRL9k3jN4U9b6ZJ8Wpf3sdKBX0lxUlfdxwLn4RfTLqeX3YWZL\ngGuByfgH8iq8Q0lNLsFz/wXe6WGImW2qdS9rdxu+r6uBv+LXUapiLAcuTOL6HO9B9+cs9mF/TgFe\nl1SG/17GmdkHybyJwNNJvi/GO620wpuRFwEvpa3rQfx64heSHkquU/0Ez+HHeLNjVYeVmvQGliVn\nj9+h+rPVC/BeoF8CM0g6rtRrr0ONtHfTcgghhFAY4gwqhBBCQYoCFUIIoSBFgQohhFCQokCFEEIo\nSJkMWJlz7du3t+7du+c7jIxt376dNm3a5DuMohS5y1zkLnORu8xlkrulS5d+Zma1DnRckAWqe/fu\nLFmypPYFC9SCBQsYMGBAvsMoSo0qd++8A3ffDbt3Q8eO/urbF/r3z8nmGlXuDrDIXeYyyZ2kNbUv\nVaAFKoSitmULTJwIkydD69bQrh1s2AC7dvn88ePhzjuheX1Gcwqh6YlrUCE0pBdfhJ494aGHYMQI\neP99+PBD2LkTNm+Gn/8c7rkHfvQj+OijfEcbQkGLAhVCQ1m4EC68EDp1gqVLYepUODJpZpf8TOrh\nh+FPf4J//QtOOgneeCO/MYdQwLIqUJIGSnpX0ipJN1Uz/zuSFkraJWl8NtsKoaCtXAnnnuvFae5c\nOPnkmpe9+GJYtgwOOwwuuMCb/0II+8i4QCXD2j8MDAJ6AZdK6pW22OfAL4B7Mo4whEK3cSMMGuRn\nSS+9tOesaX+OOQb+8hf44gsYMgTKy3MfZwhFJpszqL7AKjP7IBlQ8ln88QTfMLONZrYYfxBZCI1P\neTmcdx58/DHMnu2Fp65OPBGeeAJeew2uvz53MYZQpLLpxdeJvR9yto49I0HXm6SR+NM56dChAwsW\nLMgitPwqKysr6vjzqdhy12P6dLq9/jrLJ05k086dUN/YO3bkqJ/+lK6PPMK/27Rhw9lnZxxLseWu\nkETuMpfT3GX6rHj8MQyPp7wfCkyuYdmJwPi6rrt3795WzObPn5/vEIpWUeVu0SKzkhKzq6/Obj0V\nFWZnnGHWqpXZypUZr6aocldgIneZyyR3wBKrQy3IpolvPXs/hbNzMi2Exm/HDrjySujcGR54ILt1\nlZbCU09BixYwbJjf2BtCyKpALQZ6SuohqQX+ILBZDRNWCAVuwgR47z148knvjZetTp280P39794V\nPYSQeYEyswr8Ka4vAyuAmWa2PHkc+CgASR0lrSN5+qukdZIObYjAQ8ib+fP9RtyxY+HHP2649V51\nlfcGvOkmv8E3hCYuq6GOzGwO/jjw1GlTU77fgDf9hdA4bN8Ow4d7b71Jkxp23RJMmwbHHefbePVV\nKIl76UPTFX/9IdTHLbfA6tUwfbqPs9fQOneG+++Hv/0NHnmk4dcfQhGJAhVCXS1cCA8+6OPpnXZa\n7rZzzTVw5plw441eDENooqJAhVAXu3Z5s1vnzg3ftJeuqqmvpASuvRb8Vo0QmpwoUCHUxe23w4oV\nXjgOOST32+va1Z8l9cor8Nhjud9eCAUoClQItVm0yM+arrwSBg48cNsdOdJ7CY4fH4/mCE1SFKgQ\n9mfbNrjsMujSxbuWH0gSPP6437g7fHjcwBuanChQIezPmDGwZg3MmNEwN+TWV48e3qtv3jz47W8P\n/PZDyKMoUCHU5I9/hN//Hm69Ffr3z18cI0b4Wdytt9Z/MNoQilgUqBCq88EHcN11Xphuvjm/sUjw\n6KP+KPlLL4VPP81vPCEcIFGgQki3eTOcdZZ3854xA5plNeBKwzj4YJg5E7ZsgSuugMrKfEcUQs5F\ngQoh1c6d/gDCDz+EWbOge/d8R7THCSfA5Ml+PWrUqOg0ERq9Ajg0DKFAVFbC5Zf7iBEzZ8IPf5jv\niPY1fLh32rj9dmjVyke2kPIdVQg5EQUqBPDiNHo0PP+8P/ZiyJB8R1Sz226DsjLv3demDdx5ZxSp\n0ChFgQph61bvJTdnjo9/N25cviPaPwnuvdcfmjhpkjf13XFHvqMKocFFgQpN28qVfs1p1SqYMsV7\n7hUDyeMFuOsuWLiQg8aOzW9MITSw6CQRmqaKCh+loW9f2LQJ5s4tnuJUpaQEpk6FZ56BZcvoM2IE\nzJ6d76hCaDBRoELTYua98044wUcK/+53YfFiGDAg35FlbuhQWLqUXUceCeeeC2ec4Q87jFHQQ5HL\nqkBJGijpXUmrJN1UzXxJeiiZ/5ak72ezvRAy9v778LvfQe/eMHiwd4p47jl47TUfTqjYHXssy6ZM\n8RHQly+H00+HU0+FJ5+ETz7Jd3QhZCTja1CSSoGHgf8B1gGLJc0ys3dSFhsE9Exe/YBHkq8h5EZl\nJXz2mV9beustfy1aBG++6fNPOcVHZRg2rDBuwG1Au1u08JHPx4yBp57ya1PDhvnME0/0onXccXDs\nsf464ojo/RcKWjb/oX2BVWb2AYCkZ4HBQGqBGgw8Y2YGLJJ0uKRvmVnuDukqK/2fNI+OXrcOXngh\nrzEUlZSmqGPWrvWu3lXTzLyX2u7d/rutqICvv/avX30F27f768svYeNGv56UegNr27Zw0kne6+2i\ni6BbtwO8c3nQsqXfyDtypBfol1+Gl17ym3zLy/cs16KFF6kjjvA8tW7t91a1agXNm3sBb9YMSku9\nkJWU+Nf0opb6vkgLXvzP1sPxx/v9eAdANgWqE7A25f069j07qm6ZTsA+BUrSSGAkQIcOHViQ4aCY\nqqykf54f8NbRjIoi/UfNtw7A11W5S75aSQmUlGASVlqKNWuGlZayu3lzKlu1YnfLllQeeijlXbtS\n3q4d5W3b8lXHjmw/5hh2tW+/50Nz9epG/Qj1srKy6v9v+vWDfv1QZSUtN2yg1Ucf0XrdOlp8/jnN\nt22j+datNNu2jZJNmygpL6dk1y5KKipg925UWemvqgOGtNEr1Eiuc8X/bN1t7tePFUcf/c37Gv/u\nGkDBtHGY2TRgGkCfPn1sQDYXrcvKGiaoDC1YsICs4m/CIneZi9xlLnJXdx2SV5Vc5i6bArUe6JLy\nvnMyrb7L7GPp0qWfSVqTRWz51h74LN9BFKnIXeYid5mL3GUuk9zVqa09mwK1GOgpqQdedC4BLktb\nZhYwJrk+1Q/YWpfrT2Z2ZBZx5Z2kJWbWJ99xFKPIXeYid5mL3GUul7nLuECZWYWkMcDLQCnwhJkt\nlzQqmT8VmAOcBawCdgDXZB9yCCGEpiCra1BmNgcvQqnTpqZ8b8DobLYRQgihaYqRJHJjWr4DKGKR\nu8xF7jIXuctcznInayTdREMIITQucQYVQgihIEWBCiGEUJCiQOWApBskmaT2KdMmJIPmvivpzHzG\nV4gk3S3p38mgws9LOjxlXuSuFrUN3BycpC6S5kt6R9JySeOS6e0kzZW0MvnaNt+xFipJpZL+IWl2\n8j5nuYsC1cAkdQF+AnyUMq0Xfp/YccBAYEoy2G7YYy5wvJmdALwHTIDIXV2kDNw8COgFXJrkLeyr\nArjBzHoB/wWMTnJ1E/CKmfUEXkneh+qNA1akvM9Z7qJANbz7gV8Bqb1PBgPPmtkuM1uN3xfWNx/B\nFSoz+6uZVSRvF+GjjkDkri6+GbjZzMqBqoGbQxoz+8TMliXff4l/0HbC8/V0stjTwPn5ibCwSeoM\nnA08njI5Z7mLAtWAJA0G1pvZm2mzaho0N1RvGPBi8n3krnaRowxI6g6cDLwOdEgZ5WYDew83F/Z4\nAD8ATx01OGe5K5jBYouFpHlAx2pm3Qz8Gm/eC9XYX+7M7IVkmZvxZpg/HMjYQtMi6WDgOeB6M9um\nlJHMzcwkxf03aSSdA2w0s6WSBlS3TEPnLgpUPZnZGdVNl/Q9oAfwZvLH3hlYJqkvGQ6a29jUlLsq\nkq4GzgFOtz036EXuahc5qgdJzfHi9Acz+3My+dOqZ9VJ+hawMX8RFqz+wHmSzgJaAodKmkEOcxdN\nfA3EzN42s/8ws+5m1h1vZvm+mW3AB829RNJByeC6PYE38hhuwZE0EG86OM/MdqTMitzV7puBmyW1\nwDuVzMpzTAVJfvQ4HVhhZvelzJoFXJV8fxUQTy9MY2YTzKxz8vl2CfCqmV1BDnMXZ1AHQDKI7kz8\nacMVwGgzq8xzWIVmMnAQMDc5A11kZqMid7WraeDmPIdVqPoDQ4G3Jf0zmfZrYBIwU9JwYA1wcZ7i\nK0Y5y10MdRRCCKEgRRNfCCGEghQFKoQQQkGKAhVCCKEgRYEKIYRQkKJAhRBCKEhRoEIIIRSkKFAh\nhBAK0v8DN14hHqYhn8YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11406ca20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXl4VdW5+P95T2YyEEJImMGpVWhFBUWtrThDq8WqYLXV\n2kFKq9bvVa/1Xv1VrNprvbdWrVbF1nkCtVZUbB3aiFgRwVIntEWEBghDQhJISEKG9/fHWtHNMedk\nh5wkJ4f38zz7OXvv9a613nfvs9e7pr2XqCqGYRiGkWxE+loBwzAMw+gIc1CGYRhGUmIOyjAMw0hK\nzEEZhmEYSYk5KMMwDCMpMQdlGIZhJCXmoIw9BhEpFZFFIrJdRH7V1/r0JCIyR0QeihP+nohMiRE2\nRUTWxYl7n4hclwA1DSMu5qD6KSJytogsE5E6EakQkedF5Cgf1mHhJCIqIvsGjseJyAIRqfWF9l9F\n5MioOJk+vX+JSL2IrBGRe0RkbEDmZBFZ6sOrRORhERkZCD9PRBZ3Ys99ItIiIsMC5+709tWJyE4R\naQ4cPy8iY71NdVHbmTGymQVUAgWqemncC9wJfVFIi8i1IvKOv05zupOWqo5X1bLEaNbzRP93Q8iX\nicgPEpT3fBE5UUSyRGRjVFiWfx62ichGEbkkEXkaDnNQ/RD/ENwM/AIoBUYDtwNf70Ia+wCvAe8A\newHDgaeAF0TkiIDoEz7ds4GBwARgGXCcT+cM4BGvTzEwHmgCFovIoJC65AKnA7XAt9vPq+psVc1T\n1Txv67z2Y1WdFkiiMHA+T1XnxchqDPC+JsHb6SKSvhvRVgGXA88lWB0jPhNx//kDgXejwuYA++H+\nW8cAl4vI1F7VLpVRVdv60YZzEnXAjDgyc4CHOjivwL5+/0FgYQcydwCL/P7xQAMwKkY+AqwFLo86\nH8E9yD/3x+cBi+Poey5QDlwMvBvWJmCstyk9xHW7D2gGdvrrd7zX8wrgI6AKmA8UBeI8DmzEOc5F\nwHh/flZUWs9EX99Antf5/SnAOuCnPs0H/fmTgRVADfA34MAQtjwEzOlEZo635wFgO/AeMCkQvgY4\n3u/neF2rgfeB/wTWBWQPBt7y6cwDHmu3qzMbfD6XAW/76zgPyI6h877AK16uElchwV97Ber99T4T\nGAQ8C2zxej8LjPTy1wOtQKOXv82f3x94EdgKfAjMDHGtBwGr/f6PgBujwjcAJwaOfw481ptlQipv\n1oLqfxwBZONaO93hBFwBHM184EsikoMrxJeqanmMND6Pa73tko6qtgFP+jzC8B3gUVzBt7+ITAwZ\nLzSqeh7wMK6AyVPVl4CLgFOBo3EtyGpcS7Sd53G14xJcAf2wT2tuVFqnhFRjKFCEq23PEpGDgXuA\nHwKDgbuABSKS1Q1Tg3wdd00LgQXAbTHkrgb28dtJuPsBuC5e4I+4Ck0R7l6fHggPY8NMYCqupX4g\nrsLSEdcCL+CcwkjgNwCq+hUfPkE/bSFHgHtx13I0riJ1m5e/EngVuNDLX+hb6S/iWvslwDeB34rI\nuI4UEZHjRKQGV3Ea6fdvAS4QkRoROdr3EAwD/hGI+g9cL4KRAMxB9T8GA5Wq2tKJ3Ez/IH2yRYUX\nAxUdxKvA/S+KfF4dyQTTIIZMRSA8JiIyGtc18oiqbgJexrWoukJllK0HhIw3G7hSVdepahOu1XFG\ne/ebqt6jqtsDYRNEZGAXdQvSBlytqk2q2oBrid2lqm+oaquq3o/rHj28G3kEWayqC1W1FedgJsSQ\nmwlcr6pbfWXk1kDY4UAGcLOqNqvqE8CbgfAwNtyqqhtUdSvwDHBQDD2acQ5nuKo2qmrMcUtVrVLV\nJ1V1h6pux7Wajo4lj2vlrVHVe1W1RVX/jqtEzYiR/suqWohzzjOAEbjWYLGqFqrqK0CeF68NRN0G\n5MfRw+gC5qD6H1VAcYgxjPn+QfpkiwqvxNX+ohmGK0irfV4dyQTTIIbMsEB4PM4BVqrqCn/8MHC2\niGSEiNtOcZStK0PGGwM8FXDgK3FdQ6UikiYiN4jIRyKyDVc4QQinG4ctqtoYlf+lUZWIUbjWXCII\nDujvALJj/G+G41oK7ayNCluvqhojPIwN0Xrk0TGX47qNl/pZht+LIYeIDBCRu0Rkrb8/i4BCEUmL\nEWUMMDlKz2/hWrUdpb/Oy5wF3A9s9mlUiMhNXqzO/xYEog7EdYUaCcAcVP/jdVwN9dRupvMSHdce\nZwKvq+oOL3NYcEZeFB/ixlV2SUdEIrhuoJdD6HEusLefAbURuAnnBL4ayoruUQ5Mi3Ju2aq6Hjcp\nZDqum3MgbrwLXAEKbkwkmh3AgMBxdOEXHacc13IJ5j9AVR/thk27QwXOqbQzOipshIhIjPCE2aCq\nG1X1fFUdjusy/G2cmXuX4rqYJ6tqAdDeDRjr/pQDr0TpmaeqP4qhy0hct+RLvnI3F7jAx7vEy1Tj\nrk+wZToBN95nJABzUP0MVa0FfgbcLiKn+ppkhohME5Ebu5DUNcCRInK9iBSJSL6IXIRzGD/1eb2E\n67d/SkQmiki6l5stIt/zterLgKvETXvPFpGhwO9wtcpfB/ITHx7cjsCNexyG6/Y5CPgCbpygq918\nu8OdwPUiMsYrOEREpvuwfFxFoArndH4RFXcTsHfUuRW41l+an8kVr8sJ4G5gtohMFkeuiHxNRDrs\nIvL3ORv33Kb7axirxdAV5gP/JSKDfGXkokDY60AL8BOf/2m4+7VbNsRDRGYEKkPVOCfT5o+jr3c+\nbtypRkSKcONoQaLlnwU+JyLneDsyROTQTrqDJ+LGHgEOwc3ki+YB3P9/kE/rfNyEEyMBmIPqh6jq\nr4BLgKtws5jKgQtx/eVh0/gXcBSuxrcGVxM8HThJVV8LiJ4BLMTNvqrFzc6bhGtd4QeszwH+A1eY\nv4+bFfYlVa0KpHMkrkAJbt8HnlbVd3zteaOqbsQNRp/sC54w1Miu70GFfRflFtzkgRdEZDuwBJjs\nwx7AdWWt9zYtiYr7e2Cc7y5qv+4XA6fgZrN9i07uh6ouwxVot+EK5FXEnkAAzhk04LqdrvT753Rm\nZAiuwdn6MW6SwoMBHXcCp3m9tuJm0P2hGzbE41DgDRGpw92Xi1V1tQ+bA9zvr/dM3GsNObhu5CXA\nn6LSugU3nlgtIrf6caoTcZMjNuC6HX8JxJuQMhF4y7ce96fjltHVuFmga4Ey3MSZaF2M3UR27Vo2\nDMMwjOTAWlCGYRhGUmIOyjAMw0hKzEEZhmEYSYk5KMMwDCMp2Z0PVvY4xcXFOnbsWADq6+vJzc3t\nW4V6ALOr/5CKNkFq2pWKNkHq2bV8+fJKVR3SmVxSOqixY8eybJl75aCsrIwpU6b0rUI9gNnVO7Q2\nw5oyqPonVH8E1ashYwAMO+TTLTv6GxtRJJtNiSIV7UpFmyD17BKRtZ1LJamDMozu0rQN3vodLLkZ\ntvmP+GQMgEF7Q2MtvOu/cxBJh4O/D1++EgaOip2eYRi9jzkoI6Voa4HXbnRbUy2MORqm/QZGHAZ5\nQ6H9gz31W6DiLfjwaefIVtwLh8yCr1wFeaV9a4NhGI5QkyREZKqIfCgiq0Tkig7CRURu9eFvi8gh\ngbA14lYBXSEiHX0qxDASQm053H8s/OVK2OtY+MFSOK8M9p8O+cM+dU4AuUNg35Pga7+Fn6yCCd+B\n5XfCnRNcl6BhGH1Ppw7Kf+vrdmAaMA44Sz67hso03Lo5++E+v39HVPgxqnqQqk7qvsqG8VlWPuWc\ny8a/wzcegjP/ACMODRd34Gg4ZS78cIUbj3rgOFj8S7CPrBhG3xKmBXUYsEpVV/vvcj2G+8pzkOnA\nA+pYgvvsfbxlGgwjIajCq7+A+adB0T7ww7/Dgd/avbRKxsP5b8K4M+DlK2DeN2BnfWL1NQwjPJ1+\ni09EzgCmquoP/PE5uE/cXxiQeRa4oX2BMRF5Gfipqi4TkY9xHxltxS1sNjdGPrNwrS9KS0snPvbY\nYwDU1dWRlxdr+Zj+i9nVfVRh9R37sO7xUZQcv4nPX/4BkYzuN3tUYf2TI/nojn0oGF/L3lctYWBJ\ndgI0Ti5S8T+YijZB6tl1zDHHLA/To9YbkySOUtX1IlICvCgiH6jqomgh77jmAkyaNEnbp1Sm2vTK\ndsyu7tHWAs/MgnWPw2EXwdSbS5FIAmc3HAPvfQX+8K1CPvrZkfzobwXkhP22ej8hFf+DqWgTpK5d\nnRGmi289uy5mNtKfCyXjF39DVTcDT7HrWjKG0WXaWuDJs93Mu6Ovhqm3gPTAN1HGz4Qzn4K61Xnc\nNwXqNiU+D8MwYhPmsX4T2E9E9hKRTNx6KguiZBYA5/rZfIcDtapa4RcvywcQkVzceizvJlB/Yw9D\n2+Dp78L7j8MJ/wtT5uw6Oy/RfO5k+OL/vE31R27yxI6qzuMYhpEYOnVQqtqCWwzvz8BKYL6qvudX\nVZ3txRYCq3GLld0N/NifLwUWi8g/gKXAc7aYl7G7qMKzs+Hth+CY6+DIy3on30ETazjrGdi6Ch46\nyb3oaxhGzxNqDEpVF+KcUPDcnYF9BS7oIN5q3IqthtEtVOHP/wFv3Q1H/Td85crezX+vY2HmkzDv\nVHj0ZPjWnyAzdT6NZhhJiX3N3OgX/PVn8MYtMPn/wbHX9Y0On/sanPYIlP/NTUFvaeobPQxjT8Ec\nlJH0/O1X8Op17pt5J93Us2NOnTF+Bnz997D6RXjyLDdhwzCMnsEclJHUvPV7ePEyGDcDTr6rb51T\nOwed52YOfvAULPiBm7hhGEbisY/FGknLe4/Ds7Ng36lw2kMQSetrjT5l8k/cZImyn0FWgZ/qngTO\n0zBSCXNQRlLywR/hD2fDqCPd5IS0zL7W6LN85SporIElN0FmHhx7vTkpw0gk5qCMpOOfz8LjM2H4\nJDh7oVvHKRkRgRP/D3bWweL/cS8LH3OtOSnDSBTmoIykYtWfYf7pMHSCm8qdld/XGsVHBE6+w41D\nvXq9c1JTrjEnZRiJwByUkTR8uMC1nIaMh2+/ANkD+1qjcEgETrkLUFh0rTtnTsowuo85KCMpWHGf\nmxE3fKLr1ssZ1NcadQ2JuDWl1DupHVUw7dbkmthhGP0Nc1BGn/O3X7mp5Huf4BYazOynqwpIBL5+\nNwwYDH/7X6jfCKc9DOmpt1KHYfQK9h6U0We0NsPzFzvnNH4mnPVM/3VO7UgETrgRTvo1rPwDPHgi\nNGzta60Mo39iDsroE+o2uq+DL73Vfb7otEcgPauvtUoch/8/OP1RWLcE7joE1i/ta40Mo/9hDsro\ndf79miu0K5Y7xzT116k5VvOFb8L3Frv9e46CN251Y1SGYYTDHJTRa+yshxf+E+472n0J/PtL4Itn\n9bVWPcuIw+CHb8G+J8GfLob5p8G26OU+DcPoEHNQRq+w6s9wxxfg9f9zH309fxmUfrGvteodcorg\nm0/DCf8Hq/4Et+8Pr9/kxuAMw4iNOSijR1m/FB49BR6eCmlZcN4r7p2h/vKOU6KQCBx5Kfz4PRjz\nFXjhUpg7ET542j42axixMAdlJBxVWPMKPDQVfjfZrZ907PUw+x+ucN6TGbQ3nPUsnPmU+0TSvFPh\nt19w74G17uxr7QwjubD3oIyEUfVP+Piesfzje1DzMQwYAsf/Eib9KPk/WdSbiMD+p8LnTnZfbH/t\nBnj6u/Di5XDA6W5yxeijUnPiiGF0hVAOSkSmArcAacDvVPWGqHDx4V8FdgDnqepbYeIa/RNVqKuA\nfy+Gj/8Ca/7qHBQyhr2PgylzXGFry6LHJpLuJol84Zvw0Quw4l54+wFYfifkDXXLzI/+stuGHOC6\nCQ1jT6JTByUiacDtwAnAOuBNEVmgqu8HxKYB+/ltMnAHMDlkXCMJ0TZo2uZeMt1eAdvKobbctYy2\nvAeb3/30BdTMfNd1N+lHUDvidU6acWTfKt/PEHGz/PY9yc10/OezbjHEj/8C7zziZDJyoXh/56gG\n7w8DR0H+cMgfAbklbkwvYv0hRooR5i99GLBKVVcDiMhjwHQg6GSmAw+oqgJLRKRQRIYBY0PETSht\nrfDcj3sq9cRRseFzbH90NyMH3qX5zHs16gfd1Yf54/atrcVdo7YWaGuGliY39tHSCM31roDcWQdN\ntR0P3mcNhJLxcMAZUPIFN416+MRPC8eyMhtI6Q6ZufCFM92mCtUfwdpXYeMKqFwJa8rg7YdixM1z\n9ycz1y1Rkp7jPrOUluHW04pkuG5DSXO/m7bsT829gPgP20rUfoBdPnybxB/B7dZzlcQkk11DJ8Ch\nvVTGhnFQI4DywPE6XCupM5kRIeMCICKzgFkApaWllJWVAVBXV/fJfhi0VXj3icNDy/cVbVpElTTt\nfgJxCgwRDRQ2+slUGIkokua3iCLpimS0EUlXIpltpJW2kpPdRl52K+kFzWTkt5Ce30xmUTNZQxrJ\nKmkiPbf1k3wagFUNsGrxp3l39X71B/rcpr0gZy8Y9Q0YBbQ2RGiqymJnZSZNlVk012bQUpdOS106\nrXXptDal0dwUobExDa0X2pojaIu4rc1vrUJbaz7b3mlAVVylR3H70WiM/SSk289VkpJMdq1fW039\nuA96Ja+k6RRQ1bnAXIBJkybplClTACgrK6N9PyzHVCVYuR5gd+zqD6SiXaloE6SmXaloEySbXUP9\n1vOEcVDrcRW3dkb6c2FkMkLE/QzLly+vFJG1/rAYqAyhZ3/D7Oo/pKJNkJp2paJNkHp2jQkjFMZB\nvQnsJyJ74ZzLN4Gzo2QWABf6MabJQK2qVojIlhBxP4OqDmnfF5FlqjopjDH9CbOr/5CKNkFq2pWK\nNkHq2tUZnTooVW0RkQuBP+Omit+jqu+JyGwffiewEDfFfBVumvl348XtEUsMwzCMlCLUGJSqLsQ5\noeC5OwP7ClwQNq5hGIZhdEZ/ePVvbl8r0EOYXf2HVLQJUtOuVLQJUteuuIjaAjWGYRhGEtIfWlCG\nYRjGHog5KMMwDCMpMQdlGIZhJCXmoAzDMIykxByUYRiGkZSYgzIMwzCSEnNQhmEYRlJiDsowDMNI\nSsxBGYZhGEmJOShjj0FESkVkkYhsF5Ff9bU+PYmIzBGRGGvvgoi8JyJTYoRNEZF1ceLeJyLXJUBN\nw4iLOah+ioicLSLLRKRORCpE5HkROcqHdVg4iYiKyL6B43EiskBEan2h/VcROTIqTqZP718iUi8i\na0TkHhEZG5A5WUSW+vAqEXlYREYGws8TkcXEwRd6LSIyLHDuTm9fnYjsFJHmwPHzIjLW21QXtZ0Z\nI5tZuDV1ClT10rgXuBP6opAWkWtF5B1/neZ0Jy1VHa+qZYnRrOeJ/u+GkC8TkR8kKO/5InKiiGSJ\nyMaosJki8jcR2SEiZYnIz/gUc1D9EBG5BLgZ+AVQCowGbge+3oU09gFeA94B9gKGA08BL4jIEQHR\nJ3y6ZwMDgQnAMuA4n84ZwCNen2JgPNAELBaRQSF1yQVOB2qBb7efV9XZqpqnqnne1nntx6o6LZBE\nYeB8nqrOi5HVGOB9TYIPUIrI7qxmvQq4HHguweoY8ZmI+88fCLwbFbYV99+/obeV2iNQVdv60YZz\nEnXAjDgyc4CHOjivwL5+/0FgYQcydwCL/P7xQAMwKkY+AqwFLo86H8E9yD/3x+cBi+Poey5QDlwM\nvBvWJmCstyk9xHW7D2gGdvrrd7zX8wrgI6AKmA8UBeI8DmzEOc5FwHh/flZUWs9EX99Antf5/SnA\nOuCnPs0H/fmTgRVADfA34MAQtjwEzOlEZo635wFgO/AeMCkQvgY43u/neF2rgfeB/wTWBWQPBt7y\n6cwDHmu3qzMbfD6XAW/76zgPyI6h877AK16uElchwV97Ber99T4TGAQ8C2zxej8LjPTy1wOtQKOX\nv82f3x94EedUPgRmhrjWg4DVfv9HwI0x5H4AlPVmWbAnbNaC6n8cAWTjWjvd4QRcARzNfOBLIpKD\nK8SXqmp5jDQ+j2u97ZKOqrYBT/o8wvAd4FFcwbe/iEwMGS80qnoe8DCugMlT1ZeAi4BTgaNxLchq\nXEu0neeB/YASXAH9sE9rblRap4RUYyhQhGvJzRKRg4F7gB8Cg4G7gAUiktUNU4N8HXdNC3GrXt8W\nQ+5qYB+/nYS7H4Dr4gX+iKvQFOHu9emB8DA2zASm4lrqB+IqLB1xLfACzimMBH4DoKpf8eET9NMW\ncgS4F3ctR+MqUrd5+SuBV4ELvfyFvpX+Iq61X4Jb3fu3IjKuI0VE5DgRqcFVnEb6/VuAC0SkRkSO\njmGDkUDMQfU/BgOVqtrSidxM/yB9skWFFwMVHcSrwP0vinxeHckE0yCGTEUgPCYiMho4BnhEVTcB\nL+NaVF2hMsrWA0LGmw1cqarrVLUJ1+o4o737TVXvUdXtgbAJIjKwi7oFaQOuVtUmVW3AtcTuUtU3\nVLVVVe/HdY8e3o08gixW1YWq2opzMBNiyM0ErlfVrb4ycmsg7HAgA7hZVZtV9QngzUB4GBtuVdUN\nqroVeAY4KIYezTiHM1xVG1U15rilqlap6pOqukNVt+NaTfGcxsnAGlW9V1VbVPXvuErUjBjpv6yq\nhTjnPAMYgWsNFqtqoaq+EicvI0GYg+p/VAHFIcYw5vsH6ZMtKrwSGNZBvGG4grTa59WRTDANYsgM\nC4TH4xxgpaqu8McPA2eLSEaIuO0UR9m6MmS8McBTAQe+Etc1VCoiaSJyg4h8JCLbcIUThHC6cdii\nqo1R+V8aVYkYhWvNJYLggP4OIDvG/2Y4rqXQztqosPWqqjHCw9gQrUdeDH0vx3UbL/WzDL8XQw4R\nGSAid4nIWn9/FgGFIpIWI8oYYHKUnt/CtWo7Sn+dlzkLuB/Y7NOoEJGbYullJBZzUP2P13E11FO7\nmc5LdFx7nAm8rqo7vMxhwRl5UXyIG1fZJR0RieC6gV4Ooce5wN4istHPkLoJ5wS+GsqK7lEOTIty\nbtmquh43KWQ6rptzIG68C1wBCm5MJJodwIDAcXThFx2nHNdyCeY/QFUf7YZNu0MFzqm0MzoqbISI\nSIzwhNmgqhtV9XxVHY7rMvxtnJl7l+K6mCeragHQ3g0Y6/6UA69E6Zmnqj+KoctIXLfkS75yNxe4\nwMe7pKu2GbuHOah+hqrWAj8DbheRU31NMkNEponIjV1I6hrgSBG5XkSKRCRfRC7COYyf+rxewvXb\nPyUiE0Uk3cvNFpHv+Vr1ZcBV4qa9Z4vIUOB3QAHw60B+4sOD2xG4cY/DcN0+BwFfwI0TdLWbb3e4\nE7heRMZ4BYeIyHQflo+rCFThnM4vouJuAvaOOrcC1/pLE5GpxO9yArgbmC0ik8WRKyJfE5H8joT9\nfc7GPbfp/hrGajF0hfnAf4nIIF8ZuSgQ9jrQAvzE538a7n7tlg3xEJEZgcpQNc7JtPnj6Oudjxt3\nqhGRItw4WpBo+WeBz4nIOd6ODBE5tJPu4Im4sUeAQ3Az+aJ1TvP3JB2I+HvSlda/EQdzUP0QVf0V\ncAlwFW4WUzlwIa6/PGwa/wKOwo1LrMHVlE8HTlLV1wKiZwALcbOvanGz8ybhWlf4AetzgP/AFebv\n42aFfUlVqwLpHIkrUILb94GnVfUdX3veqKobcYPRJ/uCJww1sut7UGFruLfgJg+8ICLbgSXAZB/2\nAK4ra723aUlU3N8D43x3Uft1vxg4BTeb7Vt0cj9UdRlwPm5wvxo3jfy8OFHuxl23s4Ar/f45nRkZ\ngmtwtn6Mm6TwYEDHncBpXq+tuBl0f+iGDfE4FHhDROpw9+ViVV3tw+YA9/vrPRM3tTsH1428BPhT\nVFq34MYTq0XkVj9OdSJucsQGXLfjL4F4E1ImAm/51uP+uJmQ0ZyDuw93AF/2+3d3yWojJrJr17Jh\nGIZhJAfWgjIMwzCSEnNQhmEYRlJiDsowDMNISsxBGYZhGEnJ7nywsscpLi7WsWPHAlBfX09ubm7f\nKtQDmF39h1S0CVLTrlS0CVLPruXLl1eq6pDO5JLSQY0dO5Zly9wrB2VlZUyZMqVvFeoBzK7+Qyra\nBImzS1VZ0tDAk9u3s1dGBlNycxmXmcmu7/b2Dnav+gcisrZzqSR1UIZh9A+WNjRwW3U1yxsbKYhE\n+FN9Pb+tqaEkLY0zCwr4YWFhnzgqIzUwB2UYRpdRVa6tqmLetm0MSUvjysGDOaOggLq2Nhbt2MGf\n6ur4TXU1dW1tXFpUZE7K2C3MQRmG0WXur61l3rZtnFNQwP8rKiI74uZbFaWlcWp+PtPz8riuqop7\na2vJFOEnRWE/CmIYn2IOyjCMLvGX+nr+b+tWTsrN5fLBg4l00DoSEa4cPJhmVe6qqSFThNmDQi2w\nbBifYA7KMIzQrGxq4vLNmxmflcX1Q4Z06JzaiYhwdXExO1X5TXU1ozMy+GperJU2DOOz2HtQhmGE\nora1lQs3bmRgJMJtpaXkRDovPtJEuG7IEL6YlcX1lZVUtnS2zqZhfIo5KMMwQnHT1q1saW3llqFD\nGZIevvMl3Tup+rY2rq+q6jyCYXjMQRmG0SnLGxp4Yvt2zh04kC9kxVuhomP2zczkgkGDeKG+nj/X\n1fWAhkYqYg7KMIy47FRlTmUlI9LT+XE3Jjp8t7CQ8ZmZXFdZydbW1gRqaKQq5qAMw4jL72pqWN3c\nzP9XXMyAEONOsUgX4bqSEra1tXGDdfUZITAHZRhGTFbv3Mnc6mqm5eby5QEDup3e5zIzOb+wkOfq\n6nijoSEBGhqpjDkowzA6RFW5trKSnEiEKwYPTli6PygsZGR6OtdXVtJsK3obcQjloERkqoh8KCKr\nROSKDsJFRG714W+LyCGBsDUi8o6IrBCRZYlU3jCMnmNhfT1LGxu5uKiI4i7M2uuMbO/wPmpu5uHa\n2oSla6QenTooEUkDbgemAeOAs0RkXJTYNGA/v80C7ogKP0ZVD1LVSd1X2TCMnqaurY3/rapifGYm\nM/LzE57+Mbm5HD1gALdXV7PZ3o0yYhCmBXUYsEpVV6vqTuAxYHqUzHTgAXUsAQpFZFiCdTUMo5f4\nbXU1la2tXFVcTFoPfej1isGDaQH+1yZMGDEI024fAZQHjtcBk0PIjAAqAAVeEpFW4C5VndtRJiIy\nC9f6orQ08Bp4AAAgAElEQVS0lLKyMgDq6uo+2U8lzK7+QyraBLHtWp+RwYNDh3JkfT1blyzhsxKJ\n47iBA1moyj6vv87+TU3dTm9Pu1epTm98i+8oVV0vIiXAiyLygaouihbyjmsuwKRJk7R9ca5UW6ir\nHbOr/5CKNkHHdrWpcl5FBfk7d3Lj+PEUpqX1qA6Ht7Wxcv16/jBiBH8cObJb09hhz7pXewJh/g3r\ngVGB45H+XCgZVW3/3Qw8hesyNAwjCXl02zaWNzZyaVFRjzsncBMmfl5czPqWFm7ZurXH8zP6F2Ec\n1JvAfiKyl4hkAt8EFkTJLADO9bP5DgdqVbVCRHJFJB9ARHKBE4F3E6i/YRgJYm1zMzdt3cpROTmc\n1gMTI2IxMSeHbxYU8PC2bfy9sbHX8jWSn04dlKq2ABcCfwZWAvNV9T0RmS0is73YQmA1sAq4G/ix\nP18KLBaRfwBLgedU9U8JtsEwjG7Sqsp/b95Mpgg/HzKk11fAvaSoiKHp6fxsyxaa2tp6NW8jeQk1\nBqWqC3FOKHjuzsC+Ahd0EG81MKGbOhqG0cPcW1vLiqYmfllSQmkC33kKS24kwtXFxczeuJHbqqu5\nNIEvBhv9F/uShGHs4XzY1MRtW7dyQm4uX8vN7TM9vjxgAGfk53NPba198dwAzEEZxh7N5pYWLti0\nicK0NH5WXNzrXXvRXFlczISsLK7csoUPEjDt3OjfmIMyjD2UBhFmb9xIbWsrdwwdSlEvzNrrjEwR\nbiktpSAS4aJNm2xZjj0cc1CGsQfSrMrviotZtXMnvy4t5YDdWISwpxiSns4tpaVUtrZyyaZNNNqk\niT0Wc1CGsYexU5Wrtmzhg5wcrhkyhKMSsIxGovlidjbXDhnCm42NfL+igmprSe2RmIMyjD2IjS0t\nfHfDBp6tq+OUmhq+0YvvO3WVk/PyuKmkhJU7d3L2+vWsbW7ua5WMXsYclGHsISxpaGDGunX8c+dO\nflVSwtRt2/papU45KS+Pe4cNo66tjbPWr+fVHTv6WiWjFzEHZRgpztrmZq7esoXzKyooTEvjsREj\nmJqX19dqhWZCdjaPjBjB4LQ0Zm/cyPkVFay0GX57BL2xYGHcuIZhJJ5WVf7R2MhlmzZxcnk5C+rq\nOLOggMdGjGCfzMy+Vq/LjMrI4MmRI7m8qIj3mpo4Y/16Ltu0ibL6eptEkcJ0+sp4YMHCE3DLaLwp\nIgtU9f2AWHDBwsm4BQsnh4xrGEY3aFFlU0sL5S0tfLRzJ0sbGlja2Mi2tjZyRThv4EDOHTiQIX3w\nhYhEkinCdwoL+UZ+Pr+rqeGxbdt4vr6eHBG+lJPDQdnZ1Gdns09zM8PT03tsHSuj9wjzj/1kwUIA\nEWlfsDDoZD5ZsBBYIiLtCxaODRE3obSq8ot+sADahkGDeLWysq/VSDipaFcibdK4gYp6mfatTZVW\noBU3NbxZlSZV6tva2N7WRl1bG1WtrQTnuA1PT+f43Fwm5+TwlZwcCpLg/aZEUpCWxiWDB3NhURFL\nGxr4644dlNXX89KOHVBSwh3l5aQBBZEIhWlpDIxEGBCJkCFClggZIqThuo/SRBD4ZCPw6w52dXJ9\n5fKS6bn6XGYmZxYU9EpePb1gYZi4QOIWLGwFnh0xIrR8X6E5ObxVXd3XaiScVLSrN20KFpIR1U8K\nzogq6UCaKumq5LS1UaBKaVsbB7e2UtzSwuCWFoa0tFAUmJL9Vpy8UmURvC/7rS4SYU1zM9sKCqhM\nT6c+EqE+EmFHWhq1IrSI0Ay0iKAitAFtgIp0WHGIW5noZZLpuTqgsZHSXmoEJE2bP5ELFr6RYN16\nglRdgCwV7UpFmyA17SorK2PKoYf2tRoJJxXvVRjCOKjuLFiYESLuZ1i+fHmliKz1h8VAcrRtE4vZ\n1X9IRZsgNe1KRZsg9ewaE0YojIP6ZMFCnHP5JnB2lMwC4EI/xjSZTxcs3BIi7mdQ1SHt+yKyTFUn\nhTGmP2F29R9S0SZITbtS0SZIXbs6o1MHpaotItK+YGEacE/7goU+/E7cWlFfxS1YuAP4bry4PWKJ\nYRiGkVL06IKFseIahmEYRmf0hy9JzO1rBXoIs6v/kIo2QWralYo2QeraFRdxjR/DMAzDSC76QwvK\nMAzD2AMxB2UYhmEkJeagDMMwjKTEHJRhGIaRlJiDMgzDMJISc1CGYRhGUmIOyjAMw0hKzEEZhmEY\nSYk5KMMwDCMpMQdl7DGISKmILBKR7SLyq77WpycRkTki8lCc8PdEZEqMsCkisi5O3PtE5LoEqGkY\ncTEH1U8RkbNFZJmI1IlIhYg8LyJH+bAOCycRURHZN3A8TkQWiEitL7T/KiJHRsXJ9On9S0TqRWSN\niNwjImMDMieLyFIfXiUiD4vIyED4eSKyuBN77hORFhEZFjh3p7evTkR2ikhz4Ph5ERnrbaqL2s6M\nkc0s3Jo6Bap6adwL3Al9UUiLyLUi8o6/TnO6k5aqjlfVssRo1vNE/3dDyJeJyA8SlPd8ETlRRLJE\nZGNU2P/5Z2O7iHwgIucmIk/DYQ6qHyIilwA3A78ASoHRwO3A17uQxj7Aa8A7wF7AcOAp4AUROSIg\n+oRP92xgIDABWAYc59M5A3jE61MMjAeagMUiMiikLrnA6UAt8O3286o6W1XzVDXP2zqv/VhVpwWS\nKAycz1PVeTGyGgO8r0nwAUoR2Z3VrFcBlwPPJVgdIz4Tcf/5A4F3o8LqgVNwz8Z3gFuiK3lGN1BV\n2/rRhnsQ6oAZcWTmAA91cF6Bff3+g8DCDmTuABb5/eOBBmBUjHwEWAtcHnU+gnuQf+6PzwMWx9H3\nXKAcuBh4N6xNwFhvU3qI63Yf0Azs9NfveK/nFcBHQBUwHygKxHkc2IhznIuA8f78rKi0nom+voE8\nr/P7U4B1wE99mg/68ycDK4Aa4G/AgSFseQiY04nMHG/PA8B24D1gUiB8DXC838/xulYD7wP/CawL\nyB4MvOXTmQc81m5XZzb4fC4D3vbXcR6QHUPnfYFXvFwlrkKCv/aKcwZ1wJnAIOBZYIvX+1lgpJe/\nHmgFGr38bf78/sCLwFbgQ2BmiGs9CFjt938E3NiJ/ALg0t4oC/aEzVpQ/Y8jgGxca6c7nIArgKOZ\nD3xJRHJwhfhSVS2Pkcbnca23XdJR1TbgSZ9HGL4DPIor+PYXkYkh44VGVc8DHsYVMHmq+hJwEXAq\ncDSuBVmNa4m28zywH1CCK6Af9mnNjUrrlJBqDAWKcC25WSJyMHAP8ENgMHAXsEBEsrphapCv465p\nIa7gvC2G3NXAPn47CXc/ANfFC/wRV6Epwt3r0wPhYWyYCUzFtdQPxFVYOuJa4AWcUxgJ/AZAVb/i\nwyfopy3kCHAv7lqOxlWkbvPyVwKvAhd6+Qt9K/1FXGu/BLe6929FZFxHiojIcSJSg6s4jfT7twAX\niEiNiBzdQZwc4FBcZcBIAOag+h+DgUpVbelEbqZ/kD7ZosKLgYoO4lXg/hdFPq+OZIJpEEOmIhAe\nExEZDRwDPKKqm4CXcS2qrlAZZesBIePNBq5U1XWq2oRrdZzR3v2mqveo6vZA2AQRGdhF3YK0AVer\napOqNuBaYnep6huq2qqq9+O6Rw/vRh5BFqvqQlVtxTmYCTHkZgLXq+pWXxm5NRB2OJAB3Kyqzar6\nBPBmIDyMDbeq6gZV3Qo8AxwUQ49mnMMZrqqNqhpz3FJVq1T1SVXdoarbca2mzziNACcDa1T1XlVt\nUdW/4ypRM2Kk/7KqFuKc8wxgBK41WKyqhar6SgfR7gT+gVtB3EgA5qD6H1VAcYgxjPn+Qfpkiwqv\nBIZ1EG8YriCt9nl1JBNMgxgywwLh8TgHWKmqK/zxw8DZIpIRIm47xVG2rgwZbwzwVMCBr8R1DZWK\nSJqI3CAiH4nINlzhBCGcbhy2qGpjVP6XRlUiRuFac4kgOKC/A8iO8b8ZjmsptLM2Kmy9qmqM8DA2\nROuRF0Pfy3Hdxkv9LMPvxZBDRAaIyF0istbfn0VAoYikxYgyBpgcpee3cK3ajtJf52XOAu4HNvs0\nKkTkpg7k/xf4Aq7bsM/HOFMFc1D9j9dxNdRTu5nOS3Rce5wJvK6qO7zMYcEZeVF8iBtX2SUdEYng\nuoFeDqHHucDeIrLRz5C6CecEvhrKiu5RDkyLcm7ZqroeNylkOq6bcyBuvAtcAQpuTCSaHcCAwHF0\n4RcdpxzXcgnmP0BVH+2GTbtDBc6ptDM6KmyEiEiM8ITZoKobVfV8VR2O6zL8bZyZe5fiupgnq2oB\n0N4NGOv+lAOvROmZp6o/iqHLSFy35Eu+cjcXuMDHuyQoKyLXANOAE1V1W9esNuJhDqqfoaq1wM+A\n20XkVF+TzBCRaSJyYxeSugY4UkSuF5EiEckXkYtwDuOnPq+XcP32T4nIRBFJ93KzReR7vqZ4GXCV\nuGnv2SIyFPgdUAD8OpCf+PDgdgRu3OMwXLfPQbha6CN0vZtvd7gTuF5ExngFh4jIdB+Wj6sIVOGc\nzi+i4m4C9o46twLX+ksTkanE73ICuBuYLSKTxZErIl8TkfyOhP19zsY9t+n+GsZqMXSF+cB/icgg\nXxm5KBD2OtAC/MTnfxrufu2WDfEQkRmBylA1zsm0+ePo652PG3eqEZEi3DhakGj5Z4HPicg53o4M\nETm0k+7gibixR4BDcDP5onX+L1xl5nhVrerUSKNLmIPqh6jqr4BLgKtws5jKgQtx/eVh0/gXcBRu\nXGINrqZ8OnCSqr4WED0DWIibfVWLm503Cde6wg9YnwP8B64wfx83K+xLUQ/skbgCJbh9H3haVd/x\nteeNqroRNxh9si94wlAju74HdUnnUcDnswA3tX47sASY7MMewHVlrfc2LYmK+3tgnO8uar/uF+Om\nHLd3H8W9H6q6DDgfN7hfjZtGfl6cKHfjrttZwJV+/5zOjAzBNThbP8ZNUngwoONO4DSv11bcDLo/\ndMOGeBwKvCEidbj7crGqrvZhc4D7/fWeiXutIQfXjbwE+FNUWrfgxhOrReRWP051Im5yxAZct+Mv\ngXgTUiYCb/nW4/50PPnhF7gW5arA/++/u2q40TFi3aWGYRhGMmItKMMwDCMpMQdlGIZhJCXmoAzD\nMIykxByUYRiGkZTszgcre5zi4mIdO3YsAPX19eTm5vatQj2A2dV/SEWbIDXtSkWbIPXsWr58eaWq\nDulMLikd1NixY1m2zL1yUFZWxpQpU/pWoR7A7Oo/hLVpZ/NWarb/g6bmSjIzBpOVUUxWZgnZmaXs\n+p5rcrAn36v+RqrZJSJrO5dKUgdlGP0FVaW2/l2qty1jR+O/AchIH8S2+pW0f8wgN3ssw4dMJzMj\n+mtThmHEwxyUYewmbdpKReVz1Gz/O5kZgykpOo7CvAPJSC+gTVvZ2VxFfcPHbN76Fz5adwelg09g\nUP7EpGxNGUYyYg7KMHaD1tYGyjfNp75xDcWFX6Zk0DG7OJ6IpJGdWUJ2Zgn5Az7PhsoFVFQ+x/Yd\n/2JUyQwiEXv0DKMzbBafYXSRnc3VrN7we3Y0ljNiyKmUFh0bt1WUmVHImKHnMHTwVOp2/JN1m5/E\nLZllGEY8zEEZRhdobW1g7caHaW2tZ8ywcyjMj7XE0q6ICIMHTmbo4Kls3/EBG7YswD4zZhjxsX4G\nwwiJaivlmx6nubmaMcPOJTdnTJfTGDxwMq1tjWypLiMSyWLo4Kk2JmUYMTAHZRghUFUqKhdS3/gx\nI4ZM3y3n1M6Qwq/Q1tZIVe0SMtILKS48IoGaGkbqEKqLT0SmisiHIrJKRK7oIFxE5FYf/raIHBII\nWyMi74jIChH5zHoqhtEfqKpdQvX2tyguPIrC/FgrlodDRCgtOpH8AfuzaetLn0xPNwxjVzp1UH5B\ntNtxK0aOA84SkXFRYtOA/fw2C7gjKvwYVT1IVSd1X2XD6F0ysmrYtPVFCnIPoGTQsQlJU0QYUTKd\nzPSBlG96gpbW+oSkaxipRJgW1GHAKlVd7Rcvewy3FHaQ6cAD6lgCFIrIsATrahi9TktrPfmDV5KZ\nMYjhQ6YndLwoLZLNqNKZtLY12Mw+w+iAMA5qBG7F1nbW+XNhZRR4SUSWi8is3VXUMHobVWX95j8S\nSWtmZMkM0iLxFl/dPbKzhjKs+Kvuhd7qsoSnbxj9md6YJHGUqq4XkRLgRRH5QFUXRQt55zULoLS0\nlLKyMgDq6uo+2U8lzK7kJyf/3+QN+piqjaOpLP8A+KDH8sorGsoWfZVVH9bS3DSox/IJkkr3qp1U\ntAlS167OCOOg1gOjAscj/blQMqra/rtZRJ7CdRl+xkGp6lxgLsCkSZO0/cOIqfaRxHbMruRmR2M5\nH29YREHuOLY0F/e4TW1tX2L1+rlkDP+IfUbMJj09r0fzg9S5V0FS0SZIXbs6I0wX35vAfiKyl4hk\nAt8EFkTJLADO9bP5DgdqVbVCRHJFJB9ARHKBE4F3E6i/YSScltYdrNv0BBnpAxlefArQ8+8pRSIZ\njCydQWtbE+u2/NFe4jUMQjgoVW0BLgT+DKwE5qvqeyIyW0Rme7GFwGpgFXA38GN/vhRYLCL/AJYC\nz6nqnxJsg2EkDFVl/ZanaWmtY1TpDNLSsnst7+zMEoYOnkp9w0dU1r7Wa/kaRrISagxKVRfinFDw\n3J2BfQUu6CDeaiDct2AMIwmoqn2duh3/ZOjgqeRkDe/1/AflH0J9w2o2b/0LudmjGZA9utd1MIxk\nwb7FZxieHY3r2LT1ZfIHHEBRwWF9ooOIMHzIKWSmD3KfVWrZ3id6GEYyYA7KMIDmljrWbXqcjPQC\nRgz5ep9+Hy8tks2ooWfS2tZE+abHadPWPtPFMPoSc1DGHk9bWwvlmx6jpa2BUaUze3XcKRbZmSWM\nGDKdhqZyNlbZsK2xZ2IOytijUVU2bHmahqb1jCz5BjlZyfMBlIF54xk88Aiqty2jetvf+1odw+h1\nzEEZezSVNa9SW/8uJYOOpSD3gL5W5zOUFh1Pbs7ebKh8lu31/+xrdQyjVzEHZeyxVG9fwebqvzIw\n70CKC4/qa3U6RCTCqNKZZGeWUr75ceob1vS1SobRa5iDMvZItm5bzoYtT5ObszfDi09J6kUD0yJZ\njBn2bTLTC/n3xkdpaNrQ1yoZRq9gDsrY46iqXUpF5bPk5ezH6NKziESSf93O9LQBjBl2DmlpA1hb\n8RA7Gtf1tUqG0eOYgzL2GFSVyprX2Fj1PPkDPs+ooTP7hXNqJyO9gLHDziESyWJNxX3U1L3T1yoZ\nRo9iDsrYI2hpbaB80zw2bX2JgtxxjCydQUT6j3NqJzOjiL1HnE9O1kjWb/4Dm7b+xb7bZ6Qs/e8J\nNYwuUt/4b9ZvepKW1jqGDj6JooLJST3m1Bnt3X0Vlc9RWfMqDU0bGFY8jayMwX2tmmEkFHNQRsqy\ns7mGLTWLqNm+gsz0QvYa8f0++b5eTxCRNIYXn0JO1jA2Vb3MR+V3MLjwCIYUfplIJLOv1TOMhGAO\nykg5drbUUlmzmJptbwFCUcGhlBQd2yMr4vYlIs62/AEHsGnri87m7W9TVDCJwvyDyeiFNaUMoycJ\n5aBEZCpwC5AG/E5Vb4gKFx/+VWAHcJ6qvhUmrmEkguaWbWyrf5/auvdpaCoHIgzKP5ghg75MRvrA\nvlavR8lIz2NkyTcYVDCRLVv/yubqv7C5uoz8AZ9nYN54crPH9MoCiIaRaDp1UCKSBtwOnACsA94U\nkQWq+n5AbBqwn98mA3cAk0PGNYxQqCqtbTtobtlGc0stTTu30NBUQePODTS31AKQnVlKyaBjGZj3\nRTIzCvtY494lN3s0ucO/Q9POKqq3v0XN9hVs37ESgKyMYgZkjyYzo5isjCIyMwYj0oKq9uvxOCO1\nCdOCOgxY5dd2QkQeA6YDQSczHXjArwu1REQKRWQYMDZE3ISi2sbGqj/3VPIJI3fQeioqG/pajYTz\nqV2fnVmm7ec+mXWm/pz6mWhtqLah2opqC6ottGkzrW2NtLU10traiLLrl70z04vIyRpFUcFh5A/4\nPFmZNlEgK3MwQwefQGnRcTQ2VVDfuIb6hrVsq19Ja9un/7niUbDy4yWkpQ0gLZJDJJJBRDKQSCZC\nGiIRRNIQIiC43/bVhQNOTeKuONy7zi/1n6u+JyuzhKKCib2SVxgHNQIoDxyvw7WSOpMZETIuACIy\nC5gFUFpaSllZGQB1dXWf7IdDGTzirS7I9w1ZA5TK6k19rUbC6dyuTwssRQJ+TEDFn4ugfkMjtLWl\no20FqBbR1pJJa2s2bS1ZtLbkoNr+F94J9Mx7QV3/DyYjw4HhSKSZtPQG0tIbaG7dTk5OBIk0E4k0\nI9KMSCtE2hDaQBQRV4Fov1HuOEhyTXHfc5+r3mNnYxFvV/XOOmVJM0lCVecCcwEmTZqkU6ZMAaCs\nrIz2/fAck1DdeoLdsyv5SUW7UtEmcHYdesSUvlYjoaTyvUpFuzojjINaD4wKHI/058LIZISI+xmW\nL19eKSJr/WExUBlCz/6G2dV/SEWbIDXtSkWbIPXsGhNGKIyDehPYT0T2wjmXbwJnR8ksAC70Y0yT\ngVpVrRCRLSHifgZVHdK+LyLLVHVSGGP6E2ZX/yEVbYLUtCsVbYLUtaszOnVQqtoiIhcCf8ZNFb9H\nVd8Tkdk+/E5gIW6K+SrcNPPvxovbI5YYhmEYKUWoMShVXYhzQsFzdwb2FbggbFzDMAzD6Iz+8LHY\nuX2tQA9hdvUfUtEmSE27UtEmSF274iL2JWTDMAwjGekPLSjDMAxjD8QclGEYhpGUmIMyDMMwkhJz\nUIZhGEZSYg7KMAzDSErMQRmGYRhJiTkowzAMIykxB2UYhmEkJeagDMMwjKTEHJSxxyAipSKySES2\ni8iv+lqfnkRE5ojIQ3HC3xORKTHCpojIujhx7xOR6xKgpmHExRxUP0VEzhaRZSJSJyIVIvK8iBzl\nwzosnERERWTfwPE4EVkgIrW+0P6riBwZFSfTp/cvEakXkTUico+IjA3InCwiS314lYg8LCIjA+Hn\nicjiTuy5T0RaRGRY4Nyd3r46EdkpIs2B4+dFZKy3qS5qOzNGNrNwa+oUqOqlcS9wJ/RFIS0i14rI\nO/46zelOWqo6XlXLEqNZzxP93w0hXyYiP0hQ3vNF5EQRyRKRjVFhN4pIuYhsE5G1IvLficjTcJiD\n6oeIyCXAzcAvgFJgNHA78PUupLEP8BpunfS9cGuCPwW8ICJHBESf8OmeDQwEJgDLgON8OmcAj3h9\nioHxQBOwWEQGhdQlFzgdqAW+3X5eVWerap6q5nlb57Ufq+q0QBKFgfN5qjovRlZjgPc1CT5AKSK7\ns5r1KuBy4LkEq2PEZyLuP38g8G5U2D3AeFUtAI4EviUip/WyfqmLqtrWjzack6gDZsSRmQM81MF5\nBfb1+w8CCzuQuQNY5PePBxqAUTHyEWAtcHnU+QjuQf65Pz4PWBxH33OBcuBi4N2wNgFjvU3pIa7b\nfUAzsNNfv+O9nlcAHwFVwHygKBDncWAjznEuwhVE4FpiwbSeib6+gTyv8/tTgHXAT32aD/rzJwMr\ngBrgb8CBIWx5CJjTicwcb88DwHbgPWBSIHwNcLzfz/G6VgPvA/8JrAvIHgy85dOZBzzWbldnNvh8\nLgPe9tdxHpAdQ+d9gVe8XCWuQoK/9grU++t9JjAIeBbY4vV+Fhjp5a8HWoFGL3+bP78/8CKwFfgQ\nmBniWg8CVvv9HwE3xpEdgavwXd5ZuraF2/pcAdu6eMNgKtBCnEKZcA5qI/DdDmSO8Q93DnAD8Eqc\nfPb3ae7VQdg1wOt+/zziO6iXgRtxrcEWYGIYm+iCg/Ly90UVrBcDS4CRQBZwF/BoIPx7QL4PuxlY\nESut6OsbLYNzUC3AL316ObiCfzNuFeo04Du4Aj2rEzvCOqhG3EKiacD/AEsC4Wv41EHdALwKFAGj\ncJWLdT4sE1cJ+Q8gAzgD55zb7Yprg99fimuhFwErgdkxdH4UuBJXccgGjopzbQfjWt0D/D16HPhj\nILwM+EHgOBdXCfoubh28g3FOcFwMXY7DOdw6XEWkxv/W+/2jA7JXeDkFVuMdpW3d36yLr/8xGKhU\n1ZZO5GaKSE1wiwovBio6iFeBKyCKfF4dyQTTIIZMRSA8JiIyGucUH1HVTThndW5n8aKojLL1gJDx\nZgNXquo6VW3CFepntHe/qeo9qro9EDZBRAZ2UbcgbcDVqtqkqg24lthdqvqGqraq6v247tHDu5FH\nkMWqulBVW3Et5gkx5GYC16vqVlUtB24NhB2Oc0w3q2qzqj4BvBkID2PDraq6QVW3As8AB8XQoxnX\nDTtcVRtVNea4papWqeqTqrpDVbfjWk1Hx5LHtfLWqOq9qtqiqn8HngRmxEj/ZVUtBP7oZUbgnG2x\nqhaq6isB2RtwTvIQ3HWujaOH0QXMQfU/qoDiEGMY8/2D9MkWFV4JDOsg3jBcQVrt8+pIJpgGMWSG\nBcLjcQ6wUlVX+OOHgbNFJCNE3HaKo2xdGTLeGOCpgANfiWs9lopImojcICIficg2XOEEIZxuHLao\namNU/pdGVSJG4VobiSA4oL8DyI7xvxmOa120szYqbL2qaozwMDZE65EXQ9/Lcd3GS/0sw+/FkENE\nBojIXX5iwjZcN2ChiKTFiDIGmByl57eAoTHSX+dlzgLux7USxwAVInJTtLw6/o7rEr8mlt5G1zAH\n1f94HVdDPbWb6bxEx7XHmbiuuR1e5rDgjLwoPsSNq+ySjohEcN0vL4fQ41xgbxHZ6GdI3YRzAl8N\nZUX3KAemRTm3bFVdj5sUMp3/v717D5KqPPM4/v3NDHIVb5hRQUEpEVnW1XATMNRANOJlQ7xRo2iy\n6kbZiMIKcSEYBRVjxPtqBDGoIYnkRpQibDREpjZFqStEymgQGFDDcJGIAg6IMMOzf7xnTNPQzEz3\nzPQ5zfOp6pruPqfPeZ/unvP0eznvCX1VRxCaEyEcQCE056TbSWhyqpN+8Et/zTpCzSV1/+3M7Pkc\nYjVMXhsAAA6NSURBVMrGRkJSqXNS2rLOkpRheZPFYGabzOzbZnYCcCPwo4OM3BsPnAYMsDBAYUj0\nfKbPZx2huTq1nB3M7D8ylKULoTl9UfTj7ingpuh1tx4kjBKge32xuobxBJUwZrYNuAN4QtI3ol+S\nrSRdIOn+RmxqKjBI0jRJR0s6XNLNhITxX9G+FhE6lX8rqY+kkmi90ZKui35VTwBuVxj23kbSccDT\nQEfg4ZT9KVqeehtI+GfuT2j2ORPoTRgV2NhmvmzMAKZJ6hoV8FhJI6JlhxN+CGwhJJ170177IXBK\n2nPLCbW/YknDOXiTE8AsYLSkAQraS7pI0uEHWjn6nNsQ/m9LovcwU42hMX4JTJJ0VPRj5OaUZa8S\n+s5uifZ/KeHzyiqGg5F0RcqPoU8ISWZv9Dj9/T6cUFvZKulo4M60zaWvvwDoIemaKI5WkvrV0xzc\nhzA4BELz3dK08hZJujF63ySpP3ATDfth5hrAE1QCmdmDwK3A7YRRTOuAMYT28oZuYzVwDqFf4n3C\nL+XLgPPNbEnKqpcDCwmjr7YROtD7EmpXWBjSfQ2hE30LYRRYW2CwmW1J2c4gwgEl9XY98KKZ/SX6\n9bzJzDYBjwIXRweehtiqfc+DOtgv3FSPAvMJQ+s/JQyYGBAt+wmhKWt9FNNraa/9MdArai6qe9/H\nAv9K6EQfRT2fh5ktBb4NPE44IFcSBpRkMovwvl1JGEzwGeG9z9VUQqzvAS8T+lHqyrgbuDQq18eE\nEXTzcojhYPoBr0uqJnwuY81sbbRsCvBc9H6PJAxaaUtoRn4N+H3ath4l9Cd+IumxqJ/qa0A5sIHQ\n7Fg3YCWTPsCfo9pjT8JIyHSXEEaBfkoYvPLf0c01Ae3btOycc87Fg9egnHPOxZInKOecc7HkCco5\n51wseYJyzjkXS9lMWNnsOnXqZN26dQNgx44dtG/fPr8FagYeV3IUYkxQmHEVYkxQeHEtW7bsIzM7\ntr71YpmgunXrxtKl4ZSDiooKysrK8lugZuBx5cgMKith+XJ4803YsAEmTYLTTmvyXflnlRyFGBMU\nXlySPqh/rRyb+CQNl7RSUqWkiQdY3lPSq5I+lzQhl30594W9e+HSS6FHDxg5EqZPh1/9CsrK4N13\n810651wTyTpBRWewPwFcAPQCrpTUK221j4FbgAeyLqFz6e69F154IdSYli6F6urw1wyGDvUk5VyB\nyKUG1R+oNLO10dnmcwlzl33BzDab2RuEWYqdy92iRXDHHTBqFEybBn36QOvWcPrpsHhxSFJlZbCi\nofPFOufiKpcE1Zl9Z0Cuip5zrnmsXw9XXQW9esHMmbDP/KWEJFVREZ6/5BLY47+LnEuyrKc6UrjU\n93Az+/fo8TWEmYXHHGDdKUC1mWVs6pN0A+HaMpSWlvaZO3cuANXV1XTokGl2/uTyuBpHtbWcOXYs\nHdasYdnMmew86aSM6x6zZAn/fPvtrBo3jg0jRmRcr6H8s0qOQowJCi+uoUOHLjOzvvWumO2VDoGB\nwEspjycBkzKsOwWY0NBt9+nTx+osXrzYCpHH1UizZ5uB2Zw59a+7d6/ZkCFmX/qS2fbtOe/aP6vk\nKMSYzAovLmCpNfMVdd8ATpV0sqTDCLMEz89he84d2O7dcNdd0K9f6HuqjwQPPACbN8P9jbkCiXMu\nTrJOUBYuOT4GeIlwJdJfmtk70bWCRgNIOk5SFdGlIaKrVHZsioK7Q8gzz8D774ckld7vlEm/flBe\nDg8+GPqunHOJk9N5UGa20Mx6mFl3M5sWPTfDzGZE9zeZWRcz62jhSpRdzGx7UxTcHSJ27YJ77oFB\ng+D88xv32mnToKYmjPpzziWOz8Xn4m3WLKiqgrvvbnjtqc4pp8CYMaEG9s6BrjXnnIszT1Auvnbu\nDCfllpXBsGHZbWPyZGjXLmzHOZconqBcfD35JGzaFPqesnXMMfCd78DcubB6ddOVzTnX7DxBuXja\nuTOMwDv3XPjKV3Lb1vjxYbYJr0U5lyieoFw8zZgRholPmZL7tkpL4YYbYM6cMBrQOZcInqBc/KTW\nngYPbpptfve7UFwM993XNNtzzjU7T1AufmbOhA8/hDvvbLptdu4M110XRvRVVTXddp1zzcYTlIuX\nzz4Ltadhw+Ccc5p22xMnhmtJeS3KuUTwBOXiZebMMHKvKWtPdbp2heuvD/tYtarpt++ca1KeoFx8\n7NgBP/xhuOjgkCHNs4+pU6FtW7jttubZvnOuyXiCcvFx112h9nTPPc23j9JS+N734MUXwwUOnXOx\n5QnKxcPbb8NDD8G114Z595rTuHGhue/WW6G2tnn35ZzLmicol39mYbaHjh1b5vIYbdrAD34Ay5eH\nc6Occ7HkCcrl33PPwZ/+FPqfOnVqmX2Wl8OAAWGuvm3bWmafzrlG8QTl8mvLlnAS7cCB4TylliLB\nY4+F2Sq++c0w/Nw5FyueoFz+1NSEpPTJJ2Fqo6IW/jr27x/6vebPb96BGc65rHiCcvlhBjfeGJLD\nww/DGWfkpxxjxoQa1JQpsGBBfsrgnDsgT1AuPyZOhNmz4fvfh5tvzl85pFB7O+ssGDUKVq7MX1mc\nc/vwBOVallkYDHH//TB6dDhxNt/atoV58+Cww8LktAsX5rtEzjk8QbmWtGYNXHhhqD1dcQU8/njj\nL+PeXLp2hSVLoEsXuOiiMNPEnj35LpVzh7ScEpSk4ZJWSqqUNPEAyyXpsWj5W5K+nMv+XEJVV8Pd\nd0Pv3iEJPPII/Pzn4fIXcdKjB7z6aqjZTZ8ealO/+AVFu3fnu2TOHZJKsn2hpGLgCeA8oAp4Q9J8\nM/trymoXAKdGtwHAk9FfV8h27oS1a+GVV+B3v4OKCti9G0aODAMiTjgh3yXMrG3bcKn5oUNhwgQo\nL2dghw5w9dVw3nnQqxd07w6tWuW7pM4VvKwTFNAfqDSztQCS5gIjgNQENQL4iZkZ8JqkIyUdb2Yb\nc9jvwdXWhgNLzHWvqgrzwcWR2f73zf5xq60NQ8Rra2HXrjDJ644dsHUrgyor9z3xtWfPMAjissvC\nuU5JMXIkXH45LF7Mx/fdR+mzz4bBFAAlJdCtGxx1FBx5JBxxRJidolWrcCspCU2XRUXhb3ozZkya\nNWP9HcxSIcYEMYurd+9wVYAWkEuC6gysS3lcxf61owOt0xnYL0FJugG4AaC0tJSKigoAqqurv7jf\nEKqtZfCsWQ1eP1+OM6MmJgeq+lhdOaODrRUXY0VFWFERe1u3prZNG/a2bk1N+/ZUn302tSeeyK7S\nUraffjq7OncOr/3881CTSpriYqrHjqXj+PG0/+AD2kW3ths3UlJdTfH69ZSsXEnRnj2opoaimhpU\nN7+fGYrxCcBJ+g42VCHGBPGKa8uAAazo3r1F9pVLgmpSZvYU8BRA3759raysDICKigrq7jdYdXXT\nFq4ZZBVXAvylAOOqqKhgSIHFBIX5HSzEmCBecZVGt5aQS4JaD5yY8rhL9Fxj19nPsmXLPpL0QfSw\nE/BRDuWMK48rOQoxJijMuAoxJii8uLo2ZKVcEtQbwKmSTiYknXLgqrR15gNjov6pAcC2hvQ/mdmx\ndfclLTWzvjmUM5Y8ruQoxJigMOMqxJigcOOqT9YJysxqJI0BXgKKgdlm9o6k0dHyGcBC4EKgEtgJ\nXJt7kZ1zzh0KcuqDMrOFhCSU+tyMlPsG3JTLPpxzzh2akjCTxFP5LkAz8biSoxBjgsKMqxBjgsKN\n66Bkqee8OOecczGRhBqUc865Q5AnKOecc7EU6wRV32S0SSDpREmLJf1V0juSxkbPHy3pD5JWR3+P\nyndZsyGpWNKbkhZEjxMfVzQl168lvStphaSBSY9L0n9G37+3JT0vqU0SY5I0W9JmSW+nPJcxDkmT\nouPHSknn56fU9csQ1/ToO/iWpN9KOjJlWSLiylVsE1TKZLQXAL2AKyX1ym+pslIDjDezXsDZwE1R\nHBOBP5rZqcAfo8dJNBZYkfK4EOJ6FPi9mfUE/oUQX2LjktQZuAXoa2a9CaeFlJPMmJ4Fhqc9d8A4\nov+zcuCfotf8KDquxNGz7B/XH4DeZnYGsAqYBImLKyexTVCkTEZrZruBusloE8XMNprZn6P7nxIO\ndp0JsTwXrfYc8I38lDB7kroAFwFPpzyd6LgkHQEMAX4MYGa7zWwrCY+LcEpJW0klQDtgAwmMycz+\nF/g47elMcYwA5prZ52b2HuF8zP4tUtBGOlBcZvaymdVED18jzMQDCYorV3FOUJkmmk0sSd2As4DX\ngdKUWTU20XLTWzWlR4DbgNQZUZMe18nA34FnoqbLpyW1J8Fxmdl64AHgb4SJmreZ2cskOKY0meIo\npGPIdcD/RPcLKa6DinOCKiiSOgC/AcaZ2fbUZdEJzYka7y/pYmCzmS3LtE4S4yLUNL4MPGlmZwE7\nSGv6SlpcUZ/MCELyPQFoL+nq1HWSFlMmhRJHKkmTCV0FP8t3WVpanBNUVhPNxpGkVoTk9DMzmxc9\n/aGk46PlxwOb81W+LA0Gvi7pfULz6zBJPyX5cVUBVWb2evT414SEleS4zgXeM7O/m9keYB4wiGTH\nlCpTHIk/hkj6N+BiYJT946TVxMfVUHFOUF9MRivpMEKn4Pw8l6nRJInQn7HCzB5KWTQf+FZ0/1tA\nTK5G1jBmNsnMuphZN8Jn84qZXU3y49oErJN0WvTUVwkX4UxyXH8DzpbULvo+fpXQF5rkmFJlimM+\nUC6pdTSp9anA/+WhfFmRNJzQhP51M9uZsijRcTWKmcX2RphodhWwBpic7/JkGcM5hCaHt4Dl0e1C\n4BjCiKPVwCLg6HyXNYcYy4AF0f3ExwWcCSyNPrMXgKOSHhcwFXgXeBuYA7ROYkzA84R+tD2E2u71\nB4sDmBwdP1YCF+S7/I2Mq5LQ11R33JiRtLhyvflUR84552Ipzk18zjnnDmGeoJxzzsWSJyjnnHOx\n5AnKOedcLHmCcs45F0ueoJxzzsWSJyjnnHOx9P+s8dfmsvAwLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1142d6c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXt4XVWZ/z/f3NMkbZqmTW/QckdwuNhKARGKIAIyoHIV\nB0RErCPKbwBvo6PgCDKMIDAiFx0EBCkFRAFBEYbCcClQkOEqcrHYtEnbpE3apGmu7++PtSKHQ87J\nSU4uJ+n7eZ79nL33etda77v3Putdt72XzAzHcRzHyTXyRlsBx3Ecx+kLd1CO4zhOTuIOynEcx8lJ\n3EE5juM4OYk7KMdxHCcncQflOI7j5CTuoJytBkk1kh6VtEnSpaOtz3Ai6XxJN6cJf1nSwhRhCyXV\npol7g6QfDIGajpMWd1BjFEknS1ouqUVSnaT7JR0Qw/osnCSZpB0TjneTdLek5lhoPyxp/6Q4RTG9\n1yW1Sloh6XpJcxNkjpL0dAxvlHSLpNkJ4adJeqwfe26Q1CVpRsK5a6J9LZI6JHUmHN8vaW60qSVp\nOzFFNmcCDcBEMzs37QXuh5EupCVNk3SrpNXxfj0uacFg0zOz3c1s6RCqOKwkP7sZyC+VdMYQ5b1E\n0mGSiiXVJ4UVx//DRkn1ks4ZijydgDuoMUj8E1wOXATUANsCVwFHDyCNHYDHgReB7YCZwF3AA5L2\nSxC9I6Z7MjAJ2BNYDhwS0zkO+FXUpxrYHWgHHpM0OUNdyoBjgWbgn3rPm9kiMys3s/Jo6229x2Z2\nREISlQnny83sthRZzQFesRx4O11SwQCjlAPPAPOAKuBG4HeSyodaN+c9zCM883sALyWFnQ/sRHi2\nDga+LunwEdVuPGNmvo2hjeAkWoDj08icD9zcx3kDdoz7vwTu60PmauDRuH8o0AZskyIfAW8DX086\nn0f4I38/Hp8GPJZG31OBlcDZwEuZ2gTMjTYVZHDdbgA6gY54/Q6Nen4TeBNoBJYAVQlxbgfqCY7z\nUWD3eP7MpLTuSb6+CXn+IO4vBGqBb8Q0fxnPHwU8DzQBTwB7DOBZ2AjMS3O9lgA3AZuAl4H5CeEr\ngEPjfmnUdQPwCvA1oDZBdm/guZjObcDiXrv6syHmcx7wQryOtwElKXTeEXgkyjUQKiTEa29Aa7ze\nJwKTgXuBdVHve4HZUf5CoBvYEuV/Es/vCvwRWA+8BpyQwTWeDLwV978EXJIUvho4LOH4+8DikSwT\nxvM26gr4NsAbBocDXaQplMnMQdUDn+tD5uD45y4FLgYeSZPPrjHN7foIuwB4Mu6fRnoH9RBwCaE1\n2NVXoduXTQzAQUX5G5IK1rOBZcBsoBi4Frg1Ifx0oCKGXQ48nyqt5OubLENwUF3Af8T0SgkF/1pg\nAZAPfJZQoBdnYMtesQCelOYZ2AIcGdP+IbAsIXwF7zioi4H/JbTMtiFULmpjWBGhEvIvQCFwHME5\n99qV1oa4/zShhV4FvAosSqHzrcC3CRWHEuCANNd2CqHVPSHeo9uB3ySELwXOSDguI1SCPgcURL0b\ngN1S6HIIweG2ECoiTfG3Ne4fRHBeBtQkxDsWeHE0yobxuHkX39hjCtBgZl39yJ0gqSlxSwqvBur6\niFdHKCCqYl59ySSmQQqZuoTwlEjaluAUf2VmawjO6tT+4iXRkGTr+zKMtwj4tpnVmlk7oVA/rrf7\nzcyuN7NNCWF7Spo0QN0S6QG+Z2btZtZGaIlda2ZPmVm3md1I6B7dN10ikiYSWsAXmFlzGtHHzOw+\nM+uO8numkDsBuNDM1pvZSuDKhLB9CY7pcjPrNLM7CF2NvWRiw5VmttrM1gP3EJxrX3QSuspmmtkW\nM0s5bmlmjWZ2p5ltNrNNhFbTQankCa28FWb2CzPrMrM/AXcCx6dI/yEzqwR+E2VmEZxttZlVmtkj\nhG5XCC2+XjYSHKYzBLiDGns0AtUZjGEsiX+kv29J4Q3AjD7izSAUpBtiXn3JJKZBCpkZCeHpOAV4\n1cyej8e3ACdLKswgbi/VSba+mmG8OcBdCQ78VULrsUZSvqSLJb0paSOhcIIMnG4a1pnZlqT8z02q\nRGxDaG30iaRSQiG/zMx+2E9+iQP6m4GSFM/NTELrope3k8JWmZmlCM/EhmQ9Uo2bfZ3Qbfx0nGV4\nego5JE2QdK2kt+P9eRSolJSfIsocYEGSnp8BpqdIvzbKfJow3rc2plEn6bIo1hJ/JyZEnUToCnWG\nAHdQY48nCTXUT2SZzoP0XXs8gdA1tznK7JM4Iy+J1wjjKu9KR1IeoavjoQz0OBXYPs6AqgcuIziB\nIzOyIjtWAkckObcSM1tFmBRyDGGsahKhOxFCAQqhayeZzYQup16SC7/kOCsJLZfE/CeY2a19KSup\nmFCjrwW+mJmJGVFHcCq9bJsUNkuSUoQPyIZ0mFm9mX3BzGYS7Ptpmpl75wK7AAvMbCJwYDyf6v6s\nJHRXJ+pZbmZfSqHLbEJ3+oOxcncd8OUY75wos4FwfRJbpnsSxvucIcAd1Bgjdul8F7hK0idiTbJQ\n0hGSLhlAUhcA+0u6UFKVpApJXyE4jG/EvB4kDCrfJWmepIIot0jS6bFWfR7wHYVp7yWSpgM/J9Qq\nf5yQn2J44rYfsAOwD6HbZy/g/YRZgQPt5hsM1wAXSpoTFZwq6ZgYVkGoCDQSnM5FSXHXANsnnXue\n0PrLjzO50nU5AfwMWCRpgQJlkj4u6T1dRLFFeQdh0spnzawnczP7ZQnwLUmTY2XkKwlhTxLGzr4a\nn7NPEe7XgG3oD0nHJ1SGNhCcTK+dyde7gnAtmiRVAd9LSi5Z/l5gZ0mnRDsKJX2wn+7geYTJIQAf\nIMzkS+YmwvM/Oab1BcLYozMEuIMag5jZpcA5wHcIs5hWAmcRateZpvE6cAChxreCUBM8FviYmT2e\nIHoccB9h9lUzYQB9PqF1hYUp3acQBtEbCbPASoEPmVljQjr7EwqUxO3zwG/N7MVYe643s3rgCuCo\nWPBkQpPe/R5Upu+iXAHcTZhav4kwYaL33aKbCF1Zq6JNy5Li/jewW+wu6r3uZwP/SBhE/wz93A8z\nW04o0H5CKJDfIEwo6Yv9CeMoh/Fuez+cmalpuYBg61+BBwjjVb06dgCfinqtJ8yg+/UgbeiPDwJP\nSWoh3JezzeytGHY+cGO83icQJq2UErqRlwG/T0rrCsJ44gZJV8ZxqsOAkwgz7+p5Z8JKKuYBz8XW\n46703TL6HmEW6NuEiRmXmFmyLs4g0bu7lh3HcRwnN/AWlOM4jpOTuINyHMdxchJ3UI7jOE5OkpWD\nknS4pNckvSHpm32Ef0bSC5JelPSEpFQvCjqO4zjOuxj0JIn4QtxfgI8S3st4Bvi0mb2SILM/4SXM\nDZKOAM43s36/wFxdXW1z584dlF4jTWtrK2VlZaOtxpDiNo0NxptN480ecJtS8eyzzzaY2dT+5Ab6\nReVE9gHe6J0GKmkx4cXGvzsoM3siQb73m2f9MnfuXJYv7+uVg9xj6dKlLFy4cLTVGFLcpuGhbT28\nfj80/RU2/BXaGmH/82DbAwaXXi7YNJSMN3vAbUqFpLf7l8rOQc3i3Z9HqeWdd0j64vPA/Vnk5zhj\nltZ18IsDoPEv4bh8OlgP3HQofOoW2O3Y0dXPcXKRbLr4jgMON7Mz4vEphM+OnNWH7MHATwlfJ25M\nDo8yZxI+PElNTc28xYsXD0qvkaalpYXy8vG1JI/bNLR0t+Xzf+fsSetbZez2/Zep3KuJ/OIeOpsL\neenb72fjKxPZ4ctvMPvYVQNKd7zdp/FmD7hNqTj44IOfNbP5/QoO9PPnvRuwH/CHhONvAd/qQ24P\nwpvWO2ea9rx582ys8PDDD4+2CkOO2zR0dLWb/fIwswvyzP782/eGd2w2W/wJs/Mx+5/vDizt8Xaf\nxps9Zm5TKoDlNszLbTwD7CRpO0lFhE+I3J0ooLCUwq+BU8zsL1nk5ThjDjP47enw5gNw1HWwSx/r\nHReWwvF3wJ6fhUe/D7XJH1RynK2YQTsoC+sRnQX8gbBMwRIzezl+SHRRFPsuYU2hn0p6XtLYmPng\nOEPAK7fDi7fAwu/DBz6fWi4vH474L6iYBfcugp7+VvpynK2EbCZJYGb3ET4kmnjumoT9M4AzssnD\nccYinW3wx69BzR7w4X/tX764Ao64EpYcC0/9F+z3L8Ovo+PkOv4lCccZBp74ETT/DQ6/IrSQMmHX\nT8JOH4el34WNtcOrn+OMBdxBOc4Qs7EWHr8YdjsO5i7MPJ4Uuvp6uuH3/2/Y1HOcMYM7KMcZYh78\nZnAyH/3PgcedvB0c+G/w6p1hcoXjbM24g3KcIWTlk2FixP7nQeXcwaWx/7kh7kPfCi/zOs7Wijso\nxxkizOCP50H5DDjgPZ9Ozpz8Ilh4AdQ9B6/+un95xxmvuINynCHi9d/ByifgoO9BUZYfD/iHz8DU\n3eF/vuPTzp2tF3dQjjMEWA/8z7ehakfY+/Ts08vLh4/8ABpfg/+7Kfv0HGcs4g7KcYaAlxbDmhfC\nS7n5hUOT5i7HwKwFsPR86NoyNGk6zljCHZTjZEl3Jzz8XajZE95/4tClK8EhF8HGlbD8mv7lHWe8\n4Q7KcbLkT/8NG96Ej1wIGuJ/1HYfge0/Co/+e1hPynG2JtxBOU4WdLTAI9+HbT4EOx05PHkc9iPY\n0hTycZytCXdQjpMFj/w7tNSFl3Kl4cmjZg/4wBfgmaug4c/Dk4fj5CLuoBxnkKx7BZZdBnudDtvs\nN7x5HfzvUFgGfzhnePNxnFzCHZTjDAIzuO+s8L7ToRcPf35lU+Gg78Ib98Pr9w9/fo6TC7iDcpxB\n8PJtsOJh+MhFwXmMBPucBVU7wQPnhJmDjjPecQflOAOkfVPoapsxD+adOXL55hfBx34cxqEe/reR\ny9dxRgt3UI4zAMzg/q+EiRFHXpX5Wk9Dxc4fh3mL4PH/gMZlVSObueOMMO6gHGcALLsc/u9GOPC7\nMHvB6Ohw+I/DS8F//uH7aF45Ojo4zkjgDspxMuTNB8LXynf9JCz83ujpUVACxy8B6xR3nuTjUc74\nxR2U42RA4+twx4nhC+OfvGnovxgxUKbsDDuf+xdWPgH3f9XXjXLGJwWjrYDj5DrrXoXFR0NeAXz6\n7uyX0hgqph2ylsotu/Hkj6CtAT75y9C6cpzxgjsox0nDS7fB3Z+Hwglw0t2DXyV3uPjoJVAxAx44\nFzbVwUm/hQlTRlsrxxkavIvPcfqgawv8/v/BnSfB9D3hi38a/q9FDAYJ9jsHjlsCq5fDf+8Lr90T\nZhs6zlgnKwcl6XBJr0l6Q9J7FrlW4MoY/oKkD2STn+MMN81/gwe/BZfNhqeugAX/Dz67FCbOGm3N\n0rP78XDqQ2EsavHR8LP58OffuqNyxjaD7uKTlA9cBXwUqAWekXS3mb2SIHYEsFPcFgBXx1/HGXWs\nB1rqoe45WPUMrFoGbz0YwnY5OjinuQeNro4DYdsPwZf/DC/eAo/+AG77BJRPhzkHwpyDYJv9Qxdl\n8aTh+7Ct4wwl2YxB7QO8YWZvAUhaDBwDJDqoY4CbzMyAZZIqJc0ws7os8k1LTzf87p+HK/X3Urd6\nZzbdOnL5jQRj0iaLrYXYYrAesO7wPPR0Qf2q3VlVDp1tYemK1rWweV0IgzArb+pusP/XYP6XoHLO\nqFmSFfmFsNdpsMc/wcu3w+u/g7cfgZeXvCNTWBZahCWTw9haURkUlIaXjvMKQPlxlqKiI1OSQxsm\n5zYmn7t+GI82bSydCQtHJq9sHNQsIPE1wVre2zrqS2YW8B4HJelM4EyAmpoali5dOiilrFu8dMe+\ng4o7GHqsika1j1h+I8GYtSkWpgCSQR4o31C+QX4xHSUbySvqoaC8i/K9O6ma3EHRlA7Kd9xE+Y4t\n5JeGudrP/xX466hZkTEtLS3p/yczoOoMmPx52FJXwqY/V9DeUEz7umI61hWzubWA7oY8emrz6WnP\nw3qEdQvrEfSAWbyYid2Ew9hlOGafuzSMR5sq9ioddPk8UHJmFp+ZXQdcBzB//nxbuHDhoNM6uHGI\nlMqApUuXko2uuYjbNDYYbzaNN3tgvNr05ojZlI2DWgVsk3A8O54bqMx7ePbZZxskvZ2FbiNJNdAw\n2koMMW7T2GC82TTe7AG3KRUZdaJn46CeAXaStB3B6ZwEnJwkczdwVhyfWgA0ZzL+ZGYjtIBB9kha\nbmbzR1uPocRtGhuMN5vGmz3gNmXLoB2UmXVJOgv4A5APXG9mL0taFMOvAe4DjgTeADYDn8teZcdx\nHGdrIKsxKDO7j+CEEs9dk7BvwJezycNxHMfZOvEvSWTPdaOtwDDgNo0NxptN480ecJuyQuavmjuO\n4zg5iLegHMdxnJzEHZTjOI6Tk7iDchzHcXISd1CO4zhOTuIOynEcx8lJ3EE5juM4OYk7KMdxHCcn\ncQflOI7j5CTuoBzHcZycxB2Us9UgqUbSo5I2Sbp0tPUZTiSdL+nmNOEvS1qYImyhpNo0cW+Q9IMh\nUNNx0uIOaowi6WRJyyW1SKqTdL+kA2JYn4WTJJO0Y8LxbpLultQcC+2HJe2fFKcopve6pFZJKyRd\nL2lugsxRkp6O4Y2SbpE0OyH8NEmP9WPPDZK6JM1IOHdNtK9FUoekzoTj+yXNjTa1JG0npsjmTMI6\nNhPN7Ny0F7gfRrqQljRN0q2SVsf79bik5BWsM8bMdjezpUOo4rCS/OxmIL9U0hlDlPcSSYdJKpZU\nnxR2gqQnJG2WtHQo8nPewR3UGETSOcDlwEVADbAtcBVw9ADS2AF4HHgR2A6YCdwFPCBpvwTRO2K6\nJwOTgD2B5cAhMZ3jgF9FfaqB3YF24DFJkzPUpQw4FmgG/qn3vJktMrNyMyuPtt7We2xmRyQkUZlw\nvtzMbkuR1RzgFcuBD1BKGuhKAuWENdjmAVXAjcDvJJUPtW7Oe5hHeOb3AF5KCltPePYvHmmltgrM\nzLcxtBGcRAtwfBqZ84Gb+zhvwI5x/5fAfX3IXA08GvcPBdqAbVLkI+Bt4OtJ5/MIf+Tvx+PTgMfS\n6HsqsBI4G3gpU5uAudGmggyu2w1AJ9ARr9+hUc9vAm8CjcASoCohzu1APcFxPgrsHs+fmZTWPcnX\nNyHPH8T9hUAt8I2Y5i/j+aOA54Em4AlgjwE8CxuBeWmu1xLgJmAT8DIwPyF8BXBo3C+Num4AXgG+\nBtQmyO4NPBfTuQ1Y3GtXfzbEfM4DXojX8TagJIXOOwKPRLkGQoWEeO0NaI3X+0RgMnAvsC7qfS8w\nO8pfCHQDW6L8T+L5XYE/EpzKa8AJGVzjycBbcf9LwCUp5M4Alo5kWbA1bKOugG8DvGFwONBFmkKZ\nzBxUPfC5PmQOjn/uUkKt8JE0+ewa09yuj7ALgCfj/mmkd1APAZcQWoNdfRW6fdnEABxUlL8hqWA9\nG1gGzAaKgWuBWxPCTwcqYtjlwPOp0kq+vskyBAfVBfxHTK+UUPCvJaw2nQ98llCgF2dgy16xAJ6U\n5hnYQlgwNB/4IbAsIXwF7zioi4H/JbTMtiFULmpjWBGhEvIvQCFwHME599qV1oa4/zShhV4FvAos\nSqHzrcC3CRWHEuCANNd2CqHVPSHeo9uB3ySELwXOSDguI1SCPkdYB29vghPcLYUuhxAcbguhItIU\nf1vj/kFJ8u6ghmHzLr6xxxSgwcy6+pE7QVJT4pYUXg3U9RGvjlBAVMW8+pJJTIMUMnUJ4SmRtC3B\nKf7KzNYQnNWp/cVLoiHJ1vdlGG8R8G0zqzWzdkKhflxv95uZXW9mmxLC9pQ0aYC6JdIDfM/M2s2s\njdASu9bMnjKzbjO7kdA9um+6RCRNJLSALzCz5jSij5nZfWbWHeX3TCF3AnChma03s5XAlQlh+xIc\n0+Vm1mlmdxC6GnvJxIYrzWy1ma0H7iE4177oJHTDzjSzLWaWctzSzBrN7E4z22xmmwitpoNSyRNa\neSvM7Bdm1mVmfwLuBI5Pkf5DZlYJ/CbKzCI422ozqzSzR9Lk5QwR7qDGHo1AdQZjGEviH+nvW1J4\nAzCjj3gzCAXphphXXzKJaZBCZkZCeDpOAV41s+fj8S3AyZIKM4jbS3WSra9mGG8OcFeCA3+V0Hqs\nkZQv6WJJb0raSCicIAOnm4Z1ZrYlKf9zkyoR2xBaG30iqZRQyC8zsx/2k1/igP5moCTFczOT0Lro\n5e2ksFVmZinCM7EhWY9U42ZfJ3QbPx1nGZ6eQg5JEyRdK+nteH8eBSol5aeIMgdYkKTnZ4DpKdKv\njTKfJoz3rY1p1Em6LJVeztDiDmrs8SShhvqJLNN5kL5rjycQuuY2R5l9EmfkJfEaYVzlXelIyiN0\nvzyUgR6nAttLqo8zpC4jOIEjM7IiO1YCRyQ5txIzW0WYFHIMYaxqEqE7EUIBCqHLKZnNhC6nXpIL\nv+Q4Kwktl8T8J5jZrX0pK6mYUKOvBb6YmYkZUUdwKr1smxQ2S5JShA/IhnSYWb2ZfcHMZhLs+2ma\nmXvnArsAC8xsInBgPJ/q/qwkdFcn6lluZl9KoctsQnf6g7Fydx3w5RjvnIHa5gwOd1BjjNil813g\nKkmfiDXJQklHSLpkAEldAOwv6UJJVZIqJH2F4DC+EfN6kDCofJekeZIKotwiSafHWvV5wHcUpr2X\nSJoO/ByYCPw4IT/F8MRtP2AHYB9Ct89ewPsJswIH2s03GK4BLpQ0Jyo4VdIxMayCUBFoJDidi5Li\nrgG2Tzr3PKH1ly/pcNJ3OQH8DFgkaYECZZI+LqkiWTC2KO8gTFr5rJn1ZG5mvywBviVpcqyMfCUh\n7EnC2NlX43P2KcL9GrAN/SHp+ITK0AaCk+m1M/l6VxCuRZOkKuB7Sckly98L7CzplGhHoaQP9tMd\nPI8wOQTgA4SZfMk650sqIYxr5cXneiCtfycN7qDGIGZ2KXAO8B3CLKaVwFmE2nWmabwOHEAYl1hB\nqCkfC3zMzB5PED0OuI8w+6qZMIA+n9C6wsKU7lMIg+iNhFlgpcCHzKwxIZ39CQVK4vZ54Ldm9mKs\nPdebWT1wBXBULHgyoUnvfg8q0xruFcDdhKn1mwgTJnrfLbqJ0JW1Ktq0LCnufwO7xe6i3ut+NvCP\nhEH0z9DP/TCz5cAXgJ8QCuQ3CBNK+mJ/wjjKYbzb3g9nZmpaLiDY+lfgAcJ4Va+OHcCnol7rCTPo\nfj1IG/rjg8BTkloI9+VsM3srhp0P3Biv9wmESSulhG7kZcDvk9K6gjCeuEHSlXGc6jDgJGA1odux\nd8JKKuYBz8XW466EmZDJnEJ4lq8GPhz3fzYgq52U6N1dy47jOI6TG3gLynEcx8lJ3EE5juM4OYk7\nKMdxHCcncQflOI7j5CQD/WDliFBdXW1z584dbTUyorW1lbKystFWY0hxm8YG482m8WYPuE2pePbZ\nZxvMbGp/cjnpoObOncvy5e955SAnWbp0KQsXLhxtNYYUt2lskKlNjd3d/HbTJpa1tf39pSIBexQX\nc+zEicwsyI1iYGu+R2OJobBJ0tv9S+Wog3IcJ3ue3bKFm5ub+Z/WVrqAnYuKmBA/CNFpxrVNTVzb\n1MSHS0s5edIkPjxhQvoEHWeEcQflOOOMTjN+smEDP29qojIvj89MmsSxFRXsUFT0LrnVnZ3cuWkT\nv960iUX19XyyvJx/ra5mQp4PTTu5gTsoxxlHrO7q4mtr1vB8ezvHV1TwjSlTKE3hcGYWFvKVqiq+\nNHkyP92wgeuamnihvZ3LamrYMcmZOc5o4FUlxxknLG9r49jaWl7v6OA/p03j/KlTUzqnRAokvlpV\nxXXTp9PU08OJq1bxQEvLCGjsOOnJykFJOlzSa5LekPTNPsJ3lfSkpHZJ52WTl+M4qVne1sai+nqq\n8/O5ffZsjiwf+Erw+0+YwJ2zZrFrURHnrV3LH9xJOaPMoB1UXHflKuAIYDfg05J2SxJbD3wV+NGg\nNXQcJy29zmlGQQHXz5jBnMLBf0x7akEB182YwR7FxXzNnZQzymTTgtoHeMPM3opfPF5MWD/n75jZ\nWjN7hrBSpuM4Q0yyc5o6BFPGy/LyuNadlJMDDPpr5pKOAw43szPi8SmExcPO6kP2fKDFzFK2pCSd\nSVg+mpqamnmLFy8elF4jTUtLC+WD6E7JZdymscFfOju5drvtqOzu5qtr1jCpZyiXiIItEldNncqK\n4mK+vHYtu7a3D2n6yYzHe+Q29c3BBx/8rJnN708uZ2bxmdl1hFUrmT9/vo2Vl9v8RbyxwXizqb6r\ni2+/9RaTCgu5Ze5cpu+yy7Dk86GeHk5ZtYpfTJ/OzbNmsdMwzu4bb/cI3KZsyaaLbxXvXiZ6djzn\nOM4wsqmnhy/V17MlL4+rp09n+jB+CaIiL4+rZ8ygNC+PRXV1rO3qGra8HCeZbBzUM8BOkraTVERY\nqfLuoVHLcZy+6DTjnDVreKujgzMaGtilON2CsEPDjIICrp4+nY3RMbYOcVei46Ri0A7KzLoIy4z/\nAXgVWGJmL0taJGkRgKTpkmqJy5NLqpU0cSgUd5ytDTPj3xsaeKKtjfOnTuV9W7aMWN7vKy7mspoa\nXu/o4Lw1a+jylbidESCrvgEzuw+4L+ncNQn79YSuP8dxsuSG5mbu3LSJMysr+WRFBUtHOP8PT5jA\nv1ZX8+8NDfxnYyPfqq4eYQ2crY2cmSThOE5qHmpt5dL16/lYWRlfmTx51PQ4aeJE3u7s5KbmZuYU\nFnLypEmjposz/nEH5Tg5zsvt7Xxj7Vr+obiYi6ZOJS9+kXy0OK+qir91dvLDxka2KSz0r6A7w4Z/\ni89xcpjazk7+ub6eyvx8rqypoSQHvjSeL3HJtGnsXFTEOWvW8Mowvx/lbL2M/tPuOE6frO/u5sz6\nejrMuGb69CH5SsRQURanuE/Kz2dRfT1/6/SPxThDjzsox8lBWuOU7vquLn46fXpOLn8xraCA66ZP\np9uMM+vqaPB3pJwhxh2U4+QYHfFdp1fa2/nRtGnsXVIy2iqlZPuiIn46fTrrurv9HSlnyHEH5Tg5\nRIcZ/7JmDY+1tfG96mo+UlY22ir1y54lJVxWU8NrHR2cWVfHJndSzhDhDspxcoQtPT18pb6epZs3\n82/V1RwSmH8WAAAgAElEQVQ3cey8037QhAlcWlPDS+3tnFFXR3N392ir5IwD3EE5Tg7Q1tPDWWvW\n8HhbG9+vruakMeScevloWRmX19TwWns7p9fVscGdlJMl7qAcZ5RZ19XFGXV1PNXWxoVTp3LsGHRO\nvRxcVsZ/TZ/OXzs7OXX1alZ0dIy2Ss4Yxh2U44wiL2zZwgmrVvFaRweXTpvGMRUVo61S1nx4wgSu\nnT6d9d3dnLR6NY9u3jzaKjljFHdQjjMKmBm/3riRU1evpkjilpkzOWwcLWz3wdJSlsyaxayCAv65\nvp7rNmygxz8w6wwQd1COM8Ks7urirDVr+LeGBuaXlnLbrFkjsmzGSDOrsJCbZ87kiLIyrtiwgc/W\n1fGGd/k5A8AdlOOMEF1m3NDUxNErV/JUWxtfq6rimunTqczPH23Vho3SvDwumTaNH0ydypsdHRxb\nW8sV69ezxaeiOxmQO99OcZxxypaeHn7b0sINTU38rauLgyZM4DtTpjCzsHC0VRsRJPHJigoOmjCB\n/2xs5LqmJn6zaROnTJrE8RMnUpED3xd0chN3UI4zTPy1o4P7W1u5tbmZ9T09vL+4mP+aMoWDJ0xA\no/xF8tGgKj+fH06bxqcqKrh6wwYuXb+eazZs4PiJEzmqvBwfoXKSycpBSTocuALIB35uZhcnhSuG\nHwlsBk4zs+eyydNxcpXWnh5ebm/nybY2Hmpt5c34AdUDS0s5vbKS+SUlW6VjSuaDpaV8sLSUV9rb\nub6piV82N3NDczNTZs7kmcZGPlRayvuLi5k0jrs+ncwYtIOSlA9cBXwUqAWekXS3mb2SIHYEsFPc\nFgBXx1/HGXOYGZvN2NDdzbrublZ1dvK3ri5WdnbySns7b3Z2YoSB3fklJZw4cSKHlJUxPYe+Qp5L\n7FZczI9qaljf3c3Dra3ctnIlv2pu5sbmZgDmFhayW1ERcwoLmV1YyOyCAqYVFDA5P59yyZ39VkA2\n/5x9gDfM7C0ASYuBY4BEB3UMcJOZGbBMUqWkGWZWl0W+aek246LGxuFK/j2snjyZ/21oGLH8RoKx\nYlPaLqGEKc0GrJo8mYfXrft7HCM4nB6gm/Dc9P52mNFJ+C5eW08Pm83Y3NNDc08PHX1Mla7Jz2eX\noiI+Vl7OPxQX8w/FxeN64sNQU5Wfz7ETJzJl3To+eOCBvNjezotbtvBiezvPt7fz+9ZWkqdUFACT\n8vOZIDEhL49SiZK8PAqBQokCiXzC2lV5hEpDnkSvS1PS9i7SOL6BusSx8l8aEOXlLByhrLJxULOA\nlQnHtby3ddSXzCzgPQ5K0pnAmQA1NTUsXbp0UEp1A/fOmjWouIPBSkt5bsOGEctvJBgvNiUWJlZa\nyotNTSg6mMSCKd+MvHhcYEaBGflmFADFPT1MNaPIjLKeHsq6uynv6aGiu5vqri6mdHWRONWhG3h+\nBGwDaGlpGfT/JBdpaWnhmUcfBd7pdgHoAtYXFNBYUMDG/Hxa8vJoycujNS+P9rw82iU25+WxEeiW\n6JLoluiBsEmhQsI7lRqL55IZ6nGw8fJfSmSnvLwRe+5ypu/BzK4DrgOYP3++LVy4cNBpPTVEOmXC\n0qVLyUbXXMRtGhuMN5vGmz3gNmVLNg5qFbBNwvHseG6gMu/h2WefbZD0dha6jSTVwDhrw7tNY4Tx\nZtN4swfcplTMyUQoGwf1DLCTpO0ITuck4OQkmbuBs+L41AKgOZPxJzObmoVeI4qk5WY2f7T1GErc\nprHBeLNpvNkDblO2DNpBmVmXpLOAPxCmmV9vZi9LWhTDrwHuI0wxf4Mwzfxz2avsOI7jbA1kNQZl\nZvcRnFDiuWsS9g34cjZ5OI7jOFsn/o2R7LlutBUYBtymscF4s2m82QNuU1bI/BP4juM4Tg7iLSjH\ncRwnJ3EH5TiO4+Qk7qAcx3GcnMQdlOM4jpOTuINyHMdxchJ3UI7jOE5O4g7KcRzHyUncQTmO4zg5\niTsox3EcJydxB+VsNUiqkfSopE2SLh1tfYYTSedLujlN+MuSFqYIWyipNk3cGyT9YAjUdJy0uIMa\no0g6WdJySS2S6iTdL+mAGNZn4STJJO2YcLybpLslNcdC+2FJ+yfFKYrpvS6pVdIKSddLmpsgc5Sk\np2N4o6RbJM1OCD9N0mP92HODpC5JMxLOXRPta5HUIakz4fh+SXOjTS1J24kpsjmTsI7NRDM7N+0F\n7oeRLqQlTZN0q6TV8X49Lil5BeuMMbPdzWzpEKo4rCQ/uxnIL5V0xhDlvUTSYZKKJdUnhf0o/jc2\nSfqzpFOHIk8n4A5qDCLpHOBy4CKgBtgWuAo4egBp7AA8DrwIbAfMBO4CHpC0X4LoHTHdk4FJwJ7A\ncuCQmM5xwK+iPtXA7kA78JikyRnqUgYcCzQD/9R73swWmVm5mZVHW2/rPTazIxKSqEw4X25mt6XI\nag7wiuXAByglDXQlgXLCGmzzgCrgRuB3ksqHWjfnPcwjPPN7AC8lhbUC/0j4b3wWuCK5kudkgZn5\nNoY2wh+hBTg+jcz5wM19nDdgx7j/S+C+PmSuBh6N+4cCbcA2KfIR8Dbw9aTzeYQ/8vfj8WnAY2n0\nPRVYCZwNvJSpTcDcaFNBBtftBqAT6IjX79Co5zeBN4FGYAlQlRDndqCe4DgfBXaP589MSuue5Oub\nkOcP4v5CoBb4Rkzzl/H8UcDzQBPwBLDHAJ6FjcC8NNdrCXATsAl4GZifEL4CODTul0ZdNwCvAF8D\nahNk9waei+ncBizutas/G2I+5wEvxOt4G1CSQucdgUeiXAOhQkK89kZwBi3AicBk4F5gXdT7XmB2\nlL8Q6Aa2RPmfxPO7An8E1gOvASdkcI0nA2/F/S8Bl/Qjfzdw7kiUBVvDNuoK+DbAGwaHA12kKZTJ\nzEHVA5/rQ+bg+OcuBS4GHkmTz64xze36CLsAeDLun0Z6B/UQcAmhNdjVV6Hbl00MwEFF+RuSCtaz\ngWXAbKAYuBa4NSH8dKAihl0OPJ8qreTrmyxDcFBdwH/E9EoJBf9awmrT+YQa+AqgOANb9ooF8KQ0\nz8AWwoKh+cAPgWUJ4St4x0FdDPwvoWW2DaFyURvDigiVkH8BCoHjCM651660NsT9pwkt9CrgVWBR\nCp1vBb5NqDiUAAekubZTCK3uCfEe3Q78JiF8KXBGwnEZoRL0OcI6eHsTnOBuKXQ5hOBwWwgVkab4\n2xr3D+ojTilQBxw+UuXBeN+8i2/sMQVoMLOufuROkNSUuCWFVxP+TMnUEQqIqphXXzKJaZBCpi4h\nPCWStiU4xV+Z2RqCsxpoP35Dkq3vyzDeIuDbZlZrZu2EQv243u43M7vezDYlhO0padIAdUukB/ie\nmbWbWRuhJXatmT1lZt1mdiOhe3TfdIlImkhoAV9gZs1pRB8zs/vMrDvK75lC7gTgQjNbb2YrgSsT\nwvYlOKbLzazTzO4gdDX2kokNV5rZajNbD9xDcK590Unohp1pZlvMLOW4pZk1mtmdZrbZzDYRWk0H\npZIntPJWmNkvzKzLzP4E3AkcnyL9h8ysEvhNlJlFcLbVZlZpZo/0Ee0a4P8Iq4w7Q4A7qLFHI1Cd\nwRjGkvhH+vuWFN4AzOgj3gxCQboh5tWXTGIapJCZkRCejlOAV83s+Xh8C3CypMIM4vZSnWTrqxnG\nmwPcleDAXyW0Hmsk5Uu6WNKbkjYSCifIwOmmYZ2ZbUnK/9ykSsQ2hNZGn0gqJRTyy8zsh/3klzig\nvxkoSfHczCS0Lnp5OylslZlZivBMbEjWI9W42dcJ3cZPx1mGp6eQQ9IESddKejven0eBSkn5KaLM\nARYk6fkZYHqK9GujzKcJ431rYxp1ki7rQ/4/gfcTug1HfYxzvOAOauzxJKGG+oks03mQvmuPJxC6\n5jZHmX0SZ+Ql8RphXOVd6UjKI3S/PJSBHqcC20uqjzOkLiM4gSMzsiI7VgJHJDm3EjNbRZgUcgxh\nrGoSoTsRQgEKocspmc2ELqdekgu/5DgrCS2XxPwnmNmtfSkrqZhQo68FvpiZiRlRR3AqvWybFDZL\nklKED8iGdJhZvZl9wcxmEuz7aZqZe+cCuwALzGwicGA8n+r+rCR0VyfqWW5mX0qhy2xCd/qDsXJ3\nHfDlGO+cRFlJFwBHAIeZ2caBWe2kwx3UGCN26XwXuErSJ2JNslDSEZIuGUBSFwD7S7pQUpWkCklf\nITiMb8S8HiQMKt8laZ6kgii3SNLpsaZ4HvAdhWnvJZKmAz8HJgI/TshPMTxx2w/YAdiH0O2zF6EW\n+isG3s03GK4BLpQ0Jyo4VdIxMayCUBFoJDidi5LirgG2Tzr3PKH1ly/pcNJ3OQH8DFgkaYECZZI+\nLqkiWTC2KO8gTFr5rJn1ZG5mvywBviVpcqyMfCUh7EnC2NlX43P2KcL9GrAN/SHp+ITK0AaCk+m1\nM/l6VxCuRZOkKuB7Sckly98L7CzplGhHoaQP9tMdPI8wOQTgA4SZfMk6f4tQmTnUzBr7NdIZEO6g\nxiBmdilwDvAdwiymlcBZhNp1pmm8DhxAGJdYQagpHwt8zMweTxA9DriPMPuqmTCAPp/QusLClO5T\nCIPojYRZYKXAh5L+sPsTCpTE7fPAb83sxVh7rjezeuAK4KhY8GRCk979HtQ5/UeBmM/dhKn1mwgT\nJnrfLbqJ0JW1Ktq0LCnufwO7xe6i3ut+NmHKcW/3Udr7YWbLgS8APyEUyG8QJpT0xf6EcZTDeLe9\nH87M1LRcQLD1r8ADhPGqXh07gE9FvdYTZtD9epA29McHgacktRDuy9lm9lYMOx+4MV7vEwiTVkoJ\n3cjLgN8npXUFYTxxg6Qr4zjVYcBJwGpCt2PvhJVUzAOei63HXQkzIZO5iNCifCPhnvzrQA13+kbe\nXeo4juPkIt6CchzHcXISd1CO4zhOTuIOynEcx8lJ3EE5juM4OclAP1g5IlRXV9vcuXNHW42MaG1t\npaysbLTVGFLcprHBeLNpvNkDblMqnn322QYzm9qfXE46qLlz57J8+XteOchJli5dysKFC0dbjSFl\na7GpvaOR9RufprunHbNOeqyL8tLtqJq4D+Fd49xmvN2n8WYPuE2pkPR2/1I56qAcZ7hp2vR/1DX8\nDsMoyC8jT4UYRv3mv9Dc8jKzph5DcVE2XzVyHCdb3EE5WxXdPR3UNfyO5pYXmFAyh9nTPkVhwUQg\nfNm/ueVF6hvv581V1zJt8keYMmlf3v2VH8dxRgp3UM5WQ09PJytW38CWjnqmVh7E1MkHvqsrTxKV\nFXtQVroddQ2/Y836B5DElElpPy7uOM4wkfsd7Y4zJBirG+5lS0cd29ScyLSqhSnHmQoLKtim5kQq\nJuxKfeMDtGx+q085x3GGF3dQzlZBSflqmlteYOrkhUws26VfeUnMmvYJigunUrv2dto714+Alo7j\nJOIOyhn3tG75G+WT36Riws5MrTyw/wiR/Lxitp1+EiBW1i+mu6d9+JR0HOc9uINyxjWdXS3Urrmd\n7q4SZk375IAnPBQVTmZ2zfG0dzZQ1/C7YdLScZy+cAfljGvqG++ju6eNjQ27k59XMqg0yku3Y+rk\nA2lueZFNrX8ZYg0dx0mFOyhn3LKx9VU2tr7K1MkL6e7M7s336soPU1w0jdUN99LdvaX/CI7jZI07\nKGdc0t3dRl3DfZQUTad60n5Zp5enfGZVH01Xdwv16x8YAg0dx+mPrByUpMMlvSbpDUnf7CN8V0lP\nSmqXdF42eTnOQKhf/wBd3a3MnHo0Uv6QpFlaMospk/ajadOffOq544wAg3ZQCv/6q4AjgN2AT0va\nLUlsPfBV4EeD1tBxBkhL21s0bXqe6sr9KS2eMaRpT5u8kKLCKlY33ENPT8eQpu04zrvJpgW1D/CG\nmb1lZh3AYuCYRAEzW2tmzwCdWeTjOBnT09PB6nX3UFRYxdTKg4Y8/by8QmZWH01nVxNrNzw85Ok7\njvMOMrPBRZSOAw43szPi8SnAAjM7qw/Z84EWM0vZkpJ0JnAmQE1NzbzFixcPSq+RpqWlhfLy8tFW\nY0gZyzaVVb7JhIm1NK3Zk872yr+fH2qbyif/hZLyOprW7E1Xx8QhS3cgjOX71BfjzR5wm1Jx8MEH\nP2tm8/uTy5lv8ZnZdcB1APPnz7ex8ol6/5x+7tC2ZRVvrX6UyRXz2H37o94VNtQ2dffsyxsrr2L6\ntqvZYdbHh2ycayCM1fuUivFmD7hN2ZJNF98qYJuE49nxnOOMOGbdrGq4m4L8cmqmHDrs+eXnlTCj\n+uO0d6yhofnJYc/PcbZGsnFQzwA7SdpOUhFwEnD30KjlOAOjoelx2jvWMqP644N+IXegTCzblYoJ\n72PdhqW0dzSOSJ6OszUxaAdlZl3AWcAfgFeBJWb2sqRFkhYBSJouqRY4B/iOpFpJo9Nh74xbtnSs\nZd2GR5lYtltGH4IdSmZUH4FUyKp1d2HWPaJ5O854J6sxKDO7D7gv6dw1Cfv1hK4/xxkWeno6qV1z\nB3n5JcyYcsSI519YUMHM6qOoXXsHazcspabqkBHXwXHGK/4lCWdMU9d4P+2d65g99ZMUFIzObKlJ\n5btTWbE3DU2P0dr211HRwXHGI+6gnDFLc8tLNG36E9WVH6J8wg6jqsuMKYdTVDiF2rV30dW9eVR1\ncZzxgjsoZ0zS0bme1evuobR4NtMmHzza6pCXV8TsacfS3b2Z1evuZrDvFzqO8w7uoJwxR1f3Zv5W\nvxiUx+xpx47KO0h9UVo8g5oph7Fp82vUN97vTspxsiRnXtR1nEzo7tnC23U309G1gTnTP0NRYWX/\nkUaQKZP2obOrmcbmJ8jPm8C0qoWjrZLjjFncQTljhp6eDv5W9yvaO9awzfSTKCudO9oq9UlN1aF0\nd29mXdMj5OeXMmXSgtFWyXHGJO6gnDFBd087K9csYXN7LbOnHUfFhJ1GW6WUSGLm1H+ku6eN+sbf\nY9bDlEn7Dni5ecfZ2vExKCfnaWtfzVu119La9ldmTT2aSeXJq7rkHlJedKS7sGb9A9SuvYPunvbR\nVstxxhTegnJyFjOjsXkZa9c/SEF+OXNnnkZZybajrVbG5OUVsE3NiTQ2P8Ga9Q+xpWMN20w7npLi\nmtFWzXHGBO6gnJzDzNi0+TUamh6jrX0VFRN2ZebUoynILx1t1QaMJKorP0Rp8Wxq197Bm6uuZVL5\nPzC18sMUF1WPtnqOk9Nk5aAkHQ5cAeQDPzezi5PCFcOPBDYDp5nZc9nk6YxfOrs2smnzX2hsXkZH\nZyOFBZXMrP5HKiv2HvPjN2Wlc9hh9iIamh5n/cblNLe8wMSy3aks34Oy0rnk5RWNtoqOk3MM2kEl\nLPn+UaAWeEbS3Wb2SoLYEcBOcVsAXB1/na0YM6O7u5WOrvV0dK6nrb2O1ra3aO9sAKCkaDqzpx3L\nxLLdkMbPMGlBfhnTpxxGdeWHaGxexvrmp9nY+jIinwmlcygrmUNR4RSKC6spKqwiL69wtFV2nFEl\nmxbU35d8B5DUu+R7ooM6BrjJwhuLyyRVSpphZnVZ5JsWsx7qG/8wXMm/h7LJq6hraBux/EaCd9uU\n+mVT6w37+wuphtEDZhiGWTdmXZh10WOddHe30d2zhe6eNsLH8ANSAWUlc6is2Juy0u0pKaoZ8y2m\ndBTkl1FTdQhTJx/E5ra/0dL2Oi2b33jPEvJ5KiI/fwL5eaXk5xUjFSIVkJdXAIiKqjWsWrcRIUAQ\nr1k4TkXuXtfx/18aH5SUNY1YXtk4qFnAyoTjWt7bOupLZhbwHgeVtOQ7S5cuHaRaxpRZI9eLWDzB\naNiwZsTyGwkGZtO7CzyzWFiisG95WO/WU0hPTxnWM4nurhJ6ukro7iqlu6uEMKG0A/hz3IaWlpaW\nLJ6p4aYY2B3UTX5BGwWFm8kvaEN5neTld6G8dqTNSD1IPaAeJKOguJvGDRuQEisRY/frFf5fGhuo\nYOKI/ZdyZpLE0C75PnLfZvMlnccGblPuM97sAbcpW7JxUJks+T6oZeGfffbZBklvZ6HbSFINNIy2\nEkOM2zQ2GG82jTd7wG1KxZxMhLJxUH9f8p3gdE4CTk6SuRs4K45PLQCaMxl/MrOpWeg1okhabmbz\nR1uPocRtGhuMN5vGmz3gNmXLoB2UmXVJ6l3yPR+4vnfJ9xh+DWG13SOBNwjTzD+XvcqO4zjO1sBw\nL/luwJezycNxHMfZOhk/L5mMHteNtgLDgNs0NhhvNo03e8Btygr5omqO4zhOLuItKMdxHCcncQfl\nOI7j5CTuoBzHcZycxB2U4ziOk5O4g3Icx3FyEndQjuM4Tk7iDspxHMfJSdxBOY7jODmJOyjHcRwn\nJ3EH5Ww1SKqR9KikTZIuHW19hhNJ50u6OU34y5IWpghbKKk2TdwbJP1gCNR0nLS4gxqjSDpZ0nJJ\nLZLqJN0v6YAY1mfhJMkk7ZhwvJukuyU1x0L7YUn7J8Upium9LqlV0gpJ10uamyBzlKSnY3ijpFsk\nzU4IP03SY/3Yc4OkLkkzEs5dE+1rkdQhqTPh+H5Jc6NNLUnbiSmyOZOwjs1EMzs37QXuh5EupCVN\nk3SrpNXxfj0uKXkF64wxs93NbOkQqjisJD+7GcgvlXTGEOW9RNJhkool1SeFXSJppaSNkt6W9K9D\nkacTcAc1BpF0DnA5cBFQA2wLXAUcPYA0dgAeB14EtgNmAncBD0jaL0H0jpjuycAkYE9gOXBITOc4\n4FdRn2pgd6AdeEzS5Ax1KQOOBZqBf+o9b2aLzKzczMqjrbf1HpvZEQlJVCacLzez21JkNQd4xXLg\nA5SSBrqSQDlhDbZ5QBVwI/A7SeVDrZvzHuYRnvk9gJeSwq4HdjezicD+wGckfWqE9Ru/mJlvY2gj\nOIkW4Pg0MucDN/dx3oAd4/4vgfv6kLkaeDTuHwq0AdukyEfA28DXk87nEf7I34/HpwGPpdH3VGAl\ncDbwUqY2AXOjTQUZXLcbgE6gI16/Q6Oe3wTeBBqBJUBVQpzbgXqC43yUUBBBaIklpnVP8vVNyPMH\ncX8hUAt8I6b5y3j+KOB5oAl4AthjAM/CRmBemuu1BLgJ2AS8DMxPCF8BHBr3S6OuG4BXgK8BtQmy\newPPxXRuAxb32tWfDTGf84AX4nW8DShJofOOwCNRroFQISFeewNa4/U+EZgM3Ausi3rfC8yO8hcC\n3cCWKP+TeH5X4I/AeuA14IQMrvFk4K24/yXgkjSyswgVvq/3l65vGT7jo62AbwO8YXA40EWaQpnM\nHFQ98Lk+ZA6Of+5S4GLgkTT57BrT3K6PsAuAJ+P+aaR3UA8BlxBag119Fbp92cQAHFSUvyGpYD0b\nWAbMBoqBa4FbE8JPBypi2OXA86nSSr6+yTIEB9UF/EdMr5RQ8K8lrDadD3yWUKAXZ2DLXrEAnpTm\nGdhCWDA0H/ghsCwhfAXvOKiLgf8ltMy2IVQuamNYEaES8i9AIXAcwTn32pXWhrj/NKGFXgW8CixK\nofOtwLcJFYcS4IA013YKodU9Id6j24HfJIQvBc5IOC4jVII+R1gHb2+CE9wthS6HEBxuC6Ei0hR/\nW+P+QQmy34xyBrxFdJS+Zb95F9/YYwrQYGZd/cidIKkpcUsKrwbq+ohXRyggqmJefckkpkEKmbqE\n8JRI2pbgFH9lZmsIzurU/uIl0ZBk6/syjLcI+LaZ1ZpZO6FQP663+83MrjezTQlhe0qaNEDdEukB\nvmdm7WbWRmiJXWtmT5lZt5ndSOge3TddIpImElrAF5hZcxrRx8zsPjPrjvJ7ppA7AbjQzNab2Urg\nyoSwfQmO6XIz6zSzOwhdjb1kYsOVZrbazNYD9xCca190ErphZ5rZFjNLOW5pZo1mdqeZbTazTYRW\n00Gp5AmtvBVm9gsz6zKzPwF3AsenSP8hM6sEfhNlZhGcbbWZVZrZIwmyFxOc5AcI1zndPXEGgDuo\nsUcjUJ3BGMaS+Ef6+5YU3gDM6CPeDEJBuiHm1ZdMYhqkkJmREJ6OU4BXzez5eHwLcLKkwgzi9lKd\nZOurGcabA9yV4MBfJbQeayTlS7pY0puSNhIKJ8jA6aZhnZltScr/3KRKxDaE1kafSColFPLLzOyH\n/eSXOKC/GShJ8dzMJLQuenk7KWyVmVmK8ExsSNYj1bjZ1wndxk/HWYanp5BD0gRJ18aJCRsJ3YCV\nkvJTRJkDLEjS8zPA9BTp10aZTxPG+9bGNOokXZYsb4E/EbrEL0iltzMw3EGNPZ4k1FA/kWU6D9J3\n7fEEQtfc5iizT+KMvCReI4yrvCsdSXmE7peHMtDjVGB7SfVxhtRlBCdwZEZWZMdK4Igk51ZiZqsI\nk0KOIYxVTSJ0J0IoQCF05ySzmdDl1Ety4ZccZyWh5ZKY/wQzu7UvZSUVE2r0tcAXMzMxI+oITqWX\nbZPCZklSivAB2ZAOM6s3sy+Y2UyCfT9NM3PvXGAXYIGFCQoHxvOp7s9KQnd1op7lZvalFLrMJnSn\nPxgrd9cBX47xzkljRgGwQ3+2OpnhDmqMEbt0vgtcJekTsSZZKOkISZcMIKkLgP0lXSipSlKFpK8Q\nHMY3Yl4PEgaV75I0T1JBlFsk6fRYqz4P+I7CtPcSSdOBnwMTgR8n5KcYnrjtR/gz70Po9tkLeD9h\nVuBAu/kGwzXAhZLmRAWnSjomhlUQKgKNBKdzUVLcNcD2SeeeJ7T+8iUdTvouJ4CfAYskLVCgTNLH\nJVUkC8YW5R2EGvpnzawnczP7ZQnwLUmTY2XkKwlhTxLGzr4an7NPEe7XgG3oD0nHJ1SGNhCcTK+d\nyde7gnAtmiRVAd9LSi5Z/l5gZ0mnRDsKJX2wn+7geYTJIRC675Yn6Zsn6YvxuknSPsCXyaxi5mSA\nO6gxiJldCpwDfIcwi2klcBahdp1pGq8DBxDGJVYQasrHAh8zs8cTRI8D7iPMvmomDKDPJ7SusDCl\n+xTCIHojYRZYKfAhM2tMSGd/QoGSuH0e+K2ZvRhrz/VmVg9cARwVC55MaNK734NKV8NN5Ir/396d\nR5bK6ZoAABCKSURBVFdVXwsc/+6EkBCCKFNEpqCigggqYTBtEByBJ+JceYr2qc+yFKvV0odYrawO\ntss6i/hQ26d1oK6KGpGnBSRQ2yKQ4oAMEogsURksPiRJIdN+f+xz5RIScklC7j03+7PWWffec85N\nfjvD3ef8zj6/H1CAldbvxgomIvcWPYd1ZX0exLSs1nufAQYE3UWRn/utwHjsIvpVNPD7UNWVwH8C\nj2MfyMVYQUld8rDrKOexf7z5sYV6UDOwWEuAP2PXUSJtrAAuCdq1E6ugm9vIGBoyFHhPREqx38ut\nqrop2HYv8Gzw874CK1pph3UjLwPeqvW1HsGuJ34tIo8G16nOA64EvsC6HSMFK/UZAvwjOHs8CauE\nrO1irAp0N/A88FiwuGYg+3ctO+ecc4nBz6Ccc84lJE9QzjnnEpInKOeccwnJE5RzzrmEdKgDVraI\nLl26aE5OTrybEZOysjLat28f72Y0K48pHJItpmSLBzym+hQVFX2lql0b2i8hE1ROTg4rV65seMcE\nUFhYyKhRo+LdjGblMQGq8NlnsGaNLevWwa5dUFVlS5s20Ls39O0Lxx4LQ4dCdvZha39dku33lGzx\ngMdUHxHZ3PBeCZqgnIuLigpYsgTeeAPmzYOSkn3bunSBzp0hLc2S09698NZbUF6+b59Bg+Ccc2Ds\nWBg9GlLrG3XHORcLT1DObd4Ms2bB00/DP/8JGRlw9tlw++0weDD0728JqjZV2LEDioth6VJYuBBm\nzoQHH7Szq+uvh+uug571jRTlnDsYL5JwrdeKFXDRRdZFd//9cOaZ8PrrlqTmzYMpUyA/v+7kBCAC\n3bpBXh5Mm2YJaudOmDMHTjgBfvYz6NMHrroK1q9v2dicSwKeoFzrs2YNXHIJDBsG775ryaWkBF55\nBS68EDIzG/4a9cnMhO99DxYsgI0b7SzstddgwACYNAk2bGi+OJxLcp6gXOuxY4d1u51yip3tzJhh\niemXv7QuueYWOTMrKbFE9corlqimToXdu5v/+zmXZDxBueRXXU33ggI48UR47jm47TbYtAnuuQc6\nHPKg24euW7d9ieraa+G3v7W2PP+8XcdyztXJE5RLbh9+CHl5nPjQQ1bw8MEH8MAD9V9XOpyys60Q\n4733rHBi0iQ499z9qwWdc9/yBOWSU2Ul/PznkJsLJSWsmT4d3nnHutjibdgwWLbMKgeXL4eBA+Gx\nx6CmOad4ci78YkpQIjJGRNaLSLGITKtju4jIo8H2D0Xk9Khtn4rIRyLyvoiE4+5bF24ffQQjRlgX\n3qWXwpo1bD/3XKu6SxQpKTB5Mnz8sVUP/vCH9rhpU8Pvda6VaDBBiUgqMBMYCwwAJopI7cPQsUC/\nYLkRmFVr+2hVPVVVc5veZOfqUVNj13dyc2HLFitKeOml+HTnxapXL3jzTXj2WUusgwfDM8/4tSnn\niO0MahhQrKqbgtk15wATau0zAXhOzTLgSBHp3sxtda5+mzfbzbVTp8K4cbB6tZWSh4EIXHONXS8b\nOhRuuAEmTIBt2+LdMufiqsEZdUXkMmCMqt4QvJ4EDFfVKVH7zAN+rarvBq8XAf+lqitFpASbKrwa\n+G9VnV3P97kRO/siOzt7yJw5c5ocXEsoLS0lKysr3s1oVmGLqdvChZzw8MNQU0PxLbewdcyYA7rz\nQhNTTQ09587l2NmzqcrKYt3Uqew844w6dw1NTDFKtnjAY6rP6NGji2LqUVPVgy7AZcDTUa8nAY/X\n2mce8N2o14uA3OB5j+CxG/ABMLKh7zlkyBANi8WLF8e7Cc0uNDF9/bXqxImqoJqXp7pxY727hiam\niI8+Uh00yGK76SbVsrIDdgldTA1ItnhUPab6ACu1gTygqjF18X0O9Ip63TNYF9M+qhp53A68inUZ\nOtc0S5bY4Kwvv2zVekuW2I2xyWLgQKvwu+MOeOIJGDIEiori3SrnWlQsCWoF0E9E+opIW+BKoKDW\nPgXANUE13whgl6p+KSLtRaQDgIi0B84DVjdj+11rs2cP/PjHNlp4ejr87W/w05/aCOPJJj3dij4W\nLoTSUqtM/MUvbLoP51qBBhOUqlYBU4C3gbXAy6r6sYhMFpHJwW7zgU1AMfAUcFOwPht4V0Q+AJYD\nb6rqW80cg2stVq2yCr0HHoAf/ADef9/uKUp2Z59tBRSXXw533w0jR8Inn8S7Vc4ddjEddqrqfCwJ\nRa97Muq5AjfX8b5NwOAmttG1dhUVcN99NmZe584wf77NudSaHHUUvPgijB8PN90EgwfT8/rrLVml\n+P32Ljn5X7ZLbKtWWen1vffCZZdZ+XhrS07RJk60m3vPOYfjZ86EUaNsPirnkpAnKJeYysth+nRL\nTtu325QVL75oZ1Ct3THHQEEBa6dNs66/U06BX/3KzjSdSyKeoFzimTcPTj7ZuvUmTbIzhgm17w1v\n5UTYdv75NrfV+PFw111w2mk2v5VzScITlEscGzfCxRfbB25mppWO//730KlTvFuWuI45xkrt33jD\nKv3y8+Hqq+Gzz+LdMueazBOUi7+dO+FHP4L+/W0m2vvus2tPI0fGu2XhccEFdjY1fTr86U8239S9\n90JZWbxb5lyjeYJy8bN7N/zmN3D88fDoozaZ34YNNgV727bxbl34tG9vlY7r1tnU9TNmwHHHwSOP\n2P1jzoWMJyjX8r75xi7q5+RYMhoxwu5peuop6O5jDDdZTg7MmQN//atdy7vtNktUjz9uxSfOhYQn\nKNdySkpsFIjeve2i/hln2MR98+dbJZprXnl5sGgRLF5sw0Ddcov97O++G7ZujXfrnGuQJyh3eFVW\n2nxHEybYUfzDD8OYMbBihVXrDR8e7xYmv1GjYOlSKzrJz7duwN697Z6qBQugujreLXSuTkk4gJmL\nu5oaG9j0hRdswsDt223SwOnTbRbZnj3j3cLWR8SKTkaOtBt7H3sM/vAH6wrs1cvmo7r0Ujj11MSa\nedi1an4G5ZpHWZl11U2ebB94w4bBrFl2xP7aa/D55zbQqSen+Dv+eCuc+OIL+OMf991zdvrp1hV4\n++12ZvWvf8W7pa6V8zMo1zg7dsDKlXZjaGGhTQ1RVWWVZOefb1Vk48f7PUyJLCMDrrjClh077F6q\nuXNh5kx46CGrpMzLs5Hjhw+3UT389+laUEwJSkTGAI8Aqdjkhb+utV2C7eOAcuD7qvqPWN7rEtyu\nXVb6vW4drF1rozoUFcGWLbY9NdU+uO64wz7IzjzTPvhcuHTtCtddZ0tpKfzlL1ZgsWiR3U8VmXn7\nuONg8GAYMMDOvE46Cfr2hY4d49p8l5waTFAikgrMBM4FtgArRKRAVddE7TYW6Bcsw4FZwPAY3+ta\niqqN17Z7ty1ff203ye7caUfQ27bB1q0MXL3auuw+/dRKwiNSU617KD/fJtCLLB06xC0kdxhkZdmA\nvJFBeXftsoOS5cutuGX1auu2ranZ955OnSxR9ehhtwp07w7Z2TZ2YufOtr1jRzjiCFvS0uITmwuV\nWM6ghgHFwdQZiMgcYAIQnWQmAM8F024sE5EjRaQ7kBPDe5tXdbWVMreQ47Zsgddf37cicqRZ1+v6\nttnE3ge+rqnZ/3lkqa7ef6mqsqWy0pa9e22pqLDrCOXl+x4rK+sPRgS6dSOjfXs7Qs7Phz597Ki5\nf3979BtoW5+OHeGss2yJ2LPH5qRav94OZEpKbPn0U/j73+2A52DatrXhrDIzoV07SE9nSGWlFdO0\nbWsJLC3NJqJs08YOjiKPKSn7LyL7HmsvsP/ziOjXB9vWRAd8PiSBo9u0scrQFhBLguoBRA/stQU7\nS2ponx4xvhcAEbkRuBEgOzubwsLCGJpWx9epruY7Tz3VqPc2xtGqVDXwB60x/jNorX8qjfpH0+Af\n8NvH1FQ0JcWWNm3sdbDUtG1LTUYGmpVFTXo61enp9piRQXW7dlRnZlKdmUllVhZVRxxBVYcOVHbs\nSEXHjpCaSmlpKVlZWfu3c9s2W0KqtLS00X9TiSohYura1ZahQ/dbLVVVpO3aRdo339Am8lheTmp5\nOW3KykjZs4fUPXtI2bvXHisrqSkvp7KigpSyMqS6GqmqIqWqCmpqkJoaWxcctElwsCbBAZxEzuYi\nr2M5MIT99zsMYvl8CJsOQ4a02N9dwhRJqOpsYDZAbm6ujmpKhi4tbZ5GxaCwsJAmtTUBeUzhkGwx\nJVs8kJwxbWjBmGJJUJ8DvaJe9wzWxbJPWgzvPUBRUdFXIrI5hrYlgi7AV/FuRDPzmMIh2WJKtnjA\nY6pPn1h2iiVBrQD6iUhfLLlcCfx7rX0KgCnBNabhwC5V/VJEdsTw3gOoatdYGp8IRGSlqubGux3N\nyWMKh2SLKdniAY+pqRpMUKpaJSJTgLexUvHfqerHIjI52P4kMB8rMS/Gysz/42DvPSyROOecSyox\nXYNS1flYEope92TUcwVujvW9zjnnXEN8qKOmmx3vBhwGHlM4JFtMyRYPeExNInqYyyydc865xvAz\nKOeccwnJE5RzzrmE5AmqkUTkfhFZJyIfisirInJk1LY7RaRYRNaLyPnxbGesRORyEflYRGpEJLfW\nttDFEyEiY4J2F4vItHi3pzFE5Hcisl1EVket6yQiC0RkQ/B4VDzbeKhEpJeILBaRNcHf3a3B+lDG\nJSIZIrJcRD4I4pkRrA9lPNFEJFVEVonIvOB1i8XkCarxFgADVXUQ8AlwJ4CIDMDu9zoZGAM8EQya\nm+hWA5cAS6NXhjie6IGOxwIDgIlBPGHzP9jPPto0YJGq9gMWBa/DpAq4Q1UHACOAm4PfTVjj2guc\npaqDgVOBMSIygvDGE+1WYG3U6xaLyRNUI6nqn1W1Kni5DBslA2ww3DmquldVS7B7w4bFo42HQlXX\nqur6OjaFMp7AtwMdq2oFEBmsOFRUdSmws9bqCcCzwfNngYtatFFNpKpfRqbkUdXd2AdgD0Ial5rI\nGGtpwaKENJ4IEekJ/BvwdNTqFovJE1TzuA743+B5fQPnhlWY4wlz2xuSrapfBs+3AtnxbExTiEgO\ncBrwHiGOK+gKex/YDixQ1VDHE3gY+AkQNbdKy8WUMIPFJiIRWQgcXcemu1T19WCfu7Duihdasm2N\nEUs8LnxUVUUklPeLiEgW8Apwm6p+I9Gj+4csLlWtBk4Nrke/KiIDa20PVTwicgGwXVWLRGRUXfsc\n7pg8QR2Eqp5zsO0i8n3gAuBs3XdDWSyD68ZFQ/HUI2HjiUGY296QbSLSPRjzsjt21B4qIpKGJacX\nVHVusDr0canq/4nIYuy6YZjj+Q5woYiMAzKAI0TkeVowJu/ia6RgKvufABeqannUpgLgShFJDwbJ\n7Qcsj0cbm0mY4/l2oGMRaYsVexTEuU3NpQC4Nnh+LRCqM2CxU6VngLWq+mDUplDGJSJdI5W8ItIO\nm0V8HSGNB0BV71TVnqqag/3vvKOqV9OSMamqL41YsGKBz4D3g+XJqG13ARuB9cDYeLc1xnguxq7R\n7AW2AW+HOZ6oto/Dqiw3Yl2ZcW9TI2J4CfgSqAx+R9cDnbEKqg3AQqBTvNt5iDF9Fysi+DDqf2hc\nWOMCBgGrgnhWA/cE60MZTx3xjQLmtXRMPtSRc865hORdfM455xKSJyjnnHMJyROUc865hOQJyjnn\nXELyBOWccy4heYJyzjmXkDxBOeecS0j/D5mqWEpHNAb7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11440eb38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXmYXVWVt99fDamqpDJXqJAZGcRgMyUkgLSGlkZAFAcG\nQUFAwNiAKCCg0gIqNPIpAi0SaEUmBaIIjQiK8FggSIAEkSlNG9IJqZCpMlRSqSE1rO+PvQtOLnVv\n3arcW3WrWO/znOeec/bae691zrl77emcLTPDcRzHcQqNov5WwHEcx3G6wh2U4ziOU5C4g3Icx3EK\nEndQjuM4TkHiDspxHMcpSNxBOY7jOAWJOyjnPYmkaklPStoi6Uf9rU8+kXS5pLsyhL8qaU6asDmS\najPEvU3S93OgpuO8C3dQgwBJJ0laKKlB0ipJj0g6JIZ1WThJMkm7JY6nS3pQUn0stP8s6eCUOENi\nev+QtFXSMkm3SpqWkDla0nMxfL2kX0qalAg/VdJT3dhzm6Q2STsnzs2L9jVI2iapNXH8iKRp0aaG\nlO2ENNmcBdQBI8zsgowXuBv6o5CO92edpM2S/i7pmN6mZWZ7mVlNDtXLK6nPbhbyNZLOyFHe8yUd\nLqlM0uqUsLL4f9gsabWk83OR53sZd1ADnPgnuA64CqgGpgA3Ap/sQRq7Ak8DLwO7ABOA+4FHJR2U\nEP1NTPckYCSwD7AQ+GhM51jgV1GfKmAvoAV4StLoLHUZBnwWqAe+0HnezOaaWaWZVUZb7+08NrMj\nE0mMSpyvNLN702Q1FXjNCuBNdUklvYj2NWCSmY0gONu7kg7dyRszCM/83sArKWGXA7sTnq1DgYsk\nHdGn2g02zMy3AboRnEQDcFwGmcuBu7o4b8Bucf9O4OEuZG4Cnoz7hwFNwOQ0+QhYDlyUcr6I8Ef+\nbjw+FXgqg76nACuA84BXsrUJmBZtKsniut0GtALb4vU7LOp5CfAGsB6YD4xJxPk1sJrgOJ8E9orn\nz0pJ63ep1zeR5/fj/hygFrg4pnlnPH808CKwCfgrsHeWz8EsoBmYleF6zQfuALYArwIzE+HLgMPi\nfkXUdSPwGvANoDYhux/wQkznXuCeTru6syHmcyHwUryO9wLlaXTeDXgiytURKiTEa2/A1ni9TwBG\nAw8B66LeDxGcN8CVQHu8Pg3AT+L5PYE/ARuA14Hjs7jOo4Glcf8rwDUp4W8BhyeOvwvc09flwmDa\n+l0B33bg5sERQBsZCmWyc1CrgdO6kDk0/rkrgKuBJzLks2dMc5cuwq4Anon7p5LZQT0OXENoDbYB\nM7KxiR44qCh/W0rBeh6wAJgElAE3A3cnwk8Hhsew64AX06WVen1TZQgOqg34QUyvglDwrwVmA8XA\nFwkFelkGGx6KBa8BfwCKMjwDzcBRMe3/ABYkwpfxjoO6GvgLMAaYTKhc1MawIYRKyNeBUuBYgnPu\ntCujDXH/OUILfQywGJibRue7gW8TKg7lwCEZru1YQqt7aLxHvwYeSITXAGckjocRKkGnASVR7zpg\nehpdPkpwuA2Eisim+Ls17n+E4LwMqE7E+yzwcn+WEQN98y6+gc1YoM7M2rqRO17SpuSWEl4FrOoi\n3ipCATEm5tWVTDIN0sisSoSnRdIUglP8lZmtITirU7qLl0Jdiq0fyDLeXODbZlZrZi2EQv3Yzu43\nM7vVzLYkwvaRNLKHuiXpAC4zsxYzayK0xG42s2fNrN3Mbid0jx6YLgEzO5pQIB8FPGpmHRnye8rM\nHjazdkKLeZ80cscDV5rZBjNbAdyQCDuQ4JiuM7NWM/sN8HwiPBsbbjCzt8xsA/A7YN80erQSusom\nmFmzmaUdtzSz9WZ2n5k1mtkWQqvpI+nkCa28ZWb2CzNrM7O/AfcBx6VJ/3EzGwU8EGUmEpxtlZmN\nMrMngMooXp+Iuplwf5xe4g5qYLMeqMpiDGN+/CO9vaWE1wFdjV/sTChIN8a8Mo1x1CXidJVOXRfn\nUzkZWGxmL8bjXwInSSrNIm4nVSm2Ls4y3lTg/oQDX0xoPVZLKpZ0taQ3JG0mFE6QhdPNwDoza07J\n/4KUSsRkQmsjLdFRPAIcLinTuGNyQL8RKE/z3EwgtC46WZ4SttLMLE14Njak6lFJ11xE6DZ+Ls4y\nPD2NHJKGSrpZ0vJ4f54ERkkqThNlKjA7Rc/PA+PTpF8bZU4Ebie0EqcCqyRdG8Ua4u+IRNSRhK5Q\np5e4gxrYPEOooX5qB9N5jK5rj8cTuuYao8ys5Iy8FF4njKtsl46kIkJXx+NZ6HEK8L44A2o1cC3B\nCRyVlRU7xgrgyBTnVm5mKwmTQo4hjFWNJHQnQihAIXTtpNJI6HLqJLXwS42zgtBySeY/1MzuzlL/\nEmDXLGUzsYrgVDqZkhI2UZLShO+oDW9jZqvN7EwzmwB8Gfhphpl7FwDvB2ZbmDTy4Xg+3f1ZQeiu\nTupZaWZfSaPLJEJ3+mOxcncLcHaMd36U2Ui4PsmW6T6E8T6nl7iDGsCYWT3wHeBGSZ+KNclSSUdK\nuqYHSV0BHCzpSkljJA2XdC7BYVwc83qMMKh8v6QZkkqi3FxJp8da9YXApQrT3ssljQd+RqhV/jiR\nn2J4cjuIUMDOInT77At8kDArsKfdfL1hHnClpKlRwXGJqdvDCRWB9QSnc1VK3DXA+1LOvUho/RXH\nmVyZupwA/guYK2m2AsMkfVzSu7qIJO0Z73FFvN9fIBTKT/TA3nTMB74paXSsjJybCHuGMHb21Zjv\nZwj3q8c2dIek4xKVoY0EJ9PZhZl6vYcTJvBskjQGuCwluVT5h4A9JJ0c7SiVdEA33cEzCJNDAPYn\nzORL5Q7C8z86pnUmYezR6SXuoAY4ZvYj4HzgUsIsphXAOYT+8mzT+AdwCKHGt4xQE/ws8DEzezoh\neizwMGH2VT1hAH0moXWFhSndJxMG0dcTZoFVAB8ys/WJdA4mFCjJ7UvAf5vZy7H2vNrMVgPXA0fH\ngicbNmn796CyfRfleuBBwtT6LYQJE7Nj2B2ErqyV0aYFKXF/DkyP3UWd1/084BOEQfTP0839MLOF\nhALtJ4QCeQlhQklXiDAOtpZwz88DTjCzF9LI94QrCLb+H/AoYbyqU8dtwGeiXhsIM+h+20sbuuMA\n4FlJDYT7cp6ZLY1hlwO3x+t9PGHSSgWhG3kBYcJIkusJ44kbJd0Qx6kOBz5HmHm3mncmrKRjBvBC\nbD3uSdcto8sIs0CXEyZmXGNmqbo4PUDbdyc7juM4TmHgLSjHcRynIHEH5TiO4xQk7qAcx3GcgiRv\nDkrSEZJel7RE0iVdhO8p6RlJLZIuzJcejuM4zsCkNx+p7Jb4gtyNwL8S3o15XtKDZvZaQmwD8FV6\n8A5PVVWVTZs2LZeq5pytW7cybNiw/lYjZ7g9hctgsgXcnkInl/YsWrSozszGdSeXFwdFeDdiSee0\nUEn3EF50fNtBmdlaYK2kj2eb6LRp01i4sKvXDwqHmpoa5syZ099q5Ay3JzNm8Mfz4f2fhF0OzVmy\nWeH3prBxe9IjaXn3UnmaZq6w7MIRZnZGPD6Z8Jb3OV3IXg40mNkP06R1FuEbX1RXV8+45557cq5v\nLmloaKCyMt3XWwYebk9m6l8ZwYvn7s+o/Tayz7V/z1m62eD3prBxe9Jz6KGHLjKzmd3J5asFlTPM\n7BbCp0WYOXOmFXqNxGtNhU2u7XngtvC76cXRzNhjDsMzfjkvt/i9KWzcnh0nX5MkVrL997wmxXOO\nM2ho2giv3gu7/Atg8Or8/tbIcQYX+XJQzwO7S9pF0hDCJ0UezFNejtMvvHQXtDXDv/4Qxu8LrxR2\n77PjDDjy4qDi+kTnAH8kLFsw38xejR8WnQsgabykWuJ35OIn7UekT9VxCgczeOEWmDATdt4PPngi\nrHwWNi7tPq7jONmRt/eg4uJoe5jZrmZ2ZTw3z8zmxf3VZjbJzEbEz9ZPMrPN+dLHcXJJ7QJY+wrs\nf1Y43uuE8OutKMfJHf4lCcfpBS/cAkMq4YOfC8ejpsLkg91BOU4ucQflOD2keRO8ci/80+ehLLHS\n0QdPhLUvw1pfos5xcoI7KMfpIYt/C21NsP8Z25+ffhyoyFtRjpMr3EE5Tg958y8wtAp2nrH9+crq\nMOX8VXdQjpMT3EE5Tg958+kw3iS9O2yPT8CGJVC/ou/1cpzBhjsox+kBDWtgwz9g8oe6Dp9ySPh9\n86m+08lxBivuoBynB6z4a/hN56Cq94Yhw91BOU4ucAflOD1gxdNQPAQmzOg6vKgEJh8Uxqkcx9kx\n3EE5Tg9Y8XT4ekRJeXqZKf8cXuJt2th3ejnOYMQdlONkSWsTvLUoffdeJ1MOAeyd7kDHcXqHOyjH\nyZK3FkJHa/cOauIsKCr1cSjH2VHcQTlOlqx4OvxOPjizXOnQMEbl41COs2O4g3KcLFnxNIx9Pwwb\n173slH+Gt54Py3E4jtM73EE5ThZYRxhT6q57r5Mph0D7Nlj5fH71cpzBjDsox8mCutehaQNMydJB\ndToyH4dynN7jDspxsuDt8acsHdTQsTBuuo9DOc6O4A7KcbJgxdPhA7Fj98g+zpR/DvE62vOnl+MM\nZtxBOU43bNsK//sQTJvT9Qdi0zHlEGjZHF7adRyn57iDcpxueOFn0FgHs7/Ws3jTDgUVw4u/yI9e\njjPYcQflOBlo3wZ//X8w9cPZT5DoZMRE2Pc0WHgTbFqeH/0cZzDjDspxMvD3O2HLSjjkW72L/5Hv\nAIInrsipWo7znsAdlOOkoaMdnr4adt4fdj28d2mMnAwH/Bv8/XZY91pu9XOcwY47KMdJw2u/Cavj\nHvKtnk2OSOWQb4bPH/3533Onm+O8F8ibg5J0hKTXJS2RdEkX4ZJ0Qwx/SdL++dLFcXpKRxs8dRVU\n7Qkf+PSOpTVsHBx0ASz+rX9ZwnF6Ql4clKRi4EbgSGA6cKKk6SliRwK7x+0s4KZ86OI4PaFlMyy4\nDv5zd1jzUmw95eBfctD5UDEW7v0U/Pky2Lh0x9N0nMFOSZ7SnQUsMbOlAJLuAY4Bkr3wxwB3mJkB\nCySNkrSzma3Kk0784evQ2piv1AOr3tqDLXfnN4++ZFDa86uwbwYYtDWFd522NYQPvLZsDi/Zfuw6\neP8nc5Nv2Qj43APwxHfhye/Bk9+FibOhsjp0/5VUhNV4t6ObbsVBeW/cnoKl0x4VwdF91JzIl4Oa\nCKxIHNcCs7OQmQhs56AknUVoYVFdXU1NTU2vlXr5twfQ1pAvkwMdNob1aslrHn3JoLWns/AXFA3p\noLiineLydkbNbmLCZ1YyYs8trAZWP5Hb/Cd/C8Z9qYw1f6pmw7Nj2fRqMR0tRXS0FGEdCY9kPbBl\nkOD2FDad9qjYqDxhQZ/kmd/SOgeY2S3ALQAzZ860OXPm9DqtOX3wLkpNTQ07omOh8d6zZyQwPv+K\nnLDjSbz37s3AYnDbMyeDZO7Il4NaCUxOHE+K53oqsx2LFi2qk1TorzxWAXX9rUQOcXsKl8FkC7g9\nhU4u7ZmajVC+HNTzwO6SdiE4nc8BJ6XIPAicE8enZgP13Y0/mVkWS8X1L5IWmtnM/tYjV7g9hctg\nsgXcnkKnP+zJi4MyszZJ5wB/BIqBW83sVUlzY/g84GHgKGAJ0Aiclg9dHMdxnIFJ3sagzOxhghNK\nnpuX2Dfg7Hzl7ziO4wxs/EsSueeW/lYgx7g9hctgsgXcnkKnz+1RaMg4juM4TmHhLSjHcRynIHEH\n5TiO4xQk7qAcx3GcgsQdlOM4jlOQuINyHMdxChJ3UI7jOE5B4g7KcRzHKUjcQTmO4zgFiTsox3Ec\npyBxB+W8J5FULelJSVsk/ai/9cknki6XdFeG8FclzUkTNkdSbYa4t0n6fg7UdJx34Q5qECDpJEkL\nJTVIWiXpEUmHxLAuCydJJmm3xPF0SQ9Kqo+F9p8lHZwSZ0hM7x+StkpaJulWSdMSMkdLei6Gr5f0\nS0mTEuGnSnqqG3tuk9QmaefEuXnRvgZJ2yS1Jo4fkTQt2tSQsqVbGvAswto2I8zsgowXuBv6o5CO\n92edpM2S/i7pmN6mZWZ7mVlNDtXLK6nPbhbyNZLOyFHe8yUdLqlM0uqUsOMl/VVSo6SaXOT3Xscd\n1ABH0vnAdcBVQDUwBbgR+GQP0tgVeBp4GdgFmADcDzwq6aCE6G9iuicRlp7dB1gIfDSmcyzwq6hP\nFbAX0AI8JWl0lroMAz4L1ANf6DxvZnPNrNLMKqOt93Yem9mRiSRGJc5Xmtm9abKaCrxmBfAxSkm9\nWVXga8AkMxtBcLZ3JR26kzdmEJ75vYFXUsI2EJ79q/taqUGLmfk2QDeCk2gAjssgczlwVxfnDdgt\n7t8JPNyFzE3Ak3H/MKAJmJwmHwHLgYtSzhcR/sjfjcenAk9l0PcUYAVwHvBKtjYB06JNJVlct9uA\nVmBbvH6HRT0vAd4A1gPzgTGJOL8GVhMc55PAXvH8WSlp/S71+iby/H7cnwPUAhfHNO+M548GXgQ2\nAX8F9s7yOZgFNAOzMlyv+cAdwBbgVWBmInwZcFjcr4i6bgReA74B1CZk9wNeiOncC9zTaVd3NsR8\nLgReitfxXqA8jc67AU9EuTpChYR47Q3YGq/3CcBo4CFgXdT7IYLzBrgSaI/XpwH4STy/J/AnglN5\nHTg+i+s8Glga978CXJNG7gygpq/Lg8G49bsCvu3AzYMjgDYyFMpk56BWA6d1IXNo/HNXEGqFT2TI\nZ8+Y5i5dhF0BPBP3TyWzg3ocuIbQGmwDZmRjEz1wUFH+tpSC9TxgATAJKANuBu5OhJ8ODI9h1wEv\npksr9fqmyhAcVBvwg5heBaHgX0tYXboY+CKhQC/LYMNDseA14A9AUYZnoJmwQGgx8B/AgkT4Mt5x\nUFcDfwHGAJMJlYvaGDaEUAn5OlAKHEtwzp12ZbQh7j9HaKGPARYDc9PofDfwbULFoRw4JMO1HUto\ndQ+N9+jXwAOJ8BrgjMTxMEIl6DTCmnj7EZzg9DS6fJTgcBsIFZFN8Xdr3P9Iirw7qBxt3sU3sBkL\n1JlZWzdyx0valNxSwquAVV3EW0UoIMbEvLqSSaZBGplVifC0SJpCcIq/MrM1BGd1SnfxUqhLsfUD\nWcabC3zbzGrNrIVQqB/b2f1mZrea2ZZE2D6SRvZQtyQdwGVm1mJmTYSW2M1m9qyZtZvZ7YTu0QPT\nJWBmRxMK5KOAR82sI0N+T5nZw2bWTmgx75NG7njgSjPbYGYrgBsSYQcSHNN1ZtZqZr8Bnk+EZ2PD\nDWb2lpltAH4H7JtGj1ZCN+wEM2s2s7Tjlma23szuM7NGM9tCaDV9JJ08oZW3zMx+YWZtZvY34D7g\nuDTpP25mo4AHosxEgrOtMrNRZvZEhrycHcAd1MBmPVCVxRjG/PhHentLCa8Duhq/2JlQkG6MeWUa\n46hLxOkqnbouzqdyMrDYzF6Mx78ETpJUmkXcTqpSbF2cZbypwP0JB76Y0HqsllQs6WpJb0jaTCic\nIAunm4F1Ztackv8FKZWIyYTWRlqio3gEOFxSpnHH5IB+I1Ce5rmZQGhddLI8JWylmVma8GxsSNWj\nMo2+FxG6jZ+LswxPTyOHpKGSbpa0PN6fJ4FRkorTRJkKzE7R8/PA+DTp10aZE4HbCa3EqcAqSdem\n08vZcdxBDWyeIdRQP7WD6TxG17XH4wldc41RZlZyRl4KrxPGVbZLR1IRofvl8Sz0OAV4n6TVcYbU\ntQQncFRWVuwYK4AjU5xbuZmtJEwKOYYwVjWS0J0IoQCF0OWUSiOhy6mT1MIvNc4KQsslmf9QM7s7\nS/1LgF2zlM3EKoJT6WRKSthESUoTvqM2vI2ZrTazM81sAvBl4KcZZu5dALwfmG1h0siH4/l092cF\nobs6qWelmX0ljS6TCN3pj8XK3S3A2THe+T21zcked1ADGDOrB74D3CjpU7EmWSrpSEnX9CCpK4CD\nJV0paYyk4ZLOJTiMi2NejxEGle+XNENSSZSbK+n0WKu+ELhUYdp7uaTxwM+AEcCPE/kphie3gwgF\n7CxCt8++wAcJswJ72s3XG+YBV0qaGhUcl5i6PZxQEVhPcDpXpcRdA7wv5dyLhNZfsaQjyNzlBPBf\nwFxJsxUYJunjkoanCkraM97jini/v0AolHPR1TQf+Kak0bEycm4i7BnC2NlXY76fIdyvHtvQHZKO\nS1SGNhKcTGcXZur1Hk6YwLNJ0hjgspTkUuUfAvaQdHK0o1TSAd10B88gTA4B2J8wky9V52JJ5YTK\nQlF8rnvS+ndScAc1wDGzHwHnA5cSZjGtAM4h9Jdnm8Y/gEMI4xLLCDXlzwIfM7OnE6LHAg8TZl/V\nEwbQZxJaV1iY0n0yYRB9PWEWWAXwITNbn0jnYEKBkty+BPy3mb0ca8+rzWw1cD1wdCx4smGTtn8P\nKtsa7vXAg4Sp9VsIEyZmx7A7CF1ZK6NNC1Li/hyYHruLOq/7ecAnCIPon6eb+2FmC4EzgZ8QCuQl\nhAklXSHCONhawj0/DzjBzF5II98TriDY+n/Ao4Txqk4dtwGfiXptIMyg+20vbeiOA4BnJTUQ7st5\nZrY0hl0O3B6v9/GESSsVhG7kBYQJI0muJ4wnbpR0QxynOhz4HPAWoduxc8JKOmYAL8TW456EmZCp\nnEx4lm8C/jnu/1ePrHa2Q9t3JzuO4zhOYeAtKMdxHKcgcQflOI7jFCTuoBzHcZyCxB2U4ziOU5D0\n5iOV/UZVVZVNmzatv9XIyNatWxk2bFh/q5Ez3J7CZTDZAm5PoZNLexYtWlRnZuO6kxtQDmratGks\nXPiu1w8KipqaGubMmdPfauQMt6dwSWfLNjN+vXkzzzY1MaGkhGlDhrBLaSn7l5dTut07toXFYLo3\n4PZkQtLy7qUGmINyHCc9HWb8YetWbtiwgRVtbUwqKeGvTU00xVdJZpSXc+1OO1FV4n97Z2DgT6rj\nDAK2dnTwb6tXs7C5mT2GDGHe+PEcUlEBwJr2dp5qbOQ/1q/nhJUrua66mn8qL+9njR2ne3yShOMM\ncFrNOH/NGl5obubyqirumziRfx46FElIYnxJCceOGMFdEyZQJHHKqlU81NDQ32o7Tre4g3KcAYyZ\n8Z1163iqqYnLqqo4bsQIitKMM32grIz5EyeyT1kZ31q7lkVNTX2sreP0DHdQjjOA+fGGDTzY0MA5\no0dz7IgR3cqPLi7mP8ePZ1JpKeevXcu6tu6WEnOc/iMrByXpCEmvS1oi6ZIuwiXphhj+kqT94/nJ\nkv4s6bW4pst5iTiXS1op6cW49cWSCo4zaHhu6FB+Xl/PCSNGMHdU6hJf6RleVMR11dU0dHRw4dq1\ntPn3OJ0CpVsHFRf9uhE4EpgOnChpeorYkcDucTuL8DVfCJ/mv8DMphNW1Tw7Je6PzWzfuD28Y6Y4\nznuH1W1tzB8zhv3Kyvj22LGoh9PH9xgyhMurqljY3Mx1GzbkSUvH2TGyaUHNApaY2dL4uf17CIu3\nJTkGuMMCCwirWe5sZqs6lwCIn7hfTFgu2XGcXmJm/Pu6dbQDV+20E8W9fLfpE8OH87kRI/hFfT1/\naWzMrZKOkwOymWY+ke2XgK7lnXVyMslMJKwrBICkacB+wLMJuXMlnUJY/OsCM9uYmrmkswitMqqr\nq6mpqclC5f6joaGh4HXsCW5P4fFkZSV/HTOGT69Zw9LaWpZ2HyUtBwI148dzSW0tl65aRUU/dvcN\nhnuTxO3ZcfrkPShJlcB9wNfMbHM8fRPwPcJKmd8DfgScnhrXzG4hLLHMzJkzrdDfzPa3xwubgW7P\n8tZWLqyt5UPl5Xy0tTUntuzU3Mzn33qL56dP5zvjuv36TN4Y6PcmFbdnx8mmi28lMDlxPCmey0om\nLnl8H/BLM0uuvrnGzNrNrIOw6mRy6WjHcVLoMOPSdesolfjeuHHk6qNF+5SXc/LIkdy7ZQvP+dRz\np4DIxkE9D+wuaRdJQwjLJD+YIvMgcEqczXcgUG9mq+LyyD8HFpvZtckIknZOHH6asHy44zhpmL9l\nCy80N3PR2LFU5/hzRV8dPZrJJSV8Z906mjo6cpq24/SWbh2UmbUB5wB/JExymG9mr0qaK2luFHsY\nWAosIbSG/i2e/xBwMvAvXUwnv0bSy5JeAg4Fvp4zqxxnkLG6rY1r16/noIoKPlVZmfP0K4qKuGLc\nOFa0tfGfG981FOw4/UJW1bA4BfzhlHPzEvsGnN1FvKeg654IMzu5R5o6znsUM+O7dXV0AJdVVfV4\nSnm2zK6o4Pjhw7mzvp6PDRvGPv69Pqef8S9JOE6B88etW3misZFzRo9mcmlpXvO6YOxYdiou5tJ1\n69jmL/A6/Yw7KMcpYDa2t3PV+vXsVVbGF0aOzHt+lUVFXDZuHEtbW5nnXX1OP+MOynEKFDPj8nXr\n2Nzezveqqijpo8UGPzx0KJ+srORnmzaxuKWlT/J0nK5wB+U4BcoDDQ081tjIeWPG8P6ysj7N++Kx\nYxnlXX1OP+MOynEKkNrWVq6qq+OA8nK+2Adde6mMKi7m8qoq/mfbNq5dv77P83cccAflOAVHuxmX\nrF1LscRVO+2Udn2nfPMvw4bxhREjuHPzZh7furVfdHDe27iDcpwC46aNG/lbSwuXjh3LhBy/kNtT\nLhg7lr3Kyvj2unXUtrb2qy7Oew93UI5TQPxuyxZu2rSJT1dW8vE8vJDbU4ZIXLvTTgBcsHatj0c5\nfYo7KMcpEBY1NfHv69ZxQHk53xk3Lm8v5PaUSaWlfH/cOF5paeE769bR4U7K6SPcQTlOAbC8tZVz\n16xhUmkp11dXM6RAnFMnhw0bxldHj+Z3DQ1cUVeHuZNy+oD+7eB2HIflra18edUqioCfjh/PyOLi\n/lapS748ejTNZtyyaRPlEpf0YiVfx+kJ7qAcpx9Z1NTEuWvWvO2cpuT5U0Y7ylejk7qjvp5iiQvG\njOn1ir6O0x3uoBynn3iooYFL165lUmnpgHBOAJK4aMwY2sy4vb6e11pauHqnnRjfz7MNncGJj0E5\nTh+zrq2Nb61dy8Vr17JveTl3TZgwIJxTJ5L4dlUVV8WJE5+traXG35Ny8oBXexynj2jp6OCO+npu\n2bSJVjPNlyrQAAAgAElEQVS+NHIk54wZU3ATIrLlmOHD2aesjAvXruXsNWs4pKKCM0eNYkZ5uY9N\nOTkhqxaUpCMkvS5piaRLugiXpBti+EuS9u8urqQxkv4k6R/xd3RuTHKcwqHDjEVNTXy3ro5/efNN\nrtu4kYMqKnhw8mTOHzt2wDqnTqYNGcKvJk7ka6NH81pLC19ctYovvPUWD27Zwvr29v5WzxngdNuC\nklQM3Aj8K1ALPC/pQTN7LSF2JLB73GYDNwGzu4l7CfC4mV0dHdclwMW5M81x+pbGjg7WtbfzZmsr\nr7W08FpLCy+1tLC2vZ1yiUOHDuW4ESOYXVHR36rmlCESZ44ezckjR3L/li38or6eb65bB8AHhgzh\nwIoKdh0yhGmlpUwrLWVUUZG3sJysyKaLbxawxMyWAki6BzgGSDqoY4A74sq6CySNkrQzMC1D3GOA\nOTH+7UANeXZQP1y/nqY8v7/x1ujR/KWuLq959CWD0Z4nk/akPA8Wt+R+hxntQDvQZkabGdvMaDSj\nsaODrR0dbGhvpyElrSklJexfXs6hQ4dy6LBhDCsa3EO+5UVFnDhyJCeMGMHibdt4urGRp5uauLO+\nnraEXAnhY7SjioqoLCqivKiIMonNVVU8tGYNJRLFQJGEYLuNzt8MDq5QXN9g/O/8pa6OIuDbVVV9\nkmc2DmoisCJxXEtoJXUnM7GbuNVmtirurwaqu8pc0lnAWQDV1dXU1NRkoXLXPLTzzjTluZCwigpe\nGEQLvQ1Ge5RiT1cFWue5IrO3C8ciM4qBYjNKzCiL27iODnbp6GBUezsj29sZ09bGpG3bqEg4rOfz\nYEtDQ8MO/R/yzR5xawfWl5SwtqSEtaWlbCkqYmtxMVuLimgqKmKzRKtES3ExKzdtol2ig+0rCx0J\nhzRQXhEejP+dFzZupAj40Cuv9EmeBTFJwsxMUpfPnZndAtwCMHPmTJszZ06v86npdcwe5FFTw47o\nWGi4PYXLYLIF3J5CZzt7dt+9T/LMxkGtBCYnjifFc9nIlGaIu0bSzma2KnYHru1OkUWLFtVJWp6F\nzv1JFTB42vVuTyEzmGwBt6fQyaU9U7MRysZBPQ/sLmkXgnP5HHBSisyDwDlxjGk2UB8dz7oMcR8E\nvghcHX//uztFzGxcFvr2K5IWmtnM/tYjV7g9hctgsgXcnkKnP+zp1kGZWZukc4A/AsXArWb2qqS5\nMXwe8DBwFLAEaAROyxQ3Jn01MF/Sl4DlwPE5tcxxHMcZ0GQ1BmVmDxOcUPLcvMS+AWdnGzeeXw98\ntCfKOo7jOO8dBve81/7hlv5WIMe4PYXLYLIF3J5Cp8/tka/r4jiO4xQi3oJyHMdxChJ3UI7jOE5B\n4g7KcRzHKUjcQTmO4zgFiTsox3EcpyBxB+U4juMUJO6gHMdxnILEHZTjOI5TkLiDchzHcQoSd1DO\nexJJ1ZKelLRF0o/6W598IulySXdlCH9V0pw0YXMk1WaIe5uk7+dATcd5F+6gBgGSTpK0UFKDpFWS\nHpF0SAzrsnCSZJJ2SxxPl/SgpPpYaP9Z0sEpcYbE9P4haaukZZJulTQtIXO0pOdi+HpJv5Q0KRF+\nqqSnurHnNkltcZ2wznPzon0NkrZJak0cPyJpWrSpIWU7IU02ZxHWthlhZhdkvMDd0B+FdLw/6yRt\nlvR3Scf0Ni0z28vManKoXl5JfXazkK+RdEaO8p4v6XBJZZJWp4T9MP43tkj6H0mn5CLP9zLuoAY4\nks4HrgOuAqqBKcCNwCd7kMauwNPAy8AuwATgfuBRSQclRH8T0z0JGAnsAywkfpVe0rHAr6I+VcBe\nQAvwlKTRWeoyDPgsUA98ofO8mc01s0ozq4y23tt5bGZHJpIYlThfaWb3pslqKvCaFcDHKCX1ZmXr\nrwGTzGwEwdnelXToTt6YQXjm9wZS1z3fCnyC8N/4InB9aiXP6SFm5tsA3Qh/hAbguAwylwN3dXHe\ngN3i/p3Aw13I3AQ8GfcPA5qAyWnyEWFdr4tSzhcR/sjfjcenAk9l0PcUYAVwHvBKtjYB06JNJVlc\nt9uAVmBbvH6HRT0vAd4A1gPzgTGJOL8GVhMc55PAXvH8WSlp/S71+iby/H7cnwPUAhfHNO+M548G\nXgQ2AX8F9s7yOZgFNAOzMlyv+cAdwBbgVWBmInwZcFjcr4i6bgReA74B1CZk9wNeiOncC9zTaVd3\nNsR8LgReitfxXqA8jc67AU9EuTpChYR47Y3gDBqAE4DRwEPAuqj3QwTnDXAl0B6vTwPwk3h+T+BP\nwAbgdeD4LK7zaGBp3P8KcE038g8CF/RVeTAYt35XwLcduHlwBNBGhkKZ7BzUauC0LmQOjX/uCsIC\nk09kyGfPmOYuXYRdATwT908ls4N6HLiG0BpsA2ZkYxM9cFBR/raUgvU8YAEwCSgDbgbuToSfDgyP\nYdcBL6ZLK/X6psoQHFQb8IOYXgWh4F9LWJG6mFADXwaUZbDhoVjwGvAHoCjDM9BMWFS0GPgPYEEi\nfBnvOKirgb8AY4DJhMpFbQwbQqiEfB0oBY4lOOdOuzLaEPefI7TQxwCLgblpdL4b+Dah4lAOHJLh\n2o4ltLqHxnv0a+CBRHgNcEbieBihEnQaYU28/QhOcHoaXT5KcLgNhIrIpvi7Ne5/pIs4FcAq4Ii+\nLhcG0+ZdfAObsUCdmbV1I3e8pE3JLSW8ivBnSmUVoYAYE/PqSiaZBmlkViXC0yJpCsEp/srM1hCc\nVU/78etSbP1AlvHmAt82s1ozayEU6sd2dr+Z2a1mtiURto+kkT3ULUkHcJmZtZhZE6EldrOZPWtm\n7WZ2O6F79MB0CZjZ0YQC+SjgUTPryJDfU2b2sJm1E1rM+6SROx640sw2mNkK4IZE2IEEx3SdmbWa\n2W+A5xPh2dhwg5m9ZWYbgN8B+6bRo5XQDTvBzJrNLO24pZmtN7P7zKzRzLYQWk0fSSdPaOUtM7Nf\nmFmbmf0NuA84Lk36j5vZKOCBKDOR4GyrzGyUmT3RRbR5wN8Jq4k7vcQd1MBmPVCVxRjG/PhHentL\nCa8Duhq/2JlQkG6MeWUa46hLxOkqnbouzqdyMrDYzF6Mx78ETpJUmkXcTqpSbF2cZbypwP0JB76Y\n0HqsllQs6WpJb0jaTCicIAunm4F1Ztackv8FKZWIyYTWRlqio3gEOFxSpnHH5IB+I1Ce5rmZQGhd\ndLI8JWylmVma8GxsSNWjMo2+FxG6jZ+LswxPTyOHpKGSbpa0PN6fJ4FRkorTRJkKzE7R8/PA+DTp\n10aZE4HbCa3EqcAqSdd2If//gA8Sug37fYxzIOMOamDzDKGG+qkdTOcxuq49Hk/ommuMMrOSM/JS\neJ0wrrJdOpKKCN0vj2ehxynA+yStjjOkriU4gaOysmLHWAEcmeLcys1sJWFSyDGEsaqRhO5ECAUo\nhC6nVBoJXU6dpBZ+qXFWEFouyfyHmtndWepfAuyapWwmVhGcSidTUsImSlKa8B214W3MbLWZnWlm\nE4AvAz/NMHPvAuD9wGwLk0Y+HM+nuz8rCN3VST0rzewraXSZROhOfyxW7m4Bzo7xzk/KSroCOBI4\n3Mw298xqJxV3UAMYM6sHvgPcKOlTsSZZKulISdf0IKkrgIMlXSlpjKThks4lOIyLY16PEQaV75c0\nQ1JJlJsr6fRYU7wQuFRh2nu5pPHAz4ARwI8T+SmGJ7eDCAXsLEK3z76EWuiv6Hk3X2+YB1wpaWpU\ncFxi6vZwQkVgPcHpXJUSdw3wvpRzLxJaf8WSjiBzlxPAfwFzJc1WYJikj0saniooac94jyvi/f4C\noVDuqqupp8wHvilpdKyMnJsIe4YwdvbVmO9nCPerxzZ0h6TjEpWhjQQn09mFmXq9hxMm8GySNAa4\nLCW5VPmHgD0knRztKJV0QDfdwTMIk0MA9ifM5EvV+ZuEysxhZra+WyOdbnEHNcAxsx8B5wOXEmYx\nrQDOIfSXZ5vGP4BDCOMSywg15c8CHzOzpxOixwIPE2Zf1RMG0GcSWldYmNJ9MmEQfT1hFlgF8KGU\nP+zBhAIluX0J+G8zeznWnleb2WrgeuDoWPBkwyZt/x7U+d1HgZjPg4Sp9VsIEyZmx7A7CF1ZK6NN\nC1Li/hyYHruLOq/7eYQpx53dRxnvh5ktBM4EfkIokJcQJpR0hQjjYGsJ9/w84AQzeyGNfE+4gmDr\n/wGPEsarOnXcBnwm6rWBMIPut720oTsOAJ6V1EC4L+eZ2dIYdjlwe7zexxMmrVQQupEXECaMJLme\nMJ64UdINcZzqcOBzwFuEbsfOCSvpmAG8EFuPexJmQqZyFaFFuSTx/H2rp4Y77yDvInUcx3EKEW9B\nOY7jOAWJOyjHcRynIHEH5TiO4xQk7qAcx3GcgqQ3H6nsN6qqqmzatGn9rUZGtm7dyrBhw/pbjZzh\n9hQug8kWcHsKnVzas2jRojozG9ed3IByUNOmTWPhwne9flBQ1NTUMGfOnP5WI2e4PflhW1s96zY+\nwfixR1BcNKRXaRSKLbnC7SlscmmPpOXdS+Wxi0/SEZJel7RE0iVdhO8p6RlJLZIuzJcejlOIbNry\nNzZt+RubG1JXbHAcp5O8OKj4DawbCZ/8mA6cKGl6itgG4KvAD/Ohg+MUMlsbwzunG7f8rZ81cZzC\nJV8tqFnAEjNbGt8+v4fwLbO3MbO1ZvY84avFjvOeob2jmcaWWkqKh9HUUkvLtnX9rZLjFCR5+ZKE\nwsqqR5jZGfH4ZMKHHM/pQvZyoMHMumxJSTqL8Bl/qqurZ9xzzz051zeXNDQ0UFmZ7gPNAw+3J/cM\nqahj5LhX2Vz3AYaPXUzTlsls3ZT6Kb/uKQRbconbU9jk0p5DDz10kZnN7E6u4CdJmNkthK8HM3Pm\nTCv0QUcfGC1sCsGet+p+T/2WUmYf8Flq1/ya0pKVzNznw4QPv2dPIdiSS9yewqY/7MlXF99Ktv9k\n/6R4znHe82xtWsrQimkUqZhRw/elrb2BhsYl/a2W4xQc+XJQzwO7S9pF0hDCV4MfzFNejjNg2Na6\niW2tG6isCF16w4fuTnHxMJ8s4ThdkJcuPjNrk3QOYbnjYuBWM3tV0twYPi+uFbSQsFZQh6SvAdN9\nkS9nMNPQ9AYAlRVhbUGpmFGVe7O+/lna2rdSUjx4Xux0nB0lb2NQZvYwYe2g5Ll5if3VhK4/x3nP\nsLVpKSXFwxlS+s5q8aOG78v6+meob3iZsSMP7EftHKew8G/xOU4fYdbB1qalVFbsSnLV9PIhOzGk\ntIqGpqUZYjvOew93UI7TRzS3rKK9o5nKoe+eUj60bBJNzSvxBUQd5x3cQTlOH9E5/jSs4t0OqqJ8\nIu0djbS2beprtRynYHEH5Th9xNam/6N8yPguJ0JUlE0EoKmltq/VcpyCxR2U4/QBHdZOY0stQ8un\ndhlePqQaqYTGZn9d0HE6cQflOH1Ac8tbmLUxrKJrByUVUVE2gaYWd1CO04k7KMfpAxqb3wRgaPmU\ntDIVZRNp3raKDmvvK7Ucp6BxB+U4fcDW5jcZUjo244u4FWUTMWunpWV1H2rmOIWLOyjHyTNmRlPz\nmxlbTwBDy8N7643ezec4gDsox8k7La3raO9oZlg3DqqkeAQlxZU+DuU4EXdQjpNnGpuXA6SdwdeJ\nJCrKJtHU7FPNHQfcQTlO3mlsepOS4uGUlozqVraifCLb2jbQ1t7YB5o5TmHjDspx8oiZsTWOPyW/\nv5eOoW+/sOvdfI7jDspx8khrWz1t7Zu7nSDRSXnZBACa/IVdx3EH5Tj5pHP8aVg340+dFBeVUVa6\nk7egHAd3UI6TVxqb36SoqIyyIeOyjlNRPpHGllo6OtryqJnjFD7uoBwnjzQ2v8nQsilI2f/VRlX+\nEx0dzWzY/FweNXOcwscdlOPkiW1t9bS01jG0Irvxp06GVexCZcWu1G36C+3tzXnSznEKH3dQjpMH\nzDp4a+0DSKWMHDa9x/F3GnMY7R3N1NU/nQftHGdg4A7KcfLAuk1/YWvzMiZUHcWQ0jE9jl9RNp6R\nlf/E+voFtLZtzoOGjlP45M1BSTpC0uuSlki6pItwSbohhr8kaf986eI4fcnWpuWs2/gEIyv3ZmTl\nPr1OZ6fRh4J1sG7jEznUznEGDnlxUJKKgRuBI4HpwImSUvs5jgR2j9tZwE350MVx+pJtbfXUrr2P\nIaWj2bnqqKxezk3HkNLRjB4xk41b/saGzQu9JeW85yjJU7qzgCVmthRA0j3AMcBrCZljgDvMzIAF\nkkZJ2tnMVuVJJ9asf4wOa81X8gAMG72SVXVNec2jL3F70mEYBmYYHbS2bqK5dS3t7VsRxUwZ/yWK\ni8p2OJdxoz9MQ9MbrKr7Pav4PWVDdqJ8yHikYipHr2ZVXSPQeydYSPizVti8Y4/YueqIPskzXw5q\nIrAicVwLzM5CZiKwnYOSdBahhUV1dTU1NTW9VmrMhEVI+X23pGyoUbdxTV7z6EvcnkwEx2AmOtrL\naGsdTnvreLY1j2Ltm68Dr+con70oLt3KkPINbKvYSGPJ64AxpKKDuo3rcpRH/+PPWmHzjj3i9VfK\n+yTPfDmonGFmtwC3AMycOdPmzJmzA6ntSNzsqKmpYcd0LCzcnsJlMNkCbk+hs509u/dNnvlyUCuB\nyYnjSfFcT2W2Y9GiRXWSludEw/xRBdT1txI5xO0pXAaTLeD2FDq5tCerb3/ly0E9D+wuaReC0/kc\ncFKKzIPAOXF8ajZQ3934k5ll/72YfkLSQjOb2d965Aq3p3AZTLaA21Po9Ic9eXFQZtYm6Rzgj0Ax\ncKuZvSppbgyfBzwMHAUsARqB0/Khi+M4jjMwydsYlJk9THBCyXPzEvsGnJ2v/B3HcZyBjX9JIvfc\n0t8K5Bi3p3AZTLaA21Po9Lk9Cg0Zx3EcxyksvAXlOI7jFCTuoBzHcZyCxB2U4ziOU5C4g3Icx3EK\nEndQjuM4TkHiDspxHMcpSNxBOY7jOAWJOyjHcRynIHEH5TiO4xQk7qCc9ySSqiU9KWmLpB/1tz75\nRNLlku7KEP6qpDlpwuZIqs0Q9zZJ38+Bmo7zLtxBDQIknSRpoaQGSaskPSLpkBjWZeEkySTtljie\nLulBSfWx0P6zpINT4gyJ6f1D0lZJyyTdKmlaQuZoSc/F8PWSfilpUiL8VElPdWPPbZLaJO2cODcv\n2tcgaZuk1sTxI5KmRZsaUrYT0mRzFmFtmxFmdkHGC9wN/VFIx/uzTtJmSX+XdExv0zKzvcysJofq\n5ZXUZzcL+RpJZ+Qo7/mSDpdUJml1Stg1klbEe7Jc0rdyked7GXdQAxxJ5wPXAVcB1cAU4Ebgkz1I\nY1fgaeBlYBdgAnA/8KikgxKiv4npngSMBPYBFgIfjekcC/wq6lMF7AW0AE9JGp2lLsOAzwL1wBc6\nz5vZXDOrNLPKaOu9ncdmdmQiiVGJ85Vmdm+arKYCr1kBfIxSUm9WFfgaMMnMRhCc7V1Jh+7kjRmE\nZ35v4JWUsFuBveI9ORj4vKTP9LF+gwsz822AbgQn0QAcl0HmcuCuLs4bsFvcvxN4uAuZm4An4/5h\nQBMwOU0+ApYDF6WcLyL8kb8bj08Fnsqg7ynACuA84JVsbQKmRZtKsrhutwGtwLZ4/Q6Lel4CvAGs\nB+YDYxJxfg2sJjjOJwkFEQTnkEzrd6nXN5Hn9+P+HKAWuDimeWc8fzTwIrAJ+Cuwd5bPwSygGZiV\n4XrNB+4AtgCvAjMT4cuAw+J+RdR1I/Aa8A2gNiG7H/BCTOde4J5Ou7qzIeZzIfBSvI73AuVpdN4N\neCLK1REqJMRrb8DWeL1PAEYDDwHrot4PEZw3wJVAe7w+DcBP4vk9gT8BG4DXgeOzuM6jgaVx/yvA\nNRlkJxIqfBd1l65vGa55fyvg2w7cPDgCaCNDoUx2Dmo1cFoXMofGP3cFcDXwRIZ89oxp7tJF2BXA\nM3H/VDI7qMeBawitwTZgRjY20QMHFeVvSylYzwMWAJOAMuBm4O5E+OnA8Bh2HfBiurRSr2+qDMFB\ntQE/iOlVEAr+tYTVpYuBLxIK9LIMNjwUC14D/gAUZXgGmgkLhBYD/wEsSIQv4x0HdTXwF2AMMJlQ\nuaiNYUMIlZCvA6XAsQTn3GlXRhvi/nOEFvoYYDEwN43OdwPfJlQcyoFDMlzbsYRW99B4j34NPJAI\nrwHOSBwPI1SCTiOsibcfwQlOT6PLRwkOt4FQEdkUf7fG/Y8kZC+JcgYsJTpK33q3eRffwGYsUGdm\nbd3IHS9pU3JLCa8CVnURbxWhgBgT8+pKJpkGaWRWJcLTImkKwSn+yszWEJzVKd3FS6EuxdYPZBlv\nLvBtM6s1sxZCoX5sZ/ebmd1qZlsSYftIGtlD3ZJ0AJeZWYuZNRFaYjeb2bNm1m5mtxO6Rw9Ml4CZ\nHU0okI8CHjWzjgz5PWVmD5tZO6HFvE8aueOBK81sg5mtAG5IhB1IcEzXmVmrmf0GeD4Rno0NN5jZ\nW2a2AfgdsG8aPVoJ3bATzKzZzNKOW5rZejO7z8wazWwLodX0kXTyhFbeMjP7hZm1mdnfgPuA49Kk\n/7iZjQIeiDITCc62ysxGmdkTCdmrCfdkf8J1rs+gh9MN7qAGNuuBqizGMObHP9LbW0p4HdDV+MXO\nhIJ0Y8wr0xhHXSJOV+nUdXE+lZOBxWb2Yjz+JXCSpNIs4nZSlWLr4izjTQXuTzjwxYTWY7WkYklX\nS3pD0mZC4QRZON0MrDOz5pT8L0ipREwmtDbSEh3FI8DhkjKNOyYH9BuB8jTPzQRC66KT5SlhK83M\n0oRnY0OqHpVp9L2I0G38XJxleHoaOSQNlXRznJiwmdANOEpScZooU4HZKXp+HhifJv3aKHMicDuh\nlTgVWCXp2lR5C/yN0CV+RTq9ne5xBzWweYZQQ/3UDqbzGF3XHo8ndM01RplZyRl5KbxOGFfZLh1J\nRYTul8ez0OMU4H2SVscZUtcSnMBRWVmxY6wAjkxxbuVmtpIwKeQYwljVSEJ3IoQCFEJ3TiqNhC6n\nTlILv9Q4Kwgtl2T+Q83s7iz1LwF2zVI2E6sITqWTKSlhEyUpTfiO2vA2ZrbazM40swnAl4GfZpi5\ndwHwfmC2hQkKH47n092fFYTu6qSelWb2lTS6TCJ0pz8WK3e3AGfHeOdnMCNX9+Q9izuoAYyZ1QPf\nAW6U9KlYkyyVdKSka3qQ1BXAwZKulDRG0nBJ5xIcxsUxr8cIg8r3S5ohqSTKzZV0eqxVXwhcqjDt\nvVzSeOBnwAjgx4n8FMOT20GEP/MsQrfPvsAHCbMCe9rN1xvmAVdKmhoVHJeYuj2cUBFYT3A6V6XE\nXQO8L+Xci4TWX7GkI8jc5QTwX8BcSbMVGCbp45KGpwpK2jPe44p4v79AKJSfeFeqPWc+8E1Jo2Nl\n5NxE2DOEsbOvxnw/Q7hfPbahOyQdl6gMbSQ4mc4uzNTrPZzQWtkkaQxwWUpyqfIPAXtIOjnaUSrp\ngG66g2cQJodA6L5bmKJvkaQvx+smSbOAs8muYuakwR3UAMfMfgScD1xKmMW0AjiH0F+ebRr/AA4h\njEssI9SUPwt8zMyeTogeCzxMmH1VTxhAn0loXWFhSvfJhEH09YRZYBXAh8xsfSKdgwkFSnL7EvDf\nZvZyrD2vNrPVwPXA0bHgyYZN2v49qEw13CTXAw8SptZvIUyYmB3D7iB0Za2MNi1IiftzYHrsLuq8\n7ucBnyAMon+ebu6HmS0EzgR+QiiQlxAmlHSFCONgawn3/DzgBDN7IY18T7iCYOv/AY8SxlE6ddwG\nfCbqtYEwg+63vbShOw4AnpXUQLgv55nZ0hh2OXB7vN7HEyatVBC6kRcQJowkuZ4wnrhR0g1xnOpw\n4HPAW4Rux84JK+mYAbwQW497EmZCpvJpwizQLcBdwH/Gzekl2r472XEcx3EKA29BOY7jOAWJOyjH\ncRynIHEH5TiO4xQk7qAcx3GcgqQ3H6nsN6qqqmzatGn9rUZGtm7dyrBhw/pbjZzh9hQug8kWcHsK\nnVzas2jRojozG9ed3IByUNOmTWPhwoXdC/YjNTU1zJkzp7/VyBluT+GynS3LlsH118OIEXDmmTAp\n3fvUhctgujfg9mRC0vLupbyLz3EGNitXwtlnwx57wI03wve+B9Omwac/DU8+2d/aOc4O4Q7KcQYo\noxcuhN12g1tugS99CZYuhTfegAsvhKeegjlz4Pe/7281HafXuINynIHIm28y/XvfCw7qf/8Xbrop\ndOvtsgtcfXXo8ttvPzjxRFic7fdyHaewcAflOAONlhY47jjU1gb33RecUirDhsEDD0BFBXzyk7Bx\nY9/r6Tg7iDsoxxloXHABPPcc/3PxxWHsKR2TJ8NvfwvLl8MJJ0Bbd8uGOU5h4Q7KcQYS99wTJkNc\neCF1H/5w9/If+hDMmwd/+hP84Af5189xcog7KMcZKGzcCOecAwcfDFelrviRgdNPh2OPhSuvhBUr\nupd3nALBHZTjDBSuuCI4qZ/+FEp7ssgw8MMfht8LL8y9Xo6TJ7JyUJKOkPS6pCWSLukiXJJuiOEv\nSdo/np8s6c+SXovLNp+XiHO5pJWSXoxbX6ya6jgDk9deg5/8JLyEu88+PY8/dSpccgnMnw81NTlX\nz3HyQbcOSlIxcCNwJDAdOFHS9BSxI4Hd43YWcFM83wZcYGbTgQOBs1Pi/tjM9o3bwztmiuMMUszg\n61+HysrwIm5v+cY3gqP66ld9woQzIMimBTULWGJmS+OKmvcAx6TIHAPcYYEFwChJO5vZqs5VPuMq\nlouBiTnU33EGP7//PTz6KFx+OYzr9vNl6amogGuvhZdfDhMnHKfAyeZbfBMJy4h3Uss7S2FnkplI\nWDocAEnTgP2AZxNy50o6BVhIaGm962UNSWcRWmVUV1dTU+DdEw0NDQWvY09we/oXtbZywFe+ApMn\n8/wHP4gldO+VLaNHs/eMGQz/1rd4dsoU2kaMyKm+O8JAuzfd4fbkADPLuAHHAj9LHJ8M/CRF5iHg\nkDgkoYQAAAoCSURBVMTx48DMxHElsAj4TOJcNVBMaMVdCdzanS4zZsywQufPf/5zf6uQU9yefuba\na83A7Pe/f1dQr2156SWzoiKzr/3/9u4+tqr6DOD492kpJWUoKy4FpYxiChE1EVKZyxSBTRwkCvrH\nZFOY+BYyXsYiCtqogCZjJGqcMeNFagwSCVEWmgU1Q1Zxi90oBHl/k/lSwmtneFmRl95nf/xO5a72\n3nvannvPOe3zSU7uveeec8/z5HdPn3tOf+d3ZncstoDFrm0ysHxSA+o0w997VfV1iu8wUJr0ur83\nz9cyIlIAvAusUtW1SYXxmKo2qWoCWI47lWiMadbQAAsXwtixMG5ccJ97442u6/lrr8HBg8F9rjEB\n81OgNgPlIlImIt2BSUB1i2WqgSleb75bgFOqekREBFgB7FHVl5JXEJF+SS/vAXa2OwtjOqOFC+H0\naXjxRRAJ9rOffx66d4e5c4P9XGMClLFAqeolYAbwAa6TwxpV3SUi00RkmrfYeuAQcBB3NPQbb/5P\ncKcEx7TSnXyxiOwQke3AaOB3gWVlTNzt2+eud3r0UbjhhuA/v29fV5zWroWPPw7+840JgK8bFqrr\nAr6+xbwlSc8VmN7Ken8HWv3pp6qT2xSpMV3Jk0+6XncLFmRvG48/DkuXusfaWsiz6/ZNtNg30pio\n2bgRqqvh6aehpCR72ykqckMmbd4Mb7yRve0Y005WoIyJksZGmDbN3UJj9uzsb++BB2DkSDcE0tGj\n2d+eMW1gBcqYKHn2WThwAF5/HXr0yP728vJg+XI4dw5mzsz+9oxpAytQxkRFbS28/LI7ghozJnfb\nHTwYnnsO3nnH3eTQmIiwAmVMFHzzDUyd6m7bvnhx7rc/Z44bhHb6dDh1KvfbN6YVVqCMiYIFC2Dv\nXne6rVev3G+/oMCdVjx61BUpN9qLMaGyAmVM2FavhkWL4JFH3KgRYamocIVy1Sp44YXw4jDG4+s6\nKGNMlmzcCFOmuJ50r74adjRQWQn797vOGoMGwf33hx2R6cKsQBkTlm3bYOJEGDIE1q3LTa+9TETc\nacYvv3Tj9Q0YALfdFnZUpouyU3zGhGHnTjcA7JVXwnvvQe/eYUd0WWGhGwKprAzuvhvefz/siEwX\nZQXKmFxSdaOI33wzJBLuj3///mFH9V3Fxa5wlpbC+PHwzDPQ1BR2VKaLsQJlTK4cP+6OSGbMgNGj\nYft2uP76sKNKrazMXZv14IOu08Qdd8ChQ2FHZboQXwVKRH4uIvtE5KCIzGvlfRGRP3rvbxeR4ZnW\nFZFiEfmriBzwHr8fTErGRIiqG+vuoYdg4EB36/ZXXnG3cc/mOHtBKSqCqio31dZCeTncey9s2mRd\n0U3WZSxQIpIPvAaMA4YCvxSRoS0WGweUe9NjwJ98rDsP+FBVy3F34P1O4TMmdi5ccNczrVwJs2bB\nsGEwYgSsWQOTJ7uOEbNmBX9/p2ybOtUNwTR3Lnz0Edx+O1x3nesaX1UFu3e7cQSNCZBohl9BIvJj\nYL6q3um9fgpAVX+ftMxSoEZV3/Ze7wNGAQNTrdu8jHdjw37e+kPSxVJRUaF1dXXtydOprMz6TvRV\nfT2lUfyfQjt1uXyS9wd3s/XLU1OTmy5dgosX3fh1jY1uOnnSXeTa0HB5/aIid23Rffe5QVmvuCLQ\nXGpqahg1alSgn+lLYyO89ZbrefjJJ/D115ff690brr4a+vRx+ffs6R4LCqBbN8jPd+P/5eW5It08\nAfX19fQvLU2x0fjptPtOXp67iWYHiMgWVa3ItJyfbubXAF8lva4HfuRjmWsyrFuiqke850eBVs93\niMhjuKMySkpKqKmp8RFy625ZsYJuZ8+2e30/+qpyKW6/jtPo6vlo0h9RzctD8/O/fUwUFtLUoweJ\nwkIuFhdz4dpruVBczPmSEs4MGcJ/Bwxwf5ABtm4NPJezZ892aH/okMGD4YknIJGgqL6eXnv3Unji\nBN0bGig8eZJuZ86Qf+IE+efOkXf+PJJIIE1Nbkok3GckEkjSD4KSLv5di7rmfDQ/n3/cdVdOthmJ\n66BUVUWk1UM5VV0GLAN3BNWhX4w5uJ1AaL9qs8Tyia7OlAtYPlGXnM+oHG3TT4E6DCQfd/f35vlZ\npiDNusdEpF/SKb7jmQLZsmXLSRH5wkfMYboKOBl2EAGyfKKrM+UClk/UBZnPD/0s5KdAbQbKRaQM\nV1wmAb9qsUw1MENEVuNO4Z3yCs+JNOtWA78GFnmP6zIFoqo/8BFvqESkzs+51biwfKKrM+UClk/U\nhZFPxgKlqpdEZAbwAZAPVKnqLhGZ5r2/BFgPjAcOAo3A1HTreh+9CFgjIg8DXwC/CDQzY4wxsebr\nf1Cquh5XhJLnLUl6rsB0v+t68xuAn7YlWGOMMV2HjSQRvGVhBxAwyye6OlMuYPlEXc7zyXgdlDHG\nGBMGO4IyxhgTSVagjDHGRJIVqACIyHwROSwi27xpfNJ7T3kD5e4TkTvDjLMtMg0QHAci8rmI7PDa\npM6bF5tBikWkSkSOi8jOpHkp44/6dy1FPrHcd0SkVET+JiK7RWSXiPzWmx/L9kmTT7jto6o2dXAC\n5gNzWpk/FPgUKATKgM+A/LDj9ZFPvhfrIKC7l8PQsONqRx6fA1e1mLcYmOc9nwf8Iew408Q/EhgO\n7MwUfxy+aynyieW+A/QDhnvPewH7vZhj2T5p8gm1fewIKrsmAKtV9byq/ht3ndiIkGPyYwRwUFUP\nqeoFYDUul85gAvCm9/xNYGKIsaSlqpuA/7SYnSr+yH/XUuSTSqTzUdUjqrrVe34G2IMbezSW7ZMm\nn1Ryko8VqODM9O6FVZV0WJ9qEN2oi2vcLSmwQUS2eIMOg89BiiMsVfxxbrNY7zsiMhAYBvyTTtA+\nLfKBENvHCpRPIrJBRHa2Mk3A3f9qEHATcATo2Fj0Jii3qupNuPuRTReRkclvqjtXEdvrLOIevyfW\n+46IfA94F5itqqeT34tj+7SST6jtE4nRzONAVX/mZzkRWQ78xXvpZ6DdKIpr3P9HVQ97j8dF5M+4\nUxBtHqQ4YlLFH8s2U9Vjzc/jtu+ISAHuj/kqVV3rzY5t+7SWT9jtY0dQAfC+iM3uAZp7KVUDk0Sk\n0Bswtxz4V67ja4dvBwgWke64QX6rQ46pTUSkp4j0an4OjMW1S/MgxeBzkOKISRV/LL9rcd13RESA\nFcAeVX0p6a1Ytk+qfEJvn7B7j3SGCVgJ7AC2ew3XL+m9SlwPl33AuLBjbUNO43E9eT4DKsOOpx3x\nD8L1MvoU2NWcA9AH+BA4AGwAisOONU0Ob+NOq1zEneN/OF38Uf+upcgnlvsOcCvu9N12YJs3jY9r\n+6TJJ9T2saGOjDHGRJKd4jPGGBNJVqCMMcZEkhUoY4wxkWQFyhhjTCRZgTLGGBNJVqCMMcZEkhUo\nY4wxkfQ/Nm35nQjOy30AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114726a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import math\n",
    "from matplotlib import (cm, pyplot as plt, mlab)\n",
    "\n",
    "def visualize(word, model):\n",
    "    \"\"\" visualize the input model for a particular word \"\"\"\n",
    "    variance=np.array([np.diag(model.covars_[i]) for i in range(model.n_components)])\n",
    "    figures = []\n",
    "    for parm_idx in range(len(model.means_[0])):\n",
    "        xmin = int(min(model.means_[:,parm_idx]) - max(variance[:,parm_idx]))\n",
    "        xmax = int(max(model.means_[:,parm_idx]) + max(variance[:,parm_idx]))\n",
    "        fig, axs = plt.subplots(model.n_components, sharex=True, sharey=False)\n",
    "        colours = cm.rainbow(np.linspace(0, 1, model.n_components))\n",
    "        for i, (ax, colour) in enumerate(zip(axs, colours)):\n",
    "            x = np.linspace(xmin, xmax, 100)\n",
    "            mu = model.means_[i,parm_idx]\n",
    "            sigma = math.sqrt(np.diag(model.covars_[i])[parm_idx])\n",
    "            ax.plot(x, mlab.normpdf(x, mu, sigma), c=colour)\n",
    "            ax.set_title(\"{} feature {} hidden state #{}\".format(word, parm_idx, i))\n",
    "\n",
    "            ax.grid(True)\n",
    "        figures.append(plt.tight_layout())\n",
    "    #for p in figures:\n",
    "    #    p.show()\n",
    "    \n",
    "visualize(my_testword, model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "#####  ModelSelector class\n",
    "Review the `ModelSelector` class from the codebase found in the `my_model_selectors.py` module.  It is designed to be a strategy pattern for choosing different model selectors.  For the project submission in this section, subclass `SelectorModel` to implement the following model selectors.  In other words, you will write your own classes/functions in the `my_model_selectors.py` module and run them from this notebook:\n",
    "\n",
    "- `SelectorCV `:  Log likelihood with CV\n",
    "- `SelectorBIC`: BIC \n",
    "- `SelectorDIC`: DIC\n",
    "\n",
    "You will train each word in the training set with a range of values for the number of hidden states, and then score these alternatives with the model selector, choosing the \"best\" according to each strategy. The simple case of training with a constant value for `n_components` can be called using the provided `SelectorConstant` subclass as follow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Features: grnd-rx, grnd-ry, grnd-lx, grnd-ly\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n",
      "For Features: delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n",
      "For Features: polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly, delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly, polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n",
      "For Features: norm-polar-rr, norm-polar-rtheta, norm-polar-lr, norm-polar-ltheta\n",
      "Number of states trained in model for VEGETABLE is 3\n",
      "Number of states trained in model for BOOK is 3\n",
      "Number of states trained in model for PREFER is 3\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from my_model_selectors import SelectorConstant\n",
    "\n",
    "for features in [features_ground, features_norm_grnd, features_delta_norm_grnd, features_polar, features_custom, features_custom2]:\n",
    "    print(\"For Features: {}\".format(', '.join(features)))\n",
    "    for word in ['VEGETABLE', 'BOOK', 'PREFER']: # Experiment here with different words\n",
    "        training = asl.build_training(features)  # Experiment here with different feature sets defined in part 1\n",
    "        model = SelectorConstant(training.get_all_sequences(), training.get_all_Xlengths(), word, n_constant=3).select()\n",
    "        print(\"Number of states trained in model for {} is {}\".format(word, model.n_components))\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Cross-validation folds\n",
    "If we simply score the model with the Log Likelihood calculated from the feature sequences it has been trained on, we should expect that more complex models will have higher likelihoods. However, that doesn't tell us which would have a better likelihood score on unseen data.  The model will likely be overfit as complexity is added.  To estimate which topology model is better using only the training data, we can compare scores using cross-validation.  One technique for cross-validation is to break the training set into \"folds\" and rotate which fold is left out of training.  The \"left out\" fold scored.  This gives us a proxy method of finding the best model to use on \"unseen data\". In the following example, a set of word sequences is broken into three folds using the [scikit-learn Kfold](http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html) class object. When you implement `SelectorCV`, you will use this technique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Features: grnd-rx, grnd-ry, grnd-lx, grnd-ly\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n",
      "For Features: norm-rx, norm-ry, norm-lx, norm-ly\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n",
      "For Features: polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n",
      "For Features: delta-rx, delta-ry, delta-lx, delta-ly\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly, delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly, polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n",
      "For Features: norm-polar-rr, norm-polar-rtheta, norm-polar-lr, norm-polar-ltheta\n",
      "Train fold indices:VEGETABLE Test fold indices:[2 3 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 4 5]\n",
      "Train fold indices:VEGETABLE Test fold indices:[0 1 2 3]\n",
      "Train fold indices:BOOK Test fold indices:[ 6  7  8  9 10 11 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5 12 13 14 15 16 17]\n",
      "Train fold indices:BOOK Test fold indices:[ 0  1  2  3  4  5  6  7  8  9 10 11]\n",
      "Train fold indices:PREFER Test fold indices:[3 4 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 5 6]\n",
      "Train fold indices:PREFER Test fold indices:[0 1 2 3 4]\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "for features in [features_ground, features_norm, features_polar, features_delta, features_custom, features_custom2]:\n",
    "    print(\"For Features: {}\".format(', '.join(features)))\n",
    "    training = asl.build_training(features) # Experiment here with different feature sets\n",
    "    for word in ['VEGETABLE', 'BOOK', 'PREFER']:# Experiment here with different words\n",
    "        word_sequences = training.get_word_sequences(word)\n",
    "        split_method = KFold()\n",
    "        for cv_train_idx, cv_test_idx in split_method.split(word_sequences):\n",
    "            print(\"Train fold indices:{} Test fold indices:{}\".format(word, cv_train_idx, cv_test_idx))  # view indices of the folds\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Tip:** In order to run `hmmlearn` training using the X,lengths tuples on the new folds, subsets must be combined based on the indices given for the folds.  A helper utility has been provided in the `asl_utils` module named `combine_sequences` for this purpose."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Scoring models with other criterion\n",
    "Scoring model topologies with **BIC** balances fit and complexity within the training set for each word.  In the BIC equation, a penalty term penalizes complexity to avoid overfitting, so that it is not necessary to also use cross-validation in the selection process.  There are a number of references on the internet for this criterion.  These [slides](http://www2.imm.dtu.dk/courses/02433/doc/ch6_slides.pdf) include a formula you may find helpful for your implementation.\n",
    "\n",
    "The advantages of scoring model topologies with **DIC** over BIC are presented by Alain Biem in this [reference](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.6208&rep=rep1&type=pdf) (also found [here](https://pdfs.semanticscholar.org/ed3d/7c4a5f607201f3848d4c02dd9ba17c791fc2.pdf)).  DIC scores the discriminant ability of a training set for one word against competing words.  Instead of a penalty term for complexity, it provides a penalty if model liklihoods for non-matching words are too similar to model likelihoods for the correct word in the word set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part2_submission'></a>\n",
    "### Model Selection Implementation Submission\n",
    "Implement `SelectorCV`, `SelectorBIC`, and `SelectorDIC` classes in the `my_model_selectors.py` module.  Run the selectors on the following five words. Then answer the questions about your results.\n",
    "\n",
    "**Tip:** The `hmmlearn` library may not be able to train or score all models.  Implement try/except contructs as necessary to eliminate non-viable models from consideration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "words_to_train = ['FISH', 'BOOK', 'VEGETABLE', 'FUTURE', 'JOHN']\n",
    "import timeit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# autoreload for automatically reloading changes made in my_model_selectors and my_recognizer\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Features: grnd-rx, grnd-ry, grnd-lx, grnd-ly\n",
      "Training complete for FISH with 3 states with time 0.017429196996090468 seconds\n",
      "Training complete for BOOK with 15 states with time 7.631037918996299 seconds\n",
      "Training complete for VEGETABLE with 15 states with time 2.7356050810049055 seconds\n",
      "Training complete for FUTURE with 15 states with time 7.51266448199749 seconds\n",
      "Training complete for JOHN with 15 states with time 71.98392313999648 seconds\n",
      "\n",
      "\n",
      "For Features: norm-rx, norm-ry, norm-lx, norm-ly\n",
      "Training complete for FISH with 3 states with time 0.02001891400141176 seconds\n",
      "Training complete for BOOK with 15 states with time 7.6717120179964695 seconds\n",
      "Training complete for VEGETABLE with 13 states with time 2.695085592997202 seconds\n",
      "Training complete for FUTURE with 15 states with time 10.099434593001206 seconds\n",
      "Training complete for JOHN with 15 states with time 68.00940729099966 seconds\n",
      "\n",
      "\n",
      "For Features: polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 3 states with time 0.04143108200514689 seconds\n",
      "Training complete for BOOK with 14 states with time 8.054697480001778 seconds\n",
      "Training complete for VEGETABLE with 15 states with time 3.2600678550006705 seconds\n",
      "Training complete for FUTURE with 15 states with time 7.801963856996736 seconds\n",
      "Training complete for JOHN with 15 states with time 96.39143779699953 seconds\n",
      "\n",
      "\n",
      "For Features: delta-rx, delta-ry, delta-lx, delta-ly\n",
      "Training complete for FISH with 3 states with time 0.017349911002384033 seconds\n",
      "Training complete for BOOK with 13 states with time 10.145716072998766 seconds\n",
      "Training complete for VEGETABLE with 14 states with time 2.82975213300233 seconds\n",
      "Training complete for FUTURE with 5 states with time 3.053132804998313 seconds\n",
      "Training complete for JOHN with 15 states with time 97.38220669099974 seconds\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly, delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly, polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 3 states with time 0.012755595998896752 seconds\n",
      "Training complete for BOOK with 15 states with time 6.440526984995813 seconds\n",
      "Training complete for VEGETABLE with 15 states with time 2.3182206400015275 seconds\n",
      "Training complete for FUTURE with 15 states with time 5.326594521000516 seconds\n",
      "Training complete for JOHN with 15 states with time 105.3185355490059 seconds\n",
      "\n",
      "\n",
      "For Features: norm-polar-rr, norm-polar-rtheta, norm-polar-lr, norm-polar-ltheta\n",
      "Training complete for FISH with 3 states with time 0.01752006400056416 seconds\n",
      "Training complete for BOOK with 15 states with time 6.645680902001914 seconds\n",
      "Training complete for VEGETABLE with 15 states with time 2.3035293700013426 seconds\n",
      "Training complete for FUTURE with 15 states with time 7.480311676001293 seconds\n",
      "Training complete for JOHN with 15 states with time 80.05443284600187 seconds\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# TODO: Implement SelectorCV in my_model_selector.py\n",
    "from my_model_selectors import SelectorCV\n",
    "\n",
    "for features in [features_ground, features_norm, features_polar, features_delta, features_custom, features_custom2]:\n",
    "    print(\"For Features: {}\".format(', '.join(features)))\n",
    "    training = asl.build_training(features)  # Experiment here with different feature sets defined in part 1\n",
    "    sequences = training.get_all_sequences()\n",
    "    Xlengths = training.get_all_Xlengths()\n",
    "    for word in words_to_train:\n",
    "        start = timeit.default_timer()\n",
    "        model = SelectorCV(sequences, Xlengths, word, \n",
    "                        min_n_components=2, max_n_components=15, random_state = 14).select()\n",
    "        end = timeit.default_timer()-start\n",
    "        if model is not None:\n",
    "            print(\"Training complete for {} with {} states with time {} seconds\".format(word, model.n_components, end))\n",
    "        else:\n",
    "            print(\"Training failed for {}\".format(word))\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Features: grnd-rx, grnd-ry, grnd-lx, grnd-ly\n",
      "Training complete for FISH with 2 states with time 0.5083610430010594 seconds\n",
      "Training complete for BOOK with 5 states with time 2.3749072439968586 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 1.0230197199998656 seconds\n",
      "Training complete for FUTURE with 7 states with time 2.5702231059985934 seconds\n",
      "Training complete for JOHN with 15 states with time 23.73983343999862 seconds\n",
      "\n",
      "\n",
      "For Features: norm-rx, norm-ry, norm-lx, norm-ly\n",
      "Training complete for FISH with 2 states with time 0.46581525699730264 seconds\n",
      "Training complete for BOOK with 5 states with time 2.187444796996715 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 0.839135713998985 seconds\n",
      "Training complete for FUTURE with 7 states with time 3.470598104999226 seconds\n",
      "Training complete for JOHN with 15 states with time 21.086591559003864 seconds\n",
      "\n",
      "\n",
      "For Features: polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.37221223899541656 seconds\n",
      "Training complete for BOOK with 5 states with time 2.1939105929996003 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 0.8480037159970379 seconds\n",
      "Training complete for FUTURE with 7 states with time 2.4597269150035572 seconds\n",
      "Training complete for JOHN with 15 states with time 24.194740694998472 seconds\n",
      "\n",
      "\n",
      "For Features: delta-rx, delta-ry, delta-lx, delta-ly\n",
      "Training complete for FISH with 2 states with time 0.3874858650015085 seconds\n",
      "Training complete for BOOK with 5 states with time 2.526043418001791 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 1.1064737390042865 seconds\n",
      "Training complete for FUTURE with 7 states with time 3.0120708769973135 seconds\n",
      "Training complete for JOHN with 15 states with time 22.317160674996558 seconds\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly, delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly, polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.35506775099929655 seconds\n",
      "Training complete for BOOK with 5 states with time 2.226308760000393 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 0.8733371889975388 seconds\n",
      "Training complete for FUTURE with 7 states with time 2.474384203000227 seconds\n",
      "Training complete for JOHN with 15 states with time 21.03825439100183 seconds\n",
      "\n",
      "\n",
      "For Features: norm-polar-rr, norm-polar-rtheta, norm-polar-lr, norm-polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.39198113499878673 seconds\n",
      "Training complete for BOOK with 5 states with time 2.508924987996579 seconds\n",
      "Training complete for VEGETABLE with 5 states with time 0.8558187170056044 seconds\n",
      "Training complete for FUTURE with 7 states with time 2.4724012220031 seconds\n",
      "Training complete for JOHN with 15 states with time 20.81340588299645 seconds\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# TODO: Implement SelectorBIC in module my_model_selectors.py\n",
    "from my_model_selectors import SelectorBIC\n",
    "\n",
    "for features in [features_ground, features_norm, features_polar, features_delta, features_custom, features_custom2]:\n",
    "    print(\"For Features: {}\".format(', '.join(features)))\n",
    "    training = asl.build_training(features_norm)  # Experiment here with different feature sets defined in part 1\n",
    "    sequences = training.get_all_sequences()\n",
    "    Xlengths = training.get_all_Xlengths()\n",
    "    for word in words_to_train:\n",
    "        start = timeit.default_timer()\n",
    "        model = SelectorBIC(sequences, Xlengths, word, \n",
    "                        min_n_components=2, max_n_components=15, random_state = 14).select()\n",
    "        end = timeit.default_timer()-start\n",
    "        if model is not None:\n",
    "            print(\"Training complete for {} with {} states with time {} seconds\".format(word, model.n_components, end))\n",
    "        else:\n",
    "            print(\"Training failed for {}\".format(word))\n",
    "    print(\"\\n\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For Features: grnd-rx, grnd-ry, grnd-lx, grnd-ly\n",
      "Training complete for FISH with 2 states with time 0.6633177000039723 seconds\n",
      "Training complete for BOOK with 15 states with time 2.651067051003338 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.8423181960024522 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.9325823559993296 seconds\n",
      "Training complete for JOHN with 15 states with time 22.002903697000875 seconds\n",
      "\n",
      "\n",
      "For Features: norm-rx, norm-ry, norm-lx, norm-ly\n",
      "Training complete for FISH with 2 states with time 0.36614591399847995 seconds\n",
      "Training complete for BOOK with 15 states with time 2.149419809997198 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.8906770270041307 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.5192945119997603 seconds\n",
      "Training complete for JOHN with 15 states with time 23.771717907002312 seconds\n",
      "\n",
      "\n",
      "For Features: polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.4469728300027782 seconds\n",
      "Training complete for BOOK with 15 states with time 2.1698754399985773 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.8720005370050785 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.7038290920027066 seconds\n",
      "Training complete for JOHN with 15 states with time 21.508271152997622 seconds\n",
      "\n",
      "\n",
      "For Features: delta-rx, delta-ry, delta-lx, delta-ly\n",
      "Training complete for FISH with 2 states with time 0.4098692160041537 seconds\n",
      "Training complete for BOOK with 15 states with time 2.202282173006097 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.9708353879977949 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.7397027920014807 seconds\n",
      "Training complete for JOHN with 15 states with time 22.261935776994505 seconds\n",
      "\n",
      "\n",
      "For Features: norm-grnd-rx, norm-grnd-ry, norm-grnd-lx, norm-grnd-ly, delta-norm-grnd-rx, delta-norm-grnd-ry, delta-norm-grnd-lx, delta-norm-grnd-ly, polar-rr, polar-rtheta, polar-lr, polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.5386828539994895 seconds\n",
      "Training complete for BOOK with 15 states with time 2.690197664000152 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.8415045559959253 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.4607197740042466 seconds\n",
      "Training complete for JOHN with 15 states with time 21.240913279005326 seconds\n",
      "\n",
      "\n",
      "For Features: norm-polar-rr, norm-polar-rtheta, norm-polar-lr, norm-polar-ltheta\n",
      "Training complete for FISH with 2 states with time 0.4146465209996677 seconds\n",
      "Training complete for BOOK with 15 states with time 2.2418584290062427 seconds\n",
      "Training complete for VEGETABLE with 10 states with time 0.7771465959958732 seconds\n",
      "Training complete for FUTURE with 15 states with time 2.4506959759964957 seconds\n",
      "Training complete for JOHN with 15 states with time 21.0271422260048 seconds\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# TODO: Implement SelectorDIC in module my_model_selectors.py\n",
    "from my_model_selectors import SelectorDIC\n",
    "\n",
    "for features in [features_ground, features_norm, features_polar, features_delta, features_custom, features_custom2]:\n",
    "    print(\"For Features: {}\".format(', '.join(features)))\n",
    "    training = asl.build_training(features_norm)  # Experiment here with different feature sets defined in part 1\n",
    "    sequences = training.get_all_sequences()\n",
    "    Xlengths = training.get_all_Xlengths()\n",
    "    for word in words_to_train:\n",
    "        start = timeit.default_timer()\n",
    "        model = SelectorDIC(sequences, Xlengths, word, \n",
    "                        min_n_components=2, max_n_components=15, random_state = 14).select()\n",
    "        end = timeit.default_timer()-start\n",
    "        if model is not None:\n",
    "            print(\"Training complete for {} with {} states with time {} seconds\".format(word, model.n_components, end))\n",
    "        else:\n",
    "            print(\"Training failed for {}\".format(word))\n",
    "    print(\"\\n\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Question 2:**  Compare and contrast the possible advantages and disadvantages of the various model selectors implemented.\n",
    "\n",
    "**Answer 2:**\n",
    "\n",
    "**BIC**  \n",
    "\n",
    "In statistics, the Bayesian information criterion (BIC) or Schwarz criterion (also SBC, SBIC) is a criterion for model selection among a finite set of models; the model with the lowest BIC is preferred. It is based, in part, on the likelihood function and it is closely related to the Akaike information criterion (AIC).\n",
    "\n",
    "When fitting models, it is possible to increase the likelihood by adding parameters, but doing so may result in overfitting. Both BIC and AIC attempt to resolve this problem by introducing a penalty term for the number of parameters in the model; the penalty term is larger in BIC than in AIC.\n",
    "\n",
    "The two main limitations of BIC are:\n",
    "\n",
    "* The approximation that BIC is equal to the minimum description length criterion but with negative sign, is only valid for sample size _n_ much larger than the number _k_ of parameters in the model.\n",
    "* The BIC cannot handle complex collections of models as in the variable selection (or feature selection) problem in high-dimension\n",
    "\n",
    "**DIC**  \n",
    "\n",
    "DIC is a discriminant selector ability of a training set for one word against competing words. Instead of a penalty term for complexity, it provides a penalty if model liklihoods for non-matching words are too similar to model likelihoods for the correct word in the word set. \n",
    "\n",
    "The competitive nature of DIC makes it the best performing selection strategy in most of the cases.\n",
    "\n",
    "DIC & BIC had similar performace stats in the limited testing I ran above.\n",
    "\n",
    "\n",
    "**CV**  \n",
    "\n",
    "Cross validation holds out a random portion of the training set on which the model can be tested to help prevent over-fitting to the training data.\n",
    "\n",
    "In order to obtain good performance from this method one generally needs to use a large training set from which it is possible to hold out a larger validation set.\n",
    "\n",
    "\n",
    "References:\n",
    "\n",
    "[BIC Wikipedia]('https://en.wikipedia.org/wiki/Bayesian_information_criterion')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part2_test'></a>\n",
    "### Model Selector Unit Testing\n",
    "Run the following unit tests as a sanity check on the implemented model selectors.  The test simply looks for valid interfaces  but is not exhaustive. However, the project should not be submitted if these tests don't pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "....\n",
      "----------------------------------------------------------------------\n",
      "Ran 4 tests in 84.452s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.runner.TextTestResult run=4 errors=0 failures=0>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from asl_test_model_selectors import TestSelectors\n",
    "suite = unittest.TestLoader().loadTestsFromModule(TestSelectors())\n",
    "unittest.TextTestRunner().run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part3_tutorial'></a>\n",
    "## PART 3: Recognizer\n",
    "The objective of this section is to \"put it all together\".  Using the four feature sets created and the three model selectors, you will experiment with the models and present your results.  Instead of training only five specific words as in the previous section, train the entire set with a feature set and model selector strategy.  \n",
    "### Recognizer Tutorial\n",
    "##### Train the full training set\n",
    "The following example trains the entire set with the example `features_ground` and `SelectorConstant` features and model selector.  Use this pattern for you experimentation and final submission cells.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of word models returned = 112\n"
     ]
    }
   ],
   "source": [
    "from my_model_selectors import SelectorConstant\n",
    "\n",
    "def train_all_words(features, model_selector):\n",
    "    training = asl.build_training(features)  # Experiment here with different feature sets defined in part 1\n",
    "    sequences = training.get_all_sequences()\n",
    "    Xlengths = training.get_all_Xlengths()\n",
    "    model_dict = {}\n",
    "    for word in training.words:\n",
    "        model = model_selector(sequences, Xlengths, word, \n",
    "                        n_constant=3).select()\n",
    "        model_dict[word]=model\n",
    "    return model_dict\n",
    "\n",
    "models = train_all_words(features_ground, SelectorConstant)\n",
    "print(\"Number of word models returned = {}\".format(len(models)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "##### Load the test set\n",
    "The `build_test` method in `ASLdb` is similar to the `build_training` method already presented, but there are a few differences:\n",
    "- the object is type `SinglesData` \n",
    "- the internal dictionary keys are the index of the test word rather than the word itself\n",
    "- the getter methods are `get_all_sequences`, `get_all_Xlengths`, `get_item_sequences` and `get_item_Xlengths`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of test set items: 178\n",
      "Number of test set sentences: 40\n"
     ]
    }
   ],
   "source": [
    "test_set = asl.build_test(features_ground)\n",
    "print(\"Number of test set items: {}\".format(test_set.num_items))\n",
    "print(\"Number of test set sentences: {}\".format(len(test_set.sentences_index)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part3_submission'></a>\n",
    "### Recognizer Implementation Submission\n",
    "For the final project submission, students must implement a recognizer following guidance in the `my_recognizer.py` module.  Experiment with the four feature sets and the three model selection methods (that's 12 possible combinations). You can add and remove cells for experimentation or run the recognizers locally in some other way during your experiments, but retain the results for your discussion.  For submission, you will provide code cells of **only three** interesting combinations for your discussion (see questions below). At least one of these should produce a word error rate of less than 60%, i.e. WER < 0.60 . \n",
    "\n",
    "**Tip:** The hmmlearn library may not be able to train or score all models.  Implement try/except contructs as necessary to eliminate non-viable models from consideration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# TODO implement the recognize method in my_recognizer\n",
    "from my_recognizer import recognize\n",
    "from asl_utils import show_errors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "**** WER = 0.5561797752808989\n",
      "Total correct: 79 out of 178\n",
      "Video  Recognized                                                    Correct\n",
      "=====================================================================================================\n",
      "    2: JOHN WRITE *NEW                                               JOHN WRITE HOMEWORK\n",
      "    7: *SOMETHING-ONE *BOX GO *ARRIVE                                JOHN CAN GO CAN\n",
      "   12: *IX *WHAT *CAN CAN                                            JOHN CAN GO CAN\n",
      "   21: JOHN *WRITE *JOHN *FUTURE *CAR *TEACHER *VISIT *MARY          JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n",
      "   25: JOHN *IX IX *LIKE IX                                          JOHN LIKE IX IX IX\n",
      "   28: JOHN *WHO IX *LIKE *LOVE                                      JOHN LIKE IX IX IX\n",
      "   30: JOHN LIKE *MARY *MARY *MARY                                   JOHN LIKE IX IX IX\n",
      "   36: *VISIT *VISIT *IX *GIVE *MARY *IX                             MARY VEGETABLE KNOW IX LIKE CORN1\n",
      "   40: *MARY *GO *GIVE MARY *MARY                                    JOHN IX THINK MARY LOVE\n",
      "   43: JOHN *IX BUY HOUSE                                            JOHN MUST BUY HOUSE\n",
      "   50: *JOHN *SEE BUY CAR *JOHN                                      FUTURE JOHN BUY CAR SHOULD\n",
      "   54: JOHN SHOULD NOT BUY HOUSE                                     JOHN SHOULD NOT BUY HOUSE\n",
      "   57: *MARY *VISIT VISIT MARY                                       JOHN DECIDE VISIT MARY\n",
      "   67: *SHOULD *JOHN *BLAME BUY HOUSE                                JOHN FUTURE NOT BUY HOUSE\n",
      "   71: JOHN *FUTURE VISIT MARY                                       JOHN WILL VISIT MARY\n",
      "   74: *IX *VISIT VISIT MARY                                         JOHN NOT VISIT MARY\n",
      "   77: *JOHN BLAME *LOVE                                             ANN BLAME MARY\n",
      "   84: *JOHN *ARRIVE *NEW BOOK                                       IX-1P FIND SOMETHING-ONE BOOK\n",
      "   89: *MARY *POSS *IX *IX IX *ARRIVE *BOOK                          JOHN IX GIVE MAN IX NEW COAT\n",
      "   90: JOHN *SOMETHING-ONE IX *IX *VISIT *ARRIVE                     JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "   92: JOHN *SHOULD IX *IX *IX BOOK                                  JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "  100: *IX NEW CAR BREAK-DOWN                                        POSS NEW CAR BREAK-DOWN\n",
      "  105: JOHN *FRANK                                                   JOHN LEG\n",
      "  107: JOHN *GO *HAVE *ARRIVE *JOHN                                  JOHN POSS FRIEND HAVE CANDY\n",
      "  108: *IX *LOVE                                                     WOMAN ARRIVE\n",
      "  113: IX CAR *CAR *MARY *BOX                                        IX CAR BLUE SUE BUY\n",
      "  119: *VISIT *BUY1 IX *BOX *GO                                      SUE BUY IX CAR BLUE\n",
      "  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK\n",
      "  139: JOHN *BUY1 WHAT *MARY BOOK                                    JOHN BUY WHAT YESTERDAY BOOK\n",
      "  142: JOHN *STUDENT YESTERDAY WHAT BOOK                             JOHN BUY YESTERDAY WHAT BOOK\n",
      "  158: LOVE JOHN WHO                                                 LOVE JOHN WHO\n",
      "  167: JOHN *MARY *VISIT LOVE MARY                                   JOHN IX SAY LOVE MARY\n",
      "  171: JOHN MARY BLAME                                               JOHN MARY BLAME\n",
      "  174: *CAN *GIVE1 GIVE1 *YESTERDAY *WHAT                            PEOPLE GROUP GIVE1 JANA TOY\n",
      "  181: JOHN *BOX                                                     JOHN ARRIVE\n",
      "  184: *GIVE BOY *GIVE1 TEACHER APPLE                                ALL BOY GIVE TEACHER APPLE\n",
      "  189: JOHN *SOMETHING-ONE *VISIT BOX                                JOHN GIVE GIRL BOX\n",
      "  193: JOHN *SOMETHING-ONE *VISIT BOX                                JOHN GIVE GIRL BOX\n",
      "  199: *JOHN CHOCOLATE *GO                                           LIKE CHOCOLATE WHO\n",
      "  201: JOHN *MARY *LOVE *JOHN BUY HOUSE                              JOHN TELL MARY IX-1P BUY HOUSE\n"
     ]
    }
   ],
   "source": [
    "# TODO Choose a feature set and model selector\n",
    "features = features_ground # change as needed\n",
    "model_selector = SelectorBIC # change as needed\n",
    "\n",
    "# TODO Recognize the test set and display the result with the show_errors method\n",
    "models = train_all_words(features, model_selector)\n",
    "test_set = asl.build_test(features)\n",
    "probabilities, guesses = recognize(models, test_set)\n",
    "show_errors(guesses, test_set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "**** WER = 0.43820224719101125\n",
      "Total correct: 100 out of 178\n",
      "Video  Recognized                                                    Correct\n",
      "=====================================================================================================\n",
      "    2: *FUTURE WRITE HOMEWORK                                        JOHN WRITE HOMEWORK\n",
      "    7: JOHN *VISIT *IX CAN                                           JOHN CAN GO CAN\n",
      "   12: JOHN *CAR *VISIT *HOUSE                                       JOHN CAN GO CAN\n",
      "   21: JOHN *NEW WONT *JOHN *BUY *CAR *FUTURE *MARY                  JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n",
      "   25: JOHN *MARY IX IX IX                                           JOHN LIKE IX IX IX\n",
      "   28: JOHN *IX IX IX IX                                             JOHN LIKE IX IX IX\n",
      "   30: JOHN LIKE *MARY IX IX                                         JOHN LIKE IX IX IX\n",
      "   36: MARY VEGETABLE *GIVE *GIVE *MARY *MARY                        MARY VEGETABLE KNOW IX LIKE CORN1\n",
      "   40: JOHN *VISIT *FUTURE1 *JOHN *MARY                              JOHN IX THINK MARY LOVE\n",
      "   43: JOHN *JOHN BUY HOUSE                                          JOHN MUST BUY HOUSE\n",
      "   50: *SOMETHING-ONE *SEE BUY CAR *JOHN                             FUTURE JOHN BUY CAR SHOULD\n",
      "   54: JOHN SHOULD NOT BUY HOUSE                                     JOHN SHOULD NOT BUY HOUSE\n",
      "   57: *IX *VISIT VISIT *VISIT                                       JOHN DECIDE VISIT MARY\n",
      "   67: JOHN FUTURE *WHO BUY HOUSE                                    JOHN FUTURE NOT BUY HOUSE\n",
      "   71: JOHN *FINISH *GIVE1 MARY                                      JOHN WILL VISIT MARY\n",
      "   74: *IX *VISIT VISIT MARY                                         JOHN NOT VISIT MARY\n",
      "   77: *JOHN BLAME MARY                                              ANN BLAME MARY\n",
      "   84: *IX *BUY *HOMEWORK BOOK                                       IX-1P FIND SOMETHING-ONE BOOK\n",
      "   89: JOHN IX GIVE *IX IX NEW *BREAK-DOWN                           JOHN IX GIVE MAN IX NEW COAT\n",
      "   90: JOHN *IX IX SOMETHING-ONE WOMAN BOOK                          JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "   92: JOHN GIVE IX *IX *IX BOOK                                     JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "  100: POSS NEW CAR BREAK-DOWN                                       POSS NEW CAR BREAK-DOWN\n",
      "  105: JOHN *SEE                                                     JOHN LEG\n",
      "  107: JOHN *JOHN *HAVE *MARY *MARY                                  JOHN POSS FRIEND HAVE CANDY\n",
      "  108: WOMAN *HOMEWORK                                               WOMAN ARRIVE\n",
      "  113: IX CAR *IX *MARY *BOX                                         IX CAR BLUE SUE BUY\n",
      "  119: *PREFER *BUY1 IX *VISIT *IX                                   SUE BUY IX CAR BLUE\n",
      "  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK\n",
      "  139: JOHN *BUY1 WHAT *MARY BOOK                                    JOHN BUY WHAT YESTERDAY BOOK\n",
      "  142: JOHN BUY YESTERDAY WHAT BOOK                                  JOHN BUY YESTERDAY WHAT BOOK\n",
      "  158: LOVE JOHN WHO                                                 LOVE JOHN WHO\n",
      "  167: JOHN IX *VISIT *WOMAN MARY                                    JOHN IX SAY LOVE MARY\n",
      "  171: JOHN MARY BLAME                                               JOHN MARY BLAME\n",
      "  174: *GIVE1 *GIVE3 GIVE1 *GIRL *JOHN                               PEOPLE GROUP GIVE1 JANA TOY\n",
      "  181: JOHN ARRIVE                                                   JOHN ARRIVE\n",
      "  184: *IX BOY *GIVE1 TEACHER *GIRL                                  ALL BOY GIVE TEACHER APPLE\n",
      "  189: JOHN *IX *VISIT BOX                                           JOHN GIVE GIRL BOX\n",
      "  193: JOHN *IX *VISIT BOX                                           JOHN GIVE GIRL BOX\n",
      "  199: *JOHN CHOCOLATE *MARY                                         LIKE CHOCOLATE WHO\n",
      "  201: JOHN *MAN *WOMAN *LIKE BUY HOUSE                              JOHN TELL MARY IX-1P BUY HOUSE\n"
     ]
    }
   ],
   "source": [
    "# TODO Choose a feature set and model selector\n",
    "features = features_custom\n",
    "model_selector = SelectorBIC\n",
    "\n",
    "# TODO Recognize the test set and display the result with the show_errors method\n",
    "models = train_all_words(features, model_selector)\n",
    "test_set = asl.build_test(features)\n",
    "probabilities, guesses = recognize(models, test_set)\n",
    "show_errors(guesses, test_set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "**** WER = 0.5280898876404494\n",
      "Total correct: 84 out of 178\n",
      "Video  Recognized                                                    Correct\n",
      "=====================================================================================================\n",
      "    2: JOHN WRITE HOMEWORK                                           JOHN WRITE HOMEWORK\n",
      "    7: JOHN CAN GO CAN                                               JOHN CAN GO CAN\n",
      "   12: JOHN *WHAT *JOHN CAN                                          JOHN CAN GO CAN\n",
      "   21: JOHN *HOMEWORK *JOHN *PREFER BUT *WHAT *FUTURE *WHO           JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n",
      "   25: JOHN *IX IX *WHO IX                                           JOHN LIKE IX IX IX\n",
      "   28: JOHN *FUTURE IX *FUTURE *LOVE                                 JOHN LIKE IX IX IX\n",
      "   30: JOHN LIKE *MARY *MARY *MARY                                   JOHN LIKE IX IX IX\n",
      "   36: *IX *VISIT *GIVE *GIVE *MARY *MARY                            MARY VEGETABLE KNOW IX LIKE CORN1\n",
      "   40: JOHN *GO *DECIDE *JOHN *MARY                                  JOHN IX THINK MARY LOVE\n",
      "   43: JOHN *IX BUY HOUSE                                            JOHN MUST BUY HOUSE\n",
      "   50: *JOHN *SEE BUY CAR *JOHN                                      FUTURE JOHN BUY CAR SHOULD\n",
      "   54: JOHN SHOULD NOT BUY HOUSE                                     JOHN SHOULD NOT BUY HOUSE\n",
      "   57: *MARY *GO *GO MARY                                            JOHN DECIDE VISIT MARY\n",
      "   67: *SHOULD FUTURE *MARY BUY HOUSE                                JOHN FUTURE NOT BUY HOUSE\n",
      "   71: JOHN *FUTURE *GIVE1 MARY                                      JOHN WILL VISIT MARY\n",
      "   74: *IX *GO *GO *VISIT                                            JOHN NOT VISIT MARY\n",
      "   77: *JOHN *GIVE1 MARY                                             ANN BLAME MARY\n",
      "   84: *HOMEWORK *GIVE1 *HOMEWORK *COAT                              IX-1P FIND SOMETHING-ONE BOOK\n",
      "   89: *MAN *GIVE *IX *IX IX *ARRIVE *BOOK                           JOHN IX GIVE MAN IX NEW COAT\n",
      "   90: JOHN GIVE IX SOMETHING-ONE WOMAN *ARRIVE                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "   92: JOHN *IX IX *IX *IX BOOK                                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n",
      "  100: POSS NEW CAR BREAK-DOWN                                       POSS NEW CAR BREAK-DOWN\n",
      "  105: JOHN *SEE                                                     JOHN LEG\n",
      "  107: JOHN POSS *HAVE HAVE *MARY                                    JOHN POSS FRIEND HAVE CANDY\n",
      "  108: *LOVE *HOMEWORK                                               WOMAN ARRIVE\n",
      "  113: IX CAR *IX *MARY *JOHN                                        IX CAR BLUE SUE BUY\n",
      "  119: *MARY *BUY1 IX *BLAME *IX                                     SUE BUY IX CAR BLUE\n",
      "  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK\n",
      "  139: JOHN *ARRIVE WHAT *MARY *ARRIVE                               JOHN BUY WHAT YESTERDAY BOOK\n",
      "  142: JOHN BUY YESTERDAY WHAT BOOK                                  JOHN BUY YESTERDAY WHAT BOOK\n",
      "  158: LOVE JOHN WHO                                                 LOVE JOHN WHO\n",
      "  167: JOHN *MARY *VISIT LOVE MARY                                   JOHN IX SAY LOVE MARY\n",
      "  171: *IX MARY BLAME                                                JOHN MARY BLAME\n",
      "  174: *JOHN *GIVE3 GIVE1 *YESTERDAY *JOHN                           PEOPLE GROUP GIVE1 JANA TOY\n",
      "  181: *EAT ARRIVE                                                   JOHN ARRIVE\n",
      "  184: ALL BOY *GIVE1 TEACHER *YESTERDAY                             ALL BOY GIVE TEACHER APPLE\n",
      "  189: *MARY *GO *YESTERDAY BOX                                      JOHN GIVE GIRL BOX\n",
      "  193: JOHN *GO *YESTERDAY BOX                                       JOHN GIVE GIRL BOX\n",
      "  199: *HOMEWORK *STUDENT *GO                                        LIKE CHOCOLATE WHO\n",
      "  201: JOHN *MAN *LOVE *JOHN BUY HOUSE                               JOHN TELL MARY IX-1P BUY HOUSE\n"
     ]
    }
   ],
   "source": [
    "# TODO Choose a feature set and model selector\n",
    "features = features_polar # change as needed\n",
    "model_selector = SelectorCV # change as needed\n",
    "\n",
    "# TODO Recognize the test set and display the result with the show_errors method\n",
    "models = train_all_words(features, model_selector)\n",
    "test_set = asl.build_test(features)\n",
    "probabilities, guesses = recognize(models, test_set)\n",
    "show_errors(guesses, test_set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Question 3:**  Summarize the error results from three combinations of features and model selectors.  What was the \"best\" combination and why?  What additional information might we use to improve our WER?  For more insight on improving WER, take a look at the introduction to Part 4.\n",
    "\n",
    "**Answer 3:**\n",
    "\n",
    "![](results_summary.png)\n",
    "\n",
    "The best combination of features and model from my experiments was the first custom feature set (which was a combination of normalized ground features, delta of normalized ground features and polar features), along with the BIC model selector.\n",
    "\n",
    "The following chart shows the best performance of each feature set:\n",
    "\n",
    "![](performance.png)\n",
    "\n",
    "The combined custom feature set builds upon the accuracy of the polar feature set by combining it with the ground features and deltas. This combination increases the overall predictive power of the model compared to the standalone features.\n",
    "\n",
    "I was surprised to see that normalisation of the polar features (features_custom2) did not improve the error rate over that of the polar feature set.\n",
    "\n",
    "The following chart shows the relative performace of the model selection techniques:\n",
    "\n",
    "![](model_selection.png)\n",
    "\n",
    "According to my experiments, the BIC model selector produced the highest accuracy (lowest error rate) for the custom, grounded and delta feature sets, whereas cross validation performed better for polar and normalized grounded feature sets.\n",
    "\n",
    "\n",
    "**Improvements**\n",
    "\n",
    "It is certainly possible to improve the predictive capacity of the feature sets by progressing from a '0-gram' statistical language model (SLM) which only considers the probability of a single word being signed, to an 'n-gram' (2 or more word model) which considers the probability of a word being signed given the occurance of the previous word or words in a sentence (depending on the length of n)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part3_test'></a>\n",
    "### Recognizer Unit Tests\n",
    "Run the following unit tests as a sanity check on the defined recognizer.  The test simply looks for some valid values but is not exhaustive. However, the project should not be submitted if these tests don't pass."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "..\n",
      "----------------------------------------------------------------------\n",
      "Ran 2 tests in 32.490s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.runner.TextTestResult run=2 errors=0 failures=0>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from asl_test_recognizer import TestRecognize\n",
    "suite = unittest.TestLoader().loadTestsFromModule(TestRecognize())\n",
    "unittest.TextTestRunner().run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id='part4_info'></a>\n",
    "## PART 4: (OPTIONAL)  Improve the WER with Language Models\n",
    "We've squeezed just about as much as we can out of the model and still only get about 50% of the words right! Surely we can do better than that.  Probability to the rescue again in the form of [statistical language models (SLM)](https://en.wikipedia.org/wiki/Language_model).  The basic idea is that each word has some probability of occurrence within the set, and some probability that it is adjacent to specific other words. We can use that additional information to make better choices.\n",
    "\n",
    "##### Additional reading and resources\n",
    "- [Introduction to N-grams (Stanford Jurafsky slides)](https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf)\n",
    "- [Speech Recognition Techniques for a Sign Language Recognition System, Philippe Dreuw et al](https://www-i6.informatik.rwth-aachen.de/publications/download/154/Dreuw--2007.pdf) see the improved results of applying LM on *this* data!\n",
    "- [SLM data for *this* ASL dataset](ftp://wasserstoff.informatik.rwth-aachen.de/pub/rwth-boston-104/lm/)\n",
    "\n",
    "##### Optional challenge\n",
    "The recognizer you implemented in Part 3 is equivalent to a \"0-gram\" SLM.  Improve the WER with the SLM data provided with the data set in the link above using \"1-gram\", \"2-gram\", and/or \"3-gram\" statistics. The `probabilities` data you've already calculated will be useful and can be turned into a pandas DataFrame if desired (see next cell).  \n",
    "Good luck!  Share your results with the class!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# create a DataFrame of log likelihoods for the test word items\n",
    "df_probs = pd.DataFrame(data=probabilities)\n",
    "df_probs.head()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  },
  "nbpresent": {
   "slides": {
    "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a": {
     "id": "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a",
     "prev": null,
     "regions": {
      "3fb9ce83-fbb2-4995-832a-f8f400734ad3": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "1dbb9346-179b-4835-b430-6369d88f1a1b",
        "part": "whole"
       },
       "id": "3fb9ce83-fbb2-4995-832a-f8f400734ad3"
      }
     }
    },
    "1519a4fa-1588-4644-98de-9c43bf0aceb5": {
     "id": "1519a4fa-1588-4644-98de-9c43bf0aceb5",
     "prev": "8a712017-49b7-449f-8264-43a032ace902",
     "regions": {
      "29546121-ed11-44b7-8144-0c44e874098f": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "365590a4-6963-4812-a1cf-688f7b6bb9ff",
        "part": "whole"
       },
       "id": "29546121-ed11-44b7-8144-0c44e874098f"
      }
     }
    },
    "176eaccb-15dd-455d-bf07-504213e7aa01": {
     "id": "176eaccb-15dd-455d-bf07-504213e7aa01",
     "prev": "de6b30f4-2463-4901-92ed-aabad78e5e0f",
     "regions": {
      "1542aa9e-dc55-4b90-adef-bf5181872b42": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "5c242050-c1f7-4b3b-8103-2ea9d71a40dc",
        "part": "whole"
       },
       "id": "1542aa9e-dc55-4b90-adef-bf5181872b42"
      }
     }
    },
    "19091b36-b0e7-49b1-b501-ec05937e0da9": {
     "id": "19091b36-b0e7-49b1-b501-ec05937e0da9",
     "prev": "1983c02e-fb99-4c05-a728-e0c0ad7c06d8",
     "regions": {
      "6529a31c-8d45-425c-b1d7-d0ac6fca6a32": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "e766909d-9421-4aaf-9fb1-bc90d27e49e3",
        "part": "whole"
       },
       "id": "6529a31c-8d45-425c-b1d7-d0ac6fca6a32"
      }
     }
    },
    "1983c02e-fb99-4c05-a728-e0c0ad7c06d8": {
     "id": "1983c02e-fb99-4c05-a728-e0c0ad7c06d8",
     "prev": "176eaccb-15dd-455d-bf07-504213e7aa01",
     "regions": {
      "1c4e605d-7f22-4f30-b3fb-74b2937e7a4a": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "4d217204-e5c0-4568-bd30-12c2e41b681d",
        "part": "whole"
       },
       "id": "1c4e605d-7f22-4f30-b3fb-74b2937e7a4a"
      }
     }
    },
    "212b111f-4527-459c-8297-1db5580ee5c9": {
     "id": "212b111f-4527-459c-8297-1db5580ee5c9",
     "prev": "76898529-e49e-4663-8d02-8261dfe1d94b",
     "regions": {
      "2e4bd280-3cd6-47d0-9c81-17737b24053b": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "0c316996-9933-4b3d-82ec-259518dc8bc9",
        "part": "whole"
       },
       "id": "2e4bd280-3cd6-47d0-9c81-17737b24053b"
      }
     }
    },
    "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579": {
     "id": "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579",
     "prev": "e76e9a02-54c1-4ec9-80fb-c611ed398122",
     "regions": {
      "b5721d20-d6f8-4ddb-a5aa-eb16f0cc8893": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "313015a2-b5a9-4136-a8ea-5d011e47d840",
        "part": "whole"
       },
       "id": "b5721d20-d6f8-4ddb-a5aa-eb16f0cc8893"
      }
     }
    },
    "732f1952-ee54-46fb-8067-099512824296": {
     "id": "732f1952-ee54-46fb-8067-099512824296",
     "prev": "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a",
     "regions": {
      "f31d4597-08ad-4c46-ad52-4bd2d775c624": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "aadfec52-27ca-4541-8920-fa9253d51827",
        "part": "whole"
       },
       "id": "f31d4597-08ad-4c46-ad52-4bd2d775c624"
      }
     }
    },
    "76898529-e49e-4663-8d02-8261dfe1d94b": {
     "id": "76898529-e49e-4663-8d02-8261dfe1d94b",
     "prev": "19091b36-b0e7-49b1-b501-ec05937e0da9",
     "regions": {
      "ec1746fc-aec9-4a7c-8225-9e9ac8d45889": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "b3e539be-84e2-49ce-a183-31cfc5c7ce7c",
        "part": "whole"
       },
       "id": "ec1746fc-aec9-4a7c-8225-9e9ac8d45889"
      }
     }
    },
    "8a712017-49b7-449f-8264-43a032ace902": {
     "id": "8a712017-49b7-449f-8264-43a032ace902",
     "prev": "bed9e696-630e-4747-be1c-bc3737ba992f",
     "regions": {
      "1faab517-cd16-4c63-bb01-a67246749d7a": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "3f14ddf0-4145-4687-9c33-712c3c32520f",
        "part": "whole"
       },
       "id": "1faab517-cd16-4c63-bb01-a67246749d7a"
      }
     }
    },
    "90af992d-eb6d-4496-b2d2-6aa9a95b6a61": {
     "id": "90af992d-eb6d-4496-b2d2-6aa9a95b6a61",
     "prev": "732f1952-ee54-46fb-8067-099512824296",
     "regions": {
      "4f448bec-5be9-4553-88ae-e35ed7612f25": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "c445fbfb-b8ab-4e9a-8d13-12231a1c588f",
        "part": "whole"
       },
       "id": "4f448bec-5be9-4553-88ae-e35ed7612f25"
      }
     }
    },
    "bed9e696-630e-4747-be1c-bc3737ba992f": {
     "id": "bed9e696-630e-4747-be1c-bc3737ba992f",
     "prev": "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579",
     "regions": {
      "ac1513f0-404f-492b-8b42-0313e9a753b0": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "18dd2eee-8b6c-4a5e-9539-132d00a7c7e1",
        "part": "whole"
       },
       "id": "ac1513f0-404f-492b-8b42-0313e9a753b0"
      }
     }
    },
    "de6b30f4-2463-4901-92ed-aabad78e5e0f": {
     "id": "de6b30f4-2463-4901-92ed-aabad78e5e0f",
     "prev": "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0",
     "regions": {
      "55ec36e0-362f-4fd3-8060-7cee056039aa": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "c3cf461e-4c9e-4dec-99d2-07bfa79cbe23",
        "part": "whole"
       },
       "id": "55ec36e0-362f-4fd3-8060-7cee056039aa"
      }
     }
    },
    "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0": {
     "id": "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0",
     "prev": "1519a4fa-1588-4644-98de-9c43bf0aceb5",
     "regions": {
      "4c1e9714-9ba0-45fd-8a2f-ef80a5c85c2e": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "6534d4dc-125f-47e6-a022-cf1e0d277174",
        "part": "whole"
       },
       "id": "4c1e9714-9ba0-45fd-8a2f-ef80a5c85c2e"
      }
     }
    },
    "e76e9a02-54c1-4ec9-80fb-c611ed398122": {
     "id": "e76e9a02-54c1-4ec9-80fb-c611ed398122",
     "prev": "90af992d-eb6d-4496-b2d2-6aa9a95b6a61",
     "regions": {
      "9491b84d-193b-40ff-9321-d21eb1ba88d4": {
       "attrs": {
        "height": 0.8,
        "width": 0.8,
        "x": 0.1,
        "y": 0.1
       },
       "content": {
        "cell": "b64ec10e-fa9d-4f3f-907f-6799611ed6b1",
        "part": "whole"
       },
       "id": "9491b84d-193b-40ff-9321-d21eb1ba88d4"
      }
     }
    }
   },
   "themes": {}
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}