worksheet04_solutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# COMP90051 Workshop 4\n",
    "## Perceptron\n",
    "***\n",
    "In this worksheet, we'll implement the perceptron (a building block of neural networks) from scratch. \n",
    "Our key objectives are:\n",
    "\n",
    "* to review the steps involved in the perceptron training algorithm, \n",
    "* to assess how the perceptron behaves in two distinct scenarios (separable vs. non-separable data)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Synthetic data set\n",
    "We'll use the built-in `make_classification` function from `sklearn` to generate a synthetic binary classification data set.\n",
    "The main advantage of using synthetic data is that we have complete control over the distribution. \n",
    "This is useful for studying machine learning algorithms under specific conditions.\n",
    "In particular, we'll be varying the *degree of separability* between the two classes by adjusting the `class_sep` parameter below.\n",
    "To begin, we'll work with a data set that is almost linearly separable (with `class_sep = 2`).\n",
    "\n",
    "**Note:** In this worksheet we deviate from the standard `0`/`1` encoding for binary class labels used in `sklearn`. \n",
    "We use `-1` in place of `0` for the \"negative\" class.\n",
    "This matches the presentation in lectures and makes the gradient descent update simpler to implement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEMCAYAAADAqxFbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+QVeWZJ/Dv0+0FLmBoEBhjNyyMMVpRW4hMJi5ulUACJhFlTISJO+OmZg21UztrolkT1Cyg65Q4pDRhnS2XManUVEwCjkg0jsGfyWysjQmEXzHCJjWuoVuN+AOM0NpN8+wf5x64597z+9d7zrnfTxWF3u4+97330u9z3ud93+cVVQUREZGty3QDiIioWBgYiIjIgYGBiIgcGBiIiMiBgYGIiBwYGIiIyIGBgYiIHBgYiIjIgYGBiIgcGBiIiMjhFNMNiGPq1Kk6a9Ys080gIiqVHTt2vK6q04K+r5SBYdasWdi+fbvpZhARlYqIvBTm+5hKIiIiBwYGIiJyYGAgIiKHUs4xEBG5GRkZwcDAAN59913TTTFq3Lhx6OvrQ61Wi/XzDAxEGdi6cxDrt+3Hy4eGcEZPHTcuORvL5vaablblDQwM4NRTT8WsWbMgIqabY4Sq4o033sDAwABmz54d6xpMJRGlbOvOQdy0ZS8GDw1BAQweGsJNW/Zi685B002rvHfffRennXZaxwYFABARnHbaaYlGTQwMRClbv20/hkZGHY8NjYxi/bb9hlrUWTo5KNiSvgcMDEQpe/nQUKTHqTpUFRdffDEee+yxE4898MADuPTSS1N7jnvuuQcf+MAHICJ4/fXXU7tuM84xEKXsjJ46Bl2CwBk9dQOtIT9pzwWJCO69915cddVVWLBgAY4dO4abb74ZP/rRj1Jr8/z583HZZZfhkksuSe2arRgYiFJ245KzcdOWvY50Ur3WjRuXnG2wVdTKnguyPyd7LghAouBw3nnnYenSpbjzzjtx5MgRXHPNNTjzzDNTaTMAzJ07N7VreWFgIEqZ3alwVVKx+c0FJf2s1qxZgw9/+MMYM2aMa/meFStWYP/+9jmnG264Addcc02i504DAwNRBpbN7S10IOBy2mzngiZMmIAVK1Zg4sSJGDt2bNvXN23alPg5slSYwCAi3QC2AxhU1ctMt4eoqrJKoZRN1nNBXV1d6OpyX98TZcSwZMkS/P73v8e8efNw3333pdK2IIUJDAC+AOAFAO8z3RCiKssyhVImJueCoowYtm3blmFL3BViuaqI9AH4FIB8wiFRB+NyWsuyub2448rz0dtThwDo7anjjivPL3xw3LBhA/r6+jAwMID+/n5ce+21qT9HUUYMXwfwZQCnmm4IlQNz5PFxOe1JWc4FrV27NpPrXnfddbjuuusyubbN+IhBRC4D8Jqq7gj4vpUisl1Eth88eDCn1lERseREMjcuORv1WrfjMS6npWZFGDHMB3C5iHwSwDgA7xOR76jqXzR/k6puBLARAObNm6f5N5PSkvRunznyk+K8l1xOS0GMBwZVvQnATQAgIpcA+K+tQYGqI40VMcyRW5K8l0VfTktmGU8lUWdJo8CcVy687DnyrTsHMX/d05i96lHMX/d0YGqMxfooK4UKDKr6Y+5hqLY07varmCOPM29S5JFT1CBHxVKowEDVl8bdflrLDIvUecW5+y/qyImLA8qPgYFyldbd/rK5vXh21UK8uO5TeHbVwlhBoUidV5y7/6KOnDo5xZVF2e19+/bhoosuwtixY/G1r30tjWYGMj75TJ2lKCtiirayKc7egqK8l62KnOJqs2cz8NRtwOEBYFIfsGg10L889uWyKLs9ZcoUbNiwAVu3bo19jagYGCh3RVgRU7TOK255hqzeyyRLiv2CXKE2Ju7ZDDxyHTDSaOvhA9b/A4mCQ9plt6dPn47p06fj0UcfjX2NqBgYqCMVbfdvke7+ky4p9gpyC86ZVqzifU/ddjIo2EaGrMcTBAaAZbeJSqmIh+kUYSQFJE+zeQW5oqXvcHgg2uMRsOw2UQnFvUMvVCokI2mk2dyC3PWbdiW+bqom9VnpI7fHU5BW2W0TGBioY0W9Q0+aYnELKgBw6yPP462jIwCAnnoNay8/12iwySrNVrT0HRatds4xAECtbj2esaKPGLhclSikJMsw3ZbH3vjAbnzpgd0nggIAHBoawY0P7Da65j+rZbCFW17bvxxYugGYNAOAWH8v3ZB4fiFtr776Kvr6+nDXXXfh9ttvR19fH95+++1Mn5MjBqKQkqRY3ILKyHH3WpAjx9VoQcCsJsKLNMF+Qv/yzAJBWmW3Tz/9dAwMJJ/3iIKBgVLRCbn3JKmQqHl002v+s5oIL8oEO/ljYKDEOuUM4SQrmSbVazg0NBL4fbYuEWzdOeh4/7yCb/PjPeNrUAUOD41UNkBT9hgYKDGTyxDzHKkkWcl0ZPhY2+NdAKRLMOqSUhpVdQRXr+C7/aU38eCOwROPN89XVDVAU/YYGCgxU7uITYxU4qRC1m/bj5HR9s5/0vga1iw917EqqVlzcPUKvt977gBG1fvcqk48wEhVISKmm2GU+vybCIOrkigxU1U+y1KszStAHjo6gmVze7Fz9WJ4dWP2z3pdwy8oBD1/FY0bNw5vvPFG4o6xzFQVb7zxBsaNGxf7GhwxUGKmdhHHGalkmXryunaYSeug7/H6erdIYHAwXYY7T319fRgYGECnnws/btw49PXF36jHwECJmVqGGHWVUJapJ79rhwmcQd/j9fVPX9jrmGNoZV+jE1aNAUCtVsPs2bNNN6P0pIxDrnnz5qlbYSrqLK2dMXCys3xm38G2TnD+uqddA0lPvYZdaxYnaovXtXt76nh21cK2jnnBOdPa2gj4B9e4q5IAuL5PcQ43onITkR2qOi/w+xgYqMzcOtzWO2i7E7x+0y54/Wv/+oo5oTphL7NXPep57d6eelsAyLOjDgpa1DnCBgamkqjUWtNYbqt07Alpr9ST/fP2teKknPyubT9uX2dcrSvX5b15rBrrlFRVp+CqJCq11hpEXhOxLx8a8p0Mb+4k46x2unHJ2ah1BS+RHBoZdV2aCliBI4vzp7NeNVa0Y1IpOQYGKjW3TtzNGT11LJvbi8nja55ft/nd+c9f9zS+unUv5q972tGJL5vbi4njkg/A/TrWrTsH2543jKyL15Vl2TCFx8BAruJ2QnkLkw5p7gTXLD3Xt5PcunPQc08BYHXa3/nZ75xVUv9pN+bc+rjnSKBVT73W1oZWrR1rkrvyZXN7cceV56O3pw6BNbeQ5nxG0Y5JpeQ4x0BtslrWmUUe2m99/3HVtucJWlq7ftt+z0lkLyOjGroOUr3WjbWXn+tog9fzDR4awqxVj6K3p44j7x3znZcIem+zLF5XuHMWKDEGBmqTRe2jLA65WTa313N9v98dsV8nmfZdbq1bMGHMKa5F7ey/vVYN2fy+9vKhIeNFDIt4TColw1QStckiNZD2ITd2GiWtNImdOktz8XZvTx3rP3MBdq1ZjLtXzAFgHW/ZmppbcM403/SVnzN66sZz/Fmnqih/HDFQmyxSA2kfctM8gkmaJnHbKJdU8x4Bvzt6ANj0iwOxApLAulsvwlnKPGehWjhioDZZrGJJsmQy68nNsCubolhwzjTf6w+NjGLtw8/jhs27XCuvhqHAiVpMbpjjp7gYGKhNFqmBMMHGayVU1h1fFnfWz+w7WcTNs7rq0Ag8TvcMpbfx+gt3ljKVnvFUkojMAPCPAP4I1k3QRlX9htlWUdqpgaDVQEmL0CXht2s5ruZgEPf6duXUyeNreOfdY44zoptffyHPUqZSM14rSUTeD+D9qvpLETkVwA4Ay1T1114/w1pJ1RO1CF3a5bKD5hhqXQIIHGmfeq0bY0/pcl2q6jfHEEZrYT+WnKA0lKZWkqq+AuCVxn//QUReANALwDMwkBlZdk5B8whxRjBh22s/dstDe3FkuL3zrte6cMeV/QDa78oB94J4zaMZtzv6o8PHPDfE1brkxF6H5mswEFBejAeGZiIyC8BcAM+ZbQm1ynqtfNoroaK21z4+88hwexumTBjbtvegVVAAau7Yt+4cxK2PPO96nZ56DWsvP5dBgIwynkqyichEAD8B8LequsXl6ysBrASAmTNnXvjSSy/l3MLOlnXpZrd0S61LMHHcKTh0tH1zmNvPN3fOR947FpjiaRVUOjvpCGnrzkGsffh513Y1BwSmjSgrpUklAYCI1AA8COB+t6AAAKq6EcBGwJpjyLF5hOyXjLamWybVazjSlG7xu+N3Gx148WtvUOnsGx/Y7fr8YQTNM0wYewqWze3FV7fuxf0/+92JABVmZMZAQmkzvlxVRATANwG8oKp3mW4Puctjrfyyub14dtVCvLjuU5gw9pS29f1DI6O49ZHn25a0RtmH4NfeoB3II8cV12/aFauwYFAb7dIWzUHB5reLmSWvKQvGAwOA+QD+EsBCEdnV+PNJ040ip6hr5ZNWZ/W6s3/r6IijE/zipl2RloL6tffBHYOBO5AV/qWxvQSNrOzSFl7P7/XzpsthUDUZDwyq+lNVFVXtV9U5jT//bLpd5BRl01sad7FZ7tp1C1hxdj9H6YD9Xo8dYIPSXG5Y8pqyUIg5BiqHsEsm06jO6rapLQ1eK5XidqRhf87r9UweX8OapeeeWBXlNvqxayK5ibuai/MS5Mf4iIGqJ427WLcRSk/d/fQ1W29AZygCz4AVd4QS9udaX8/k8TX01Gs4dHQE67ftx9adg67pOgD4t2dO8RyZHR0+1vZ40K5wzktQEAYGSl1aE9XNk9HPrlrYtumrWbeIZ8cKWOcieK3Mts+Dbv3ZWpd4HgUK+N/Ju7Ffz90r5uDdkeM4NDTi6JgB4NMX9rZNgP/yd4ddj/m8acvetk1yPfVaYF0rzktQEAYGSl1WRd2Wze3F/DOnuH7ts386w3OeoFsE6z9zgeeIwj4PunWEsv6qC7Bz9WJ8fcWcttcjAP79R2c6OuCwE+5+HfMz+w6GWpXk9VrtZa9+OC9BQTjHQKnLsqjb/Z+/CF/duhffe+4ARlXRLYLP/ukM3L7sfMxe9ajrzxxXPfHcfuUrWncnr9+2H9dv2oUzeur49IW9eGbfQc/XE2WndZyOufVrSTp3HsVJQRgYKBNxa/s0T4r2jK9BFW3HYt6+7Hzcvuz8tp8N6vDCBiy3Tv7BHYO+KZooE+5B7QzTaSfp3HkUJwVhYOgAZVmB0tohN+fPw+wA9urwFpwzDfPXPR369cdZVRXlDj6oYw7TaSfp3Fmmm4IwMFSc6YPiowjaSxDUObt1eAvOmYYHdwxGev1x0jRR7uDDdMxhivKF+T4vrNZKfgpTRC8KnscQXtbF79LkV8SumQChO8I4rz/Oz7jVQqrXuhOffGdfm3f3lIawRfS4KqniyrQCJezkZ5S193Fef5xVVVkchwpwzwGZwcBQcWU6KN5vH4KbMGvv47z+1k7+cxN/jh0Tv4hlPzgXuPs8YM9mz59r3neRxl19R+052LPZen/X9vi+z5Q9BoaKK9NB8V67g/0qngaNfOK+/hOd/NVHsFb+F8YPvQJAgcMHgEeuS6XTCrPvoUwjvkT2bLbe18MHkPb7TNFx8rniyrYCxWtS1CvvHzTySfz6n7oNGGl53pEhvLrlZvxsdH7s9zHsooCO2XPg8T7jqduA/uVm2tTBGBg6QBVWoCRdnhn39evhAdcRy3R93ffgoKBAFHZJbMfsOTg8EO1xyhQDA5VC3Dv/pCt6fo+pOB0H2x5/WU/D0MgovrTZeapb2JFA2BRR2UZ8sU3qa6SRXB6n3DEwUGlEvfNPYw/HHcNX4Y7afRgvwyceO6pj8HfHrPTGqKrjmmFHAlH3PVQuELRatNqaU2hOJ9Xq1uOUO04+U2WlsaJn+/s+jlUj12Lg+FQcV8HA8alYNXItHj5+ses1w44EyrQoIBf9y4GlG4BJMwCI9ffSDZxfMIQjBqqsNFb0WDn+YTw8fLHv99nXDDsS6JgU0Z7N1gTy4QErLbRotXdn37+cgaAgGBiostJY0dPcgfudLW1fM8pkceVTRPYSVDs9ZC9BBRgACo6pJKqstNI19p4Gv/0UzaW7s9gBXUp+S1Cp0DhioMpKO13jNQKZPL7muGblRwJhcQlqaTEwUKkFLUdNs5P2ShOtWep95GhHy3IJapS5C4qMgYFKK++S4nlPGJe+qmpWS1A5d5E5lt2m0ipTSfGosizjnass7uzvPs9jJDIDuP5Xya5dcWHLbnPEQKVV5QJzcU6RK6QslqBy7iJzXJVEpVWmkuJRVTnoJeY1R8HyGalhYKDSqvLu4SoHvcQWrbbmKpqxfEaqGBiotKq8Z6DKQS8xls/IXCEmn0XkUgDfANAN4D5VXef3/Zx8pk5Q+lVJVDilmXwWkW4Afw/g4wAGAPxCRB5W1V+bbRmRWdwoR6YUIZX0EQC/VdV/VdVhAN8HcIXhNhERdawiBIZeAM2LkgcajxERkQFFCAyhiMhKEdkuItsPHmw/UYuIiNJRhMAwCGBG0//3NR5zUNWNqjpPVedNmzYtt8YREXUa45PPAH4B4CwRmQ0rIPw5gKvNNomIHJpLW9QnW48NvcUCdhVlPDCo6jER+RsA22AtV/2Wqj5vuFlEZGstWjf05smvsYBdJRUhlQRV/WdV/aCqnqmqf2u6PUTUxO3AnWY8fCd7ezZbxQPX9lh/79mc6dMZHzEQUcGFKU7HAnbZMVBmvBAjBiIqsDDF6VjALjsGjkhlYCACshmq5zz8z4xb0bpmLGCXLQNlxhkYiOyh+uEDAPTkUD1JR57FNdMSNWC1Fq2rT7H+sIBdPgyUGS9EEb2oWESPUpXFiWBFPWWsNV8NWHf87NyLK8XPLGwRPY4YqDM13zW7deBAsqF6UU8ZM5CvpoQMlBmPvSpJRL6iqnem2RiiXLjdgbmJOlRv3gQmXYCOtn+P6UnaogYs8pfFEak+QgcGEWlORAqAOQAYGKh8gtblA9EnVFuDjVtQKMIk7aQ+jxQXVxXRSVFSSW+r6vLGn6sAPJlVo4gy5Xt3HHOo7hVspDv+NbNg6ljMqqzQ6hBRUkmtO5JvSbMhRLnxvGtOMDHsFWz0OLD2ULxrZsEOTHbKK49aRwY2aJ143jxfZ4UEBgYR+QaAL6rqi82Pq+qbHj9CVCytHcRZi4Hd321f5ZHkrjmtFE3czizKz+Wcr/ad8M6qHaaCUUWESSX9AcDDIjIBAERkiYg8m22ziAKETU247SfY/V3ggqvTW+WxZzMwfKT9ca9g49X2qHsfTlxnErBlpfk9E16vy8SEN1dfJRI4YlDVr4rI1QB+LCLDAN4BsCrzlhF5iXI36NVB/ObxdPYTeK1wqk8BPnFne3v82h7lzrrteTXcz2XF73WlOeEddmTE1VeJBI4YRGQRgM8DOAJgKoDrVPV/Z90wIk9R7gaz7iD8Vji5deYP/Sfvtnu29QDwwxvCP++Jn8uxE/T7TNKa8I4yojKwW7hKwqSSbgHw31T1EgCfAbBJRBZm2ioiP1E6+6w7CK+2DL3p7LDsTs1tGat9Hb82bf+mMziE6fTz7AT9PpOgDVph04JRbghMrb6qiMDAoKoLVfWnjf/eC+ATAG7PumFEnqJ09ll3EH6db3OHFXSHb6dF/IrV7fh2uOcF8u8Egz6T/uVW6m7tIevv5qAQdhQQ5YbAwG7hKolcEkNVXwGwKIO2EIUTpbPPuoPw63ybOyy/O3y77XZbvTSPNlyDiFh/megE4wbgKKOAqKM/r2BEgWLVSlLVgOQmUYaidvZZdhD9yxuVRl00d1henZd0O9vev7yxKc6DfSft9h5cuRFYe9hMJxg3AEcZBTA9lBtWV6XOE7SyJepegjDVL6NUyPzhDdacgpuwVTWTbu7Ka3NY1Cq03LSWSNjqqgwMVcdfJKegDjpuieMw73OUz8IvOEi3taPa73mSlGnOszQ3y4DnioGB+EvnJugOtUjnKKztQdv+hFZun2fS15D3e8Cbl9zwPAbi7k83QTltv70EeRd/C7Pc1O3zTLp3I+/NYZwkLhwGhirj7s92QStb/DrjvEtNBC1ftbV+nkn3bnBzWMdjYKgy/oK3C1rZEtQZ5znial3p47VaqfXzTLp6h6t/Oh4DQ5XxF7xd0LJKx9c9tN6hZ3nWQHOa5c/uDfd5Jt27wc1hHY+Tz1VX5Ym9rF9bmEnYvCf4q/x5Uua4KomKL0knl1WH3Nym+mRg+B1gdNj7OYq0iokoQNjAEOUENwrCuzlvQYflRD1IJYvDX1qDzdCbQFfN2tk89Jb7Z5r3BD//jVEOGBjSwhOj2p3oxA7AquPTGJ0ePgBs/xYSnSGQRYfsFmyOjwBjJgBfedH9Z9I8ayAI/41RToxOPovIehHZJyJ7ROQhEekx2Z5EuGfAyVE1E2jfqOWRwgzbsae54sqePHbr4IPalOcEP/+NUU5Mr0p6AsB5qtoP4P8CuMlwe+LjngGnMAfJuAnbsWdy+IuH+mTvr4VdwdN8DOetU6y/o65g4r8xyonRVJKqPt70vz+DdRBQNrLOzeaZUiiDUJ1VU3oJsPL5w0esZZ9hDrQHktcnihvAWtsSpcieXT47aioo739jnM/oWKZHDM3+CsBjmVw56iHrcXDPgFOYg2Tm/dXJO+36FEDEmvAN+xn1L7fe30l9Vuf11G3up6b5fe5hAljraWxR+QWfKKmgPP+N5fE7Q4WVeWAQkSdF5Fcuf65o+p5bABwDcL/PdVaKyHYR2X7w4MFojcgjN8tNQU5hDpK57K6Tm7fGTHAuCwWCP6OgzivM5x72bru1U2ze1HbnbOuP1wa3oOATNhWU578xzmd0tMxTSar6Mb+vi8jnAFwGYJH6bKpQ1Y0ANgLWPoZIjcgrNxuUUugkkVM9MSZ+g5ashvncF60GtqxEYBXT5uu6LWs9cW2X9JBXCsgWJRUU999Y1LQQ5zM6mulVSZcC+DKAy1X1aGZPxJpBZkRK9Xjw+oz2bA4OJmE+9/7lCAwKrdcNmpdovbP2q7+UR7oxTlqIvzMdzfQcwz0ATgXwhIjsEpF7M3kW5v+Ti1MPKE6qp5nXZ2Rf14vdeYX93P3qIrldN8xdc/P3tNZfsovh5ZVujJMW4u9MRzO9KukDuTxR2LQGuYu7sSpuqgewOs2zFlvfu2Wl8zPzCyjNnVfYz33R6vbyGn7XDUoN2d/TLK8UkNv3x0kL8Xemo7FWEgWLWw/I8wQysSac/a7r1lnbdYr85gSu/Id0OuCzFgO/edy9U3Sr09QsrSJ6UetBeX3/KXXnPIiN9Zw6DmsldZKs15vHnYgMWnfv1fkvWu0/2vC87oz4rzvKHX3r3bS9Ac6rnlJcUetBeX3/KXXrfXV7n4lcMDCUXR71c+JurPLr+Jvb5xbUtqx0v+bhAeDKjf7XzUMeK9CiBmSvx4fest4zpoUoJAaGssuiymiroA7eS5g8tVcH6xeMOiX/HTUgB71nVXt/KDMMDGWXx3rzJB1x3A4pzGgjytkNZQwiUQNy3ABO1IKBoezyqp+T5I4zTsec1qigzKWqvd4DoDFx3/K+dMpIijLHVUlll/fRklGZbl/VTljL+v0s6+iKQgm7Ksn0BjdKKkr9nCwPrfdiuuZOEUo7pPm+Z/l+snAeNTCVVAVh0jymUiqmO+b6ZPc1/H5nLNjSuHtO+33P8v3MYyEDlQJHDJ3C1J17WWvuuN09b1kZ/YCdtN/3LN9P00GcCoOBoVPk+UvfnDoZPgJ0j3F+Pc+VMkNvRXvc5lp2o+nM6rAplrTf9yxrGJU1iFPqGBg6RV6/9K132kNvAqrWQTwmzqmI+7qDOu6wd/1ez1OfHG/eIcszGVg4jxo4x9Ap8lrj7nanfXzEOojnKy+m+1xhxH3dYQrlhbnrd3v+rhow/M7JuY+o8w5ZbVbjcldqYGDoFGn80oeZjC1anjru6w5TcTXMaMvt+YePtE+I5z3J6/VZcoc0gYGhsyTdpBZmdU3eB9aHEed1Ozr0A7COJW3a8xNltNX6/Gt73L8vr+BZ5k1/lAvOMVA4YVfXVClP3b+8cSb1YasIXVp5fdOTvKb3llDhMTBQOGFTRHkeWJ+noGNKozAdPIuW7qPCYSqJwomSIoqTuvHKeae1yaxIG9VMT/IWMd1HhcJaSRROljV6vK59wdXA7u8me8602l2lmkum61eRMayVROnKMkXklfPe8e3kufC08ulVSr9UNd1HqWEqicLLaimjV+eqo9G+P8r3Ru3QvdIv0mXdgZetU+WyVPLBEQOZ51XQTrrdH4+SC4+6AsirEqrbhDFgBS9WIKWKYWAgs/ZstnYBt+qqARd+LvnqnSgrgPzKTtvpF7dgxaWeVDEMDGTWU7cBo8Ptj489FbjsruS58Cj59KD5iP7lgB53f54yzjUQeeAcA5nl1aHa1U+j5sL9Sj3EbUvz41zqSR2AIwYyK81dwElPIAvTFte5BgHOWhy9vUQFxcBA2QlzpGWau4CTLk0N05b+5db+CkjTN6m134IT0FQRDAxlZuIM57DC3r2nuaY+6dLUsG35zeNwFNQDOAFNlcI5hrIqeoXMKOcHp7WmPo38f5i2VGmzG5GLQowYRORLIqIiMtV0W0qj6BUyTXSeeRWnM10dlShjxgODiMwAsBjA70y3pVSKftdqovPMq9RDWgGoyKlA6mhFSCXdDeDLAH5guiGlEidtkkaV0bDyOkq0VR6lHtI6Da/IqUDqaEYDg4hcAWBQVXeLSOD3U5OoHa9fRwSkHzBMl5bOWtIAFGUOhihnmQcGEXkSwOkuX7oFwM2w0khhrrMSwEoAmDlzZmrtK62oHa9XR/TYV4BjQ9ncubJQm7eipwKpoxk7j0FEzgfwFICjjYf6ALwM4COq+qrfz/I8hhjW9qBtiaWfMp4zUCZVOt+BSqPw5zGo6l5Vna6qs1R1FoABAB8OCgoUU9RJ37B3rlWaQM3ztZg+3pPIRxEmnylLJyacD8Dards0aqjVgVPqwNCb7T8XJpBUaQI179dS9TkYKjUe7Vllbkc42sFh0oyTd6dxj3msUjqkSq+FyEPYVBJHDFXmNuFsB4XWzi7OnWuVJlCr9FqIEmJgqLKwnV3c1UP/IXDqAAAGcElEQVRVKkFdpddClJDxnc+Uoax3H1dpArVKr4UoIQaGKsu6s8urBEUeqvRaiBLi5HPV5VkGg4gKjZPPZCnz7mMGNSIjGBiomKq0R4KoZDjHQNlIuou46OdNEFUYRwyUvjTu9rmvgMgYjhgofWnc7fOUNCJjGBgofWnc7XNfAZExDAyUvjTu9rmvgMgYzjFQ+tI61rPMS22JSowjBkof7/aJSo0jBsoG7/aJSosjBiquKp0OR1QiHDFQMXHnM5ExHDFQMXHnM5ExDAxUTNz5TGQMAwMVE3c+ExnDwEDFxJ3PRMYwMFAxcS8EkTFclUTFxb0QREZwxEBERA4MDERE5MDAQEREDgwMRETkwMBAREQOxgODiPwXEdknIs+LyN+Zbg8RUaczulxVRBYAuALABar6nohMN9keIiIyP2L4awDrVPU9AFDV1wy3JzmWiiaikjMdGD4I4N+JyHMi8hMR+RPD7UnGLhV9+AAAPVkqmsGBiEok81SSiDwJ4HSXL93SeP4pAD4K4E8AbBaRP1ZVdbnOSgArAWDmzJnZNTgJv1LR3MFLRCWReWBQ1Y95fU1E/hrAlkYg+LmIHAcwFcBBl+tsBLARAObNm9cWOAqBpaKJqAJMp5K2AlgAACLyQQBjALxutEVJsFQ0EVWA6cDwLQB/LCK/AvB9AP/BLY1UGiwVTUQVYHS5qqoOA/gLk21IlT2P8NRtVvpoUp8VFDi/QEQlwrLbaWOpaCIqOdOpJCIiKhgGBiIicmBgICIiBwYGIiJyYGAgIiIHKeO2ARE5COClpoemoswb49rx9RRf1V4TX0+xpfV6/o2qTgv6plIGhlYisl1V55luR1r4eoqvaq+Jr6fY8n49TCUREZEDAwMRETlUJTBsNN2AlPH1FF/VXhNfT7Hl+noqMcdARETpqcqIgYiIUlKZwCAiV4nI8yJyXERKuxpBRC4Vkf0i8lsRWWW6PUmIyLdE5LVGWfXSE5EZIvKMiPy68W/tC6bblISIjBORn4vI7sbrudV0m9IgIt0islNEfmi6LWkQkf8nIntFZJeIbM/jOSsTGAD8CsCVAP7FdEPiEpFuAH8P4BMAPgTgsyLyIbOtSuTbAC413YgUHQPwJVX9EKzjaP9zyT+f9wAsVNULAMwBcKmIfNRwm9LwBQAvmG5Eyhao6py8lqxWJjCo6guqut90OxL6CIDfquq/Ns6q+D6AKwy3KTZV/RcAb5puR1pU9RVV/WXjv/8Aq/PpNduq+NTyTuN/a40/pZ50FJE+AJ8CcJ/ptpRZZQJDRfQCOND0/wMoccdTZSIyC8BcAM+ZbUkyjbTLLgCvAXhCVUv9egB8HcCXARw33ZAUKYDHRWSHiKzM4wlLdVCPiDwJ4HSXL92iqj/Iuz3UmURkIoAHAXxRVd823Z4kVHUUwBwR6QHwkIicp6qlnBMSkcsAvKaqO0TkEtPtSdHFqjooItMBPCEi+xqj8cyUKjCo6sdMtyFjgwBmNP1/X+MxKggRqcEKCver6hbT7UmLqh4SkWdgzQmVMjAAmA/gchH5JIBxAN4nIt9R1VIfH6yqg42/XxORh2ClnDMNDEwlFcsvAJwlIrNFZAyAPwfwsOE2UYOICIBvAnhBVe8y3Z6kRGRaY6QAEakD+DiAfWZbFZ+q3qSqfao6C9bvztNlDwoiMkFETrX/G8Bi5BC4KxMYROTPRGQAwEUAHhWRbabbFJWqHgPwNwC2wZrY3Kyqz5ttVXwi8j0A/wfA2SIyICL/0XSbEpoP4C8BLGwsHdzVuDstq/cDeEZE9sC6KXlCVSuxxLNC/gjAT0VkN4CfA3hUVX+U9ZNy5zMRETlUZsRARETpYGAgIiIHBgYiInJgYCAiIgcGBiIicmBgICIiBwYGIiJyYGAgSqBxPsPHG/99u4j8D9NtIkqqVLWSiApoDYDbGgXO5gK43HB7iBLjzmeihETkJwAmArhEVf/QqGnzPwEMA/ixqt5vtIFEETGVRJSAiJwPq+bQcOPwHsA6SfCfVPXz4AiCSoiBgSgmEXk/gPthnbL3jojYx5j24eSBS6Mm2kaUBAMDUQwiMh7AFlhnQL8A4L/Dmm8ArJP3+hr/zd8xKh3OMRClrDHHcA+AdwH8lHMMVDYMDERE5MBhLhEROTAwEBGRAwMDERE5MDAQEZEDAwMRETkwMBARkQMDAxEROTAwEBGRAwMDERE5/H+ngInt16Y2CQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "class_sep = 2 \n",
    "#class_sep = 0.5 # uncomment this later\n",
    "\n",
    "from sklearn.datasets import make_classification\n",
    "X, Y = make_classification(n_samples=200, n_features=2, n_informative=2, \n",
    "                           n_redundant=0, n_clusters_per_class=1, flip_y=0,\n",
    "                           class_sep=class_sep, random_state=1)\n",
    "Y[Y==0] = -1 # encode \"negative\" class using -1 rather than 0\n",
    "plt.plot(X[Y==-1,0], X[Y==-1,1], \"o\", label=\"Y = -1\")\n",
    "plt.plot(X[Y==1,0], X[Y==1,1], \"o\", label=\"Y = 1\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"$x_0$\")\n",
    "plt.ylabel(\"$x_1$\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Is the perceptron a suitable classifier for this data set?\n",
    "\n",
    "**Answer:** *Strictly speaking, the perceptron is only guaranteed to converge to a \"reasonable\" solution if the data is linearly seperable.\n",
    "This data is not quite linearly seperable, so one could argue that the perceptron is unsuitable.*\n",
    "  \n",
    "In preparation for training and evaluating a perceptron on this data, we randomly partition the data into train/test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set has 134 instances. Test set has 66 instances.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33, random_state=90051)\n",
    "print(\"Training set has {} instances. Test set has {} instances.\".format(X_train.shape[0], X_test.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Definition of the perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall from lectures that a perceptron is a binary classifier which maps an input vector $\\mathbf{x}$ to a binary ouput given by\n",
    "$$\n",
    "\\begin{align*}\n",
    "f(\\mathbf{x}; \\mathbf{w}, b) &= \\begin{cases}\n",
    "    1 & \\mathrm{if} \\ s(\\mathbf{x}; \\mathbf{w}, b) \\geq 0, \\\\\n",
    "    -1 & \\mathrm{otherwise},\n",
    "\\end{cases}\n",
    "\\end{align*}\n",
    "$$\n",
    "where $s(\\mathbf{x}; \\mathbf{w}, b) = \\mathbf{w} \\cdot \\mathbf{x} + b$. \n",
    "Here $\\mathbf{w}$ is a vector of weights (one for each feature) and $b$ is the bias term.\n",
    "\n",
    "Let start by implementing the weighted sum function $s(\\mathbf{x}; \\mathbf{w}, b)$ below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def weighted_sum(X, w, b):\n",
    "    \"\"\"\n",
    "    Returns an array containing the weighted sum s(x) for each instance x in the feature matrix\n",
    "    \n",
    "    Arguments:\n",
    "    X : numpy array, shape: (n_instances, n_features)\n",
    "        feature matrix\n",
    "    w : numpy array, shape: (n_features,)\n",
    "        weights vector\n",
    "    b : float\n",
    "        bias term\n",
    "    \"\"\"\n",
    "    return np.dot(X, w) + b # fill in\n",
    "\n",
    "def predict(X, w, b):\n",
    "    \"\"\"\n",
    "    Returns an array containing the predicted binary labels (-1/1) for each instance in the feature matrix\n",
    "    \n",
    "    Arguments:\n",
    "    X : numpy array, shape: (n_instances, n_features)\n",
    "        feature matrix\n",
    "    w : numpy array, shape: (n_features,)\n",
    "        weights vector\n",
    "    b : float\n",
    "        bias term\n",
    "    \"\"\"\n",
    "    return np.where(weighted_sum(X, w, b) >= 0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Perceptron training algorithm\n",
    "We're now going to implement the perceptron training algorithm presented in lectures.\n",
    "The algorithm is essentially an application of *sequential gradient descent* (a.k.a. *online stochastic gradient descent*) to minimise the following empirical loss:\n",
    "$$\n",
    "L[\\mathbf{w}, b] = \\frac{1}{n} \\sum_{i = 1}^{n} \\max(0, -y \\cdot s(\\mathbf{x}; \\mathbf{w}, b))\n",
    "$$\n",
    "It's *sequential* in that the weights/bias are updated for each training instance—one at a time.\n",
    "After iterating through all of the training instances, we say that we've completed an *epoch*.\n",
    "Typically, multiple epochs are required to get close to the optimal solution.\n",
    "\n",
    "Let's now write a function called `train` which implements sequential gradient descent.\n",
    "For your reference, the function should implement the following pseudocode:\n",
    "\n",
    "> repeat $n_\\mathrm{epochs}$ times\n",
    "\n",
    "> >   for each $(\\mathbf{x}, y)$ pair in the training set\n",
    "\n",
    "> > > if the model prediction $\\hat{y} = f(\\mathbf{x})$ and $y$ differ, make a weight update\n",
    "\n",
    "> return $\\mathbf{w}$ and $b$\n",
    "\n",
    "**Note:** the weight update in the inner-most loop is given by $\\mathbf{w} \\gets \\mathbf{w} + \\eta y  \\mathbf{x}$ and $b \\gets b + \\eta y$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train(X, Y, n_epochs, w, b, eta=0.1):\n",
    "    \"\"\"\n",
    "    Returns updated weight vector w and bias term b\n",
    "    \n",
    "    Arguments:\n",
    "    X : numpy array, shape: (n_instances, n_features)\n",
    "        feature matrix\n",
    "    Y : numpy array, shape: (n_instances,)\n",
    "        target class labels relative to X\n",
    "    n_epochs : int\n",
    "        number of epochs (full sweeps through X)\n",
    "    w : numpy array, shape: (n_features,)\n",
    "        initial guess for weights vector\n",
    "    b : float\n",
    "        initial guess for bias term\n",
    "    eta : positive float\n",
    "        step size (default: 0.1)\n",
    "    \"\"\"\n",
    "    for t in range(n_epochs):\n",
    "        for i in range(X.shape[0]):\n",
    "            yhat = predict(X[i,:], w, b) # fill in\n",
    "            if yhat != Y[i]:\n",
    "                w += eta * Y[i] * X[i,:] # fill in\n",
    "                b += eta * Y[i] # fill in\n",
    "    return w, b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test your implementation by running it for 5 epochs.\n",
    "You should get the following result for the weights and bias term:\n",
    "`w = [ 0.26746342 -0.96011853]; b = -0.2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w = [ 0.26746342 -0.96011853]; b = -0.2\n"
     ]
    }
   ],
   "source": [
    "# Initialise weights and bias to zero\n",
    "w = np.zeros(X.shape[1]); b = 0.0\n",
    "\n",
    "w, b = train(X_train, Y_train, 5, w, b)\n",
    "print(\"w = {}; b = {}\".format(w, b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've trained the perceptron, let's see how it performs.\n",
    "\n",
    "Below we plot the data (training and test sets) along with the decision boundary (which is defined by $\\{\\mathbf{x}: s(\\mathbf{x}; \\mathbf{w}, b) = 0$\\}).\n",
    "\n",
    "**Note:** It's not necessary to understand how the `plot_results` function works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEMCAYAAADAqxFbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOW5B/DfM8OErCSsVhKQiBYpARMEVAIpi2WRRWorVqSWq1eKVy8KFQq4ofe2lxaveBEpVbDe1uoVFLCCCkWIQePGJqIEkaWQQCWgCWQj23v/ODmTOTPnzJxz5pw5M5Pn+/nkE2Yyy5sB3ue87/u8z0tCCDDGGGMyl9MNYIwxFl04MDDGGFPgwMAYY0yBAwNjjDEFDgyMMcYUODAwxhhT4MDAGGNMgQMDY4wxBQ4MjDHGFDgwMMYYU2jndAPM6NKli+jVq5fTzWCMsZiye/fus0KIrqEeF5OBoVevXti1a5fTzWCMsZhCRP/Q8zieSmKMMabAgYExxpgCBwbGGGMKMbnGwBiLXQ0NDSgtLUVdXZ3TTYlbiYmJyMrKgsfjMfV8DgyMsYgqLS1FWloaevXqBSJyujlxRwiBc+fOobS0FNnZ2aZeg6eSGGMRVVdXh86dO3NQsAkRoXPnzmGNyDgwMMYijoOCvcL9fDkwMMbaDCEEhg0bhrffftt737p16zBu3DjDr3XnnXeiW7duyMnJ0fX4wsJCFBcXG36fXbt2Yfbs2YafFw4ODIyxqLZxbxnyl2xH9oLNyF+yHRv3lpl+LSLCqlWrMHfuXNTV1aGqqgqLFi3Cs88+a/i1ZsyYgXfeeUf344MFhsbGRs3nDRo0CMuXLzfcvnDw4jNjLGpt3FuGhes/R21DEwCgrKIWC9d/DgCYkpdp6jVzcnIwadIk/O53v0N1dTXuuOMO9O7d2/DrFBQU4Pjx47oee/z4caxatQputxsvvfQSnnnmGaxZswaJiYnYu3cv8vPz8bOf/Qz3338/6urqkJSUhD/96U/o06cPCgsL8eSTT2LTpk1YvHgxTpw4gaNHj+LEiRN44IEHbBlNcGBgjEWtpVsOeYOCrLahCUu3HDIdGADgsccew8CBA5GQkOAtrzN8+HBcuHAh4LFPPvkkbrjhBtPvBUhlfGbNmoXU1FQ8+OCDAIA1a9agtLQUxcXFcLvdOH/+PHbu3Il27dph27ZtWLRoEV5//fWA1yopKcGOHTtw4cIF9OnTB/fcc4/ptFQtHBgYY1HrVEWtofv1SklJwa233orU1FS0b98eALBz586wXtOMW265BW63GwBQWVmJX/ziFzh8+DCICA0NDarPmTBhAtq3b4/27dujW7du+Oabb5CVlWVpuzgwMGaxjXvLsHTLIZyqqEX3jCTMG9snrKvbtqx7RhLKVIJA94yksF/b5XLB5WpdZg02YujTpw8mTZoEAJg1axZmzZoV9vsDUoCSPfLIIxg5ciQ2bNiA48ePY8SIEarPkQMZALjd7qDrE2ZxYGDMQnbMibdl88b2UXyeAJDkcWPe2D6Wv1eoEcO+fft0vc6KFSsAAPfdd5/i/rS0NJw/f17zeZWVlcjMlP6NvPjii7reyy6clcSYhYLNiTPjpuRl4r9u7o/MjCQQgMyMJPzXzf2jIsjedtttuP7663Ho0CFkZWVhzZo1AKQ1gM6dOwc8ftKkSdiwYQNyc3NVg9D8+fOxcOFC5OXl2TIKMIKEEI42wIxBgwYJPo+BRaPsBZuh9j+KABxbMiHSzYlKBw8eRN++fZ1uhm0mTpyI9evXIyEhwdF2qH3ORLRbCDEo1HN5xMCYhbTmvq2YE2exYdOmTY4HhXBxYGDMQvPG9kGSx624z645ccbswoGBMQtFw5z4qveOoPjIWcV9xUfOYtV7RyLWBhbboiYriYjcAHYBKBNCTHS6PYyZNSUv09HF0QFZ6bjv5b1YMS0PQ3t3QfGRs97bjOkRNYEBwP0ADgLo4HRDGItlQ3t3wYppebjv5b2Yfm1PvPTxCW+QYEyPqJhKIqIsABMArHa6LYzFg6G9u2D6tT2xfPvXmH5tTw4KzJCoCAwAngYwH0Cz0w1hsc3KSpyxrPjIWbz08QnMHnUFXvr4RMCaQ1sVTtntdevWoV+/fnC5XNCTLl9RUYGVK1eaaueNN96IiooKU8+1guOBgYgmAjgjhNgd4nEziWgXEe0qLy+PUOuYk4x28vKu47KKWgi07jpua8HBd01h7pg+3mmlmA0O+9cCy3KAxRnS9/1rTb9UOGW3c3JysH79ehQUFOh6r2CBIdQGtrfeegsZGRm63scO0bDGkA9gMhHdCCARQAciekkIMd33QUKI5wA8B0gb3CLfTBZJZkpL2FWJM1rorcG0v7RSsaYgrznsL62MvSml/WuBN2cDDS31kipPSrcBYMBUUy9ptuy20U15CxYswJEjR5Cbm4sf/ehHmDBhAh555BF07NgRJSUl+OqrrzBlyhScPHkSdXV1uP/++zFz5kwAUjXWXbt2oaqqCuPHj8ewYcNQXFyMzMxMvPHGG0hKsndfjOOBQQixEMBCACCiEQAe9A8KrO0x08nbVYkzGhgJlLN+GNjJDe3dJfaCAgC8+0RrUJA11Er3mwwMQGTKbi9ZsgQHDhzw1lgqLCzEnj17cODAAWRnZwMAXnjhBXTq1Am1tbUYPHgwfvKTnwSU0zh8+DBeeeUVPP/885g6dSpef/11TJ9ubxfpeGBgTI2ZTt7OSpx20TsKcHo05FjF2MpSY/fr5FTZ7SFDhniDAgAsX74cGzZsAACcPHkShw8fDggM2dnZyM3NBQBcc801ug8HCkdUBQYhRCGAQoebwaKAmU4+3Eqcke78jIwCnBwNOVoxNj1Lmj5Suz9MRspuh3tQj8y3zHZhYSG2bduGDz/8EMnJyRgxYgTq6uoCnuNfZru21v6/c8cXnxlTY6a0RDi7jp1YuDZSidXJGkyOVowd/Sjg8fsdPUnS/RbbuXMn9u3bF/AVKiiUlZVh9OjRAfenpaWpBhpZZWUlOnbsiOTkZJSUlOCjjz4K+3ewCgcGFpXMdvJT8jLxwYJROLZkAj5YMEr3Fa0TnZ+RUYCTNZgcXbsZMBWYtBxI7wGApO+Tloe1vmDWhg0bkJWVhQ8//BATJkzA2LFjAQCnT59Gu3aBky+dO3dGfn4+cnJyMG/evICfjxs3Do2Njejbty8WLFiA6667zvbfQS8uu80YnCmXnb9ku+p0WWZGEj5YMCrgfqumuoy+jlY73URoFsJwW+Kt7PaKFSvQs2dPTJ482emmKIRTdjuq1hgYc4oTC9dG10SsqMFkZr1ArZ0A0NRyUdnWT6nzP6ktHvBUEmNwZqrGiUqsZqbM/NvpJgp4DJ9SF194xMAYWq909UyxGJ2KCfb4SFdiNbte4NvO7AWbTb0Gix0cGBhroaeTNjoVo/X4Xf/4Fps+O42K2gYAQMdkDx6b1M/2IGHFlFks7hdhxvBUEmMGGJ2K0Xr8Sx+d8AYFAPiupgHzXvvM9rpOVkyZ8Sl18Y8DA4u4WD5hzOhUjJHplYYmYfs8vRXrGtFwSh2zF08lsYiL5RPGjE6jpCW2w/m64JU0fcmBRG1dAmhdA8lI9kAIoLK2wXC6qBXrGk6fUmeWEALDhw/HQw89hPHjxwOQymmvWbMG77zzjqHXuvPOO7Fp0yZ069YNBw4cCPn4wsJCJCQkYOjQoYbbffz4cRQXF2PatGmGn2sGjxhYxPmeMPbU1kOKIBHtjE6j/Py6ywy9fkayR3UX9rx1n2Hea5957/uupgEVtQ1xX17c6tFlOGW3/c2YMcNQMCksLERxcbHh9wGkwPDyyy+beq4ZHBiYI+w+YcyuA3uMTqPMG3cV7h3RG66WDE8XATf07QaPKzDlEwCq6hrx+JtfBKxLNDQLNDRpb0aN13RReXQpBwd5dDkgK930a/qW3X7iiSd0l932V1BQgE6dOul67PHjx7Fq1SosW7YMubm52LlzJ8rLy/GTn/wEgwcPxuDBg/HBBx8AAN577z3k5uYiNzcXeXl5uHDhAhYsWICdO3ciNzcXy5YtM9xWo3gqiTnC/4Sx63p3tiw42F30zeg0yrxxV8HtIizf/jXuG3kF5o7pg417yzDn1X0Bu60bmgW+q2lQfZ1Q4jFd1K7zqyNRdttXr169MGvWLKSmpuLBBx8EAEybNg1z5szBsGHDcOLECYwdOxYHDx7Ek08+iWeffRb5+fmoqqpCYmIilixZgieffBKbNm0Kqx16cWBgEee7pjC0dxdc17uzpdNJRkpUW1VmYtV7RzAgK13R/uIjZ7G/tBIDstIDguCUvEzMeXWfuV9QQ7ymi/qOLmePusKSfyNOld32tW3bNnz55Zfe2+fPn0dVVRXy8/Mxd+5c3H777bj55puRlRV+JVmjeCqJRVywE8asoDdDyMqKqlpTHm4XNI/Z1OrIM5I8AesYHhfB41affgKAdi7ClNzuivtiJdMrFLvOr1Yruy1P4fh+bdu2DSdPnvTeXrVqlSXv39zcjI8++shbxbWsrAypqalYsGABVq9ejdraWuTn56OkpMSS9zOCRwws4uw+YUwrcygj2YP8Jdu9o4Oa+kbVkcXjb35heNSgNeXhHwTPnL8IF4Bpz3+MjskeeFyEhubWCaUkjxuLJ/cDELgL2/c+/6ykKbnd8cqnJ5F/ZZeYy/QKxu7Rpa9QIwb5JLZQVqxYASCwhlJaWhrOnz/vvT1mzBg888wz3sqr+/btQ25uLo4cOYL+/fujf//++PTTT1FSUoIePXoELeFtNQ4MLO6oFX3zuAlVdY3e+Xu1wCH7rqYBG/eWYUpepqGpJv8pjzPnL+IvH/7D+9yRV3XF67vLvO36rqYBHjchI8mjmnaq9j5Bq6Be2cXyuXinRfP51bfddhsKCwtx9uxZZGVl4fHHH8ddd92FkpIS5OfnBzx+0qRJ+OlPf4o33ngDzzzzDJYvX457770XAwYMQGNjIwoKCrBq1So8/fTT2LFjB1wuF/r164fx48fD5XLB7Xbj6quvxowZMzBnzhxbfzcuu83i0sa9ZVj8ty+8u4uJACP/1DNbOmm16qdaWUgL1+/Hpv2n8S9De2HNB8fQ0ChQ39Ts/TkBqqW9tcpsm/HU1kPewDR3jPmdyMHWTNRGfEbEW9ltfxMnTsT69euRkJDgaDvCKbvNawwsbl1sbO2UjV7/nKqoNVT+ovjIWWzafxoAcF3vzkhq51YEBUA9KMjvJb9GOGsCVs7F25Em2lZs2rTJ8aAQLp5KYpax8yrTKLVOXY3WVXz3jCTNReyyilrkPr4VRNJ0UIfEdsjMSMIff34NAOCXf9mNCwZ2O3fPSApYEzD6WVo9F29XmiiLDTxiYJaJpqtMPTn9SR43br+up+ZO5oxkj+ZzK2obvOsV5+sacfhMFc6cvwgAaPAbKfhSyyvqlOLB9NUf49vqevx89SfotWAzVhcdxd1/3qX7s7Qj08vuTYgsevGIgVnGqqtMK/YWaGUmqR1HOeiyTqrvt/hvX+h+v8ZmgUc2HgAI8LhdKLiyK7Z++Y3iMUkeN35yTSZ2lJR736tTigefl7Vmqsinop2trkeC24W7/7wLd+Vn40/Fx3F515SA9w02igg308vOTYgsunFgYJYKdzOS0V3LWlMuA3tm4Nvqel0Lx1o7mStrje1AvnCxEYkeF16YMRhDe3fB0ndKsLLwCARaF7On5GUqNr5NX/2x5uvVNzWjfbt2WL79ayR6XJg44NKIFR+MZJooiz48lcQsFe4CqNHzDrSmr267tqfp0tAb95Yh74mtmovFwVxsaMYHh6W2zBt3FRZNuApJHjeW3jLAGxTuenEXir4qx/TVH6M5xJvIwcbjdqFf9/SIFR+0exMii24cGJhlfK8y/Xf56mX0XINglVqn5GXigwWjcGzJBHywYJTuoDDvtc9M1ysSAJ4tPIKl70i7Ve8e3htrZgxStK/vpWkoPnIuZFCQzRx+Of7482tw38t7ASAi8/6zftg74LWH9u4S8SQCqwkhMGzYMLz99tve+9atW4dx48aFfO66devQr18/uFwu6EmXr6iowMqVK0239emnn0ZNTY3p54eDAwOzjBVXmVplIrpnJGlWTLVykXTplkNBq5jq9ZeP/uH9s2/7Bl3WEXtPVOh+nTE/uAQvfXwCALBiWh7e/OyULeUhotb7TwPHipT3HSuS7jchnLLbOTk5WL9+PQoKCnS9FweGMBBRDyLaQURfEtEXRHS/021i5lhxlal13sHIq7pq1jWyMn/fqgqlvumqvu3bdvCboFNUbpLyllwE3DuiN567Y5B3RPTFqUps+eKbsEZkMSdzILBuRmtwOFYk3c4caPolzZbd7tu3L/r00b9pcMGCBThy5Ahyc3O9ZS+WLl2KwYMHY8CAAXjssccAANXV1ZgwYQKuvvpq5OTk4NVXX8Xy5ctx6tQpjBw5EiNHjjT3i4YhGhafGwH8Sgixh4jSAOwmor8LIb4M9UQWf+TpHv8sIa21h//c9CWaAcsWSbWymWQeFwGEkKMKeeTjv4i7fPvXms+Rd0D7L6jLI6/nio5GbXkI22QXALe8KAWDQXcBu9ZIt7P1XbVriUTZ7SVLluDAgQPeGktbt27F4cOH8cknn0AIgcmTJ6OoqAjl5eXo3r07Nm/eDACorKxEeno6nnrqKezYsQNdukT+79bxwCCEOA3gdMufLxDRQQCZADgwxAEzqadqWUJaJarPVtfj5buv1d1ZhmrPvLF9MO+1zzQ7/luH9FCkt2Yke1BV1xhQCE8ueuc7vbZxbxncRN6U1IDfu6U6qpHUUyuLD0at7AIpKBT9HiiYH3ZQAJwpu71161Zs3boVeXlSFllVVRUOHz6M4cOH41e/+hV+/etfY+LEiRg+fLit7dDD8cDgi4h6AcgDoJ3Dx2KGlQfmaF3JZ2YkqU5faQWFUO2Rv89du091cXhHSTlu7H8pfn79Zd4OPFiw8X3MwvWfawaFBDfh2cIj2LjvlOkzIeLWsSJppFAwX/qePdyS4KBWdtuug3oAaeF74cKF+OUvfxnwsz179uCtt97Cww8/jNGjR+PRRx8N+/3CETWBgYhSAbwO4AEhxHmVn88EMBMAevbsGeHWMTOMHJgTimrFVBehpr4R2Qs2a45GfDttl8rVulp7gh2iU1ZRi7v/vAvP3zFI8fhQv4/acZ2AtJbgJkJ9ywglWPC06lChmCKvKcjTR9nDlbctZHbEUFZWhjvuuAPvvvuu4v60tDRFoBk7diweeeQR3H777UhNTUVZWRk8Hg8aGxvRqVMnTJ8+HRkZGVi9erXi+U5MJTm++AwAROSBFBT+KoRYr/YYIcRzQohBQohBXbt2jWwDmSlGU0+D8T9rOSPJA7TUKtI6ZMf/IB6tq3W19gQrh9HQKLzlL/QUvtu4t0wz/bVZQDENBajv27DyUKGYUrZHGQTkNYeyPRFvyoYNG5CVlYUPP/wQEyZMwNixYwEAp0+fRrt2gdfYnTt3Rn5+PnJycjBv3jyMGTMG06ZNw/XXX4/+/fvjpz/9KS5cuIDPP/8cQ4YMQW5uLh5//HE8/PDDAICZM2di3Lhxjiw+O152m4gIwP8C+FYI8YCe53DZ7diQv2S75vRPuGWmtV7bt+RFTX2jrv0I/u3ZuLcM89Z9FtBh+z9n6S0DdC10a7U1lONLJoR8DStLdkdKvJXdXrFiBXr27InJkyc73RSFcMpuR8NUUj6AnwP4nIjk8fsiIcRbDraJWUDrPAN5YVZmZopEa9QhjwqMdMT+7Vm65VDQoCC//n0v78U9Iy4PmRUUbISkVd1VTlsN9RpWpdcy8/xPaosHjgcGIcT7UC86yWKcVuqpb6dvdoE6VFqpEUu3HMKcV/d526e3s/3h97vgD4VHQ9Yq0jxqNMnjPUjIn/+0l9ZraG0IbJPrEcwyUbHGwOJXqLIURmsjydQ2wpnlP2+fnqS9viDrlOzBxr2ncM+Iy0Omi2q1lQjoqLGWkenX4Wtt/PMf7QBteD2CWYYDA3OU2SkS/8Vo/6kXs2obmkCEgE5YfnUXAYMu64hvaxowJS8Tfyg8GnLnsdzWDL+A811Ng2YF15FXBSZYJHpa/7tmJHk0iwKaDbaR5PTaZrwL9/PlwMAcFaw2Uii+o5H/nnq15nxkl5QEpLR3w6UzdlTUNARUZl12ay4WjL8KC2+8CkfPVmP2qCvw3lfl3jUGAJq1nOS2prQPnLnVWsrYUVLu/bM8AvBdSPc9ttRftK9HJCYm4ty5cxwcbCKEwLlz55CYmGj6NRxfY2Btm94F6lCm5GVi0/5T2HbwTMDPbh3cA24XYfn2r5HWvh2qLjaie0YSqi82qs7xd89I8u5NkOfq57y6D51TElDb2ITn7xgUUH5Dz1qJkY7Z97FG94MYXY+ItKysLJSWlqK8vDz0g5khzc3NEEKAiNDQ0IDi4mJUVVWhqqpKVz0oGQcG5ig9C9S+5DpC8kE38oE1+0sr0TWtPa6/vBNOfFvrfa0pud3xbU09tnzxjbfI3l/vuNZboiJYUPL/uXyqmryHwbf8xl8+/EfIztvIgrlvJ250BGBVsLWLx+NBdna2082IKUIIXLx40dvJV1VV4cKFC6iurlbc19ysHEm63W6kpqYiJSUFHk/otTMZBwYWlNFD6c3Qs3NYJh/Mc8+Iy73f/1B4FPeMuNxbedS/rf4nkd314i7MHXMl7h4utX/plkMoq6hFh8R2eOKmHEWw8u/s65uaFZ29XH7jd2+XqLbXt/NW3b3tJsBvk5t/J250BGA02DLn1dfXKzp4ta/GxkbFc1wuF1JSUpCamopLLrkEvXv3RmpqquIrMTERZGL9jQMDC0ruiCNxnKQevgfz/PD7XfDbzSWYkpeJp7cdRlI7N25//mNFR6h2RsTcMVfiqa2H0a97OqbkZaJbh/aqG9WMXKnr6by1Omy1+/wL+xkdARgJtsxeDQ0NAVf2/l/19fUBz5M7/c6dO+Oyyy4L6PSTkpIUtZ6sxIGBBeXbEU+/tide+viE4+f++h58M6RXR2zYW4Z2LkL1xcD5fbVRzd3De6Nf9/SQv5ORK3W9nbdWhy3ft+q9I+jWob3iZ906tMcNfbthz4kKHgFEmaamJlRVVXk7frXpnbq6uoDnJSUlITU1Fenp6cjMzAzo9JOTk+F2W5OObQYHBhaSb0c8e9QVjpd5lg+++XFed2zcewpJHhdqG5Rzq6GK9en5nYxcqVs1fRNshPbMNPOH00TU/rXAu08AlaVAehYw+lFgwFSnW2VYc3OzopOvrq7GhQsXFJ1+bW3ghUP79u29Hfwll1zineNPS0vz/lmttlI0ie7Wsajgf0Ladb07OxYc5I5SXltYNOEq/GZz6Pl9tdcJ9Tv5d/a/SP0E8z2vIvmNfwKFgR2eFdM30ThCM2T/WuDN2UBDy2dfeVK6DURVcBBCoLa2NqCj9/2qqakJSKn1eDzeTr9Lly4BV/qpqamGFnmjFQcGFpTa4m04J6SFS14z8F07+GPhUZytDpyj1VqcNfI7eTv7/Wuxct1x7LvYEUPdp4HKk2h849/xyTcu7E/I070Qr2cxP9pGaIa8+0RrUJA11Er3RygwaGXw+E75BMvgSU1NRVZWlmqnLx/qE+84MLCg1BZvnTxO0rfzlD088QeGFmfN/E41bz+KXHTEfQ2zsQLLMdT9JT6pvxz3bm/Cs/+aDkBfp69nMT+aRmiGVZYau9+E+vp6xZW+PMXj2+lHMoMnHjledtsMLrvN/NldNK55cQZcEChu+gHua5iN6e5teKnpBjzjeQb5/1EMIHAk4n9bJt+vNlWk9zWi1rIcafrIX3oPYM6BkE9vaGgImr1TXV0dkMFDREhOTla9wvddzOVOP7bKbjMWNr3z+2YDyKnmzshyncVQ95fo2/gPLG+6GbPd63EZfQOgdWSgZ30g2FRRtI3QDBv9qHKNAQA8ScDoR70ZPMG+Ll68GPCScgZPRkaG6hRPSkqKbWmbbRUHBtZmhHMG9eqE6ZjfsBL/0TAde8QVSMRFvNA0Foc8fTFo5xE8tfUw1swYpGt9INhUkdpahdYZ1tHEm8HTdTgweBE6710BT+0Z1CV0xmddp+Dg7mrUvv/HgOf5ZvB873vfU73adzJts63iwMDajHDOoM6dMBOzXk/G7ubeaIdm/ML1Nv7YPBnb6vti6+YSLJpwlXfqJ9j6QLQt5ushhEBNTQ2ady7D+bQrcDa1r/cKP+mfnyK54hA+TRjqk8HTDkh7AJ5OrRk8vTSmeOIhgycecWBgbUY4VUen5GXi3YM/QtKRczhbXY+V4mb06pKCo2er0a97B9w9vLeuTj/apoqEEKirqws6py9n8HSv/w5jK+fi0/TbcCa5Dy53leHaf67BF30fxMCsgUhNTUVaWpp3kbetZPDEI158Zm2Glecmz3l1LzbsPYUhvTri6/Jqb+dud10pI4QQumrwNDUpR1Eul8s7dy9f2csdfseKz5G+9X5g8F2gXS8At7wIZBdE/Hdj5vDiM2vT1BaZrao6+vzOI9i49xR+nNcd73111lvQT2uh2a6RQKgMnqqqKjQ0KMuK+2bwdOnSBb169VJM7aSlpSEpKSlIBk828N3nQNHvgYL5HBTiFI8YWNzRKqf9Xzf3BxBe2YriI2cV1Vl9d2I3NasvHpsRbgaP1lfYGTzHioB1M4BBdwG71pgbMcRJyYxYpHfEwIGBxR0rp4z8WVGGvKmpCTU1NQEd/fYj57HhaDMq6gkd3E0oSP8OP0it8T4vMTExoJOPaAaPHBTkYOB/Ww//khmAlM46aTkHhwjgqSTWZtl5tGWodFI5gyfYlb5aDZ6v6jpgc3kGGoR0NX++qR3+XtkNA6/Jwo8H9oiODJ6yPcogkF0g3S7boz8wREHJDBYaBwYWd+w62jJUBo+cxeNfg6ddu3beK/oePXoorvjlipsjn3ofDULZ5otNAms+PYs7Rw8Iq92WGfZA4H3ZBcamkiJQMoOFjwMDiztmFpnDzeBJTU3FpZdequjsfQuvhSrHYOcoJ6qkZ2mUzMiKfFuYJg4MLO6onY0w94be+GGvZJw4cUJRbM23+JpaBo88j9+1a1dkZ2cr5vZDZ/DoZ9coJ+q4olHMAAAZSUlEQVQEKZnBogcHBhbzGhsbFSdoVVVVIaOqCgtzalqmd/6Jbz4pwSufKJ8np2127NgRWVlZioNU0tLSkJycHLEaPFal0kY9eR2Bs5KiWlQEBiIaB+B/ALgBrBZCLHG4SSxK+Gbw+JdaDnaKlpzB06FDB3Tv3l01bTOaavBYdQJcTBgwlQNBlHM8MBCRG8CzAH4EoBTAp0T0NyHEl862jNmtubkZNTU1mlM7cgDwl5CQ4O3gu3btqtrpO57BY4IVJ8AxZgXHAwOAIQC+FkIcBQAi+j8ANwHgwBDD5AwetaMT5YNVampqdGfw+H4lJCQ49Fsx1jZEQ2DIBOCbplAK4Fr/BxHRTAAzAaBnz56RaRlTpXZ0olrn75/B43a7vQu3atM7ejN4GGP2iobAoIsQ4jkAzwHSzmeHmxPX9KRt+h+d6JvB061bN9VO36oMHsaYvaIhMJQB6OFzO6vlPmaDxsbGkFf6ajV45AyeTp06oWfPnqpHJ/IpWjHIt25RUkfpvtrvOFuojYuGwPApgCuJKBtSQPgZgGnONik2NTU1KRZu/ef0q6qqUFdXF/A8ufBaeno6MjMzA+rwRFsGD7OIf92i2m9bf1Z5UvoZwMFBSxwXA3Q8MAghGonoPgBbIKWrviCE+MLhZkUdOYMnVA0ef74ZPFpTPO3aOf7PgDlBrW6RL65hpM0/qMZZII2KHkEI8RaAt5xuh1OEEKitrQ05xeNfeM03g0dteoczeFhQeuoTcQ0jdXFeDDAqAkM84wyeGBTHUwQKWnWL/B/DAsV5MUAODGEyk8HjcrmQkpKClJQUzuCJNlZNETgVXN5/GsgcqKx4eqxIKo3tXx1VrW6RL65hpC3OiwFyYAgiVAZPVVUV6uvrA54nd/qcwRODrJgicHL+OXOg9mE6/vzrFnFWkn5xXgywzZ7gppXB4/ullcHjW1rZv8xycnIyZ/DEssUZANT+TxCwuELfayzL0bia7AHMORBO6/Sx4vhNFloMTjnafoIbEf1aCPE7s8+3k9kMnvbt26tu0vINApzBE4d8/4OTCxBNgY/RM0XgfR2NeftIzT9nF0hBoej3QMF8Dgp2ieNigLp7OSJa63sTQC4ARwJDc3Mzzpw5Y2kGT1paWkwWXmNh8p/2UQsKeqYI1M4y9hep+edjRdJIoWC+9D17OAcHZoiRy9/zQoh/lW8Q0R9saI8u586dw2uvvea97Xa7vR282gYtzuBhmrRy+ckNiGb9UwSh9gREav7Zd00hu0AKCr63w2VkcZvFLCOB4Td+tx+ysiFGpKamYvz48d4AwBk8zDSt6R3RrH9NIdjrANLaQqTmn8v2KINAdoF0u2yPNYHByOK2GRx4okLI1Bgi+h8iIiHEMd/7hRDfaj3HbklJScjOzkbXrl2RnJzMQYEZt3+ttEisutAM49M+Wo+XF5wHTG19z8UZ0vf9a5Vt8b8/WLu1HjvsgcAAkF1gXacqB5p1M4Dtv7F2NAK0Bp5jRdJtOfBkDrTm9ZkuenImLwD4GxGlAAARjSWiD+xtFmMmvf90a6ciO1Yk3S+T1wO0FonNTPtcOQbS0pvG6yjeU7SmsG6aq36/f4e/fy3wu2xg/d2hH2uFYJ+j7+L2oLusXb+wO/AwXUIGBiHEwwBeAVDYEhDmAlhgd8MYM0XPFWew9YD0HsCk5camfV67E9j7FwSMPrrltI4UNsxS3x+x+0X1+9f/UnpdoDWo1KoM0uU9FlYL9jn6L277BxAteoI2YG/gYbromUoaDeBuANUAugCYLYTYaXfDGDNFzxWn5noAtU77GHGsCGgK3OiIiuOtnbpathOgfT+agYNvtqbABlvYtiMNVutzBFr/POqh1sfoCQ56p4nMBh5mGT1TSQ8BeEQIMQLATwG8SkSjbG0VY+EIdcWpuR5gMp20+qz2/aE6dQqyGbKpvnV/RTB2pcGqfY7BFrdDrX/oCdq+i9lGAw+zjJ6ppFFCiPdb/vw5gPEA/tPuhjFmWqgrztGPSvP/vsJJJw0WaIJ16p4k4JoZgW3xJe+qDfYadqXBqn2OWovbHbrrWysJFbSDBR4WMYYL9gghTgMYbUNbGAufnivOAVOldYT0HgDI3LqCr9GPAm6/8ubuBOl+rU6d3NJ7TnxK+q71X1HeR6EWPJI6hdfuYIxeuQerMeX/usGCtt1ZVUwXU5XchBBBxsaMOUjvFeeAqdJ6wuIKqeN99wn1KRA9qaRp35MCQUpXACR9dydI92uNTn68qrVDT/sekJAslePw5U6QHnP+VGAgu/l54NctGeR6U10B/QvARq/c9ZSh5mmimNFmi+gxnWKwUJghaqUsPEktV/HQ/pnvZxBqU1aoz1B+/tFC4P1l0ua6lK7SiODc4ZYd2Cqb5IK1XevvyH9ntP9ts/QUDuTNa47TW0SPAwPTZqbjiTXBOjQg8lVS5Y665/VAySblz/w/e7NVXO2ovtoW/q3EAb2BgQ8FYNr0zhvHsmBTIJo/O2nPpjKgdXHWPygAgZ+92VPE7NgnYPW6DXMU15Bm2uL8+EIAoU/i0todbdfBO/LirBbfz97sKWJ2VV+N4zLUbQ2PGJg2q/P9o1Gw1FWtbCDAnpGT73y/PJXlz/ezN5N2ywvATAcODEyb1fn+0SjYFIj8My2+V+9GCuFp8c0E0vPZm5m+4X0CTAdefGbBOZ2VZOb9rW5zqEVeuxZenf7sWdzhrKR40xY6Cf90xv1rgTfuVdYhCtXhWtVJ+7Yl1GvaccYzp3YyG3BWUjzRKtlsV2aME/avBT5cAfzvJGDpFdLtLYsCi9OFmtu3KpPKt+DbgKnAdf/WugHNf8rGjkV6PpeAOcjRrCQiWgpgEoB6AEcA/IsQwsCxWW1EsM4uHkYN/lfk1eXAxnuA5kb1xwfrcK3qpLMLpDpGf75J2mBGLmDYHPX1FbPZQaHeX14YtnK/AWM6OD1i+DuAHCHEAABfAVjocHuiU7ynjaoFvuZGBBx8IwvW4YabSSWXjNi/FvhopRQUAOn7RyvVR2laC8U9hiizffavlUZDi9P1LVDzuQTMIY4GBiHEViGEfFn4EYD4yIPUW49Gr3hPG9UMcEK9OF19tXb2T6hsnlB/N/IUzpZF+qektLKDrpnROh0kr5dUl7f8zjqmA+04l8Dqf5ssLjk9YvB1J4C3nW6EJayeH473tFGtACcXo5OL0yV1AoRoOclMY61F7qSTOrXe187nswv1d1O2Bxg2t7UD96e14c23KJ982I/vdNDmXxlbL7FrvwGvXTAdbA8MRLSNiA6ofN3k85iHADQC+GuQ15lJRLuIaFd5ucZ/2mhh9bm18V5uQCvwjf0tcNsrwPX3SR1uQgrQ3KB8nFbn2uhztV/7bWsACfV3kzkQeP8pwOXRbu/+tcorb+8ehnRgSU9loJKngy6eV38trdGSXfsN+ExlpoPj6apENAPALwGMFkLU6HlOzKSrbv+NND9cMF+66mPatNJxfdM2F2cg4FxlAABJgUOmJ3002N9N8Qpga5C/r/QewJSVUqd6zQxp7UErlVW+Im9qUA8OVhbkM5LSzP8226SYSFclonEA5gOYrDcoxAw+t9YavlMfetdatKZ75KvzYH83x4qkEcOAW7XbVFnaeuX9/jLttQjf6aAJ/x24XmLldKCRlGb+t8lCcHqNYQWANAB/J6J9RLTK4fZYg+vRGFvkDNap+U59XHp14HP9O9f9axE0mynU3428xvD1NqB9B+3XAaS2yVlL/ipLldNBA6YCNz3bsl4C66cDjZyg1tb/bbKQHJ9KMiPqp5J416qxA2GMTP1cNRE4/Zn0+JSu0jqE3LkeKwJenQ7UVao0iICbn5NOQwv2d+Pbzgv/DL7z+lhR6z6HYG03Qu90kP/jtEZJ/tNs/G+zTdM7lcRlt+2g9h8su6BtLfAZ2aAVap+G/9THLS9K96+bIR2LKT9m3QyNoAAAQvvq3Pfvxn/RF5BSV6vLlaeoye83bI76GoOZKSL/jX7yyAkIfnJb5UlIoySVizz/aTb+t8l04MDA7LuK9N2gVTBfu/MJtnPYf6SRPbz1tlrg2fhvwU9kC8X/99U6Y8A3gHS9quXq/SSQmA7c+KS5KSK9O9zVHgeBgOAQTynNLKKcXmNg0cCu3Ha9i5zB9mkES9tU2xkcqT0fwx5obZN3D0MlsOCE+XUDvTvcg20IjNeUZhZRPGJg9tTlCXal7/+6cuelN9VSnvpQO4nM6GtFE701lzQfZ+NZ1KxN4cVn1srK3HYz01NG8vCNLG6H8z6RpFbe2+UB2qcBtd+1thWw5/wHFvf4PAZmjNyxOlXJ0+g5CmbXRew6VMcqvkErqSNQX6WeFQVEZ3BjUY0DA9NP79W3nVfadhx24+T7yML5zKxsK6epMsTIzmcWJfTU5bH7sKBIlRY38z5mz3MO9zOz8jPh4nnMAA4MTJlhI8suUF5JWnUympZIlRZP6mjsfrXOff3dwO+yQ3fw4X5mVn4mXDyPGcCBgeljxxW9b9mMaC0trrpnAMqKrVrC/cys/kz44B+mEwcGpo8dV/RGzlW2Su13xu4P1omHuvrX+mzIpW9ayupy61w8j+nE+xiYPqMfVc/mCeeK3n//xO4XgTvesPdK1uj5zEHrECF44FD7zABANLU8V6PkhS+tnddGGdlXwto8HjEwfcxevYZauI309IbR6Rm1x/sKNmLy/8zIHfgYK9dpZGqVbT9/Heg7yfqDf1hc4hED08/o1aueonBqu5ftDA5Gd0bL97/965YjRX3oGTH5fmaLM9QfY3XmlTxF55t+XPJma/FBGRfPYxp4HwOzT6g8/HB2LzvBuyfhpHT1L5qUFVdDieQeCqc3LLKoxGW3mfNCZeWEKpDnz3+z2JVjgMNbzW0eM7PxTP65ntLYauxYp9Git7ItYyp4jYHZJ1Qmk579EzK1/QS71pjbPBbOxrNw9iZYnWUUDGcgsTDwiIHZx8orZK39BL7Uzi7Q+1p6nxvu3gSrsoyC4QwkFiYeMTD7WHmFrLfj1fO4cDp3rVFQShf186ydoKfECWNB8IiB2cuKK+T3n5Y63ury0I/Vs+FOz14GraJzPYYAVd8oK566E6QRR7TUHeLjO1mYeMTAol/mQKnjdScEf5zeaSo9exm0is5dMwO46Vkgpat0f/sOUrtue4U7XhY3ODCw6JddIHW87gSpIwakjnnQXfqmqfw3fMnlNxLTtZ8brOjcgKnAvK+lhd2L56XX4qDA4ghPJbHYkF0gdcBmTphT2/C1+0Xg1peCd+jBUj6PFQEfrZQCVdHvgd1/Asb+lg/LYXGBRwwsNoSTfmm25LTWex4rAl65TVpnuHheuq+6HHjjXuvOp2DMQRwYWHQIVlPJN/1y1EOtnbzR4GCkJlOw9yzbI61J+C5AA9Jtq+seMeYADgzM/AllVr5/sA1nVqRfGh1xBHvPYQ8A1WfVn2d13SPGHBAVtZKI6FcAngTQVQih8T+uFddKspB/oTug9cD5SM2X211DyI6aTJE+O5oxC8TMmc9E1APAGAAnnG5Lm2T3kZ162H3esx0bvsI5Xc3pERpjIURDVtIyAPMBvOF0Q9okI52ymcJzehg9PMcoOzZ8GS3fLdNTipwxhzkaGIjoJgBlQojPiMjJprRdejtltQ7tjXulP/uerSDPwRsRyaqjVjKzqzucOk2MRYjtU0lEtI2IDqh83QRgEQBd//uJaCYR7SKiXeXlOkojMH30TomodWhN9cCWRdKf5Xl7M2UhIll11Gl2T5sxZgHbRwxCiBvU7iei/gCyAcijhSwAe4hoiBDinyqv8xyA5wBp8dm+FrcxeqdEtDqu6nJpb0C4h8FEoupoNLB72owxCzg2lSSE+BxAN/k2ER0HMEhPVhKzmJ5OWatDk3f+6j0Mxq51CqvZ1c5YnTZjbYrjWUksyskZNJUnAfitA7kTANGsf29AOAfkRJKd7WxL02YsZkXFPgajeB9DhKjtcQABEFIRu4ba1qqievYGxEruf6y0kzGD+MxnFj7VU9OE1EEO/lfleQWhzmsGYmfhNVbayZhNODAwbcE6SDN7A2Jl4TVW2smYTXiNgWnT6gjNdpDh7BaOpFhpJ2M24cDAtFndQcbKwmustJMxm/DiMwsuVtJLGWMh8eIzs0Y0bzzjoMWYLTgwMOeZ6eC5GB1jtuE1BuYss5vJoqFcOGNxigMDc5bZDp73GjBmGw4MzFlmO3irU2kZY14cGJizzHbwvNeAMdtwYGDOMtvB814DxmzDWUnMWWaPyJSfy4GAMctxYGDO4w6esajCU0ksNr3/dOD5D8eKpPsZY2HhwMBiU+ZA6fwHOTiEc+Y0Y0yBp5JYbJLPf1g3Axh0V/hnTjPGvHjEwGJXdoEUFIp+L33noMCYJTgwsNh1rEgaKeg9c5oxpgsHBhabfM+YHvVQ67QSBwfGwsaBgcWmsj3KNQXfM6cZY2HhxWcWm8ycOc0Y04VHDIwxxhQ4MDDGGFPgwMAYY0yBAwNjjDEFxwMDEf07EZUQ0RdE9Hun28MYY22do1lJRDQSwE0ArhZCXCSibk62hzHGmPMjhnsALBFCXAQAIcQZh9sT3favBZblAIszpO/71zrdIsZYHHI6MHwfwHAi+piI3iOiwQ63J3rtXwu8ORuoPAlASN/fnM3BgTFmOdunkohoG4DvqfzooZb37wTgOgCDAawlosuFEELldWYCmAkAPXv2tK/B0erdJ4CGWuV9DbXS/XzIDWPMQrYHBiHEDVo/I6J7AKxvCQSfEFEzgC4AylVe5zkAzwHAoEGDAgJH3KssNXY/Y4yZ5PRU0kYAIwGAiL4PIAHAWUdbFK3Ss4zdzxhjJjkdGF4AcDkRHQDwfwB+oTaNxACMfhTwJCnv8yRJ9zPGmIUcTVcVQtQDmO5kG2KGvI7w7hPS9FF6lhQUeH2BMWYxrq4aSwZM5UDAGLOd01NJjDHGogwHBsYYYwocGBhjjClwYGCMMabAgYExxpgCBwbGGGMKHBgYY4wpcGBgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpkCxWOWaiMoB/MPpdkA6VCgWzo+IlXYC3Fa7xEpbY6WdQGy29TIhRNdQD47JwBAtiGiXEGKQ0+0IJVbaCXBb7RIrbY2VdgLx3VaeSmKMMabAgYExxpgCB4bwPOd0A3SKlXYC3Fa7xEpbY6WdQBy3ldcYGGOMKfCIgTHGmAIHhjAQ0S1E9AURNRNRVGYnENE4IjpERF8T0QKn26OFiF4gojNEdMDptgRDRD2IaAcRfdnyd3+/023SQkSJRPQJEX3W0tbHnW5TKETkJqK9RLTJ6bYEQ0THiehzItpHRLucbo8WIsogoteIqISIDhLR9Xqex4EhPAcA3AygyOmGqCEiN4BnAYwH8AMAtxHRD5xtlaYXAYxzuhE6NAL4lRDiBwCuA3BvFH+mFwGMEkJcDSAXwDgius7hNoVyP4CDTjdCp5FCiNwoT1n9HwDvCCGuAnA1dH62HBjCIIQ4KIQ45HQ7ghgC4GshxFEhRD2A/wNwk8NtUiWEKALwrdPtCEUIcVoIsaflzxcg/UfLdLZV6oSkquWmp+UrahcViSgLwAQAq51uSzwgonQABQDWAIAQol4IUaHnuRwY4lsmgJM+t0sRpZ1YLCKiXgDyAHzsbEu0tUzN7ANwBsDfhRBR21YATwOYD6DZ6YboIABsJaLdRDTT6cZoyAZQDuBPLdNzq4koRc8TOTCEQETbiOiAyldUXnmzyCCiVACvA3hACHHe6fZoEUI0CSFyAWQBGEJEOU63SQ0RTQRwRgix2+m26DRMCDEQ0jTtvURU4HSDVLQDMBDAH4QQeQCqAehaZ2xnZ6vigRDiBqfbEIYyAD18bme13MfCQEQeSEHhr0KI9U63Rw8hRAUR7YC0jhONC/z5ACYT0Y0AEgF0IKKXhBDTHW6XKiFEWcv3M0S0AdK0bbStNZYCKPUZJb4GnYGBRwzx7VMAVxJRNhElAPgZgL853KaYRkQEac72oBDiKafbEwwRdSWijJY/JwH4EYASZ1ulTgixUAiRJYToBenf6fZoDQpElEJEafKfAYxBFAZbIcQ/AZwkoj4td40G8KWe53JgCAMR/ZiISgFcD2AzEW1xuk2+hBCNAO4DsAXSIulaIcQXzrZKHRG9AuBDAH2IqJSI7nK6TRryAfwcwKiWVMV9LVe50ehSADuIaD+ki4S/CyGiOg00RlwC4H0i+gzAJwA2CyHecbhNWv4dwF9b/g3kAvitnifxzmfGGGMKPGJgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpsCBgTHGmAIHBsYYYwocGBgLQ8vZDD9q+fN/EtEzTreJsXBxrSTGwvMYgCeIqBukSquTHW4PY2Hjnc+MhYmI3gOQCmCEEOJCS/2clQDqARQKIf7qaAMZM4inkhgLAxH1h1STqL7l4B5AOtXvNSHE3eARBItBHBgYM4mILgXwV0in4lURkXw0aRZaD0hqcqJtjIWDAwNjJhBRMoD1kM5/PgjgPyCtNwBSHfyslj/z/zEWc3iNgTGLtawxrABQB+B9XmNgsYYDA2OMMQUe5jLGGFPgwMAYY0yBAwNjjDEFDgyMMcYUODAwxhhT4MDAGGNMgQMDY4wxBQ4MjDHGFDgwMMYYU/h/MtNYrCZcMlMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_results(X_train, Y_train, X_test, Y_test, score_fn, threshold = 0):\n",
    "    # Plot training set\n",
    "    plt.plot(X_train[Y_train==-1,0], X_train[Y_train==-1,1], \"o\", label=\"Y=-1, train\")\n",
    "    plt.plot(X_train[Y_train==1,0], X_train[Y_train==1,1], \"o\", label=\"Y=1, train\")\n",
    "    plt.gca().set_prop_cycle(None) # reset colour cycle\n",
    "\n",
    "    # Plot test set\n",
    "    plt.plot(X_test[Y_test==-1,0], X_test[Y_test==-1,1], \"x\", label=\"Y=-1, test\")\n",
    "    plt.plot(X_test[Y_test==1,0], X_test[Y_test==1,1], \"x\", label=\"Y=1, test\")\n",
    "\n",
    "    # Compute axes limits\n",
    "    border = 1\n",
    "    x0_lower = X[:,0].min() - border\n",
    "    x0_upper = X[:,0].max() + border\n",
    "    x1_lower = X[:,1].min() - border\n",
    "    x1_upper = X[:,1].max() + border\n",
    "\n",
    "    # Generate grid over feature space\n",
    "    resolution = 0.01\n",
    "    x0, x1 = np.mgrid[x0_lower:x0_upper:resolution, x1_lower:x1_upper:resolution]\n",
    "    grid = np.c_[x0.ravel(), x1.ravel()]\n",
    "    s = score_fn(grid).reshape(x0.shape)\n",
    "\n",
    "    # Plot decision boundary (where s(x) == 0)\n",
    "    plt.contour(x0, x1, s, levels=[0], cmap=\"Greys\", vmin=-0.2, vmax=0.2)\n",
    "\n",
    "    plt.legend()\n",
    "    plt.xlabel(\"$x_0$\")\n",
    "    plt.ylabel(\"$x_1$\")\n",
    "    plt.show()\n",
    "    \n",
    "plot_results(X_train, Y_train, X_test, Y_test, lambda X: weighted_sum(X, w, b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** How well does the decision boundary separate the points in the two classes? Where do you think the decision boundary should go? \n",
    "\n",
    "**Answer:** *The decision boundary is acceptable for `class_sep=2`, but very poor for `class_sep=0.5`.*\n",
    "\n",
    "To evaluate the perceptron quantitatively, we'll use the error rate (proportion of misclassified instances).\n",
    "The error rate is a reasonable evaluation measure to use for this data since the classes are well balanced.\n",
    "\n",
    "Complete the `evaluate` function below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.022388059701492536\n"
     ]
    }
   ],
   "source": [
    "def evaluate(X, Y, w, b):\n",
    "    \"\"\"\n",
    "    Returns the proportion of misclassified instances (error rate)\n",
    "    \n",
    "    Arguments:\n",
    "    X : numpy array, shape: (n_instances, n_features)\n",
    "        feature matrix\n",
    "    Y : numpy array, shape: (n_instances,)\n",
    "        target class labels relative to X\n",
    "    w : numpy array, shape: (n_features,)\n",
    "        weights vector\n",
    "    b : float\n",
    "        bias term\n",
    "    \"\"\"\n",
    "    return np.mean(predict(X, w, b) != Y) # fill in\n",
    "\n",
    "print(evaluate(X_train, Y_train, w, b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code block above computes the error rate on the training data, which is not a good idea in general. (Why?)\n",
    "\n",
    "Compute the error rate on the test set instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    }
   ],
   "source": [
    "print(evaluate(X_test, Y_test, w, b)) # fill in"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** How does this compare to the error on the training set? Is it what you expected?\n",
    "\n",
    "**Answer:** *The error on the test set is actually smaller than the error on the training set (for `class_sep=2`). This suggests we may not have a problem with overfitting (but it's difficult to say for such a small sample size).*\n",
    "\n",
    "Let's now examine how the train/test error rates vary as a function of the number of epochs.\n",
    "Note that careful tuning of the learning rate is needed to get sensible behaviour.\n",
    "Setting $\\eta(t) = \\frac{1}{1+t}$ where $t$ is the epoch number often works well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEOCAYAAAB4nTvgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X10VPW97/H3NwmQGMKDQEWeGgSrDVIRU6ytKFYL2NriqVC1eqtUL6ddpbb12F56b2+19NijXWfV9iq3XRZRsa6itaVSq9WrdtWqRyUoPoSHEh8qkSgPYhAEQ5Lv/WPvCZPJTGYIszNk9ue1VtbM7Nkz+zcMK5/8vr/9+21zd0RERLpTUugGiIjI4U9hISIiWSksREQkK4WFiIhkpbAQEZGsFBYiIpKVwkJERLJSWIiISFYKCxERyaqs0A3Il+HDh3t1dXWhmyEi0qesWbNmu7uPyLZf0YRFdXU1dXV1hW6GiEifYmb/zGU/laFERCQrhYWIiGSlsBARkayKZsxCRPq2/fv309jYyL59+wrdlKJUXl7OmDFj6NevX49er7AQkcNCY2MjVVVVVFdXY2aFbk5RcXd27NhBY2Mj48eP79F7qAwlIoeFffv2MWzYMAVFBMyMYcOGHVKvTWEhIocNBUV0DvXfVmHRnfqVsGdHoVshIlJwCotM9u6E310GL91T6JaISC/YsWMHU6ZMYcqUKYwcOZLRo0d3PG5pacnpPebPn8/GjRsjbmlhaIA7kw/eC25bdWaGSBwMGzaMtWvXAnDttdcycOBArr766k77uDvuTklJ+r+zb7vttry3q7W1lbKysoyPM8nW1oOlnkUmLXuC27bWwrZDRAqqoaGBmpoaLr74YiZNmkRTUxMLFiygtraWSZMmsXjx4o59TzvtNNauXUtraytDhgxh0aJFnHjiiZx66qls3bq1y3vv3r2byy67jGnTpnHSSSfxpz/9CYClS5dy3nnnceaZZzJr1iweeeQRZsyYwbnnnsvkyZMB+OlPf8oJJ5zACSecwE033ZSxrfminkUmibBo31/YdojE0I/+VM+6Lbvy+p41owZxzecn9ei1GzZsYPny5dTW1gJw/fXXc+SRR9La2sqZZ57J3Llzqamp6fSa5uZmzjjjDK6//nquuuoqli1bxqJFizrts3jxYmbPns3tt9/Ozp07OeWUU/jMZz4DwPPPP8/atWsZOnQojzzyCHV1daxbt45x48bxzDPPcNddd7F69WpaW1uZNm0aM2bMoKKioktb80U9i0xadge37epZiMTdhAkTOv3y/e1vf8vUqVOZOnUq69evZ926dV1eU1FRwTnnnAPAySefzOuvv95ln4cffpjrrruOKVOmcOaZZ7Jv3z7eeOMNAGbOnMnQoUM79j311FMZN24cAE888QTnn38+FRUVVFVVcd555/H3v/89bVvzRT2LTDrKUOpZiPS2nvYAolJZWdlxf9OmTfziF7/g2WefZciQIVxyySVp5y/079+/435paSmtrV3/8HR3/vjHPzJhwoRO2x9//PFOx0xtQ65tzSf1LDLpKEOpZyEiB+zatYuqqioGDRpEU1MTDz30UI/fa9asWR3jDRCUnnIxffp0Vq5cyd69e9m9ezf33Xcf06dP73E7cqGeRSaJMpR6FiKSZOrUqdTU1HD88cfz4Q9/mE996lM9fq9rrrmGb3/720yePJn29nYmTpzIfffdl/V106ZN46KLLuLjH/84AF//+teZPHkyDQ0NPW5LNubukb15b6qtrfW8XvzoqZvg4R/AyfPh8z/P3/uKSFrr16/nox/9aKGbUdTS/Rub2Rp3zzrIoTJUJjobSkSkg8IiE82zEBHpoLDIRD0LEZEOCotMdOqsiEgHhUUmHZPy2grbDhGRw4DCIhOVoUREOigsMlEZSiRW8rFEOcCyZct46623ImxpYWhSXiaawS0SK7ksUZ6LZcuWMXXqVEaOHNmjdvR0SfJc9+sphUUmWkhQREJ33HEHS5YsoaWlhU9+8pPcfPPNtLe3M3/+fNauXYu7s2DBAo466ijWrl3LBRdcQEVFBc8++2ynNaI2bdrEwoUL2b59O5WVlSxdupSPfOQjXHLJJVRVVbFmzRpmzJhB//79eeONN3jllVcYP348v/71r/na177Gc889R79+/fj5z3/O6aefztKlS7n//vtpbm6mpKSERx99NLJ/A4VFJipDiRTOg4vgrZfy+54jJ8M51x/0y15++WVWrlzJU089RVlZGQsWLGDFihVMmDCB7du389JLQTvfffddhgwZwk033cTNN9/MlClTurzXggULWLp0KRMmTODJJ59k4cKFPPzwwwA0NTXx9NNPU1JSwg9+8AM2bNjA448/Tnl5OTfccAMDBgzgpZdeor6+ns9+9rNs2rQJ6LyUeZQUFplogFtEgEceeYTVq1d3LPu9d+9exo4dy6xZs9i4cSNXXnkln/vc55g5c2a37/Puu+/y9NNPc/7553dsS16Jdt68eZ2uajdnzhzKy8uBYEny7373uwBMmjSJUaNGdawDlbqUeVQiDQszmw38AigFlrr79SnPDwCWAycDO4AL3P11M+sHLAWmhm1c7u7/EWVbO2lvg9a9wX3N4BbpfT3oAUTF3fnqV7/Kj3/84y7Pvfjiizz44IMsWbKE3//+99xyyy3dvs/w4cM7xkVSHW5LkqeK7GwoMysFlgDnADXARWZWk7Lb5cBOd58I3AjcEG6fBwxw98kEQfKvZlYdVVu7SPQqQD0LkZg7++yzueeee9i+fTsQnDX1xhtvsG3bNtydefPmsXjxYp577jkAqqqqeO+997q8z9ChQzn66KNZuXIlAO3t7bzwwgs5tWH69OncddddQLAYYFNTExMnTszHx8tZlKfOTgMa3P1Vd28BVgBzUvaZA9wR3r8XOMvMDHCg0szKgAqgBcjvNRa70yks1LMQibPJkydzzTXXcPbZZ/Oxj32MmTNn8vbbb7N582ZOP/10pkyZwvz58/nJT34CwPz587niiivSnnK7YsUKfvWrX3HiiScyadIk7r///pza8M1vfpO9e/cyefJkLr74YpYvX95p4Lw3RLZEuZnNBWa7+xXh4/8GnOLuC5P2eTncpzF8/ApwCtAM3AmcBRwBfMfdM/fvyPMS5dsb4OaToaQMqkbBd/I80CYiXWiJ8ugV4xLl04A2YBQwHvg3MzsmdSczW2BmdWZWt23btvwdPXHabPlglaFERIg2LN4ExiY9HhNuS7tPWHIaTDDQ/WXgL+6+3923Ak8CXZLP3W9x91p3rx0xYkT+Wp4oQ5UP0amzIiJEGxargWPNbLyZ9QcuBFal7LMKuDS8Pxd4zIO62BvApwHMrBL4BLAhwrZ2lgiLiiEasxDpRcVy5c7D0aH+20YWFu7eCiwEHgLWA/e4e72ZLTazL4S73QoMM7MG4CpgUbh9CTDQzOoJQuc2d38xqrZ2sT+pZ6GwEOkV5eXl7NixQ4ERAXdnx44dHfM2eiLSeRbu/gDwQMq2Hybd30dwmmzq63an295rOspQg1WGEuklY8aMobGxkbyOP0qH8vJyxowZ0+PXawZ3Op3KUAoLkd7Qr18/xo8fX+hmSAaH69lQhZV8NpS3Q3t7YdsjIlJgCot0WvaAlUL/cBq9xi1EJOYUFum07IH+A6GkX/BYpSgRiTmFRTotu4NeRWkYFhrkFpGYU1ik07InCIuOnoXKUCISbwqLdBJhURqeLKaehYjEnMIinS5jFupZiEi8KSzSSYxZlIQ9Cw1wi0jMKSzS6ShDJQa41bMQkXhTWKTTMcCtnoWICCgs0kuMWZRqzEJEBBQWXbl3HbNQGUpEYk5hkap1X7AelMpQIiIdFBapEivOJpehNM9CRGJOYZGqIywqtTaUiEhIYZEqOSw6BrjbCtceEZHDgMIiVXIZqqQ0uK8ylIjEnMIiVeLCRypDiYh0UFik6uhZHKEBbhGRkK7BnSq5DJWgMQsRiTn1LFJ1KkNpnoWICKhn0VXy2VDeHtxXGUpEYk49i1SJsOh3hK5nISISUlikatkdBkWprpQnIhJSWKRKLE8OSWMW6lmISLwpLFJ1CgvNsxARAYVFV4lrWYCulCciElJYpEpcywLC5T5MPQsRiT2FRarkMhQEvQuNWYhIzCksUqWGRUmZzoYSkdhTWKRKHrOAYJBbPQsRiTmFRar9qWUo9SxERBQWqbqUofppgFtEYk9hkaytFVr3pZShyrTqrIjEXqRhYWazzWyjmTWY2aI0zw8ws7vD558xs+qk5z5mZv9lZvVm9pKZlUfZViAoQYHKUCIiKSILCzMrBZYA5wA1wEVmVpOy2+XATnefCNwI3BC+tgz4DfA1d58EzACi/43dkiYsVIYSEYm0ZzENaHD3V929BVgBzEnZZw5wR3j/XuAsMzNgJvCiu78A4O473D36WtD+vcFtWcWBbaX91LMQkdiLMixGA5uTHjeG29Lu4+6tQDMwDPgI4Gb2kJk9Z2bfS3cAM1tgZnVmVrdt27ZDb3FbS3CbWOYDNGYhIsLhO8BdBpwGXBze/ouZnZW6k7vf4u617l47YsSIQz9qR1j0P7CtpExlKBGJvSjD4k1gbNLjMeG2tPuE4xSDgR0EvZDH3X27u78PPABMjbCtgUS5KTksVIYSEYk0LFYDx5rZeDPrD1wIrErZZxVwaXh/LvCYuzvwEDDZzI4IQ+QMYF2EbQ2kLUNpBreISGTX4Hb3VjNbSPCLvxRY5u71ZrYYqHP3VcCtwJ1m1gC8QxAouPtOM/sZQeA48IC7/zmqtnZIV4YqLYOW9yM/tIjI4SyysABw9wcISkjJ236YdH8fMC/Da39DcPps70lXhiopU89CRGLvcB3gLoyMZSiNWYhIvCkskmUqQ+lKeSIScwqLZGnLUOpZiIgoLJKlK0Pp1FkREYVFJxkHuDWDW0TiTWGRLGNYqGchIvGmsEimMpSISFoKi2Rp14bSDG4REYVFso4yVPI8i1KFhYjEnsIiWVsLWGkQEAkqQ4mIKCw6aWvpXIKCA/Ms3AvTJhGRw4DCIlnb/q5hkShJ6fRZEYkxhUWytpbO4xUQnDoLOn1WRGJNYZEsbRkqERYa5BaR+FJYJGvb37VnkXisQW4RiTGFRTL1LERE0soaFmZWamb/2RuNKbh0YaGehYhI9rBw9zbgtF5oS+GlK0OpZyEikvNlVZ83s1XA74A9iY3u/odIWlUomeZZgMJCRGIt17AoB3YAn07a5kCRhUW6eRZlB54TEYmpnMLC3edH3ZDDQlsL9K/svK2jZ6GwEJH4yulsKDMbY2YrzWxr+PN7MxsTdeN6nQa4RUTSyvXU2duAVcCo8OdP4bbi0u0At5b7EJH4yjUsRrj7be7eGv7cDoyIsF2F0e08C/UsRCS+cg2LHWZ2STjnotTMLiEY8C4uKkOJiKSVa1h8FfgS8BbQBMwFim/QO20ZSgPcIiJZz4Yys1Lgi+7+hV5oT2F1u+qsxixEJL5yncF9US+0pfA0z0JEJK1cJ+U9aWY3A3fTeQb3c5G0qlDS9ixUhhIRyTUspoS3i5O2OZ1ndPd93Q5wa7kPEYmvXMYsSoBfuvs9vdCewmlvA29Lc+psafi8ehYiEl+5jFm0A9/rhbYUVmJMImMZSj0LEYmvXE+dfcTMrjazsWZ2ZOIn0pb1traW4FbzLEREush1zOKC8PYbSdscOCa/zSmgjp6FligXEUmVU8/C3cen+ckaFGY228w2mlmDmS1K8/wAM7s7fP4ZM6tOeX6cme02s6tz/UA91tGzSL0Gt06dFRHpNizM7HtJ9+elPPeTLK8tBZYA5wA1wEVmVpOy2+XATnefCNwI3JDy/M+AB7s7Tt5kKkPpSnkiIll7Fhcm3f9+ynOzs7x2GtDg7q+6ewuwApiTss8c4I7w/r3AWWZmAGZ2HvAaUJ/lOPmRCIOMZSj1LEQkvrKFhWW4n+5xqtHA5qTHjeG2tPu4eyvQDAwzs4HA/wB+lOUY+ZOxDKV5FiIi2cLCM9xP9zifrgVudPfd3e1kZgvMrM7M6rZt23ZoR8xUhjIDK1XPQkRiLdvZUCea2S6CXkRFeJ/wcXmW174JjE16PCbclm6fRjMrAwYTLH1+CjDXzH4KDAHazWyfu9+c/GJ3vwW4BaC2tvbQwivT2VAQjFtogFtEYqzbsHD30kN479XAsWY2niAULgS+nLLPKuBS4L8Ilj1/zN0dmJ7YwcyuBXanBkXeZSpDJbZp1VkRibFc51kcNHdvNbOFwENAKbDM3evNbDFQ5+6rgFuBO82sAXiHzgPqvStTGQqCnoXKUCISY5GFBYC7PwA8kLLth0n39wHzUl+Xsv+1kTQuVXdlqNJ+KkOJSKzlutxH8euuDFXSTz0LEYk1hUVCtjKUTp0VkRhTWCR0W4Yq0wxuEYk1hUWCylAiIhkpLBK6K0OV9lMZSkRiTWGR0O2kPM3gFpF4U1gkZC1DqWchIvGlsEjIWoZSz0JE4kthkZAIg5J0PQudDSUi8aawSGhrCUKhJM0/iXoWIhJzCouEtpb0JSjQ2lAiEnsKi4S2/ekHtyEc4NaqsyISXwqLhO56FqW6noWIxJvCIqHbMpRmcItIvCksErotQ2khQRGJN4VFQltL+tNmQQsJikjsKSwS2varDCUikoHCIqGtJXMZSvMsRCTmFBYJWQe4VYYSkfhSWCR0W4YqVc9CRGJNYZGQrQylnoWIxJjCIiFbGcrbwL132yQicphQWCR0N8+itOzAPiIiMaSwSMi2kCDo9FkRiS2FRUK2eRagcQsRiS2FRUK3Zahwu5b8EJGYUlgkqAwlIpKRwiKhuzJUR89CYSEi8aSwSOhunoV6FiIScwqLhGzzLEBXyxOR2FJYQBAC3tb9lfJAZSgRiS2FBRwIge6uwQ0qQ4lIbJUVugGF1tS8l989sY4rgb82vMua9zd22eeYnW/xReA3T77CW1XlkbVl+MD+XPrJaswssmMkrPnnO/x1w7aOxyeNG8JZHz0q8uOKSN8U+7DYuusDlj+5iSv7w99eaebOTa902We6beOL/WDV8/9kjUcTFu3uuMOpE4Zz3MiqSI6RbPH963lh87uUlhht7c6Rlf1Z84OzeyWoRKTviX1YnDh2CHWLzoCfwbXnTeHa2s923enVSlgO9/z3j0P1aZG04x9vv8fMGx+nfktz5GHR2tbOhqZdXH7aeP73uTXc8dTrXLOqnrd27ePowRWRHltE+qZIxyzMbLaZbTSzBjNblOb5AWZ2d/j8M2ZWHW7/jJmtMbOXwttPR9lO2lqC2wLOszhmeCUDykpYt2VXZMdIeG37Hj5obWfSqEEAHbe9cWwR6ZsiCwszKwWWAOcANcBFZlaTstvlwE53nwjcCNwQbt8OfN7dJwOXAndG1U4gaYA72wzu6Jb7KCst4fiRVdT3wi/sxDFqwpA4/uhBmNErxxaRvinKnsU0oMHdX3X3FmAFMCdlnznAHeH9e4GzzMzc/Xl33xJurwcqzGxAZC3t6FlkmZQX8amzNaMGU7+lGY/4uhnrmnbRv6yECSMGAjBwQBnVwyrVsxCRjKIMi9HA5qTHjeG2tPu4eyvQDAxL2ed84Dl3/yD1AGa2wMzqzKxu27ZtqU/nLtcyVMSrztaMGsSufa28+e7eSI9Tv6WZ446qol/pga+/5uhB1Dc1R3pcEem7Dut5FmY2iaA09a/pnnf3W9y91t1rR4wY0fMDZS1D9c48i8TYQZTlIHdn3ZZdHcdKqBk1iM3v7KV5r+aSiEhXUYbFm8DYpMdjwm1p9zGzMmAwsCN8PAZYCXzF3buez5pP2cpQvbRE+fEjqzCLdqC5qXkfO9/f3zFekZB4vL5JpSgR6SrKsFgNHGtm482sP3AhsCpln1UEA9gAc4HH3N3NbAjwZ2CRuz8ZYRsD2cpQJaXBbcQ9iyP6l3HM8MpIexaJIErtWeiMKBHpTmRhEY5BLAQeAtYD97h7vZktNrMvhLvdCgwzswbgKiBxeu1CYCLwQzNbG/58KKq25lyG6oW1oSaNGhzpX/f1W3ZhBseP7BwWH6oqZ/jAATojSkTSinRSnrs/ADyQsu2HSff3AfPSvO7fgX+Psm2d5FqG6oXLqtaMGsSqF7awc08LQyszhNchWNfUzPhhlVQO6PrVTxo1iHUqQ4lIGof1AHevyVqGin6eRcKkiMcO6rfs6jJekVAzahCb3n6PD1q1FLuIdKawgBxWne29Jcprjo7ujKjmvftp3Lk3c1gcPYjWdmfT27vzfmwR6dsUFnAQ8yyiD4thAwcwclB5JOWgA4Pbg9M+r0FuEclEYQE5zODu3Svl1YwaRP2W/E+QSwRQoveSqnpYJUf0L9W4hYh0EftVZ4EczoYKT52tuw02PZz7+35kNpx+9UE3Z9KoQfztH9t4ZN3blJbkb8nwx/+xjQ9VDWBEVfqVU0pKjI8ePYhnXnuHv27Ymrfjiki0RlQN4ITR6SsG+aKwgOw9CzM4dSFsXZf7e25dD88t71FYnDRuCG3tzhXL6w76tdnMmtT9BY5OGjuEpU+8xvzbV+f92CISjXM/djQ3f3lqpMdQWED2MQuAWdcd3Hs+9L9g9a3gHoTNQTjzuA/x4Lem80Fr+8EdMwcTRlR2+/zVs47j8yeOItqlDEUkn4ZUZPhDN48UFnCgDFWSx3/wgR+C1r3wwXtQnn6MIBOzoBxUCOX9Sjlx7JCCHFtEDl8a4IagZ1FSBiV5/OcYGJZ79hzCargiIocJhQUEYdFdCaonBoark+x+O7/vKyJSAAoLCMpQmQa3eyrRs1BYiEgRUFhAMNku7z2LRFjoFFQR6fsUFhBNGariSLBS9SxEpCgoLCCaMlRJCVSOUFiISFFQWEA0PQsIBrlVhhKRIqCwgLBnEUVYHKWwEJGioLCAsGcRwQxIhYWIFAmFBURbhtqzFdrzv2yHiEhvUlhAtGWo9lbYuzP/7y0i0osUFhBhGUqzuEWkOCgsIMIylGZxi0hxUFhANPMsQLO4RaRoKCwgwp7FiOBWPQsR6eMUFhBdWAwYBGXlwRlRIiJ9mMICoitDmWkWt4gUBYUFRNezgHBinspQItK3KSwgunkWoFncIlIUFBYQ3TwLCMtQ6lmISN+msIDoy1Dv7wh6LyIifZTCor0NvD3CsAhnce/ZFs37i4j0AoVFW0twG1kZShPzRKTvU1h0hEVEPYvKxPpQCgsR6bsUFomxhKjLUBrkFpE+TGEReRlKYSEifV+kYWFms81so5k1mNmiNM8PMLO7w+efMbPqpOe+H27faGazImtk1GWofhUwYLDKUCLSp0UWFmZWCiwBzgFqgIvMrCZlt8uBne4+EbgRuCF8bQ1wITAJmA383/D98i9RhiqJqGcBmmshIn1elD2LaUCDu7/q7i3ACmBOyj5zgDvC+/cCZ5mZhdtXuPsH7v4a0BC+X/5FXYYCzeIWkT6vLML3Hg1sTnrcCJySaR93bzWzZmBYuP3plNeOjqSVUZehIOhZbPgzLEn9+CIieTDxbJh1XaSHiDIsImdmC4AFAOPGjevZmwwYBDXnwaBReWxZipMvA2+L7v1FJN6i/P0VijIs3gTGJj0eE25Lt0+jmZUBg4EdOb4Wd78FuAWgtrbWe9TKYRPgS3dk3+9QHHNG8CMi0kdFOWaxGjjWzMabWX+CAetVKfusAi4N788FHnN3D7dfGJ4tNR44Fng2wraKiEg3IutZhGMQC4GHgFJgmbvXm9lioM7dVwG3AneaWQPwDkGgEO53D7AOaAW+4a46johIoVjwh3zfV1tb63V1dYVuhohIn2Jma9y9Ntt+msEtIiJZKSxERCQrhYWIiGSlsBARkawUFiIiklXRnA1lZtuAfx7CWwwHtuepOX1FHD8zxPNz6zPHx8F+7g+7+4hsOxVNWBwqM6vL5fSxYhLHzwzx/Nz6zPER1edWGUpERLJSWIiISFYKiwNuKXQDCiCOnxni+bn1meMjks+tMQsREclKPQsREckq9mFhZrPNbKOZNZjZokK3JwpmNtbM/mpm68ys3sy+FW4/0sz+n5ltCm+HFrqtUTCzUjN73szuDx+PN7Nnwu/87nAJ/aJhZkPM7F4z22Bm683s1Dh812b2nfD/98tm9lszKy/G79rMlpnZVjN7OWlb2u/XAv8n/PwvmtnUnh431mFhZqXAEuAcoAa4yMxqCtuqSLQC/+buNcAngG+En3MR8Ki7Hws8Gj4uRt8C1ic9vgG40d0nAjuBywvSquj8AviLux8PnEjw2Yv6uzaz0cCVQK27n0BwWYQLKc7v+nZgdsq2TN/vOQTXAzqW4Kqiv+zpQWMdFsA0oMHdX3X3FmAFMKfAbco7d29y9+fC++8R/PIYTfBZE5cJvAM4rzAtjI6ZjQE+BywNHxvwaeDecJei+txmNhg4neBaMbh7i7u/Swy+a4Lr81SEV908AmiiCL9rd3+c4Po/yTJ9v3OA5R54GhhiZkf35LhxD4vRwOakx43htqJlZtXAScAzwFHu3hQ+9RZwVIGaFaWfA98D2sPHw4B33b01fFxs3/l4YBtwW1h6W2pmlRT5d+3ubwL/CbxBEBLNwBqK+7tOlun7zdvvuLiHRayY2UDg98C33X1X8nPh5WyL6tQ4MzsX2Oruawrdll5UBkwFfunuJwF7SCk5Fel3PZTgr+jxwCigkq6lmliI6vuNe1i8CYxNejwm3FZ0zKwfQVDc5e5/CDe/neiShrdbC9W+iHwK+IKZvU5QYvw0QT1/SFiqgOL7zhuBRnd/Jnx8L0F4FPt3fTbwmrtvc/f9wB8Ivv9i/q6TZfp+8/Y7Lu5hsRo4Njxjoj/BgNiqArcp78I6/a3Aenf/WdJTq4BLw/uXAvf1dtui5O7fd/cx7l5N8N0+5u4XA38F5oa7FdXndve3gM1mdly46SyCa9kX9XdNUH76hJkdEf5/T3zuov2uU2T6flcBXwnPivoE0JxUrjoosZ+UZ2afJahrlwLL3P26Ajcp78zsNODvwEscqN3/T4Jxi3uAcQQr9n7J3VMHzoqCmc0Arnb3c83sGIKexpHA88Al7v5BIduXT2Y2hWBAvz/wKjCf4A/Dov6uzexHwAUEZ/89D1xBUJ8vqu/azH4LzCBYXfZRn8xHAAACT0lEQVRt4Brgj6T5fsPgvJmgJPc+MN/d63p03LiHhYiIZBf3MpSIiORAYSEiIlkpLEREJCuFhYiIZKWwEBGRrBQWIiKSlcJCRESyUliIpGFmbWa2Nuknb0t6m1l18rUIcnzNGDO7IF9tEDlYZdl3EYmlve4+pdCNSHIWwTVX7i50QySe1LMQyVHYI9hgZneFV6C718yOCJ+7KrxC28tm9u2k13wlvELZC2Z2Z9LblZrZr8Mruz1sZhXdHPc04GfA3LCXc0xkH1IkAy33IZKGmbURrKWV8B8Ea2m9Bpzm7k+a2TIOLFZ3O8FVCC3c7xKgBVgJfNLdt5vZkeF6PdVAA8FV3daa2T3AKnf/TTft+QvB2lYHVb4SyReVoUTS61KGCn/Jb3b3J8NNvyG4lOd+YKW77wn3+wMwneCaAr9z9+0AKQv3vebua8P7a4DqLO05DtjQ0w8jcqhUhhI5OKld8Z52zZNXPm2jmz/czGw4wdLSrZn2EYmawkLk4Iwzs1PD+18GniBY/v288FoKlcC/hNseA+aZ2TAAMzsy25ub2aNmlnrZy2pgS57aL9IjKkOJpFdhZmuTHv8F+BWwEfhG0njFL939fTO7HXg23Hepuz8PYGbXAX8Lx0CeBy7LdEAzKwEmAqnXmdgADA9Pt13g7k8d6ocTOVga4BbJUThmcb+7nxDR+58AfNXdr4ri/UUOhcJCJEdRh4XI4UxhISIiWWmAW0REslJYiIhIVgoLERHJSmEhIiJZKSxERCQrhYWIiGSlsBARkawUFiIiktX/ByXu2//Vg9cAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w_hat = np.zeros(X_train.shape[1]); b_hat = 0\n",
    "n_epochs = 100\n",
    "\n",
    "# Initialize arrays to store errors for each epoch\n",
    "train_error = np.empty(n_epochs)\n",
    "heldout_error = np.empty(n_epochs)\n",
    "\n",
    "for t in range(n_epochs):\n",
    "    # here we use a learning rate, which decays with each epoch\n",
    "    eta = 1./(1+t)\n",
    "    w_hat, b_hat = train(X_train, Y_train, 1, w_hat, b_hat, eta=eta)    \n",
    "    train_error[t] = evaluate(X_train, Y_train, w_hat, b_hat)\n",
    "    heldout_error[t] = evaluate(X_test, Y_test, w_hat, b_hat)\n",
    "\n",
    "plt.plot(train_error, label = 'Train error')\n",
    "plt.plot(heldout_error, label = 'Test error')\n",
    "plt.legend()\n",
    "plt.xlabel('Epoch, $t$')\n",
    "plt.ylabel('Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Does the test error track the training error closely? \n",
    "\n",
    "**Answer:** *Yes, the train/test errors are quite close. This suggests we may not have a problem with overfitting.*\n",
    "\n",
    "**Question:** Has the model changed significantly after training for more epochs (i.e. more than 5)? You can plot the new decision boundary by running the code block below.\n",
    "\n",
    "**Answer:** *The answer depends on the data set. For `class_sep=2` the model is relatively stable and converges quite rapidly. However for `class_sep=0.5`, the model is highly unstable.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEMCAYAAADAqxFbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8FPX9P/DXezebA3JxtiUBiWiBAjHBgEcg5bCA5ZDaihWppfqV4lergoXijX6/7ZcWfx6IlCJYv63Vr6CAFQ8oQgTFo+EwoECRQ0igENCcJOT6/P6YzGZnd2Z3ZnZmZ3fzfj4eeSQ7mZ39ZJN83vO53h8SQoAxxhiTuZwuAGOMsejCgYExxpgCBwbGGGMKHBgYY4wpcGBgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpsCBgTHGmEKC0wUwo3v37qJv375OF4MxxmLKzp07zwoheoQ6LyYDQ9++fVFSUuJ0MRhjLKYQ0Vd6zuOuJMYYYwocGBhjjClwYGCMMaYQk2MMjLHY1dTUhLKyMjQ0NDhdlLiVnJyM7OxseDweU8/nwMAYi6iysjKkpaWhb9++ICKnixN3hBA4d+4cysrKkJOTY+oa3JXEGIuohoYGdOvWjYOCTYgI3bp1C6tFxoGBMRZxHBTsFe77y4GBMdZhCCEwYsQIvPPOO95ja9aswYQJEwxf69Zbb0XPnj0xePBgXecXFxdjx44dhl+npKQEd999t+HnhYMDA2Msqq3fXY7CRVuQs+AtFC7agvW7y01fi4iwfPlyzJ07Fw0NDaitrcUDDzyA5557zvC1Zs6ciXfffVf3+cECQ3Nzs+bzCgoKsGTJEsPlCwcPPjPGotb63eW4f+1e1De1AADKK+tx/9q9AICp+Vmmrjl48GBMnjwZv//971FXV4dbbrkF/fr1M3ydoqIiHDt2TNe5x44dw/Lly+F2u/HSSy/h2WefxapVq5CcnIzdu3ejsLAQP/3pT3HPPfegoaEBKSkp+POf/4z+/fujuLgYTzzxBDZs2ICFCxfi+PHjOHLkCI4fP457773XltYEBwbGWNRavPGgNyjI6ptasHjjQdOBAQAeffRRDB06FImJid70OiNHjkRNTU3AuU888QSuueYa068FSGl8Zs+ejdTUVPz6178GAKxatQplZWXYsWMH3G43qqursX37diQkJGDz5s144IEH8Prrrwdc68CBA9i6dStqamrQv39/3HHHHaanpWrhwMAYi1onK+sNHderc+fOuPHGG5GamoqkpCQAwPbt28O6phk33HAD3G43AKCqqgo///nPcejQIRARmpqaVJ8zceJEJCUlISkpCT179sTp06eRnZ1tabk4MDBmsfW7y7F440GcrKxHr8wUzBvfP6y7246sV2YKylWCQK/MlLCv7XK54HK1D7MGazH0798fkydPBgDMnj0bs2fPDvv1ASlAyR5++GGMHj0a69atw7FjxzBq1CjV58iBDADcbnfQ8QmzODAwZiE7+sQ7snnj+yveTwBI8bgxb3x/y18rVIthz549uq6zdOlSAMBdd92lOJ6Wlobq6mrN51VVVSErS/obefHFF3W9ll14VhJjFgrWJ86Mm5qfhf+5fgiyMlNAALIyU/A/1w+JiiB700034aqrrsLBgweRnZ2NVatWAZDGALp16xZw/uTJk7Fu3Trk5eWpBqH58+fj/vvvR35+vi2tACNICOFoAcwoKCgQvB8Di0Y5C96C2n8UATi6aGKkixOV9u/fj4EDBzpdDNtMmjQJa9euRWJioqPlUHufiWinEKIg1HO5xcCYhbT6vq3oE2exYcOGDY4HhXBxYGDMQvPG90eKx604ZlefOGN24cDAmIWioU98+fuHsePwWcWxHYfPYvn7hyNWBhbbomZWEhG5AZQAKBdCTHK6PIyZNTU/y9HB0dzsDNz18m4snZ6Pq/t1x47DZ72PGdMjagIDgHsA7AeQ7nRBGItlV/frjqXT83HXy7sx44o+eOmT494gwZgeUdGVRETZACYCWOl0WRiLB1f3644ZV/TBki1fYsYVfTgoMEOiIjAAeBrAfACtTheExTYrM3HGsh2Hz+KlT47j7jGX4KVPjgeMOXRU4aTdXrNmDQYNGgSXywU90+UrKyuxbNkyU+X84Q9/iMrKSlPPtYLjgYGIJgE4I4TYGeK8WURUQkQlFRUVESodc5LRSl5edVxeWQ+B9lXHHS04+I4pzB3X39utFLPBoXQ18NRgYGGm9Ll0telLhZN2e/DgwVi7di2Kiop0vVawwBBqAdvbb7+NzMxMXa9jh2gYYygEMIWIfgggGUA6Eb0khJjhe5IQYgWAFYC0wC3yxWSRZCa1hF2ZOKOF3hxMpWVVijEFecyhtKwq9rqUSlcDb94NNLXlS6o6IT0GgNxppi5pNu220UV5CxYswOHDh5GXl4cf/OAHmDhxIh5++GF06dIFBw4cwL/+9S9MnToVJ06cQENDA+655x7MmjULgJSNtaSkBLW1tbj22msxYsQI7NixA1lZWXjjjTeQkmLvuhjHA4MQ4n4A9wMAEY0C8Gv/oMA6HjOVvF2ZOKOBkUA5+/uBldzV/brHXlAAgPcebw8KsqZ66bjJwABEJu32okWLsG/fPm+OpeLiYuzatQv79u1DTk4OAOCFF15A165dUV9fj2HDhuHHP/5xQDqNQ4cO4ZVXXsHzzz+PadOm4fXXX8eMGfZWkY4HBsbUmKnk7czEaRe9rQCnW0OOZYytKjN2XCen0m4PHz7cGxQAYMmSJVi3bh0A4MSJEzh06FBAYMjJyUFeXh4A4PLLL9e9OVA4oiowCCGKARQ7XAwWBcxU8uFm4ox05WekFeBka8jRjLEZ2VL3kdrxMBlJux3uRj0y3zTbxcXF2Lx5Mz766CN06tQJo0aNQkNDQ8Bz/NNs19fb/zt3fPCZMTVmUkuEs+rYiYFrI5lYnczB5GjG2LGPAB6/n9GTIh232Pbt27Fnz56Aj1BBoby8HGPHjg04npaWphpoZFVVVejSpQs6deqEAwcO4OOPPw77Z7AKBwYWlcxW8lPzs/DhgjE4umgiPlwwRvcdrROVn5FWgJM5mBwdu8mdBkxeAmT0BkDS58lLwhpfMGvdunXIzs7GRx99hIkTJ2L8+PEAgFOnTiEhIbDzpVu3bigsLMTgwYMxb968gO9PmDABzc3NGDhwIBYsWIArr7zS9p9BL067zRicSZdduGiLandZVmYKPlwwJuC4VV1dRq+jVU43EVqFMFyWeEu7vXTpUvTp0wdTpkxxuigK4aTdjqoxBsac4sTAtdExEStyMJkZL1ArJwC0tN1UdvRd6vx3aosH3JXEGJzpqnEiE6uZLjP/crqJAs7hXeriC7cYGEP7na6eLhajXTHBzo90Jlaz4wW+5cxZ8Japa7DYwYGBsTZ6KmmjXTFa55d89TU2fHYKlfVNAIAunTx4dPIg24OEFV1msbhehBnDXUmMGWC0K0br/Jc+Pu4NCgDwzfkmzHvtM9vzOlnRZca71MU/Dgws4mJ5hzGjXTFGuleaWoTt/fRWjGtEwy51zF7clcQiLpZ3GDPajZKWnIDqhuCZNH3JgURtXAJoHwPJ7OSBEEBVfZPh6aJWjGs4vUudWUIIjBw5Eg8++CCuvfZaAFI67VWrVuHdd981dK1bb70VGzZsQM+ePbFv376Q5xcXFyMxMRFXX3214XIfO3YMO3bswPTp0w0/1wxuMbCI891h7MlNBxVBItoZ7Ub52ZUXGbp+ZieP6irseWs+w7zXPvMe++Z8Eyrrm+I+vbjVrctw0m77mzlzpqFgUlxcjB07dhh+HUAKDC+//LKp55rBgYE5wu4dxuzasMdoN8q8CQNw56h+cLXN8HQRcM3AnvC4Aqd8AkBtQzMee/PzgHGJplaBphbtxajxOl1Ubl3KwUFuXeZmZ5i+pm/a7ccff1x32m1/RUVF6Nq1q65zjx07huXLl+Opp55CXl4etm/fjoqKCvz4xz/GsGHDMGzYMHz44YcAgPfffx95eXnIy8tDfn4+ampqsGDBAmzfvh15eXl46qmnDJfVKO5KYo7w32Hsyn7dLAsOdid9M9qNMm/CALhdhCVbvsRdoy/B3HH9sX53Oea8uidgtXVTq8A355tUrxNKPE4XtWv/6kik3fbVt29fzJ49G6mpqfj1r38NAJg+fTrmzJmDESNG4Pjx4xg/fjz279+PJ554As899xwKCwtRW1uL5ORkLFq0CE888QQ2bNgQVjn04sDAIs53TOHqft1xZb9ulnYnGUlRbVWaieXvH0Zudoai/DsOn0VpWRVyszMCguDU/CzMeXWPuR9QQ7xOF/VtXd495hJL/kacSrvta/Pmzfjiiy+8j6urq1FbW4vCwkLMnTsXN998M66//npkZ4efSdYo7kpiERdshzEr6J0hZGVGVa0uD7cLmttsalXkmSmegHEMj4vgcat3PwFAgoswNa+X4liszPQKxa79q9XSbstdOL4fmzdvxokTJ7yPly9fbsnrt7a24uOPP/ZmcS0vL0dqaioWLFiAlStXor6+HoWFhThw4IAlr2cEtxhYxNm9w5jWzKHMTh4ULtribR2cb2xWbVk89ubnhlsNWl0e/kHwTPUFuABMf/4TdOnkgcdFaGpt71BK8bixcMogAIGrsH2P+c9KmprXC6/88wQKL+0eczO9grG7dekrVItB3oktlKVLlwIIzKGUlpaG6upq7+Nx48bh2Wef9WZe3bNnD/Ly8nD48GEMGTIEQ4YMwT//+U8cOHAAvXv3DprC22ocGFjcUUv65nETahuavf33aoFD9s35JqzfXY6p+VmGupr8uzzOVF/AXz/6yvvc0QN64PWd5d5yfXO+CR43ITPFozrtVO11gmZBvbS75X3xTovm/atvuukmFBcX4+zZs8jOzsZjjz2G2267DQcOHEBhYWHA+ZMnT8ZPfvITvPHGG3j22WexZMkS3HnnncjNzUVzczOKioqwfPlyPP3009i6dStcLhcGDRqEa6+9Fi6XC263G5dddhlmzpyJOXPm2PqzcdptFpfW7y7Hwr9/7l1dTAQY+VPPaquk1bKfas1Cun9tKTaUnsIvru6LVR8eRVOzQGNLq/f7BKim9tZKs23Gk5sOegPT3HHmVyIHGzNRa/EZEW9pt/1NmjQJa9euRWJioqPlCCftNo8xsLh1obm9UjZ6/3Oyst5Q+osdh89iQ+kpAMCV/bohJcGtCAqAelCQX0u+RjhjAlb2xdsxTbSj2LBhg+NBIVzclcQsY+ddplFqlboarbv4XpkpmoPY5ZX1yHtsE4ik7qD05ARkZabgTz+7HADwy7/uRI2B1c69MlMCxgSMvpdW98XbNU2UxQZuMTDLRNNdpp45/SkeN26+so/mSubMTh7N51bWN3nHK6obmnHoTC3OVF8AADT5tRR8qc0r6trZgxkrP8HXdY342cpP0XfBW1i57Qhu/0uJ7vfSjpledi9CZNGLWwzMMlbdZVqxtkBrZpLadpQFF3VVfb2Ff/9c9+s1two8vH4fQIDH7ULRpT2w6YvTinNSPG78+PIsbD1Q4X2trp092FvePlNF3hXtbF0jEt0u3P6XEtxWmIM/7ziGi3t0DnjdYK2IcGd62bkIkUU3DgzMUuEuRjK6almry2Von0x8Xdeoa+BYayVzVb2xFcg1F5qR7HHhhZnDcHW/7lj87gEsKz4MgfbB7Kn5WYqFbzNWfqJ5vcaWViQlJGDJli+R7HFhUu53IpZ8MJLTRFn04a4kZqlwB0CN7neg1X110xV9TKeGXr+7HPmPb9IcLA7mQlMrPjwklWXehAF4YOIApHjcWHxDrjco3PZiCbb9qwIzVn6C1hAvIgcbj9uFQb0yIpZ80O5FiCy6cWBglvG9y/Rf5auX0X0NgmVqnZqfhQ8XjMHRRRPx4YIxuoPCvNc+M52vSAB4rvgwFr8rrVa9fWQ/rJpZoCjfwO+kYcfhcyGDgmzWyIvxp59djrte3g0AEen3n/39fgHXvrpf94hPIrCaEAIjRozAO++84z22Zs0aTJgwIeRz16xZg0GDBsHlckHPdPnKykosW7bMdFmffvppnD9/3vTzw8GBgVnGirtMrTQRvTJTNDOmWjlIunjjwaBZTPX668dfeb/2LV/BRV2w+3il7uuM+9638NInxwEAS6fn483PTtqSHiJqffA0cHSb8tjRbdJxE8JJuz148GCsXbsWRUVFul6LA0MYiKg3EW0loi+I6HMiusfpMjFzrLjL1NrvYPSAHpp5jaycv29VhlLf6aq+5du8/3TQLio3SfOWXATcOaofVtxS4G0RfX6yChs/Px1WiyzmZA0F1sxsDw5Ht0mPs4aavqTZtNsDBw5E//76Fw0uWLAAhw8fRl5enjftxeLFizFs2DDk5ubi0UcfBQDU1dVh4sSJuOyyyzB48GC8+uqrWLJkCU6ePInRo0dj9OjR5n7QMETD4HMzgPuEELuIKA3ATiL6hxDii1BPZPFH7u7xnyWkNfbw3xu+QCtg2SCp1mwmmcdFACFkq0Ju+fgP4i7Z8qXmc+QV0P4D6nLLa8W2I1GbHsI2OUXADS9KwaDgNqBklfQ4R99du5ZIpN1etGgR9u3b582xtGnTJhw6dAiffvophBCYMmUKtm3bhoqKCvTq1QtvvfUWAKCqqgoZGRl48sknsXXrVnTvHvnfreOBQQhxCsCptq9riGg/gCwAHBjigJmpp2qzhLRSVJ+ta8TLt1+hu7IMVZ554/tj3mufaVb8Nw7vrZjemtnJg9qG5oBEeHLSO9/utfW7y+Em8k5JDfi527KjGpl6amXywaiVUyQFhW1/AIrmhx0UAGfSbm/atAmbNm1Cfr40i6y2thaHDh3CyJEjcd999+E3v/kNJk2ahJEjR9paDj0cDwy+iKgvgHwA2nP4WMywcsMcrTv5rMwU1e4rraAQqjzy57mr96gODm89UIEfDvkOfnbVRd4KPFiw8T3n/rV7NYNCopvwXPFhrN9z0vSeEHHr6DappVA0X/qcM9KS4KCWdtuujXoAaeD7/vvvxy9/+cuA7+3atQtvv/02HnroIYwdOxaPPPJI2K8XjqgJDESUCuB1APcKIapVvj8LwCwA6NOnT4RLx8wwsmFOKKoZU12E843NyFnwlmZrxLfSdqncrauVJ9gmOuWV9bj9LyV4/pYCxfmhfh617ToBaSzBTYTGthZKsOBp1aZCMUUeU5C7j3JGKh9byGyLoby8HLfccgvee+89xfG0tDRFoBk/fjwefvhh3HzzzUhNTUV5eTk8Hg+am5vRtWtXzJgxA5mZmVi5cqXi+U50JTk++AwAROSBFBT+JoRYq3aOEGKFEKJACFHQo0ePyBaQmWJ06mkw/nstZ6Z4gLZcRVqb7PhvxKN1t65WnmDpMJqahTf9hZ7Ed+t3l2tOf20VUHRDAerrNqzcVCimlO9SBgF5zKF8V8SLsm7dOmRnZ+Ojjz7CxIkTMX78eADAqVOnkJAQeI/drVs3FBYWYvDgwZg3bx7GjRuH6dOn46qrrsKQIUPwk5/8BDU1Ndi7dy+GDx+OvLw8PPbYY3jooYcAALNmzcKECRMcGXx2PO02ERGA/wXwtRDiXj3P4bTbsaFw0RbN7p9w00xrXds35cX5xmZd6xH8y7N+dznmrfksoML2f87iG3J1DXRrlTWUY4smhryGlSm7IyXe0m4vXboUffr0wZQpU5wuisL+/fsxYMAA1NfXo6amBklJSejSpYuutNvR0JVUCOBnAPYSkdx+f0AI8baDZWIW0NrPQB6YlZnpItFqdcitAiMVsX95Fm88GDQoyNe/6+XduGPUxSFnBQVrIWlld5WnrYa6hlXTa5l5/ju1RZIQAq2trWhpaQn4XFdXhxUrVqClRfr/y83N1X1dxwODEOIDqCedZDFOa+qpb6VvdoA61LRSIxZvPIg5r+7xlk9vZfv973bHH4uPhMxVpLnVaIrHu5GQP/9uL61raC0I7JDjEXFICKFa6ft+7U8eVHe5XBgyZAjS0tKQlpaGrl276n5dxwMDi2+hBmbNDlCrtUbMkitcOShlBKmwZV07ebB+90k8MHFAyOmiWmUlArp08qh2d2X5Vfh6W1+AtbPBmL1aW1s17/jlr/3J23x6PB4kJSXB7XZ7j/nOtDp9+jSGDjW3EJADA3OU2S4S/9aI2owjM+qbWpDscSHF41ZUwnKXj4uAoX26oOSrb/Cj/Cz8sfgIBvXKCBoc5LL6bjUKSAPnLo228ugBgRMskj0ub5kyUzxYOGWQakVv5WwwuwghQBT/HQWhKn21it/tdsPtdiMxMTGg0ne73bret3DHjjkwMEcZ7SLx5dsaWb+7HHNe3aPaX9+9cyLqm1tQ39iiK3Fd5fkmPHVjXkBXzL+rG+B2AX8sPuJNv+E7xhCs+2ZqfhYWbzwY0BLRKs/WAxXer/1bAIBy21J/0T4ekZycjHPnzqFbt24xHRyEEN6uHq3K37+CJiLFHb9/pe9yucJ+T4QQOHfuHJKTk01fgwMDc5SRLpJgpuZnYUPpSWzefybgezcO6w23i7Bky5dIS0pA7YVm9MpMQd2FZtUuo16ZKd6gI1f2c17dg25tAeb5WwoC0m/o6b4xUjH7nmu0BRBOsI2E7OxslJWVoaKiIvTJDmttbfUO8Mqffb9WI3fnyEHA/3MkJCcnIzs72/TzOTAwR+kZoPYl5xGSN7qRN6wpLatCj7QkXHVxVxz/ut57ral5vfD1+UZs/Py09y7/b7dc4b3DDxaU/L8v76omr2HwTb/x14++Cll5Gxkw963EjbYArAq2dvF4PMjJyXG6GN6ZOzU1NaoftbW1AZV/cnKydzA3LS0NqampSEtLQ3p6uje9Riy3gmQcGFhQRjelN0PPymGZvDHPHaMu9n7+Y/ER3DHqYm/mUf+y+u9EdtuLJZg77lLcPlIq/+KNB1FeWY/05AQ8ft1gRbDyr+wbW1oVlb2cfuP37xxQLa9v5a26ettNgN8iN/9K3GgLwGiwjVfNzc2KSt630q+ursb58+cDuno6deqEtLQ09OzZE/369VMEgbS0NHg82gsf4wkHBhaUXBFHYjtJPXw35vn+d7vjd28dwNT8LDy9+RBSEty4+flPFBWh2h4Rc8ddiic3HcKgXhmYmp+FnulJqgvVjNyp66m8tSpstWP+if2MtgCMBNtY1djYqHm3X1NTg/p65e+DiLx3+NnZ2d6vfT/cbrfGq3UsHBhYUL4V8Ywr+uClT447vu+v78Y3w/t2wbrd5UhwEeouBPbvq7Vqbh/ZD4N6ZYT8mYzcqeutvLUqbPnY8vcPo2d6kuJ7PdOTcM3Anth1vLLDtACEELhw4YL37t7/jr+mpgYXLlxQPMflcnkr+L59+wZU+p07d45YH3+s48DAQvKtiO8ec4njaZ7ljW9+lN8L63efRIrHhfomZV9wqOmZen4mI3fqVnXfBGuhPTvd/OY0EVW6GnjvcaCqDMjIBsY+AuROU5wihMD58+c1+/ZramrQ1KScGJCQkOCt5L/1rW8FVPydOnWKi/79aMCBgYXkv0Palf26ORYc5IpSHlt4YOIA/Pat0P37atcJ9TP5V/Y/T/0U8z2votMb/waKAys8K7pvorGFZkjpauDNu4Gmtve+6gRa3/gVjh05gq8yrlBU/v6rdpOSkpCWloaMjAxkZ2cHDO7Gy8BuLODAwIJSG7wNZ4e0cMljBr5jB38qPoKzdY0B52oNzhr5mbyVfelqLFtzDHsudMHV7lNA1Qk0v/ErfHrahdLEfN0D8XoG86OthaampaXFe2cvd/VUV1fj6k8WoFOTMiC7WhrQvfSPeL+3dJffvXt35OTkBNzxJyYmOvTTMH8cGFhQaoO3Tm4n6Vt5yh6a9D1Dg7Nmfqbz7zyCPHTBXU13YymW4Gr3F/i08WLcuaUFz/1HBgB9lb6ewfxoaKE1NTUFHdj136SeiNC5c2ekNJ1TvV5aaxV+8YtfRKLozAKOp902g9NuM392J41rXZgJFwR2tHwPdzXdjRnuzXip5Ro863kWhf+1A0BgS8T/sUw+rtZVpPca4ZIHdrUGdxsaGhTnu1wu1Vk8vgO7brcbeGowUHUi8AUzegNz9llWfmYOEcVM2m3Gwqa3f99sADnZ2g3ZrrO42v0FBjZ/hSUt1+Nu91pcRKcBtLcM9IwPBOsqsqKFJoTw5uDXGtxtbFR2vSUkJHgr/h49eqgO7Oqa0TP2EeUYAwB4UqTjLGZwYGAdRjhZR1cmzsD8pmX4r6YZ2CUuQTIu4IWW8TjoGYiC7Yfx5KZDWDWzQNf4QLCuIrWxCv89rFtbWzVn9MgVf3Nzs+IaiYmJ3kHcrKwsxcrd9PR0JCcnWzOwKw/Gh5iVxKIbBwbWYYSTdTRv4izMfr0Tdrb2QwJa8XPXO/hT6xRsbhyITW8d8KbfDjU+oGfgWx7YVRvclY/7p2pISUlBamoqunbtiosuuijgjj8pSbk2wpQPngayhir3Wj66Tdpmc4TP5ou50zgQxDgODKzDCCfr6NT8LLy3/wdIOXwOZ+sasUxcj77dO+PI2ToM6pWO20f201Xpl5ZV4ZlpuRjY1Y3jx48jo6EGd+Yl4/WtJfj3Z1IroK6uLuD1O3fu7J2/369fP29unoimasgaCqyZ2b4H89Ft7Y9ZXOHAwDqMcLOO+i4wm/PqbqzbfRLD+3bBlxV1ijGGgt7pOHfuHHq5a3HflZn4+wefoeZLKW9PUk0N9n5ej70+1yUi5KWmwuVK887f90/UFhWpGnKKpCCwZiZQcBtQsqo9SLC4woGBxSW1QWYrso4KIbB867+wfvdJXHNJOj49UYVJOR7MevETTL+oHvu/rMIev1QNOW43zp2TKviYT9WQUyQFhW1/AIrmc1CIUzxdlcUdrXTa/3P9EADB01YES9VQU1ODvWcuYPW/u2FERiWGp9fgq4YkvHm2B0Z/uwmexGRc/73AqZwpKSnxs2JX7j4Kp8WgI2UGs4fe6aocGFjcKVy0RbXLKCszBdvnj1JNyOZ7zH9gNykpydun//7pRAzqlYrCS9qndO4qq0VpefhpyO1eixE23zEF/zEGvcHBP2UGIE1nnbyEg0MEcGBgHVJzczMufWhGa+97AAAZy0lEQVSj6hafgMBvLjqhmYNf6yMSA7vBWjlRExz0zkoKhhfAOYoXuLG4JKdq0ErFfP78eaS5s1DdEvin3TWZMHTo0ICB3YQE5/8NwplKGzFqlX9OkbGupKoyY8eZI5z/j2CsjW8Ofq2PYDn4+/Tpg/T0dOC0wDMfVeBCs3JXtEeuG4IroqWS9RPOVNqYkpGt0WIwvz8xsx4HBhYxoVI1WJWDvwBAr15R3l/vJ9yptDGDU2bEBA4MzDKtra0Bm6v7rtitqalRzcGfmpqKjIwMRaoG+cNsqoZY29rSiqm0MYFTZsSEqAgMRDQBwDMA3ABWCiEWOVwkpsI3B7/aR11dnWqqhrS0NHTr1k0xh19etWtJqoY4YNUOcDGBU2ZEPccDAxG5ATwH4AcAygD8k4j+LoT4wtmSdTzywK58l+9/xx8sVcO3v/1t1Rk90TCwGytirZXD4lc0/NcOB/ClEOIIABDR/wG4DgAHBouFGtgNloO/d+/eAfn4oyZVA2PMUtEQGLIA+E5TKANwhf9JRDQLwCwA6NOnT2RKFkOEEGhoaNCcxllTUxOQg9/tdnsr+bBy8DPG4ko0BAZdhBArAKwApAVuDhcn4oQQ3oFdra4etRz88l1+r169Au744ypVA2PMMtEQGMoB9PZ5nN12rENpaWkJmNHjP7vHf2A3OTkZaWlpqjn4U1NTkZyc7NBPw2KGb96ilC7SsfpveLZQBxcNgeGfAC4lohxIAeGnAKY7WyTrNTc3B0zh9M3RU1dXp5mqoWfPnujXr58jqRpYHPPPW1T/dfv3qk5I3wM4OGiJ42SAjgcGIUQzEd0FYCOk6aovCCE+d7hYhjU2NgYd2K2vVy5eIiJv1448f19+LCds44FdZqv3HlcuNPPXVC+dEyeVnaX8g2qcBVLHAwMACCHeBvC20+XQ4puqQWtw1z9Vg9vt9lb0MZ+Dn8UnPfmJOIeROrWgGkeBNCoCg9OC5eCXu338B3Y9Ho+34vdP1ZCens4Du7EsjrsIFLTyFvmfwwLFeTLADhEYfFM1qN3x19bWqqZqSEtLQ0ZGhne7xdTUVKSnp3tX7HLFH4es6iJwKrgYSY2tlrfIF+cw0hbnyQDjIjC0tLRobroiP/Yf2JVTNXTv3h05OTkBXT2JiYkO/TTMUVZ0ETjZ/5w1VHszHX/+eYt4VpJ+cZ4MMCY36hk4cKB45plnFDn4fRGRN1WD/4Aup2pgQS3MBFS3+SFgYaW+azi9GY0V22+y0GKwy9H2jXqI6DdCiN+bfX44zp8/jzNnznhz8KsN7PKMHqab7z84uQDREniOni4C73U0+u0j1f+cUyQFhW1/AIrmc1CwSxwnA9QdGIhote9DAHkAHAkMPXr0wIwZM5x4aRZv/Lt91IKCni4Ctb2M/UWq//noNqmlUDRf+pwzkoMDM8TIfMlqIcS0to8bAGy2q1CMRYzWXH5yAyCp+0fPRvWh1gREqv/Zd0xhzIPS5zUzpeNW+ODpwGsd3SYdZ3HDSGD4rd/jB60sCGOO0OreEa3SmMKcffq6C4J1E+kNLlYo36UcU8gpkh6X77Lm+vLgthwc5ECUNdSa63PgiQohAwMRPUNEJIQ46ntcCPG11nMYi3qlq6VBYtWBZhjv9tE6Xx5wzp3W/poLM6XPpauVZfE/HqzcWueOuDew2yinKHCqqllyoFkzE9jyW+UMKCvYHXiYLnpaDDUA/k5EnQGAiMYT0Yf2Fosxk/TcccrjAVqDxGa6fS4dB2noTeM6itcU7VNYN8xVP+5f4ZeuBn6fA6y9PfS5Vgj2PvoObhfcZu34hd2Bh+kSMjAIIR4C8AqA4raAMBfAArsLxpgpeu44g40HmOn2ee1WYPdfEdD66Dm4vaWwbrb6+oidL6ofX/tL6bpAe1CpV2mky2ssrBbsffQf3NY7fqG3m8jOwMN00dOVNBbA7QDqAHQHcLcQYrvdBWPMFD13nJrjAaR/TMHX0W1AS2Pg8cpj7ZW62mwnQPs4WoH9b7ZPgQ02sG3HNFit9xEwP7itt5vIbOBhltHTlfQggIeFEKMA/ATAq0Q0xtZSMRaOUHecmuMBJqeT1p3VPh6qUqcg621aGtvXVwRj1zRYtfcx2OB2qPEPPUHb7llVTBc9XUljhBAftH29F8C1AP7b7oIxZlqoO86xj0j9/77CmU4aLNAEq9Q9KcDlMwPL4kteVRvsGnZNg1V7H7UGt9N76RsrCRW07Z5VxXQxnPdZCHEKwFgbysJY+PTcceZOk8YRMnrD0FoFLWMfAdx+ubXcidJxrUqd3NJrTnpS+qz1ryinWlALHild7ZsGa/TOPViOKf/rBgvads+qYrqY2hBACBGkbcyYg/TeceZOk8YTFlZKFe97j6t3geiZSpr2bSkQdO4BgKTP7kTpuFbr5EfL2yv0tG8DiZ2kdBy+3InSOdUnAwPZ9c8Dv2mbQa53qiugfwDY6J27njTU3E0UM2IyiV5BQYEoKSlxuhgdQwwmCjNELZWFJ6XtLh7a3/N9D0Klug71HsrPP1IMfPCUtLiucw+pRXDukPQ4o3fg84KVXet35Fs5+2dfDWf2j57EgUZSgjNb6E2ix4GBaTNT8cSaYBUaEPksqXJF3ecq4MAG5ff833uzWVztyL7aEf5W4oDewMB7SzJtevuNY1mwLhDN752wZ1EZ0D446x8UgMD33uwuYnasE7B63IY5ijclYNrifPtCAKF34tJaHW3Xxjvy4KwW3/fe7C5idmVfjeM01B0NtxiYNqvn+0ejYFNXtWYDAfa0nHz7++WuLH++772Zabc8AMx04MDAtFk93z8aBesCkb+nxffu3UgiPC2+M4H0vPdmum94nQDTgQefWXBOz0oy8/pWlznUIK9dA69Ov/cs7vCspHjTESoJ/+mMpauBN+5U5iEKVeFaVUn7liXUNe3Y45mndjIb8KykeKKVstmumTFOKF0NfLQU+N/JwOJLpMcbHwhMTheqb9+qmVS+Cd9ypwFX/mf7AjT/Lhs7Bul5XwLmIEdnJRHRYgCTATQCOAzgF0KISifLFJWCVXbx0GrwvyOvqwDW3wG0NqufH6zCtaqSzimS8hj95TppgRm5gBFz1MdXzM4OCvX68sCwlesNGNPB6RbDPwAMFkLkAvgXgPsdLk90ivdpo2qBr7UZARvfyIJVuOHOpJJTRpSuBj5eJgUFQPr88TL1VprWQHHv4crZPqWrpdbQwgx9A9S8LwFziKOBQQixSQgh3xZ+DCA+5kFavW9tvE8b1QxwQj05XWOd9uyfULN5Qv1u5C6cjQ/o75LSmh10+cz27iB5vKSuou1n1tEdaMe+BLynMtPB6RaDr1sBvON0ISxhdf9wvE8b1QpwcjI6OTldSldAiLadzDTGWuRKOqVr+7EEn/cu1O+mfBcwYm57Be5Pa8Gbb1I+ebMf3+6gt+4zNl5i13oDHrtgOtgeGIhoMxHtU/m4zuecBwE0A/hbkOvMIqISIiqpqND4p40WVu9bG+/pBrQC3/jfATe9Alx1l1ThJnYGWpuU52lVrs0+d/v1X7cHkFC/m6yhwAdPAi6PdnlLVyvvvL1rGDKARX2UgUruDrpQrX4trdaSXesNeE9lpoPj01WJaCaAXwIYK4Q4r+c5MTNddctvpf7hovnSXR/TpjUd13fa5sJMBOyrDAAgKXDI9EwfDfa72bEU2BTk95XRG5i6TKpUL58pjT1oTWWV78hbmtSDg5UJ+YxMaea/zQ4pJqarEtEEAPMBTNEbFGIG71trDd+uD71jLVrdPfLdebDfzdFtUosh90btMlWVtd95f/CU9liEb3fQxP8XOF5iZXegkSnN/LfJQnB6jGEpgDQA/yCiPUS03OHyWIPz0Rgb5AxWqfl2fXznssDn+leupasRdDZTqN+NPMbw5WYgKV37OoBUNnnWkr+qMmV3UO404Lrn2sZLYH13oJEd1Dr63yYLyfGuJDOiviuJV60a2xDGSNfPgEnAqc+k8zv3kMYh5Mr16Dbg1RlAQ5VKgQi4foW0G1qw341vOWv+HXzl9dFt7escgpXdCL3dQf7nabWS/LvZ+G+zQ9PblcRpt+2g9g+WU9SxBviMLNAKtU7Dv+vjhhel42tmSttiyuesmakRFABAaN+d+/5u/Ad9AWnqal2Fchc1+fVGzFEfYzDTReS/0E9uOQHBd26rOgGplaRyk+ffzcZ/m0wHDgzMvrtI3wVaRfO1K59gK4f9Wxo5I9sfqwWe9f8ZfEe2UPx/Xq09BnwDSI8BbXfvJ4DkDOCHT5jrItK7wl3tPAgEBId4mtLMIsrpMQYWDeya2653kDPYOo1g0zbVVgZHas3HiHvby+Rdw1AFLDhuftxA7wr3YAsC43VKM4sobjEwe/LyBLvT97+uXHnpnWopd32o7URm9FrRRG/OJc3zbNyLmnUoPPjM2lk5t91M95SRefhGBrfDeZ1IUkvv7fIASWlA/TftZQXs2f+BxT3ej4EZI1esTmXyNLqPgtlxEbs21bGKb9BK6QI01qrPigKiM7ixqMaBgemn9+7bzjttOza7cfJ1ZOG8Z1aWlaepMsTIymcWJfTk5bF7s6BIpRY38zpm93MO9z2z8j3h5HnMAA4MTDnDRpZTpLyTtGpnNC2RSi2e0sXYcbXKfe3twO9zQlfw4b5nVr4nnDyPGcCBgeljxx29b9qMaE0trrpmAMqMrVrCfc+sfk944x+mEwcGpo8dd/RG9lW2Sv03xo4Hq8RD3f1rvTfk0tctZXW6dU6ex3TidQxMn7GPqM/mCeeO3n/9xM4XgVvesPdO1uj+zEHzECF44FB7zwBAtLQ9VyPlhS+tlddGGVlXwjo8bjEwfczevYYauI1094bR7hm1830FazH5v2fkDjzHynEamVpm272vAwMnW7/xD4tL3GJg+hm9e9WTFE5t9bKdwcHoymj5+Du/adtS1IeeFpPve7YwU/0cq2deyV10vtOPD7zZnnxQxsnzmAZex8DsE2oefjirl53gXZNwQrr7Fy3KjKuhRHINhdMLFllU4rTbzHmhZuWESpDnz3+x2KXjgEObzC0eM7PwTP6+ntTYauwYp9GiN7MtYyp4jIHZJ9RMJj3rJ2Rq6wlKVplbPBbOwrNw1iZYPcsoGJ6BxMLALQZmHyvvkLXWE/hS27tA77X0PjfctQlWzTIKhmcgsTBxi4HZx8o7ZL0Vr57zwqnctVpBnbur72ftBD0pThgLglsMzF5W3CF/8LRU8dZVhD5Xz4I7PWsZtJLO9R4O1J5WZjx1J0otjmjJO8Tbd7IwcYuBRb+soVLF604Mfp7ebio9axm0ks5dPhO47jmgcw/peFK6VK6bXuGKl8UNDgws+uUUSRWvO1GqiAGpYi64TV83lf+CLzn9RnKG9nODJZ3LnQbM+1Ia2L1QLV2LgwKLI9yVxGJDTpFUAZvZYU5twdfOF4EbXwpeoQeb8nl0G/DxMilQbfsDsPPPwPjf8WY5LC5wi4HFhnCmX5pNOa31mke3Aa/cJI0zXKiWjtVVAG/cad3+FIw5iAMDiw7Bcir5Tr8c82B7JW80OBjJyRTsNct3SWMSvgPQgPTY6rxHjDmAAwMzv0OZla8fbMGZFdMvjbY4gr3miHuBurPqz7M67xFjDoiKXElEdB+AJwD0EEJo/Me141xJFvJPdAe0bzgfqf5yu3MI2ZGTKdJ7RzNmgZjZ85mIegMYB+C402XpkOzeslMPu/d7tmPBVzi7qzndQmMshGiYlfQUgPkA3nC6IB2SkUrZTOI5PYxunmOUHQu+jKbvlulJRc6YwxwNDER0HYByIcRnRORkUTouvZWyWoX2xp3S1757K8h98EZEMuuolcys6g4nTxNjEWJ7VxIRbSaifSof1wF4AICu/34imkVEJURUUlGhIzUC00dvl4hahdbSCGx8QPpa7rc3kxYikllHnWZ3txljFrC9xSCEuEbtOBENAZADQG4tZAPYRUTDhRD/VrnOCgArAGnw2b4SdzB6u0S0Kq66CmltQLibwUQi62g0sLvbjDELONaVJITYC6Cn/JiIjgEo0DMriVlMT6WsVaHJK3/1bgZj1ziF1ewqZ6x2m7EOxfFZSSzKyTNoqk4A8BsHcicColX/2oBwNsiJJDvL2ZG6zVjMiop1DEbxOoYIUVvjAAIgpCR2TfXtWUX1rA2Ilbn/sVJOxgziPZ9Z+FR3TRNSBTnsP5T7FYTarxmInYHXWCknYzbhwMC0BasgzawNiJWB11gpJ2M24TEGpk2rIjRbQYazWjiSYqWcjNmEAwPTZnUFGSsDr7FSTsZswoPPLLhYmV7KGAuJB5+ZNaJ54RkHLcZswYGBOc9MBc/J6BizDY8xMGeZXUwWDenCGYtTHBiYs8xW8LzWgDHbcGBgzjJbwVs9lZYx5sWBgTnLbAXPaw0Ysw0HBuYssxU8rzVgzDY8K4k5y+wWmfJzORAwZjkODMx5XMEzFlW4K4nFpg+eDtz/4eg26ThjLCwcGFhsyhoq7f8gB4dw9pxmjClwVxKLTfL+D2tmAgW3hb/nNGPMi1sMLHblFElBYdsfpM8cFBizBAcGFruObpNaCnr3nGaM6cKBgcUm3z2mxzzY3q3EwYGxsHFgYLGpfJdyTMF3z2nGWFh48JnFJjN7TjPGdOEWA2OMMQUODIwxxhQ4MDDGGFPgwMAYY0zB8cBARL8iogNE9DkR/cHp8jDGWEfn6KwkIhoN4DoAlwkhLhBRTyfLwxhjzPkWwx0AFgkhLgCAEOKMw+WJbqWrgacGAwszpc+lq50uEWMsDjkdGL4LYCQRfUJE7xPRMIfLE71KVwNv3g1UnQAgpM9v3s3BgTFmOdu7kohoM4Bvq3zrwbbX7wrgSgDDAKwmoouFEELlOrMAzAKAPn362FfgaPXe40BTvfJYU710nDe5YYxZyPbAIIS4Rut7RHQHgLVtgeBTImoF0B1Ahcp1VgBYAQAFBQUBgSPuVZUZO84YYyY53ZW0HsBoACCi7wJIBHDW0RJFq4xsY8cZY8wkpwPDCwAuJqJ9AP4PwM/VupEYgLGPAJ4U5TFPinScMcYs5Oh0VSFEI4AZTpYhZsjjCO89LnUfZWRLQYHHFxhjFuPsqrEkdxoHAsaY7ZzuSmKMMRZlODAwxhhT4MDAGGNMgQMDY4wxBQ4MjDHGFDgwMMYYU+DAwBhjTIEDA2OMMQUODIwxxhQ4MDDGGFPgwMAYY0yBAwNjjDEFisUs10RUAeArp8sBaVOhWNg/IlbKCXBZ7RIrZY2VcgKxWdaLhBA9Qp0ck4EhWhBRiRCiwOlyhBIr5QS4rHaJlbLGSjmB+C4rdyUxxhhT4MDAGGNMgQNDeFY4XQCdYqWcAJfVLrFS1lgpJxDHZeUxBsYYYwrcYmCMMabAgSEMRHQDEX1ORK1EFJWzE4hoAhEdJKIviWiB0+XRQkQvENEZItrndFmCIaLeRLSViL5o+93f43SZtBBRMhF9SkSftZX1MafLFAoRuYloNxFtcLoswRDRMSLaS0R7iKjE6fJoIaJMInqNiA4Q0X4iukrP8zgwhGcfgOsBbHO6IGqIyA3gOQDXAvgegJuI6HvOlkrTiwAmOF0IHZoB3CeE+B6AKwHcGcXv6QUAY4QQlwHIAzCBiK50uEyh3ANgv9OF0Gm0ECIvyqesPgPgXSHEAACXQed7y4EhDEKI/UKIg06XI4jhAL4UQhwRQjQC+D8A1zlcJlVCiG0Avna6HKEIIU4JIXa1fV0D6R8ty9lSqROS2raHnraPqB1UJKJsABMBrHS6LPGAiDIAFAFYBQBCiEYhRKWe53JgiG9ZAE74PC5DlFZisYiI+gLIB/CJsyXR1tY1swfAGQD/EEJEbVkBPA1gPoBWpwuigwCwiYh2EtEspwujIQdABYA/t3XPrSSiznqeyIEhBCLaTET7VD6i8s6bRQYRpQJ4HcC9Qohqp8ujRQjRIoTIA5ANYDgRDXa6TGqIaBKAM0KInU6XRacRQoihkLpp7ySiIqcLpCIBwFAAfxRC5AOoA6BrnDHBzlLFAyHENU6XIQzlAHr7PM5uO8bCQEQeSEHhb0KItU6XRw8hRCURbYU0jhONA/yFAKYQ0Q8BJANIJ6KXhBAzHC6XKiFEedvnM0S0DlK3bbSNNZYBKPNpJb4GnYGBWwzx7Z8ALiWiHCJKBPBTAH93uEwxjYgIUp/tfiHEk06XJxgi6kFEmW1fpwD4AYADzpZKnRDifiFEthCiL6S/0y3RGhSIqDMRpclfAxiHKAy2Qoh/AzhBRP3bDo0F8IWe53JgCAMR/YiIygBcBeAtItrodJl8CSGaAdwFYCOkQdLVQojPnS2VOiJ6BcBHAPoTURkR3eZ0mTQUAvgZgDFtUxX3tN3lRqPvANhKRKWQbhL+IYSI6mmgMeJbAD4gos8AfArgLSHEuw6XScuvAPyt7W8gD8Dv9DyJVz4zxhhT4BYDY4wxBQ4MjDHGFDgwMMYYU+DAwBhjTIEDA2OMMQUODIwxxhQ4MDDGGFPgwMBYGNr2ZvhB29f/TUTPOl0mxsLFuZIYC8+jAB4nop6QMq1Ocbg8jIWNVz4zFiYieh9AKoBRQoiatvw5ywA0AigWQvzN0QIyZhB3JTEWBiIaAiknUWPbxj2AtKvfa0KI28EtCBaDODAwZhIRfQfA3yDtildLRPLWpNlo3yCpxYmyMRYODgyMmUBEnQCshbT/834A/wVpvAGQ8uBnt33N/2Ms5vAYA2MWaxtjWAqgAcAHPMbAYg0HBsYYYwrczGWMMabAgYExxpgCBwbGGGMKHBgYY4wpcGBgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpvD/AdAyVATZTKejAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_results(X_train, Y_train, X_test, Y_test, lambda X: weighted_sum(X, w_hat, b_hat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Repeat with class overlap\n",
    "By now you've probably concluded that the perceptron performs well on this data (`class_sep=2`), which is to be expected as it's roughly linearly separable. \n",
    "However, we know (from lectures) that the perceptron can fail on non-linearly separable data.\n",
    "To test this scenario, re-generate the synthetic data set with `class_sep=0.5` and repeat Sections 2–4.\n",
    "\n",
    "**Question:** What do you find? Pay particular attention to plot of the error error vs. training epochs. Do you notice anything unusual?\n",
    "\n",
    "**Answer:** As mentioned in the previous answer, the model becomes highly unstable—the optimisation algorithm doesn't really converge to a good \"approximate\" solution. As a result, the train/test errors fluctuate wildly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Comparison with logistic regression\n",
    "We've seen that the perceptron is not robust to binary classification problems with overlapping classes. \n",
    "But how does logistic regression fare in this case?\n",
    "\n",
    "Run the code block below to fit a logistic regression model using `sklearn`. \n",
    "You may wish to switch off regularisation (alter the `C` parameter) for a fairer comparison with the perceptron."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot the decision boundary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEMCAYAAADAqxFbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt4VNW5P/DvO8OEhCQkQKCVhJhALSIBEw14CaRcLKBcpNZLRWo5WikerQotFO/oOW2x+FMPIqUK6mmtHkEBK16gKDEoXgoEIkooBigkUA1qQhISclu/P3b2ZPbM3jN779l79szk/TxPnjCTuawMsN693rXWu0gIAcYYY0zmcroBjDHGogsHBsYYYwocGBhjjClwYGCMMabAgYExxpgCBwbGGGMKHBgYY4wpcGBgjDGmwIGBMcaYAgcGxhhjCj2cboAZGRkZIicnx+lmMMZYTNm1a9dJIUT/UI+LycCQk5ODnTt3Ot0MxhiLKUT0Lz2P41QSY4wxBQ4MjDHGFDgwMMYYU4jJOQbGWOxqbW1FVVUVmpubnW5K3EpMTERWVhY8Ho+p53NgYIxFVFVVFVJTU5GTkwMicro5cUcIga+//hpVVVXIzc019RqcSmKMRVRzczP69evHQcEmRIR+/fqFNSLjwMAYizgOCvYK9/PlwMAY6zaEEBgzZgzeeust733r1q3DlClTDL/WTTfdhAEDBiAvL0/X40tKSrBjxw7D77Nz507ccccdhp8XDg4MjLGotrGsGkVL30Xu4jdQtPRdbCyrNv1aRIRVq1ZhwYIFaG5uRkNDA+655x489dRThl9rzpw5ePvtt3U/PlhgaGtr03xeYWEhli9fbrh94eDJZ8ZY1NpYVo2713+KptZ2AEB1bRPuXv8pAGBmQaap18zLy8P06dPxyCOPoLGxETfeeCOGDBli+HWKi4tx5MgRXY89cuQIVq1aBbfbjRdeeAFPPvkk1qxZg8TERJSVlaGoqAg/+clPcOedd6K5uRlJSUl47rnnMHToUJSUlODRRx/Fpk2bsGTJEhw9ehSHDh3C0aNHcdddd9kymuDAwBiLWss2H/AGBVlTazuWbT5gOjAAwIMPPogLLrgACQkJ3vI6Y8eORX19fcBjH330UVx22WWm3wuQyvjMmzcPKSkp+PWvfw0AWLNmDaqqqrBjxw643W6cOnUK27dvR48ePbB161bcc889ePXVVwNeq6KiAtu2bUN9fT2GDh2KW2+91fSyVC0cGBhjUet4bZOh+/VKTk7Gddddh5SUFPTs2RMAsH379rBe04xrrrkGbrcbAFBXV4ef/exnOHjwIIgIra2tqs+ZOnUqevbsiZ49e2LAgAH48ssvkZWVZWm7ODAwZrGNZdVYtvkAjtc2YWB6EhZOHhrW1W13NjA9CdUqQWBgelLYr+1yueBydU2zBhsxDB06FNOnTwcAzJs3D/PmzQv7/QEpQMnuv/9+jB8/Hhs2bMCRI0cwbtw41efIgQwA3G530PkJszgwMGYhO3Li3dnCyUMVnycAJHncWDh5qOXvFWrEsGfPHl2vs2LFCgDA7bffrrg/NTUVp06d0nxeXV0dMjOlfyPPP/+8rveyC69KYsxCwXLizLiZBZn4/VUjkJmeBAKQmZ6E3181IiqC7PXXX49LLrkEBw4cQFZWFtasWQNAmgPo169fwOOnT5+ODRs2ID8/XzUILVq0CHfffTcKCgpsGQUYQUIIRxtgRmFhoeDzGFg0yl38BtT+RxGAw0unRro5UWn//v0YNmyY082wzbRp07B+/XokJCQ42g61z5mIdgkhCkM9l0cMjFlIK/dtRU6cxYZNmzY5HhTCxYGBMQstnDwUSR634j67cuKM2YUDA2MWioac+Kr3KrGj8qTivh2VJ7HqvcqItYHFtqhZlUREbgA7AVQLIaY53R7GzJpZkOno5OjIrDTc/mIZVswqwKVDMrCj8qT3NmN6RE1gAHAngP0AejvdEMZi2aVDMrBiVgFuf7EMsy/KxgsfH/UGCcb0iIpUEhFlAZgKYLXTbWEsHlw6JAOzL8rG8ne/wOyLsjkoMEOiIjAAeALAIgAdTjeExTYrK3HGsh2VJ/HCx0dxx4Tv4YWPjwbMOXRX4ZTdXrduHYYPHw6XywU9y+Vra2uxcuVKU+284oorUFtba+q5VnA8MBDRNABfCSF2hXjcXCLaSUQ7a2pqItQ65iSjnby867i6tgkCXbuOu1tw8J1TWDBpqDetFLPBoXwt8HgesCRd+l6+1vRLhVN2Oy8vD+vXr0dxcbGu9woWGEJtYHvzzTeRnp6u633sEA1zDEUAZhDRFQASAfQmoheEELN9HySEeBrA04C0wS3yzWSRZKa0hF2VOKOF3hpM5VV1ijkFec6hvKou9lJK5WuB1+8AWjvrJdUdk24DwMhrTb2k2bLbRjflLV68GJWVlcjPz8cPf/hDTJ06Fffffz/69OmDiooK/POf/8TMmTNx7NgxNDc3484778TcuXMBSNVYd+7ciYaGBlx++eUYM2YMduzYgczMTLz22mtISrJ3X4zjgUEIcTeAuwGAiMYB+LV/UGDdj5lO3q5KnNHASKCc94PATu7SIRmxFxQA4J2Hu4KCrLVJut9kYAAiU3Z76dKl2Ldvn7fGUklJCXbv3o19+/YhNzcXAPDss8+ib9++aGpqwqhRo/DjH/84oJzGwYMH8dJLL+GZZ57Btddei1dffRWzZ9vbRToeGBhTY6aTt7MSp130jgKcHg05VjG2rsrY/To5VXZ79OjR3qAAAMuXL8eGDRsAAMeOHcPBgwcDAkNubi7y8/MBABdeeKHuw4HCEVWBQQhRAqDE4WawKGCmkw+3EmekOz8jowAnR0OOVoxNy5LSR2r3h8lI2e1wD+qR+ZbZLikpwdatW/Hhhx+iV69eGDduHJqbmwOe419mu6nJ/r9zxyefGVNjprREOLuOnZi4NlKJ1ckaTI5WjJ34AODx+x09SdL9Ftu+fTv27NkT8BUqKFRXV2PixIkB96empqoGGlldXR369OmDXr16oaKiAh999FHYv4NVODCwqGS2k59ZkIkPFk/A4aVT8cHiCbqvaJ3o/IyMApysweTo3M3Ia4Hpy4G0QQBI+j59eVjzC2Zt2LABWVlZ+PDDDzF16lRMnjwZAHDixAn06BGYfOnXrx+KioqQl5eHhQsXBvx8ypQpaGtrw7Bhw7B48WJcfPHFtv8OenHZbcbgTLnsoqXvqqbLMtOT8MHiCQH3W5XqMvo6Wu10E6FDCMNtibey2ytWrEB2djZmzJjhdFMUwim7HVVzDIw5xYmJa6NzIlbUYDIzX6DWTgBo77yo7O6n1Pmf1BYPOJXEGJxJ1ThRidVMysy/nW6igMfwKXXxhUcMjKHrSldPisVoKibY4yNdidXsfIFvO3MXv2HqNVjs4MDAWCc9nbTRVIzW43f+6xts2nsCtU2tAIA+vTx4cPpw24OEFSmzWNwvwozhVBJjBhhNxWg9/oWPjnqDAgB8e7oVC1/Za3tdJytSZnxKXfzjwMAiLpZPGDOaijGSXmltF7bn6a2Y14iGU+qYvTiVxCIulk8YM5pGSU3sgVPNwStp+pIDidq8BNA1B5LeywMhgLqmVsPLRa2Y13D6lDqzhBAYO3Ys7r33Xlx++eUApHLaa9aswdtvv23otW666SZs2rQJAwYMwL59+0I+vqSkBAkJCbj00ksNt/vIkSPYsWMHZs2aZfi5ZvCIgUWc7wljj205oAgS0c5oGuWnF59t6PXTe3lUd2EvXLcXC1/Z673v29OtqG1qjfvy4laPLsMpu+1vzpw5hoJJSUkJduzYYfh9ACkwvPjii6aeawYHBuYIu08Ys+vAHqNplIVTzsVt44bA1bnC00XAZcMGwOMKXPIJAA3NbXjo9c8C5iVaOwRa27U3o8brclF5dCkHB3l0OTIrzfRr+pbdfvjhh3WX3fZXXFyMvn376nrskSNHsGrVKjz++OPIz8/H9u3bUVNTgx//+McYNWoURo0ahQ8++AAA8N577yE/Px/5+fkoKChAfX09Fi9ejO3btyM/Px+PP/644bYaxakk5gj/E8YuHtLPsuBgd9E3o2mUhVPOhdtFWP7uF7h9/PewYNJQbCyrxvyX9wTstm7tEPj2dKvq64QSj8tF7Tq/OhJlt33l5ORg3rx5SElJwa9//WsAwKxZszB//nyMGTMGR48exeTJk7F//348+uijeOqpp1BUVISGhgYkJiZi6dKlePTRR7Fp06aw2qEXBwYWcb5zCpcOycDFQ/pZmk4yUqLaqjITq96rxMisNEX7d1SeRHlVHUZmpQUEwZkFmZj/8h5zv6CGeF0u6ju6vGPC9yz5N+JU2W1fW7duxeeff+69ferUKTQ0NKCoqAgLFizADTfcgKuuugpZWeFXkjWKU0ks4oKdMGYFvSuErKyoqpXycLugecymVkeenuQJmMfwuAget3r6CQB6uAgz8wcq7ouVlV6h2HV+tVrZbTmF4/u1detWHDt2zHt71apVlrx/R0cHPvroI28V1+rqaqSkpGDx4sVYvXo1mpqaUFRUhIqKCkvezwgeMbCIs/uEMa2VQ+m9PCha+q53dHC6pU11ZPHQ658ZHjVopTz8g+BXp87ABWDWMx+jTy8PPC5Ca0dXQinJ48aSGcMBBO7C9r3Pf1XSzPyBeOkfx1B0TkbMrfQKxu7Rpa9QIwb5JLZQVqxYASCwhlJqaipOnTrlvT1p0iQ8+eST3sqre/bsQX5+PiorKzFixAiMGDEC//jHP1BRUYFBgwYFLeFtNQ4MLO6oFX3zuAkNzW3e/L1a4JB9e7oVG8uqMbMg01CqyT/l8dWpM/jLh//yPnf8uf3x6q5qb7u+Pd0Kj5uQnuRRXXaq9j5Bq6Cek2F5Lt5p0Xx+9fXXX4+SkhKcPHkSWVlZeOihh3DzzTejoqICRUVFAY+fPn06rr76arz22mt48sknsXz5ctx2220YOXIk2traUFxcjFWrVuGJJ57Atm3b4HK5MHz4cFx++eVwuVxwu904//zzMWfOHMyfP9/W343LbrO4tLGsGkv+9pl3dzERYOSfemZnJ61W/VRrFdLd68uxqfwE/uPSHKz54DBa2wRa2ju8PydAtbS3VpltMx7bcsAbmBZMMr8TOdicidqIz4h4K7vtb9q0aVi/fj0SEhIcbUc4Zbd5joHFrTNtXZ2y0euf47VNhspf7Kg8iU3lJwAAFw/ph6QebkVQANSDgvxe8muEMydgZS7ejmWi3cWmTZscDwrh4lQSs4ydV5lGqXXqarSu4gemJ2lOYlfXNiH/oS0gktJBvRN7IDM9CX/66YUAgF/8ZRfqDex2HpieFDAnYPSztDoXb9cyURYbeMTALBNNV5l61vQnedy44eJszZ3M6b08ms+tbWr1zlecam7Dwa8a8NWpMwCAVr+Rgi+1dUV9kz2YvfpjfNPYgp+u/gQ5i9/A6tJDuOXPO3V/lnas9LJ7EyKLXjxiYJax6irTir0FWiuT1I6jLDy7r+r7LfnbZ7rfr61D4P6N+wACPG4Xis/pjy2ff6l4TJLHjR9fmIltFTXe9+qb7MGn1V0rVeRT0U42tiDB7cItf96Jm4ty8dyOIxjcPzngfYONIsJd6WXnJkQW3TgwMEuFuxnJ6K5lrZTLBdnp+KaxRdfEsdZO5romYzuQ68+0IdHjwrNzRuHSIRlY9nYFVpZUQqBrMntmQaZi49vs1R9rvl5Lewd69uiB5e9+gUSPC9NGnhWx4oORXCbKog+nkpilwp0ANXregVb66vqLsk2Xht5YVo2Ch7doThYHc6a1Ax8clNqycMq5uGfquUjyuLHsmpHeoHDz8ztR+s8azF79MTpCvIkcbDxuF4YPTItY8UG7NyGy6MaBgVnG9yrTf5evXkbPNQhWqXVmQSY+WDwBh5dOxQeLJ+gOCgtf2Wu6XpEA8FRJJZa9Le1WvWXsEKyZU6ho37CzUrGj8uuQQUE2d+xg/OmnF+L2F8sAICJ5/3k/GBLw2pcOyYj4IgKrCSEwZswYvPXWW9771q1bhylTpoR87rp16zB8+HC4XC7oWS5fW1uLlStXmm7rE088gdOnT5t+fjg4MDDLWHGVqVUmYmB6kmbFVCsnSZdtPhC0iqlef/noX94/+7av8Ow+KDtaq/t1Jp33Hbzw8VEAwIpZBXh973FbykNErfefAA6XKu87XCrdb0I4Zbfz8vKwfv16FBcX63ovDgxhIKJBRLSNiD4nos+I6E6n28TMseIqU+u8g/Hn9tesa2Tl+n2rKpT6Llf1bd/W/V8GTVG5SVq35CLgtnFD8PSNhd4R0WfH67D5sy/DGpHFnMwLgHVzuoLD4VLpduYFpl/SbNntYcOGYehQ/ZsGFy9ejMrKSuTn53vLXixbtgyjRo3CyJEj8eCDDwIAGhsbMXXqVJx//vnIy8vDyy+/jOXLl+P48eMYP348xo8fb+4XDUM0TD63AfiVEGI3EaUC2EVEfxdCfB7qiSz+yOke/1VCWnMP/73pc3QAlk2Saq1mknlcBBBCjirkkY//JO7yd7/QfI68A9p/Ql0eeT1deihqy0PYJrcYuOZ5KRgU3gzsXCPdztV31a4lEmW3ly5din379nlrLG3ZsgUHDx7EJ598AiEEZsyYgdLSUtTU1GDgwIF44403AAB1dXVIS0vDY489hm3btiEjI/J/t44HBiHECQAnOv9cT0T7AWQC4MAQB8wsPVVbJaRVovpkYwtevOUi3Z1lqPYsnDwUC1/Zq9nxXzd6kGJ5a3ovDxqa2wIK4clF73zTaxvLquEm8i5JDfi9O6ujGll6amXxwaiVWywFhdI/AMWLwg4KgDNlt7ds2YItW7agoEBaRdbQ0ICDBw9i7Nix+NWvfoXf/OY3mDZtGsaOHWtrO/RwPDD4IqIcAAUAtNfwsZhh5YE5WlfymelJqukrraAQqj3y9wVr96hODm+rqMEVI87CTy8529uBBws2vo+5e/2nmkEhwU14qqQSG/ccN30mRNw6XCqNFIoXSd9zx1oSHNTKbtt1UA8gTXzffffd+MUvfhHws927d+PNN9/Efffdh4kTJ+KBBx4I+/3CETWBgYhSALwK4C4hxCmVn88FMBcAsrOzI9w6ZoaRA3NCUa2Y6iKcbmlD7uI3NEcjvp22S+VqXa09wQ7Rqa5twi1/3olnbixUPD7U76N2XCcgzSW4idDSOUIJFjytOlQopshzCnL6KHes8raFzI4YqqurceONN+Kdd95R3J+amqoINJMnT8b999+PG264ASkpKaiurobH40FbWxv69u2L2bNnIz09HatXr1Y834lUkuOTzwBARB5IQeGvQoj1ao8RQjwthCgUQhT2798/sg1kphhdehqM/1nL6UkeoLNWkdYhO/4H8Whdrau1J1g5jNY24S1/oafw3cayas3lrx0CijQUoL5vw8pDhWJK9W5lEJDnHKp3R7wpGzZsQFZWFj788ENMnToVkydPBgCcOHECPXoEXmP369cPRUVFyMvLw8KFCzFp0iTMmjULl1xyCUaMGIGrr74a9fX1+PTTTzF69Gjk5+fjoYcewn333QcAmDt3LqZMmeLI5LPjZbeJiAD8L4BvhBB36XkOl92ODUVL39VM/4RbZlrrtX1LXpxuadO1H8G/PRvLqrFw3d6ADtv/OcuuGalrolurraEcWTo15GtYWbI7UuKt7PaKFSuQnZ2NGTNmON0UhXDKbkdDKqkIwE8BfEpE8vj9HiHEmw62iVlA6zwDeWJWZiZFojXqkEcFRjpi//Ys23wgaFCQX//2F8tw67jBIVcFBRshaVV3lZethnoNq5bXMvP8T2qLB44HBiHE+1AvOslinNbSU99O3+wEdahlpUYs23wA81/e422f3s72B9/PwB9LDoWsVaR51GiSx3uQkD//tJfWa2htCOyW8xHMMlExx8DiV6iyFEZrI8nUNsKZ5Z+3T0vSnl+Q9e3lwcay47h13OCQy0W12koE9NGYy8j06/C1Nv75j3aAbjwfwSzDgYE5ymyKxH8y2j/1YlZTazuIENAJy6/uIqDw7D745nQrZhZk4o8lh0LuPJbbmu4XcL493apZwXX8uYELLBI9Xf9d05M8mkUBzQbbSHJ6bjPe+X6+7e3tqKurQ0NDg+7nO55KYt2b0RSJL99lohvLqjH/5T2q+fqM5AQ0tbWjqaVdV+G62tOtePy6/IBUzL9PNcPtAv5YcshbfsN3jiFY+mZmQSaWbT4QkDrSas+2ihrvn/3TbYDy2FJ/0T4fkZiYiK+//hr9+vUDWRTQuyshBDo6OtDR0YH29nZ0dHSgra0N3377Lb755hs899xzaGqS/t7PP/983a/LgYE5Su8EdSgzCzKxqfw4tu7/KuBn140aBLeLsPzdL5DaswcazrRhYHoSGs+0qeb4B6YneYOO3NnPf3kP+nUGmGduLAwov6FnrsRIx+z7WKP7QcIJtpGQlZWFqqoq1NTUhH5wNyd3/MG+q2lpaUFdXR1ycnKQkpKClJQUGFnmz4GBOUrPBLUvuY6QfNCNfGBNeVUd+qf2xCWD++LoN03e15qZPxDfnG7B5s++9F7l//XGi7xX+MGCkv/P5VPV5D0MvuU3/vLhv0J23kYmzH07caMjAKuCrV08Hg9yc3OdbobjWlpa0NDQ4P2qr69HY2Mj6uvrvfe1tyv/TblcLm9Hr/aVmpqKhISEsEdiHBhYUEYPpTdDz85hmXwwz63jBnu//7HkEG4dN9hbedS/rf4nkd38/E4smHQObhkrtX/Z5gOorm1C78QeePjKPEWw8u/sW9o7FJ29XH7jkbcqVNvr23mr7t52E+C3yc2/Ezc6AjAabJn12tra0NjY6O3wfQOA/NXS0qJ4DhGhV69eSElJQUZGhuJqX+70k5KSIpJ+48DAgpI74kgcJ6mH78E8P/h+Bn73RgVmFmTiia0HkdTDjRue+VjREaqdEbFg0jl4bMtBDB+YhpkFmRjQu6fqRjUjV+p6Om+tDlvtPv/CfkZHAEaCLTOmo6PD2+lrfcl5fV+JiYlITU1F7969kZmZGXC1n5ycrKjd5CQODCwo34549kXZeOHjo46f++t78M3onD7YUFaNHi5C45nA/L7aqOaWsUMwfGBayN/JyJW63s5bq8OW71v1XiUG9O6p+NmA3j1x2bAB2H20lkcANhNCoLm5WfMqv6GhAY2NjQG5fY/Hg9TUVG8uXy3No1Y2I1rFTkuZY3w74jsmfM/xMs/ywTc/KhiIjWXHkeRxoalVuUonVLE+Pb+TkSt1q9I3wUZoT84yfzhNRJWvBd55GKirAtKygIkPACOvdbpVAIAzZ84EvdJXy+u73W7vFb3alb5v6e54wYGBheR/QtrFQ/o5FhzkjlKeW7hn6rn47Ruh8/tqrxPqd/Lv7H+W8gkWeV5Gr9f+DZQEdnhWpG+icYRmSPla4PU7gNbOz77umHQbsD04tLW1hez0tfL6qamp6N+/P3JzcwM6/Ujl9aMJBwYWlNrkbTgnpIVLnjPwnTv4U8khnGxsCXis1uSskd/J29mXr8XKdUew50wfXOo+AdQdQ9trv8QnX7pQnlCgeyJez2R+tI3QDHnn4a6gIGttku4PIzAEy+vLaZ/m5uaA5yUlJSElJQVpaWlRn9ePJhwYWFBqk7dOHifp23nK7pt2nqHJWTO/0+m3HkA++uD21juwAstxqftzfNIyGLe9246nfp4GQF+nr2cyP5pGaIbVVRm7H1Jev6mpSbWzl79Onz4dkNdPSEjwdvADBgwIWMGTnJwcU3n9aOJ42W0zuOw282d30biOJelwQWBH+3m4vfUOzHZvxQvtl+FJz5Mo+q8dAAJHIv63ZfL9aqkiva8RtR7Pk9JHftpTB6Lq6rc1O/6ODuUckZzXD7VenxkTS2W3GQub3vy+2QByvKMfslwncan7cwxr+xeWt1+FO9zrcTZ9CaBrZKBnfiBYqijaRmih+Of1PbmzkVP+GNwdZ7yPaYUH2zAGX3Qedk9ESE5O9l7pDx48OKDTT0xM7HZ5/WjCgYF1G+GcQb06YTYWta7Ef7XOxm7xPSTiDJ5tn4wDnmEo3F6Jx7YcxJo5hbrmB4KlitTmKrTOsLZbe3s7Tp8+rXmVr57X743z+lyDUafeRK/Wb3EmMQP/Pm8ucof9CCM6O/1evXpxXj/KcWBg3UY4Z1DnT52Lea/2wq6OIeiBDvzM9Rb+1DEDW1uGYcsbFbhn6rne1E+w+YFomcyX8/rB1uv75/XzG99DfVIu6jPyvVf7ZzUfRJ+mI2gd/Z/eyVzfvH4igJyI/VbMKhwYWLcRTtXRmQWZeGf/D5FU+TVONrZgpbgKORnJOHSyEcMH9sYtY4fo6vQjkSoSQuhar6+W15c3aQ0aNCgwr3/yXHg2/hy46Arp7OXDpcC630tnMGfyZrt4wpPPrNuw8tzk+S+XYUPZcYzO6YMvahq9nbvddaUAoLW1NWSn39qqrBrrcrm8dXjkzt//S1de/3ApsG4OUHgzsHONFBRyiy373Zi9ePKZdWtqk8xWVR19ZnslNpYdx48KBuK9f570FvTTmmg2MhKQ8/rBUjxa6/VTU1PRp08f1at9y/L6ucVSUCj9A1C8iINCnOIRA4s7WuW0f3/VCADhla3YUXlSUZ3Vdyd2e4f65LFMCKGYzNWqw+OvZ8+eSE5O1rzST0lJgdttzTGnIVkxYojikhnxTu+IgQMDiztWpoz8aW1i23usFv9xcZbqJi3fHbv+ef0ePXp4O/dPTyViQ2U7vm7qwIAUD3457mxcd9FgeDyhz6COCDkoyMHA/7Ye/iUzAMCTBExfzsEhAjiVxLotO462lPP6UwcnoKGhBp98ckgRBNwNDXj28zbFc1wul3e9/ne+8x0MGTJEsVZfLr5GRNhYVo0X1n/qLQb4ZUMrfrv5MFKSU6Knimr1bmUQyC2Wblfv1h8YbCqZwazFgYHFHaMH27S3tyuu6tXy+2fOnAl4njyZ27dvX2RnZytKMciHqujN64ezlDZixtwVeF9usbFUkomSGSzyODCwuKM2yZzYw4UbC9Kxd+9e1To8/nr27Om9uv/ud7+rWofHyry+HaOcqJSWpVoyA2lZkW8L08SBgcVM/1kmAAAZd0lEQVQstfX69fX1SG5owFXZZ7DpKKGu1YXe7nYU9/4W7ZWH8UGlMq/ve6Xv2/lHOq9vdJQTsyY+oD7HMPEB59rEAnBgYFGrtbU16LLNhoYGtLWp5/ULM1IwLidFtdOX8/rRxKqltFFPnkfgVUlRLSoCAxFNAfA/ANwAVgshljrcJGaz9vb2kJu0guX1+/Xrh7PPPlt1vX60dfp6WHUCXEwYeS0HgijneGAgIjeApwD8EEAVgH8Q0d+EEJ872zJmVkdHR8j1+sHy+qmpqTjrrLNUD1WJ2Hp9B1hxAhxjVnA8MAAYDeALIcQhACCi/wNwJQAODFFIPiw91CatYOv11a70ncjrM8bURUNgyATgu0yhCsBF/g8iorkA5gJAdnZ2ZFrWDbW0tGgenygv6VTL68udu9qVfrTm9Rlj6qIhMOgihHgawNOAtPPZ4ebEJLW8vu/kbmNjY0BeXz4sXSuvL6/X506fsfgRDYGhGsAgn9tZnfcxA9Ty+v4repqaApdDJiYmIiUlBb1798bAgQMDdub26tUrrvP63Z5v3aKkPtJ9Td/yaqFuLhoCwz8AnENEuZACwk8AzHK2SdHFP68vp3X8r/b96155PB5vR5+RkRGwMzc5OZnz+t2Zf92ipm+6flZ3TPoZwMFBSxwXA3Q8MAgh2ojodgCbIS1XfVYI8ZnDzYoo37y+1rr99nZluQTfvL7alX5KSgoSEhI4xcO0qdUt8sU1jLT5B9U4C6SOBwYAEEK8CeBNp9thh7a2Nu+krVan39LSoniOb14/IyMDOTk5AZ0+5/VZ2PTUJ+IaRurivBhgVASGWNXR0aEovqb2FSyvn5aWhszMTNX1+nxYuoPiOEWgoFW3yP8xLFCcFwPkwKBBPiw91Hr9YHn9/v37qy7d9D0snUUZq1IETgWX958AMi9QVjw9XCqVxvavjqpWt8gX1zDSFufFALttD6XnsHT/vL7b7fZO3Kpd6cvr9VkMsyJF4GT+OfMC7cN0/PnXLeJVSfrFeTHAuDzBra2tLWiHX19fH3BYOhF5D1XR+uK8fjewJB2A2v8JApbU6nuNx/M0riYHAfP3hdM6faw4fpOFFoMpR9tPcCOi3wghHjH7/HB0dHTgxIkTmp2+1mHpnNdnqnz/g5MLEO2Bj9GTIvC+jkbePlL559xiKSiU/gEoXsRBwS5xXAxQd2AgorW+NwHkA3AkMHz99dfYsGGD93ZCQoJmXl9er895fabKP+2jFhT0pAjUzjL2F6n88+FSaaRQvEj6njuWgwMzxEhveUoI8XP5BhH90Yb26JKamopp06Yp1uszZorWWn5yA6JDf4og1J6ASOWffecUcouloOB7O1xGJrdZzDKSO/mt3+17rWyIEYmJicjOzkbfvn05KLDwaKV3RIc0pzB/n750QbA0UdogYPryyKQdqncrg0BusXS7erc1ry9Pbh8ulW7LgSjzAmte//0nul5bdrhUup9FTMjAQET/Q0QkhDjse78Q4hut5zAW9crXSpPEqhPNMJ720Xq8POE88tqu91ySLn0vX6tsi//9wdqt9dgxdwWODHKLrbualwPNujnAu7+1djQC2B94mC56Rgz1AP5GRMkAQESTiegDe5vFmEl6rjjl+QCtSWIzaZ9zJkGaetN4HcV7iq4lrJsWqN/v3+GXrwUeyQXW3xL6sVYI9jn6Tm4X3mzt/IXdgYfpEjIwCCHuA/ASgJLOgLAAwGK7G8aYKXquOIPNB5hJ+7xyE1D2FwSMPgbkdY0UNsxT3x+x63n1+9f/QnpdoCuoNKkM0uU9FlYL9jn6T277BxAtetNEdgYepoueVNJEALcAaASQAeAOIcR2uxvGmCl6rjg15wNI/5yCr8OlQHtL4P21R7o6dbXVToD2/egA9r/etQQ22MS2HctgtT5HoOvPE+7teoye4KA3TWQ28DDL6Ekl3QvgfiHEOABXA3iZiCbY2irGwhHqilNzPsDkctLGk9r3h+rUKchZF+0tXfsrgrFrGaza5xhscjvU/IeeoO27qspo4GGW0ZNKmiCEeL/zz58CuBzAf9vdMMZMC3XFOfEBKf/vK5zlpMECTbBO3ZMEXDgnsC2+5F21wV7DrmWwap+j1uR274H65kpCBW27V1UxXQxv9RVCnAAw0Ya2MBY+PVecI6+V5hHSBgGg8JeTTnwAcPstm3YnSPdrderklt5z2mPSd63/ivI+CrXgkdTXvmWwRq/cg9WY8n/dYEHb7lVVTBdTNSCEEEHGxow5SO8V58hrpfmEJbVSx/vOw+opED1LSVO/KwWC5P4ASPruTpDu1xqd/GhVV4ee+l0goZdUjsOXO0F6zKnjgYHsqmeA33SuINe71BXQPwFs9MpdTxlqThPFjLgsoscsFIOFwgxRK2XhSeq8iof2z3w/g1C7gUN9hvLzD5UA7z8uba5L7i+NCL4+2LkDe1Dg84K1XevvyH9ntP9ts/QUDuRd047TW0SPAwPTZqbjiTXBOjQg8lVS5Y46+xKgYpPyZ/6fvdkqrnZUX+0O/1bigN7AwOVEmTa9eeNYFiwFovmzY/ZsKgO6Jmf9gwIQ+NmbPUXMjn0CVs/bMEdxyVGmLc6PLwQQ+iQurd3Rdh28I0/OavH97M2eImZX9dU4LkPd3fCIgWmzer1/NAq2dFVrNRBgz8jJN98vp7L8+X72Zpbd8gQw04EDA9Nm9Xr/aBQsBSL/TIvv1buRQnhafFcC6fnszaRveJ8A04Enn1lwTq9KMvP+Vrc51CSvXROvTn/2LO7wqqR40x06Cf/ljOVrgdduU9YhCtXhWtVJ+7Yl1GvaccYzL+1kNuBVSfFEq2SzXStjnFC+FvhwBfC/04Fl35Nub74nsDhdqNy+VSupfAu+jbwWuPg/uzag+ads7Jik53MJmIMcXZVERMsATAfQAqASwH8IIWqdbFNUCtbZxcOowf+KvLEG2Hgr0NGm/vhgHa5VnXRusVTH6M9XShvMyAWMma8+v2J2dVCo95cnhq3cb8CYDk6PGP4OIE8IMRLAPwHc7XB7olO8LxtVC3wdbQg4+EYWrMMNdyWVXDKifC3w0UopKADS949Wqo/StCaKB41WrvYpXyuNhpak6Zug5nMJmEMcDQxCiC1CCPmy8CMA8bEO0upza+N92ahmgBPqxelaGrVX/4RazRPq70ZO4Wy+R39KSmt10IVzutJB8nxJY03n76wjHWjHuQR8pjLTwekRg6+bALzldCMsYXV+ON6XjWoFOLkYnVycLqkvIETnSWYacy1yJ53Ut+u+Hj6fXai/m+rdwJgFXR24P60Nb75F+eTDfnzTQW/8yth8iV37DXjugulge2Agoq1EtE/l60qfx9wLoA3AX4O8zlwi2klEO2tqNP7TRgurz62N93IDWoFv8u+A618CLrld6nATkoGOVuXjtDrXNp+r/aZvugJIqL+bzAuA9x8DXB7t9pavVV55e/cwpAFLs5WBSk4HnTml/lpaoyW79hvwmcpMB8eXqxLRHAC/ADBRCHFaz3NiZrnqu7+V8sPFi6SrPqZNazmu77LNJekIOFcZAEBS4JDpWT4a7O9mxwpgS5C/r7RBwMyVUqd64Rxp7kFrKat8Rd7eqh4crCzIZ2RJM//b7JZiYrkqEU0BsAjADL1BIWbwubXW8E196J1r0Ur3yFfnwf5uDpdKI4aR12m3qa6q68r7/ce15yJ800FT/1/gfImV6UAjS5r53yYLwek5hhUAUgH8nYj2ENEqh9tjDa5HY2ySM1in5pv6OOv8wOf6d67laxF0NVOovxt5juGLrUDP3tqvA0htk1ct+aurUqaDRl4LXPlU53wJrE8HGjlBrbv/22QhOZ5KMiPqU0m8a9XYgTBGUj/nTgNO7JUen9xfmoeQO9fDpcDLs4HmOpUGEXDV09JpaMH+bnzbWf/v4DuvD5d27XMI1nYj9KaD/B+nNUryT7Pxv81uTW8qictu20HtP1hucfea4DOyQSvUPg3/1Mc1z0v3r5sjHYspP2bdHI2gAABC++rc9+/Gf9IXkJauNtYoT1GT32/MfPU5BjMpIv+NfvLICQh+clvdMUijJJWLPP80G//bZDpwYGD2XUX6btAqXqTd+QTbOew/0sgd23VbLfBs/M/gJ7KF4v/7ap0x4BtA+p/befV+DEhMA6541FyKSO8Od7XHQSAgOMTTkmYWUU7PMbBoYNfadr2TnMH2aQRbtqm2MzhSez7G3NXVJu8ehjpg8VHz8wZ6d7gH2xAYr0uaWUTxiIHZU5cn2JW+/+vKnZfepZZy6kPtJDKjrxVN9NZc0nycjWdRs26FJ59ZFyvXtptJTxlZh29kcjuc94kktfLeLg/QMxVo+rarrYA95z+wuMfnMTBj5I7VqUqeRs9RMDsvYtehOlbxDVpJfYCWBvVVUUB0BjcW1TgwMP30Xn3beaVtx2E3Tr6PLJzPzMq28jJVhhjZ+cyihJ66PHYfFhSp0uJm3sfsec7hfmZWfiZcPI8ZwIGBKVfYyHKLlVeSVp2MpiVSpcWT+hi7X61zX38L8Ehu6A4+3M/Mys+Ei+cxAzgwMH3suKL3LZsRraXFVfcMQFmxVUu4n5nVnwkf/MN04sDA9LHjit7IucpWafrW2P3BOvFQV/9anw259KWlrC63zsXzmE68j4HpM/EB9dU84VzR+++f2PU8cONr9l7JGj2fOWgdIgQPHGqfGQCI9s7napS88KW189ooI/tKWLfHIwamj9mr11ATt5FObxhNz6g93lewEZP/Z0buwMdYOU8jU6ts++mrwLDp1h/8w+ISjxiYfkavXvUUhVPbvWxncDC6M1q+/63fdB4p6kPPiMn3M1uSrv4Yq1deySk63+XHFa93FR+UcfE8poH3MTD7hFqHH87uZSd49yQck67+Rbuy4mookdxD4fSGRRaVuOw2c16oVTmhCuT5898sds4k4OAWc5vHzGw8k3+upzS2GjvmabTorWzLmAqeY2D2CbWSSc/+CZnafoKda8xtHgtn41k4exOsXmUUDK9AYmHgEQOzj5VXyFr7CXypnV2g97X0PjfcvQlWrTIKhlcgsTDxiIHZx8orZL0dr57HhdO5a42CkjPUz7N2gp4SJ4wFwSMGZi8rrpDff0LqeBtrQj9Wz4Y7PXsZtIrODRoNNHyprHjqTpBGHNFSd4iP72Rh4hEDi36ZF0gdrzsh+OP0pqn07GXQKjp34RzgyqeA5P7S/T17S+26/iXueFnc4MDAol9usdTxuhOkjhiQOubCm/Wlqfw3fMnlNxLTtJ8brOjcyGuBhV9IE7tnTkmvxUGBxRFOJbHYkFssdcBmTphT2/C163nguheCd+jBlnweLgU+WikFqtI/ALueAyb/jg/LYXGBRwwsNoSz/NJsyWmt9zxcCrx0vTTPcOaUdF9jDfDabdadT8GYgzgwsOgQrKaS7/LLCfd2dfJGg4ORmkzB3rN6tzQn4TsBDUi3ra57xJgDODAw8yeUWfn+wTacWbH80uiII9h7jrkLaDyp/jyr6x4x5oCoqJVERL8C8CiA/kIIjf9xXbhWkoX8C90BXQfORypfbncNITtqMkX67GjGLBAzZz4T0SAAkwAcdbot3ZLdR3bqYfd5z3Zs+ArndDWnR2iMhRANq5IeB7AIwGtON6RbMtIpmyk8p4fRw3OMsmPDl9Hy3TI9pcgZc5ijgYGIrgRQLYTYS0RONqX70tspq3Vor90m/dn3bAU5B29EJKuOWsnMru5w6jQxFiG2p5KIaCsR7VP5uhLAPQB0/e8norlEtJOIdtbU6CiNwPTRmxJR69DaW4DN90h/lvP2ZspCRLLqqNPsTpsxZgHbRwxCiMvU7ieiEQByAcijhSwAu4lotBDi3yqv8zSApwFp8tm+FnczelMiWh1XY420NyDcw2AiUXU0GtidNmPMAo6lkoQQnwIYIN8moiMACvWsSmIW09Mpa3Vo8s5fvYfB2DVPYTW72hmraTPWrTi+KolFOXkFTd0xAH7zQO4EQHTo3xsQzgE5kWRnO7tT2ozFrKjYx2AU72OIELU9DiAAQipi19rUVVVUz96AWFn7HyvtZMwgPvOZhU/11DQhdZCjfq48ryDUec1A7Ey8xko7GbMJBwamLVgHaWZvQKxMvMZKOxmzCc8xMG1aHaHZDjKc3cKRFCvtZMwmHBiYNqs7yFiZeI2VdjJmE558ZsHFyvJSxlhIPPnMrBHNG884aDFmCw4MzHlmOnguRseYbXiOgTnL7GayaCgXzlic4sDAnGW2g+e9BozZhgMDc5bZDt7qpbSMMS8ODMxZZjt43mvAmG04MDBnme3gea8BY7bhVUnMWWaPyJSfy4GAMctxYGDO4w6esajCqSQWm95/IvD8h8Ol0v2MsbBwYGCxKfMC6fwHOTiEc+Y0Y0yBU0ksNsnnP6ybAxTeHP6Z04wxLx4xsNiVWywFhdI/SN85KDBmCQ4MLHYdLpVGCnrPnGaM6cKBgcUm3zOmJ9zblVbi4MBY2DgwsNhUvVs5p+B75jRjLCw8+cxik5kzpxljuvCIgTHGmAIHBsYYYwocGBhjjClwYGCMMabgeGAgol8SUQURfUZEf3C6PYwx1t05uiqJiMYDuBLA+UKIM0Q0wMn2MMYYc37EcCuApUKIMwAghPjK4fZEt/K1wON5wJJ06Xv5WqdbxBiLQ04Hhu8DGEtEHxPRe0Q0yuH2RK/ytcDrdwB1xwAI6fvrd3BwYIxZzvZUEhFtBfBdlR/d2/n+fQFcDGAUgLVENFgIIVReZy6AuQCQnZ1tX4Oj1TsPA61Nyvtam6T7+ZAbxpiFbA8MQojLtH5GRLcCWN8ZCD4hog4AGQBqVF7naQBPA0BhYWFA4Ih7dVXG7meMMZOcTiVtBDAeAIjo+wASAJx0tEXRKi3L2P2MMWaS04HhWQCDiWgfgP8D8DO1NBIDMPEBwJOkvM+TJN3PGGMWcnS5qhCiBcBsJ9sQM+R5hHceltJHaVlSUOD5BcaYxbi6aiwZeS0HAsaY7ZxOJTHGGIsyHBgYY4wpcGBgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpsCBgTHGmAIHBsYYYwocGBhjjClwYGCMMabAgYExxpgCxWKVayKqAfAvp9sB6VChWDg/IlbaCXBb7RIrbY2VdgKx2dazhRD9Qz04JgNDtCCinUKIQqfbEUqstBPgttolVtoaK+0E4rutnEpijDGmwIGBMcaYAgeG8DztdAN0ipV2AtxWu8RKW2OlnUAct5XnGBhjjCnwiIExxpgCB4YwENE1RPQZEXUQUVSuTiCiKUR0gIi+IKLFTrdHCxE9S0RfEdE+p9sSDBENIqJtRPR559/9nU63SQsRJRLRJ0S0t7OtDzndplCIyE1EZUS0yem2BENER4joUyLaQ0Q7nW6PFiJKJ6JXiKiCiPYT0SV6nseBITz7AFwFoNTphqghIjeApwBcDuA8ANcT0XnOtkrT8wCmON0IHdoA/EoIcR6AiwHcFsWf6RkAE4QQ5wPIBzCFiC52uE2h3Algv9ON0Gm8ECI/ypes/g+At4UQ5wI4Hzo/Ww4MYRBC7BdCHHC6HUGMBvCFEOKQEKIFwP8BuNLhNqkSQpQC+MbpdoQihDghhNjd+ed6SP/RMp1tlTohaei86en8itpJRSLKAjAVwGqn2xIPiCgNQDGANQAghGgRQtTqeS4HhviWCeCYz+0qRGknFouIKAdAAYCPnW2Jts7UzB4AXwH4uxAiatsK4AkAiwB0ON0QHQSALUS0i4jmOt0YDbkAagA815meW01EyXqeyIEhBCLaSkT7VL6i8sqbRQYRpQB4FcBdQohTTrdHixCiXQiRDyALwGgiynO6TWqIaBqAr4QQu5xui05jhBAXQErT3kZExU43SEUPABcA+KMQogBAIwBd84w97GxVPBBCXOZ0G8JQDWCQz+2szvtYGIjIAyko/FUIsd7p9ughhKglom2Q5nGicYK/CMAMIroCQCKA3kT0ghBitsPtUiWEqO78/hURbYCUto22ucYqAFU+o8RXoDMw8Ighvv0DwDlElEtECQB+AuBvDrcpphERQcrZ7hdCPOZ0e4Ihov5ElN755yQAPwRQ4Wyr1Akh7hZCZAkhciD9O303WoMCESUTUar8ZwCTEIXBVgjxbwDHiGho510TAXyu57kcGMJARD8ioioAlwB4g4g2O90mX0KINgC3A9gMaZJ0rRDiM2dbpY6IXgLwIYChRFRFRDc73SYNRQB+CmBC51LFPZ1XudHoLADbiKgc0kXC34UQUb0MNEZ8B8D7RLQXwCcA3hBCvO1wm7T8EsBfO/8N5AP4nZ4n8c5nxhhjCjxiYIwxpsCBgTHGmAIHBsYYYwocGBhjjClwYGCMMabAgYExxpgCBwbGGGMKHBgYC0Pn2Qw/7PzzfxPRk063ibFwca0kxsLzIICHiWgApEqrMxxuD2Nh453PjIWJiN4DkAJgnBCivrN+zkoALQBKhBB/dbSBjBnEqSTGwkBEIyDVJGrpPLgHkE71e0UIcQt4BMFiEAcGxkwiorMA/BXSqXgNRCQfTZqFrgOS2p1oG2Ph4MDAmAlE1AvAekjnP+8H8F+Q5hsAqQ5+Vuef+f8Yizk8x8CYxTrnGFYAaAbwPs8xsFjDgYExxpgCD3MZY4wpcGBgjDGmwIGBMcaYAgcGxhhjChwYGGOMKXBgYIwxpsCBgTHGmAIHBsYYYwocGBhjjCn8f0oXbmVpm1hKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_results(X_train, Y_train, X_test, Y_test, lambda X: clf.decision_function(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** How does the logistic regression classifier compare to the perceptron?\n",
    "\n",
    "**Answer:** *The logistic regression classifier seems to produce a more reasonable \"approximate\" solution in the presence of class overlap. Note that the support vector machine (formerly known as the perceptron of optimal stability) was designed to address this problem.*\n",
    "\n",
    "**Question:** Compute the error rate for the logistic regression classifier and compare it to the error rate for the perceptron (for `class_sep=0.5`). *Hint: you may wish to use `clf.score(...)`*\n",
    "\n",
    "**Answer:** The error rate is lower for logistic regression (around 0.23). The error rate for the perceptron fluctuates, but never seems to reach a comparably low value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1.0 - clf.score(X_test, Y_test) # fill in"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}