From eccd89815759aed337bccd37029094f0c703585f Mon Sep 17 00:00:00 2001 From: Anirudh Dagar Date: Thu, 22 Oct 2020 23:49:51 +0530 Subject: [PATCH 1/2] [PyTorch]: Add Lec 1 Neural Nets --- .../MLA-CV-Lecture1-Neural-Networks.ipynb | 481 ++++++++++++++++++ 1 file changed, 481 insertions(+) create mode 100644 notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb diff --git a/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb b/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb new file mode 100644 index 0000000..3a8a0df --- /dev/null +++ b/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb @@ -0,0 +1,481 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![MLU Logo](../data/MLU_Logo.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Machine Learning Accelerator - Computer Vision - Lecture 1\n", + "\n", + "## Neural Networks with PyTorch\n", + "\n", + "In this notebook, we build, train and validate a Neural Network in [PyTorch](https://pytorch.org/docs/stable/index.html), an open source machine learning framework that accelerates the path from research prototyping to production deployment with a clear, concise, and simple API. \n", + "\n", + "1. Implementing a neural network with PyTorch \n", + "2. Loss Functions\n", + "3. Training\n", + "4. Example - Binary Classification" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Implementing a neural network with PyTorch\n", + "(Go to top)\n", + "\n", + "Let's implement a simple neural network with two hidden layers of size 64 using the sequential container (Adding things in sequence). We will have 3 inputs, 2 hidden layers and 1 output layer. Some drop-outs attached to the hidden layers." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Sequential(\n", + " (0): Linear(in_features=3, out_features=64, bias=True)\n", + " (1): Tanh()\n", + " (2): Dropout(p=0.4, inplace=False)\n", + " (3): Linear(in_features=64, out_features=64, bias=True)\n", + " (4): Tanh()\n", + " (5): Dropout(p=0.3, inplace=False)\n", + " (6): Linear(in_features=64, out_features=1, bias=True)\n", + ")\n" + ] + } + ], + "source": [ + "import torch\n", + "from torch import nn\n", + "\n", + "net = nn.Sequential(\n", + " nn.Linear(3, 64), # Linear layer-1 with 64 out_features and input size 3\n", + " nn.Tanh(), # Tanh activation is applied\n", + " nn.Dropout(p=0.4), # Apply random 40% drop-out to layer_1\n", + " nn.Linear(64, 64), # Linear layer-2 with 64 units and input size 64 \n", + " nn.Tanh(), # Tanh activation is applied\n", + " nn.Dropout(p=0.3), # Apply random 30% drop-out to layer_2\n", + " nn.Linear(64, 1)) # Output layer with single unit\n", + "\n", + "print(net)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The weight parameters of the `Linear` layer in pytorch are initialized with a modified form of the Xavier Initialization. Using these weights as a start, we can later apply optimization such as SGD to train the weights. As a result, using a strategic technique to initialize the weights is crucial. \n", + "\n", + "Here is a full list of [Initializers](https://pytorch.org/docs/stable/nn.init.html). The commonly used one is called *Xavier initilaization*, which can keep the scale of gradients roughly the same in all the layers. (Here are more technical details of [Xavier initilaization](https://d2l.ai/chapter_multilayer-perceptrons/numerical-stability-and-init.html#xavier-initialization).)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Sequential(\n", + " (0): Linear(in_features=3, out_features=64, bias=True)\n", + " (1): Tanh()\n", + " (2): Dropout(p=0.4, inplace=False)\n", + " (3): Linear(in_features=64, out_features=64, bias=True)\n", + " (4): Tanh()\n", + " (5): Dropout(p=0.3, inplace=False)\n", + " (6): Linear(in_features=64, out_features=1, bias=True)\n", + ")" + ] + }, + "execution_count": 2, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def xavier_init_weights(m):\n", + " if type(m) == nn.Linear:\n", + " torch.nn.init.xavier_uniform_(m.weight)\n", + "\n", + "net.apply(xavier_init_weights)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can easily access them with `net[layer_index]`:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Linear(in_features=3, out_features=64, bias=True)\n", + "Tanh()\n", + "Dropout(p=0.4, inplace=False)\n", + "Linear(in_features=64, out_features=64, bias=True)\n", + "Tanh()\n", + "Dropout(p=0.3, inplace=False)\n", + "Linear(in_features=64, out_features=1, bias=True)\n" + ] + } + ], + "source": [ + "print(net[0])\n", + "print(net[1])\n", + "print(net[2])\n", + "print(net[3])\n", + "print(net[4])\n", + "print(net[5])\n", + "print(net[6])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Loss Functions\n", + "(Go to top)\n", + "\n", + "We can select [loss functions](https://d2l.ai/chapter_linear-networks/linear-regression.html#loss-function) according to our problem. A full list of supported `Loss` functions in PyTorch are available [here](https://pytorch.org/docs/stable/nn.html#loss-functions). \n", + "\n", + "Let's go over some popular loss functions and see how to call a built-in loss function:\n", + "\n", + "\n", + "__Binary Cross-entropy Loss:__ A common used loss function for binary classification. \n", + "\n", + "```python\n", + "loss = nn.BCELoss()\n", + "```\n", + "\n", + "__Categorical Cross-entropy Loss:__ A common used loss function for multi-class classification. \n", + "\n", + "```python\n", + "loss = nn.CrossEntropyLoss()\n", + "```\n", + "\n", + "__MSE Loss:__ One of the most common loss functions for regression problems. \n", + "\n", + "```python\n", + "loss = nn.MSELoss()\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Training\n", + "(Go to top)\n", + "\n", + "`torch.optim` module provides necessary optimization algorithms for neural networks. We can use the following `Optimizers` to train a network using [Stochastic Gradient Descent (SGD)](https://d2l.ai/chapter_optimization/sgd.html) method and learning rate of 0.001.\n", + "\n", + "```python\n", + "from torch import optim\n", + "optimizer = optim.SGD(net.parameters(), lr=0.001)\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Example - Binary Classification\n", + "(Go to top)\n", + "\n", + "In this example, we will train a neural network on a dataset that we randomly generated. We will have two classes and train a neural network to classify them." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.datasets import make_circles\n", + "X, y = make_circles(n_samples=750, shuffle=True, random_state=42, noise=0.05, factor=0.3)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First let's plot the simulated dataset." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEXCAYAAACH/8KRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABBMElEQVR4nO3dfXxT9dk/8M9J2qQhDdKWtJst0Buf8PYJmE7mbz+QbVhLW4qtMDe0WihgQWHUH1JbXQWtCGIZD6O3Ahu7peo6qjCdoN46tynsVsC5oogPW4G2SB9SpA1p2ibf3x9pQh7OSU5yTpKT9nq/Xrw0zenJ1dM01/k+XV+OMcZACCGEhEgV7QAIIYTENkokhBBCJKFEQgghRBJKJIQQQiShREIIIUQSSiSEEEIkoURCiB/Nzc24+uqrkZ+fj/z8fOTl5aGgoAB79+4N+L1bt27F//zP/4Qlrvnz58NkMoXl3IQEKy7aARCidAkJCdi3b5/rcUtLC+677z7odDpkZWUJft///u//4vLLLw9LTB988EFYzktIKCiREBKk9PR0LFu2DDt37sSVV16JNWvW4MKFC2hra8OECRPwq1/9Cnv27MGxY8ewfv16qNVqXH755bzHabVabN68GW+//Tbi4+ORlJSEtWvXIjU1FV9//TWqq6tx7tw52Gw23HPPPbjzzjvxyCOPAADuvfdePP/88/jud78b5StChj1GCBF0+vRpNnHiRJ+vf/HFF+yGG25gTz/9NNu7dy9jjLG+vj6Wm5vLDhw4wBhj7O6772b79+9njDHB41pbW9nkyZOZ1WpljDG2c+dO9vbbb7P+/n42c+ZMduzYMcYYY+fPn2fZ2dns448/ZowxduWVV7LOzs6w/uyEiEUtEkJCwHEcEhISsHLlSnzwwQfYvn07mpqa0NbWhgsXLvgcL3RcWloaJkyYgDvuuANTp07F1KlT8YMf/ABfffUVTp06hYqKCtc5ent78dlnn2HixIkR/EkJCYwSCSEhaGxsxJVXXomysjLYbDZkZ2fj1ltvxZkzZ8B4ytcJHadSqbB79240Njbi0KFDeOqpp3DzzTdjzpw5GDlypMfYTEdHBwwGQyR/TEJEoVlbhATp3//+N7Zt24b58+fj/fffx9KlSzFz5kxwHIdPPvkENpsNAKBWqzEwMAAAgsd9/vnnyM3NxWWXXYbFixfjvvvuw4kTJ/Af//Ef0Gq1rkRy5swZ5Obm4tixYz7nJiTaqEVCSAC9vb3Iz88HAKhUKmi1WpSVleHWW2/FihUrsHTpUlxyySXQ6XS46aabcOrUKQDA9OnTsW7dOvT39wseN2fOHGRnZ6OwsBAjRoxAQkICHn30UWg0Gmzbtg3V1dXYsWMHBgYGsHz5cnzve98DAMyYMQM///nPsW3bNlx55ZVRuzaEAADH+NrhhBBCiEjUtUUIIUSSqCSSrVu3IicnBzk5OVi/fr3P88ePH0dhYSGysrJQWVlJfcGEEKJgEU8kBw8exPvvv49XX30Ve/fuxaeffoq3337b45iVK1fisccew5tvvgnGGOrr6yMdJiGEEJEinkiMRiPKy8uh0WgQHx+Pyy67DK2tra7nW1pa0Nvb65orX1BQgAMHDkQ6TEIIISJFfNbWFVdc4fr/pqYmvPHGG3j55ZddX2tra4PRaHQ9NhqNOHv2bERjJIQQIl7UBtu//PJLzJ8/H6tWrUJmZqbr63yTyDiOi2BkhBBCghGVdSRHjhzBsmXLUFFRgZycHI/n0tLS0NHR4Xrc3t6O1NTUoM7f1WWG3a7sWc0pKYno7OyJdhgBUZzyojjlFQtxxkKMKhWHpCR9yN8f8URy5swZLF26FBs3bsQPfvADn+fT09Oh1Wpx5MgRfO9738PevXsxderUoF7DbmeKTyQAYiJGgOKUG8Upr1iIMxZilCLiiWTnzp2wWq14+umnXV+766678O6772LZsmW47rrrsGHDBjz66KMwm834z//8TxQVFUU6TEIIISINyZXtnZ09ir8DMBoNaG/vjnYYAVGc8qI45RULccZCjCoVh5SUxJC/n2ptEUKGDMYYurra0dfXC0AZN5NtbSrY7fZohzGIg0aTgKQko6yTmCiREEKGjJ6eb8FxHNLSMsBxyqgAFRenwsCAMhIJY3acO9eBnp5vYTCMku28yrjShBAiA4ulBwbDKMUkEaXhOBUMhiRYLPLOIqOrTQgZMux2G9Rq6mjxR62Og91uk/WclEgIIUMKLWD2LxzXhxIJIYRE0AMPLMLRo4fDdv729jbk52eF7fx8KJEQQsgQcejQ+3jwwfvR2dkZ0delzkRCyLDX0BCH6motWlo4pKczVFZaUVgobR8kxhhqa7fgb397D2q1GrNmFWDu3J+5nh8YGMCzzz6Nf/3ra5hMJowdOw5PPbUeAwMDePzxSlcymD9/IX74w2l4+eXd2L//T1CpOFx99TV4+OFKn9d8/fV9eOqp9SgquktS7MGiREIIGdYaGuJQVpYAi8UxdtDczKGsLAFAr6Rk8uc/v4PGxk9QV1eP3t4+LFlSgh//eIbr+WPH/om4uHg899xvYbfbsWzZ/Th06ANYLBZ85zuX4plnNqGp6d/405/+iClT/g92796FvXsPQKVSoaZmHdrb22A0etYhrK5+JuR4paBEQggZ1qqrta4k4mSxcKiu1kpKJP/4xxH86EczoNFooFLFYdeuFz2enzhxMkaOvAQNDfU4daoJzc2nYbFYcO211+O5536Njo42/OAHP8R99y1AXFwcrr32epSUFOH//t9pKCiY45NEoonGSAghw1pLC/8sJqGvixUX53mffuZMKywWi+vx++//BWvWPIaEhATMnDkLN9wwCYwxjBkzFi++uAczZmTjk08+xsKF94IxhrVrn8X/+3/lYIzhoYeW4eOPj0iKT06USAghw1p6On8pFaGvi3XDDZPxl7+8i4GBfvT29uKhhx5Ee3ub6/nDhz/Ej370E+TkzEJKSgo++eRj2O02NDT8Hjt3Pocf/egneOihcnR1deHcuXOYN+9OjB9/OUpK7sdNN92Mr7/+UlJ8cqKuLULIsFZZafUYIwEAnc4x4C7FtGnT8fnnn+Hee+fBZrNjzpyfYezYca7n8/LuwOrVlfjzn/8H8fEaXHPNtWhtbcW8eUV4/PFKFBX9FHFxcZg/fxGSkpKQn1+AhQuLoNUmIC3tO5g5M09SfHKi6r9REgsVQQGKU24Up7y84/zmm5P4znfG+fkOfuGYteWkpFpbTt7Xiar/EkKIRIWFA7IljuGIxkgIIYRIQomEEEKIJJRICCGESBK1RNLT04Pc3Fw0Nzf7PLd161ZMnz4d+fn5yM/PR11dXRQiJIQQIkZUBts/+eQTPProo2hqauJ9/tixY6ipqcGkSZMiGxghhJCgRaVFUl9fj6qqKqSm8i/xP3bsGLZv3468vDysWbMGVqu0+dyEEELCJyqJpLq6GjfeeCPvc2azGVdffTVWrVqFV199FefPn8e2bdsiHCEhhIRHuPYjsdls2LBhLe65Zy7uvnsu6utfDPxNMlHcOhK9Xo/t27e7Hs+fPx8VFRVYsWKF6HNIWVgTSUajIdohiEJxyovilJd7nG1tKsTFBX9/HP+H30P3xONQtTTDnp4By2OPo3/OT2WL0T0mjuOgVocWpz9/+tM+dHefR11dPaxWK4qL78bkyd/DhAlX+xyrUqlk/f0qLpG0trbi4MGDuPPOOwE4avp7Fz8LhFa2y4filBfFKS/vOO12e9CryLUN9dCXPQhusKCiuvk09L94AN12Bmvh3JBjE9qPhDEGm82O3t4+WfcjGTduPK6++lrY7UB8vBaXXpqOM2fO4PLLr/KJzW63e1w3qSvbFTf9NyEhAc888wxOnz4Nxhjq6uowY8aMwN9ICCEh0FevdiURJ85igb56taTzuu9H8vzzv8Mbb7yGzs4O1/Pu+5H8/vevwmrtxaFDH+Cvf30P3/nOpfjNb3bjl798Ap988g8MDAxg9+5d2LnzBezcuRsqlcqjACQAXHvtdRg//jIAQGPjJ/jss09xww2TJf0MYimmRbJw4UIsW7YM1113HdasWYPS0lL09/dj8uTJKC4ujnZ4hJAhStXiuwTB39fFitZ+JB9/fASPP16BqqonMHLkSEk/g1hRTSTvvvuu6//dx0WysrKQlRXZzesJIcOTPT0D6ubTvF+Xgm8/klGjklyP33//L9ix4znMmXMXZs6chXPnznnsR/L3vx/CBx/8FS+/vBt1dXuwdu2z+PTTRvz97wfx0EPL8MtfPoFJk77n8Rp/+cu72LDhaaxe/RQmT+af0BQOiuvaIoSQSDJXVoHpdB5fYzodzJVVks4b6f1Ijh//FBs2PI2NG38d0SQCKKhrixBCosE5oK6vXu2atWWurJI00A5Efj+S3/1uJ2w2G5588mICLClZjB/+cJqkn0MM2o8kSmJ1VoxSUZzyitU4Q92PJJyGw34k1LVFCCFEEkokhBBCJKFEQggZUoZgb72swnF9KJEQQoYMlUoNm422zPXHZhuASqWW9ZyUSAghQ4ZOl4ju7nNgTFmD20rBmB3d3V3Q6eStR0jTfwkhQ0Zi4iXo6mrH2bPNAJTRxaVSqWC3KyWxcdBoEpCYeImsZ6VEQggZMjiOQ3Iyf+mQaImVqdRSUNcWIYQQSSiREEIIkYQSCSGEEEkokRBCCJGEEgkhhBBJKJEQIjNtQz2SJ1+D0WmXIHnyNdA21Ec7JELCihIJIX4EmxS0DfUwlD0IdfNpcIxB3XwahrIHL35fXZ3H+fSryijpkJhH60gIEeBMCs79vJ1JAYDgXhWJlQ/73//7oWVQX7jgOp/utzvADR4n5vyEKBG1SAgRoK9ezZsUEisf5j1e21APzmTifU7V0uxIJoNJxHU+r+M4iwWGBxaH3DKhbjUSDVFLJD09PcjNzUVzc7PPc8ePH0dhYSGysrJQWVmJgQEqwkYiT9Xi+94EAM5k4u2SSqx82CcxOLGkJMHz+ZzfZvPsDhNB21CPlKsyYSgt8exWW7IQ+lVlos/j4tUFRwmJ+BOVRPLJJ5/gZz/7GZqamnifX7lyJR577DG8+eabYIyhvp7exCQ4rjvz1JEY/d0kjE4dKfiByHcXr19VBgiU2+YA6Hbt9PzAXr5EsDUCAGCAPT1DdPwe3WF+aBvqkTLBkUBUXSbfFg5j0O3aGXRSwqJFwuM8hHiJSiKpr69HVVUVUlN9a+K0tLSgt7cXEydOBAAUFBTgwIEDEY6QxDJtQz0My0odH4Rw3OFzAO8HIu/geGmJx9gFH84ryXB9ff6PP9cFc2WVnyN8BWrBOGNXmXwTiMdrMyYqKTnxdsGJTGxkmGJRNH36dHb69GmPrx09epTdddddrsdNTU3stttui3RoJJbs3s3YuHGMcZzjv3o9Y472BP+/lBTHP3/HyP1v3DhHrKF83+7d/D/3uHHiz8Nx4q8nx0k/BxlWFDdri/F0J3Ccv/stX52dPbDblVFCWkisVARVepzeM6tw8iQYfAex3bHOTr/Py43pdOgufwzW9m4YVSogmJLiJ0+CLVyI7u5en5lco0+dEv1z2NIzYP6vndBXr4aqpRn29AyYK6tgLZzr6Mpz+zo3KgmqLt9uOlt6BkwKey8o/f0JxEaMKhWHlJTQ9yhR3KyttLQ0dHR0uB63t7fzdoERAgjMrArwPZFKIgyAXa9Hd82Wi0kghH0pvLuVnGM6QmM4PnHodLDOyOJd36JfVebzdc7cA8TH+5wj2K45MnwoLpGkp6dDq9XiyJEjAIC9e/di6tSpUY6KKJXYmVDRwAFgScmeLYlx40I6l/Pn9BjT8XM8G/xnyxiD7pot0L79Ju9UZt1vd/h+va8PGDkStowxYBznOgetbSFCFJNIFi5ciMbGRgDAhg0bsHbtWmRnZ8NisaCoqCjK0RGlCmYmVCDh6Az1SXTV1WA6XdDncf6cfC0wPt5JRnAqs9AJOjthrqxCx9lvYTr6KSUR4hfH+AYlYhyNkchHiXG69+mzUUngzD2Ou2gJmEYDy7wiaN9+0/Ghq1KBs9l8jwP/h6/Q120ZY2A6+qnrsdFowHnnWEWAVoX7ubtrd8BaOBejU0cG3TXHdDqwBB3vuIff74uPR/fmWt4k4j2u4hxviTQlvj+9xUKMQ26MhBB/vKfrqrpMjrGClBQwjoM9ORlMJf5tzQDYk5PRvWkbzOtqYDr6KTrOfovurc/5tByYTgdLcYmjywcAU6td3UeW4hLe4/nGFayFc2E6+im6a3eIap04u8e0DfVAkBNPAEcXFjgE3RLi+vuRWOFYxe++1iZlQubF6dW0zoSAam2RGKJtqIdh6SJwXgPWXH8/0NkJqNWORYFBJBIA6Py8yedrzrtrvrtu8zr+8wx8f0pQd+ner8GSksB1dzt+nkFMp0PPU+tdx3mvXxGL6+pC97btMDywmLelJfx9Jp+ZcXwLL50TAqgLbHiirq0oiYXmLhDdOH26sM5/G9SHoBj2pGR0nmiS9Zz+BLqe/rqMRqddEnIicXaxBXsOBsCeMQbq5tOBj+U4dJz9NqT4QhULf0exECN1bZEhwbtMife0VFWXSXIS8f74ZAB6ZxdIOqfcnN1efIPcYicWMK/uL/cutlAmJ6hEJJFgz03FJYcWSiQk6vSrymBYstCjz123a6eo2Uli8Q2GcwC0b78p22uEm7myynccxusYptPBct8Cwam7fOfwhwMAtTrgcSw+3mc8SChZBNyzhcQc6tqKklho7gLhj1PbUA/DkoUhd9n4w9RqwG6HPT1DcIZUpLtjpF5P764v64ws10wzsbOnXOcIYtYYOM7v78iu0YKlpjpaL26r993Pz3Q6dNdsgb56NW9XmfcMNzFi4e8oFmKU2rVFiSRKYuHNBYQ/zpSrMoOeliqG80PL+aGaPPka2T68pFDS713omnhjAZIIIDz92ZstYwxULc285wslqSvpegqJhRhpjITELG1DPTg/SSSUWwH31dzud+a83ULDvOyHqK4y+FY65iN2UrKz5cRHzsWlJLIokZCwCTSgqq9eLbwRVKgvynG8K7GthXPRXbOFyn644bsmfVNv9Risl7sumbM7ji9hWWdkyfxqJFKoaytKYqG5C4Qep09VXlzsbgIQsI/e3wpyxMd7rLVwF+muqmAp/fcutrsrFAwAS04GJ7B/Co2RRI/Uri1akEjCQmi/c8PSRYDdLulOt3tzLRIrHgbntSPgcO+qkoOYIphMowEYE0zmfs/vZxdJsdOMifJQ1xYJC8EigSKSCIuPF16drlbDWjgXnSea0F27w9Utg3Hjhn1XlRyEximYWn2xS3DTNnRvrnWVihEr4M2DiGnGRJkokZCwYElJwR0/+M8+Qo/uzbWw3Duftx/dUlTseuy+eA9NTZREZCA0KaF763MeiyQ96oWJOK+ohCNz1QISOZRIiOy0DfXguoPrE+ac/5hj/YF5XY2jEOJgYUSmVsNSXALzuhrZ4yUXOQfgMW6cqEkJ1sK5jgF6P+dkAJheH/C1WVJyaEGTqKPB9iiJhQE4ILQ4pQ7YDtVBV2DoxqlfVQbdf//Wp1XBkpJdRSe9J194YxoNujdtC6plGQvXMxZipHUkRHGk7lqo5F0PCT/zuhp0nOlCR9t5j3+dJy52OTKV2m/Lhevr89hSmMQOSiREdqKLC0r8fhIbtA31MCxfApW5J+CAu/uWwlTUMXZQIiFBC/RHHkxhQBYf7/mYpvAOOfrq1aJ3sLSnZ1BRxxgUlUTy2muvYebMmZgxYwbq6up8nt+6dSumT5+O/Px85Ofn8x5DooP3j/yBxRj9nVEYnToSo7+bBO1Lux3buwY4FweAGQy02nyIE9tVyTQamCurHGuEeNYgUbeXckV8QeLZs2exceNGvPLKK9BoNLjrrrtw88034/LLL3cdc+zYMdTU1GDSpEmRDo8EwLvQ0H2A1WaD5q/viV5wyHV18e5QSIYOe3qGuMkXjCHuw78L1l9TtTT7VD/G02uB22bJHDEJVsRbJAcPHsSUKVMwatQojBgxAllZWThw4IDHMceOHcP27duRl5eHNWvWwGq1RjrMYc/ZfQWVyqP7SszdZTCr1oNdb0Jig3v3J3fB7CjpHwDX3w/db3cIl80ZleTTGsaiRdTlpQARb5G0tbXBaDS6HqempuKf//yn67HZbMbVV1+NVatWIT09HeXl5di2bRtWrFgh+jWkTGOLJKPREO0Q+NXVAQ8tAy5cAACom09j5EPLAEMCMHYscPKkbC+l4jjZroNir6eXIR+n1/uHM5kAjQZISADMZr/f6u8mRKXiAO/pwxcuYOTTTwD3Lwgt1giJld95qCKeSPiWrXBu1Ub1ej22b9/uejx//nxUVFQElUhoHYk0yeWPQD34IeBy4QJs5Y/AXFkFw/IlogdPnQSLMJpM6JDhOij5eroTirOhIQ7V1Vq0tHBIT2eorLSisHAgChE6SLmevO+fvj7YUtNg+vcZxzFBrjViANDZyf8eOnVKlvdQuMTCezPm1pGkpaWho6PD9bitrQ2pqamux62trdizZ4/rMWMMcXFUWzKShLqvXF8fkO8DbjhM9W1oiMPkyXqkpSUiM9Px2Pv5srIENDerwBiH5mYVysoSfI6LFQHfPwhxy1+Ov73CRlH3aLRFPJHccsstOHToEEwmEywWC9566y1MnTrV9XxCQgKeeeYZnD59Gowx1NXVYcaMGZEOc1jzt/FQYsXD4Aa3UQ0GB3jscwEMj6m+3kni5En4JInqai0sFs9rY7FwqK7WRjpcWYjZuMpjLxSxJxYqwiH3pikkaFFpkaxYsQJFRUWYPXs2cnNzcf3112PhwoVobGxEcnIy1qxZg9LSUtx+++1gjKG4uDjwiYlseAv3xceDu2D2u6NhQIwN6am+7i2PyZP1ru4qviRRUaF1HdvczP9J2NISm5+QYnej9Cj8KNDaEIPr6gr5e4k8qNZWlCi939Q5zVLd0gz7qCRw5p6gx0W8hXPTqXBeTzHjFw0NcVi2LAH9/Rc/EOPjGRxbdght0eX/wzMjw46jR/0PToeL1OvpPU3XXFnlumngey7uw79Dt2unqG19vdFmZtJJHSOhRBIlsfDmAhxx2saMlbxrnnN3xHC1QMJ1PZ1dU+6tCp2Ooaam15VMGhrisGRJAhgLLWHwiY9n2Ly5N2oD7uG6nlJ3zvQxYgTOP7tZ0S3bWPhbp0TCgxKJfIxGA5hKFfSdItNowPSJ4M51+dyRhkO4rufkyXo0N/v2ACcl2aHXA83NHDgOAkkkdBzHkJTE0NUVnVlc4bqeQrO1GAbLyPdZwZnNgTc/AwC1GtyiRWhfvU72OOUUC3/rMTdriyif+2IyZGYGPSuGcRy6N21D54kmj82QYoH3OIfQ+EVXFzeYYDjZkwjgSEwm09CYxeVOcOdMAKouE1QikojzeM5mA373O1qQqACUSIgH71paOHkSXJfJZ2Wyc0dDPiwpKWYSB3AxeaSmJmLJEs9puMIiOxBusXAoLU1wDeLHKtmne1+44Kj1NljnbXTqSKoWHAWUSIgH3lpaAGCzwZ6c7EggHOfa0ZCPkmfReLc4Vq3Suqbn8rcuhMY9osGR3JYvj93WSbDrR8TgbDZXC4UDqFpwFFAiIR5UAoPqHAZLXajVAcdLlLrIkG/h365d8T7TcwOL7rTcvj5H6yQtLRF33qnzmXIM8E9FVoJQ1o+EkrapWnBk0WB7lCh1AG70d5M8q/kGicXHo3tzbcS7tsRcT6GB89jmOStMp2O4665+vPxyvN+ZZoFE4v3JN4PLHQPAkpPBmUwhpW4GoKPtvJQQZaHUv3V3NNhO5CUhiQCO/UWUOj4Sqwv8/PNd7Pjf/+3bylLiSnmP1gnHwZ6U7Og+dS5Yrd2Bzs+bYM8YE9oLcBx1b0UIJRJycZZW6kjJ51Ly+Eh6On8rleOU3XoNltC9gBITqXN1e8fZb9F5ogmdn/vO9DNXVoXWvcUYdW9FCCWSYc5jlhak9/4rdXwEACorrdDpvD+SGBgD9Hp/89Bii0rgr1ookSqdlBau2N0ZiTSUSIY5vlla3kQPiiq8CGNh4QBqanoxYoQdF38qR/oMsE1GTLHbAbXa87em0zkWNcYiKd1TVBk4MiiRDHNy7XjI1OqYKMJYWDgAq5Wv7aW8bp/QcbDZgORkOziOISPDHtRAu9Loq1eH/NvhzD00ThIBlEiGOTm6ophOh+6tzyk+iThJnE8QIzh8++3QSI5Suqe4vj4aJ4kASiTDUCj7aXtjHAcGxFw5eP/rKSL1wRuZsQqbjRsSJVak3uzQOEn4USIZZrxLoKhMpqBv0RkAMAZ7xpiwF2MMldCCvMpKLcKTMMQN1ms0QHFxPzIy7KKOl4sSp/+KFeqsLSclTwAZKiiRDDOCJVCC4BxhUGopCr4V7MuXJ+Cqq/QwmeROIgxqNUNxcT9qa3t9ZoXFxzOPsYrf/AZYt86Ko0fNvMeHM7k0N3OKWuUulpQbFQaAM5sV9x4daiiRDDPBNPPFfKRxFgsSKx4OPaAw4NuVsK+PQ1eXo56WvDjYbBxefjkeAFBT04uMjIuJY/PmXnz+uRlnz/bg6FEz5s27+J3OWWTuxxcX90Oj8XflpSQaR1Jdtiz2urkEFyWqVD5XxL1t6KwqrMQbnqEkYImUgYEBxMV5vum+/fZbXHLJJWENTAoqkSJMcD8IjvOoocV0OljumocRv38RuHDB7zkZgO7aHVHt4nK/nmlpiWEp7R6ImB0NxfzeGxri8MADCbDZfH8GlYrBbpf+syUl2XHihHCsSivrwbshFjxvC1x7msCRPLxFaydFpV1LPmErkXLs2DFMnz4dkyZNwi9+8Qv09PS4nrvvvvtCfkEAeO211zBz5kzMmDEDdXV1Ps8fP34chYWFyMrKQmVlJQYGYnPaohIJ7adtuW+Bz37qA9+fAuh0rjs8JrDSjQMUNTMmWgvv5Fo5Xlg4gK1bfbu9dDqGhAR5frauLmXO6HKfCOJeDt67nApTq3kncDO9Htw5/uoKNOgePoKJpLq6Go8//jjee+89xMXFoaSkBH2De3ZLqfN49uxZbNy4ES+++CL27duH3//+9/jqq688jlm5ciUee+wxvPnmm2CMob6emqRy8f6DdCYN87oaV6kKc2UVEisfhqG0BOjsvLjiXa0W7FhR0h8p/wr28JMzgTm7vZKTnYPyjiRy4YIyE4AcvCeCeI/BuZdTgd3Oew7nPvB8aHFi+Agmkt7eXkybNg0pKSnYsGEDUlNT8cgjj0h+wYMHD2LKlCkYNWoURowYgaysLBw4cMD1fEtLC3p7ezFx4kQAQEFBgcfzRDr3P0j3mkbahnqk/Md3YSgtgYqn4irX3y9Yf0NJM2Pcxx4iNTMqXCvHHWM9jn9dXSpwMuURR0kYZeGdCGKxwLBkoWPjqtSRSJmQCW1DveD7zbmtM4uP93mOFieGj2Aisdvt6OzsdD1et24dvvrqK/z6178GJ+Hd3NbWBqPR6HqcmpqKs2fPCj5vNBo9nifhoW2oh2FZaeCtTu12364xjQac2ezTHRFNhYUDOHrULNsHLz9HSyFcK8f5Jg0wxslSZFKrwJnAgtvwMuZqFatMJhiWL4F1RhaYRuNxHNNoXNPRmcHgex5anBg2glM35s+fj9mzZ+PJJ5/EtGnToNPpUFtbi7vvvlvSBztft5h7Ygr0vBhSBo0iyWj0fbNHzdNPAP39AQ/jxo0DqquBykrg1CkgORnc+fPgBgc31c2nMfKhZYAhAR5TlCKA73qOHQucPBmuV+QwbhzQ1MQBEL/rn9jfe0sL/9cZ45CSArjd5wXNZFIFjCPi70+Rvyyurw8j/vgq4PVZwTGGkYYEwGgABKpQq1uao/J3p6i/9TAQTCT5+fm47rrrcODAAUybNg0AcOmll+IPf/iD63Eo0tLScPjwYdfjtrY2pKamejzf0dHhetze3u7xvBg0ayt4o0+eDDgxlul06C5/DNbbZgG3zQIwOAvM+xPtwgXYyh+BafCYSBC6nuXlcVi2LAH9/eFpmpw6xdDe3hP4wEHB/N7T0/k34kpKsg8Olkv5mRj+67+EW1HReH9qyx/zu9GVOzY4duehv9/1vktOz+CdnWhLz4Apwj+X0v7W+YR1Y6vx48fj9ddfR1VVFfr6+tDc3IzS0lLccsstIb/gLbfcgkOHDsFkMsFiseCtt97C1KlTXc+np6dDq9XiyJEjAIC9e/d6PE/kF6grigGwJyXzlkIR6o5QyuB7YeEADIbw3VSEc4YY36SB+HiG8+c5GaYAK2+lu/c2vKFcWedW0UKzE5VcnTqWBVyQuGfPHgwMDKCwsBDz5s3DnDlz8Pzzz4f8gmlpaVixYgWKioowe/Zs5Obm4vrrr8fChQvR2NgIANiwYQPWrl2L7OxsWCwWFBUVhfx6JLCA1VXVavQ8tZ53nYi/QU+lCNdU13CXZudbsKhSgXd9SSiUvNGVPWOM4HvSe/2It+TJ1wAA7+xEJZbzGQoCLm/lOA4ajQYWiwV2u13SQLtTXl4e8vLyPL62fft21/9PmDABe/bskfw6RJxArQfOZoO+ejXvH6G5ssp3oZjC7vzS0xmam+UvjTJ2rD1spdkbGuJQXa1FSwuH9HSGbdt6AQClpQmyvYaSN7oSek8GSiLupXu6a7bAXFkFffVqqFqaXQPtlEzkF7BFkpeXh56eHuzbtw+7d+/Gyy+/jNLS0kjERiJEzPx6VfNp3hlZQutSlPTH6mg1yP2hyeHECRVSUxMl1a/iKy7JVyusrCwBFRXyFZyMj1f2RldSW7TO0j3+1qUQ+QQskbJ3717Mnj3b9bi/vx8bN27Eww8rq76SOxpsF0/bUA/D8iXgBhebBsJ0OnTXbAEAJFY+DM7kmK3FNFpw/X2OmTRqNSxFxTCvqwlb3N4CXc+rrtIP1toKD52O4a67+vH223FobuagVjuKKmdkOD6wnS0X9zidCcN9iq9z9Tp/rIHux/25+PeQlMTw1FNWv62paL8/+d6XTKMBS0x0VKwWQehqRbpUSrSvpRhSB9sD3ka5JxEAiI+PV3QSIcHRV68WnUSAwQVipSWASgXObXUx1+d2d2uzQffbHQAQ0WTiz1NPWX0+tOVksXDYtSveVePLWZm/uZlDWVkCgIszpJzdVo7uNs7nPMKTlqTEzkGlcu5PL+E0keR9j9vXB5sxDVxXl0dduGApZSLIUBKwRRKLqEUi3ui0S3j/KN2rp4aKqdXoOMM/n19uYoshCn2Ah5/jiiYmcrBaWYDpyFJaHuLodMzvIspovT+1DfWOMY3m07xXIJgrQy0S8cI6/ZcMfUJ90RwAycvCFbanbWRWuwtxJK+eHohY08Ih3KVdlLjRlUetLYFjgvnVsaRkmgIcIZRIhjm++fZOUroPAAAhbOEbCeGZrST/YH64KW36L1+trVAxnQ49T61X/ESQoSK2drchsnP+URlKS2T96GIALEXFMp5RPpWV4RgvUdaHshhKm/4rx9gFA8D0iejZ8CvXe5sSR/hRi4TAWjhXeAc6AQwAi4tzrUC2a7QX9y2BY1+Ige9PkT1WOfiWaB9+wr2YMhRyLGLlALCkJEoeEUaJhAAQKCnh53gOABhDd+0OdLSdR8+mXwM63cUqrWazoufsFxYO4PPP/e9mGNuEfnvhq1YsVbDvQSE0KyvyKJEQABcXFtqTkkXP2OJsNleyENpLQulluzMyhlqLhCEpyY6pU23w/RhmKC7ux9GjZsUlEYB/cauluERwDE+IksrzDBeUSIiLtXCuY6vSIL7HmSyUXrxRCP9uirGZXDQahtraXpw4YcaePRYUF/dDrXZ0NqrVjiSybp2yurO8eW+6NvD9KWAJF7d7tusT/f52aFZWdFAiIR5C+eBXNZ8GS+Ivs6L0u0O+wohTp9qgUnmM+EC5ycURW3KyHZs2eXZXrVtnxZkzPWhr68GZMz2KTyLe9KvKYFiyEKouk6vLlLPb0Df1VjCvOdwMgD2Zv0I1CT+atUU82AX2cQCEF3hxANDd7dgp0b2kRYzcHRYWDgTs6mloiMPSpQkylG+XT6BFhbFM21AP3a6dPlPQOYsFcf/6Gt3btkNfvRrqlmbYBrfXpQQSPdQiIR6E9rtmGo3f/mquvx9Mnzhk5+wXFg7g17/uhVarhFaKcgfM5aKvXi24jknVfNrVBQa7Haajnw6Z91msokRCPFgL56J7c63HoLs9ORndm7bBvK4G3TVbBD8+uS4TVC3NsA/RO8TCwgGcPu3oKmpr60FtraNLLDwJRficGRlMsQPmodI21CN58jUYnXYJkidf49qgipdCF7oOZ5RIiA9r4Vx0nmhCR9t5gDF0ft4EwLFhkKG0RPD7OGBYlet2llypre3lGbAPhu9g/1VX2XnXuShx/YdUHqVRBt8/fuvYKKz0DqFEQvzQNtQDmZkYnToShiULXTWQxIwSxMLUX7m4D9gH2+Wl0zlmU7kP9tfW9uJvf7uAzz83u1o9zueGYncW79RxxoSvIscN+ZuUWEPVf6NE6RVBnXeJUmofMQD2jDER6eZS0vVsaIhDZaUWJpN7yuWvZVtbq8zEEMnrKVSBGhCe4MHUanRvfQ4j71+gmN+7ECW9N4XEXPXf1tZWzJs3D7fffjtKS0thNvuuLm5tbcWkSZOQn5+P/Px8LFiwINJhDntyFNBz3/Z0ON1BOlfNO8dShBY9ZmQwRSaRcPIeC9E21PudIi5YBXhwMSzq6sITKAlKxBPJ6tWr8fOf/xwHDhzAtddei23btvkc09jYiLy8POzbtw/79u3Dzp07Ix3msCfnQkLOYoHhgcUeHx7DCd+ix6E41hEI31iIobQE3NmzIU1X4CwWYPly2eMkwYtoIunv78dHH32ErKwsAEBBQQEOHDjgc1xjYyO++OILFBQUoKioCCdOnIhkmAShLST0NzrA2WweA/H6VWU+d6ZDleeiRwzZsY5AeMdCAKj6+0KvndzZOaTfO7Eioomkq6sLiYmJiItzrIM0Go04e/asz3FarRazZ8/GK6+8ggULFmDp0qXoC2I7WCIdbwE9jvOfLCB+IF63a6fnnekQ7/5yzvCy2zHkpu6KFa5yOcNlUoeShW2wff/+/Vi7dq3H1zIzM9HU1IS//vWvAICBgQFMmjQJjY2Nfs81a9YsrF+/HhMmTAhHqERIXR1QWQmcOgWMHQtUVzsenzwZntcbNw5oagrPuUn0jR4NdHbKf16OA+x2+c9LRAtbiZTs7GxkZ2d7fK2/vx8333wzbDYb1Go12tvbkZqa6vO9L7zwAnJzc5E0WL+JMeZqxYhBs7ZkctssGOfN84hz9D33hG0LJ3bqFDpCvCYxcT0xvONMYUx0F0gwe7Pb0jNgUvA1jYXfeUzN2oqPj8eNN96IN954AwCwd+9eTJ061ee4jz76CHv27AEAfPjhh7Db7Rg/fnwkQyUCgh07YRoNmErc20zpBR6JNFxXl/hjxR44YkRM1HMb6iI+a6uqqgr19fWYOXMmDh8+jF/84hcAgJdeegmbNm0CAFRWVuLgwYPIzc3FunXr8Oyzz0Il8sOIhJe5ssqn8iofBjjqbW3aJmo/iVgp8Dgc8U3ZDYUcNwpMpYI9OdlVzw3PPz/kSvHEIlqQGCWx0NwF+OMceecsaP76nt+7RsZx6Dj7LQBgdOpIgeV4DnIsWozl66lEzjj5FqYynS6ogpzOjc9Ug6VP3Bcfit1EDXC0brs3bfN43Vi4nrEQY0x1bZHYp22oh+aDvwX8wxdz98kBAMdB1dIMffXqIT1rK1ZJ3fnSY+0IBkufDM7+s2WMQXftDnTX7hDXatUnUutDoSiRENFcd6cBiuY5u6mcXSL+cIwNmynAsSiYnS/5usAE62glJwMADEsWQl+9Gpa75jm2IPATC3dO/BgLiSxKJES0QGVTXHeZNVsAwONOVAzOYkFixcPSAyWyEWpZen+dd9X6koWC5eA5k8njWN3LdTBXVjlaJ0HGQqKPEgkRzd+CMqbTobt2h2uToVBrdXFdJt5V73IN+JLg8G10xuLjfSZGCLU8BGtleT8e7C6zFs51bKDmvZUuTcZQNEokRDShO0KmVvsMvoa6ipkDfFe9P7AYhtKSkFbCUwKSgfcsvf5+GEpLPK6n342ovAi1OJzvGfO6GnRv2z5kd9sciiiRENF4y6bodOje+pzPH7mUbgiffbptNsE7WH94u1toHCYo+urV4LzKEzlL4bjXTfO7EZVI7u8Z51a6HWe/pa10YwAlEiKatXAuumu2iLpT5Es6clM1n3a1NpCZ6ZMgpM44IoFblpzFAt3vfiO4nwjv9wDUdTXEhK1EChmarIVzRd0dOo8xPLCYd5aX4IZFXusMAlE7u1ROnnTsT+H22sHMOCL87OkZF6+x4EEh1LliDLaMMVC1NMOenhGRzc9I+FCLhISNtXCu3w8Z53RPpla7ZnxZ7lsgbk0BhAdsncTOOCLCrDOyAu4VEkqnlj1jDHVdDSHUIiFhJXRH6/wg4TPw/SkXV0LD94PKX0E/99aGubKKd1U2daH4p22oBx5dhdGDlXpFrTr3c5z3c/Q7GHqoRULCSmiA3t8HiXOg1Z4xhvfDyd8Hmz09wzVTy7BkIViCzqM203Ce/SNmBpt+VRkMpSVAZ6fo/WUg4jiagTW0UYuEhJXzA0NfvTro/vBgxzIYx8E6I8ujFcJ1mRzdZ8nJw7of3rtmlnPBYNyHf4d5XY3rGN2unfJvE6BWC7Y+ydBAiYSEndgBesCtwF9LM6BSAQHKsXhgDLrf7vAdO4FjJbX3YPxw4F4w0ee6MAbdrp0Y+P6Ui4tIg5joIGbPEAbAUlQcZNQk1lDXFlEM73UfnM0WcKDXXaCumHBP/dU21CPlqkyMTh2J0akjkTLBd0pyuOlXlWH0d5McMaRdAsPSRX7L1HCMua5J0LPZ1GrBp5yTKCzFJa4WDxm6KJEQxeBd9wHhldChcF97IudKd21DPQzLSqHqMrkSmspkgmH5koglk5F3znK0yAYXcHKMgRMxNdd5TeCnNeL9DIuPh6WoWPh3w3HoONNFSWSYoERCFCNQLS+58K10l1pKRV+9Glx/v8/Xub4+Sa0gsXHpV5UF3CPGH6FWCwNgj9f4tj44DgPfn+Kq4uuNplgPL5RIiGIIrvsYnOkjR8uEb+1JYsXDkkup+EuCoS6AFFviReogudD3OfcLYWlpPotKnQmyp3p90LPyyNBDiYQohr+pwtbCubBnjAnL63JdJt5SKoYlC0WPd/i7A3c+p22oBzIzRbd6xJZ4CXaQXCznQkF/FQKCKZtDhq6oJZJNmzZhy5YtvM/19fVh5cqVyM7Oxh133IGvv/46wtGRaAj0ocSbaMIYj7MMupjxDqEV4EytBnfBjNGpI2FYshA4eZK3dcHXhSVUUVfVfBqj0y5BylWZSJmQKUvlXR9uXVmBKgRQgUUS8UTS3d2NiooK/OY3vxE85oUXXoBOp8P+/ftRUVGB8vLyCEZIosnfhxJfohEi9gMzqFlhXuMd7jOkvKcdM+e5bTaoTCbX4LfH+SwWGEpLHEnGu0z+8iXCcQyeS9Vlcp07EFcJmuKSgDsRYjBup1AWlZLhJeKJ5J133kFmZiaKi4Xnlr/33nuYNWsWAOCmm25CV1cXWltbIxUiUTDvROOvu8u5ot2uT/Q7uyiYsQVnN49+VZnnDCnv0yLwdGR/x3F9fbItDGTx8a5Nx8zramA6+mngfdLHjXP9L3VfkUAinkhmz56NRYsWQe1nDnpbWxuMRqPrsdFoxDfffBOJ8EiMMVdW+ZQkBwanDY/Qo+Pst+j8dyv6pt7qO4UV8DvllY+zBAvfwkcl4PsZLXff6/rQ9ykfw5NkmU4HVFd7fI26r4g/YVvZvn//fqxdu9bja+PHj8euXbtCOp9KJT7npaQkhvQakWY0GqIdgiiKjvP+BUBpCe9T6pbmi7H/5c9AXR1QWQmcPOloiQQ7QK3RQJ2Xi5EPLZMYdPjwtYxGvPs2RhgNjp//oWXAhQuO57pMwIgRQGkp8MYbwKlTwNix4KqrgXnzYPQ5uzIp+v05KBZilCJsiSQ7OxvZ2dkhfW9qaira29sxbrB53d7ejtTUVNHf39nZA7s9nMOw0hmNBrS3d0c7jIBiIc7kjDG8FYZt6Rkwucd+2yzgtllInnxN4D024HV3r1LBMq8I2tdeh3rwg1hpBPd4OXUKHe3dSC5/xDf2Cxdge+11n1pYRkDxv3cgNt6fsRCjSsVJugFX5PTfadOmYd++fQCAw4cPQ6vV4tJLL41yVESpzJVVjjtrN/4Gg4NZ1+Eaw7DbofvtjqBmSLli4TjHwj6BxXsBv1/kcUJdbc7ZVbTRFwkXxSSSl156CZs2bQIA3HPPPejr60NOTg6qq6uxfv36KEdHlMxaOBd4/nnRg8FiV10LDaDz4RubcM6U6t62HR1t59H5eROQkiLqtT3OlZwM5mdM0W8cbgmVNvoi4cIxFoaVTFFGXVvyGYpxepdUB8RVshUitD2wjWfzLuNbfwRbuNBnoaFQHEynQ3fNFhiWLBQ1pmNPTgYboect2c/7cw+e3zvpDsXfe7TEQoxDsmuLkHDim84qen3FIAZcXMsi8AHP22U0b57Ha9uTkj023nLF4dWyEtNqYDodeqrXB7UOh6bxEjlQiyRKYuEuBRh+cYodiHdvbQh9D2+LJMQ4eVsT8fFgBgO4rq6gNgwTY7j93sMpFmKkFgkhMhJThsV7ID8SK795WxOba9H5eROt7SBRRzskEuKGb2tg64wsaN9+U3CrYCnbCQcbGyULokSUSAjxwveBbV4X/PcQMlxQ1xYhhBBJKJEQQgiRhBIJIYQQSSiREEIIkYQSCSGEEEkokRBCCJGEEgkhhBBJKJEQQgiRhBIJIYQQSSiREEIIkYQSCSGEEEkokRBCCJGEEgkhhBBJolb9d9OmTVCpVHjwwQd9nmttbUVOTg7Gjh0LABg9ejR27twZ6RAJIYSIEPFE0t3djbVr1+JPf/oTSkpKeI9pbGxEXl4e1qxZE+HoCCGEBCviXVvvvPMOMjMzUVxcLHhMY2MjvvjiCxQUFKCoqAgnTpyIYISEEEKCEfFEMnv2bCxatAhqtVrwGK1Wi9mzZ+OVV17BggULsHTpUvT19UUwSkIIIWJxjDHvLallsX//fqxdu9bja+PHj8euXbsAAFu2bAEA3jESb7NmzcL69esxYcIE2eMkhBAiTdjGSLKzs5GdnR3S977wwgvIzc1FUlISAIAxhrg48aF2dvbAbg9LfpSN0WhAe3t3tMMIiOKUF8Upr1iIMxZiVKk4pKQkhv79MsYim48++gh79uwBAHz44Yew2+0YP358lKMihBDCJ2rTf7299NJLaGtrw/Lly1FZWYny8nLs27cPWq0Wzz77LFQqReY8QggZ9sI2RhJN1LUlH4pTXhSnvGIhzliIcUh2bRFCCIkdlEgIIYRIQomEEEKIJJRICCGESEKJhBBCiCSUSAghhEhCiYQQQogklEgIIYRIQomEEEKIJJRICCGESEKJhBBCiCSUSAghhEhCiYQQQogklEgIIYRIQomEEEKIJJRICCGESEKJhBBCiCSUSAghhEhCiYQQQogkEU8kR44cQWFhIfLz83HvvfeipaXF55i+vj6sXLkS2dnZuOOOO/D1119HOkxCCCEiRTyRrFy5EtXV1di3bx/y8vLw5JNP+hzzwgsvQKfTYf/+/aioqEB5eXmkwySEECJSXCRfrK+vD8uXL8eECRMAAFdddRV2797tc9x7772H5cuXAwBuuukmdHV1obW1FZdeeqmo11GpOPmCDiOKU14Up7woTvkoPUap8UU0kWg0GuTn5wMA7HY7tm7dip/85Cc+x7W1tcFoNLoeG41GfPPNN6ITSVKSXp6AwywlJTHaIYhCccqL4pRXLMQZCzFKEbZEsn//fqxdu9bja+PHj8euXbvQ19eH8vJyDAwMYPHixaLOp1LRvABCCFGisCWS7OxsZGdn+3zdbDajtLQUo0aNQm1tLeLj432OSU1NRXt7O8aNGwcAaG9vR2pqarhCJYQQIkFUBtvHjRuHTZs2QaPR8B4zbdo07Nu3DwBw+PBhaLVa0d1ahBBCIotjjLFIvdhnn32GO+64A5dffjni4hyNodTUVGzfvh0vvfQS2trasHz5clitVvzyl7/EsWPHoNFo8OSTT+Kaa66JVJiEEEKCENFEQgghZOihEWxCCCGSUCIhhBAiCSUSQgghklAiIYQQIknMJ5JYKwK5adMmbNmyhfe51tZWTJo0Cfn5+cjPz8eCBQsiHN1F/uJUwvVsbW3FvHnzcPvtt6O0tBRms5n3mGhdz9deew0zZ87EjBkzUFdX5/P88ePHUVhYiKysLFRWVmJgYCBisbkLFOfWrVsxffp01zXkOyYSenp6kJubi+bmZp/nlHItAf9xKuVabt26FTk5OcjJycH69et9ng/perIYN336dHb8+HHGGGN/+MMf2P333+9zzI4dO9hjjz3GGGPsww8/ZHfeeWdEY2SMsfPnz7NHHnmEXX/99Wzz5s28xxw4cMAVZ7SIiVMJ13PRokXs9ddfZ4wxtnXrVrZ+/XqfY6J1Pb/55hs2ffp01tXVxcxmM8vLy2NffvmlxzE5OTns448/Zowx9sgjj7C6ujpFxrl48WJ29OjRiMfm7h//+AfLzc1l11xzDTt9+rTP80q4lowFjlMJ1/KDDz5gP/3pT5nVamV9fX2sqKiIvfXWWx7HhHI9Y7pFwlcE8syZMz7Hvffee5g1axYAzyKQkfTOO+8gMzMTxcXFgsc0Njbiiy++QEFBAYqKinDixIkIRuggJs5oX8/+/n589NFHyMrKAgAUFBTgwIEDPsdF63oePHgQU6ZMwahRozBixAhkZWV5xNfS0oLe3l5MnDjRb/zRjhMAjh07hu3btyMvLw9r1qyB1WqNeJz19fWoqqrirW6hlGsJ+I8TUMa1NBqNKC8vh0ajQXx8PC677DKPv91Qr2dMJxKpRSAjafbs2Vi0aBHUarXgMVqtFrNnz8Yrr7yCBQsWYOnSpejr64tglOLijPb17OrqQmJiomtRq9FoxNmzZ32Oi9b19L4+qampHvHxXT+++MMtUJxmsxlXX301Vq1ahVdffRXnz5/Htm3bIh5ndXU1brzxRt7nlHItAf9xKuVaXnHFFa4k0dTUhDfeeAPTpk1zPR/q9Yxo9V8pYqUIpL84A3nwwQdd/z9t2jQ8++yz+Ne//uVqcclJSpx8Ink9MzMzfY7jON8y2JG8nu4Yzxpf9/gCPR8pgeLQ6/XYvn276/H8+fNRUVGBFStWRCQ+MZRyLQNR2rX88ssvsXjxYqxatcrj7ynU6xkziSRWikAKxSnGCy+8gNzcXCQlJQFw/FKdd91ykxJntK9nf38/br75ZthsNqjVasHXj+T1dJeWlobDhw+7Hre1tXnEl5aWho6ODtfjaBUlDRRna2srDh48iDvvvBNA5K5fMJRyLQNR0rU8cuQIli1bhoqKCuTk5Hg8F+r1jOmuLWBoFYH86KOPsGfPHgDAhx9+CLvdjvHjx0c5Kl/Rvp7x8fG48cYb8cYbbwAA9u7di6lTp/ocF63recstt+DQoUMwmUywWCx46623POJLT0+HVqvFkSNH/MYf7TgTEhLwzDPP4PTp02CMoa6uDjNmzIh4nP4o5VoGopRreebMGSxduhQbNmzwSSKAhOsp23SAKPj000/ZlVdeyWbOnMlmzZrFZs2axUpKShhjjL344ovsV7/6FWOMsd7eXvbwww+zmTNnstmzZ7Njx45FLebNmzd7zIZyj/Obb75h9913H8vJyWEFBQWu2WjR4C9OJVzP5uZmdvfdd7Ps7Gw2f/58du7cOZ84o3k9//jHP7KcnBx22223seeff54xxlhJSQn75z//yRhj7Pjx46ywsJDdfvvtrKysjFmt1ojFFkycBw4ccD1fXl4etTgZc8zQdM6GUuK1dBKKUwnX8oknnmATJ050fV7OmjWLvfjii5KvJxVtJIQQIknMd20RQgiJLkokhBBCJKFEQgghRBJKJIQQQiShREIIIUQSSiSERBhjDOXl5di5c2e0QyFEFpRICImgr7/+Gvfeey/2798f7VAIkQ0lEkLC4NVXX8WPf/xjmM1mXLhwAdnZ2di7dy/q6upQUFAQcnkaQpSIFiQSEiYPPfQQDAYD+vr6oFar8cQTT7ieKy8vxxVXXBHVzcsIkYuyKrARMoSsXr0a+fn5SEhIwCuvvBLtcAgJG+raIiRMOjs7YbVacf78ebS1tUU7HELChlokhIRBf38/ysrKsHz5ctjtdpSVleHFF1/k3eaAkFhHLRJCwqCmpgZGoxFz5szBT3/6U4waNQobN26MdliEhAUNthNCCJGEWiSEEEIkoURCCCFEEkokhBBCJKFEQgghRBJKJIQQQiShREIIIUQSSiSEEEIkoURCCCFEkv8PilQlNHfC/o4AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sns\n", + "\n", + "def plot_dataset(X, y, title):\n", + " \n", + " # Activate Seaborn visualization\n", + " sns.set()\n", + " \n", + " # Plot both classes: Class1->Blue, Class2->Red\n", + " plt.scatter(X[y==1, 0], X[y==1, 1], c='blue', label=\"class 1\")\n", + " plt.scatter(X[y==0, 0], X[y==0, 1], c='red', label=\"class 2\")\n", + " plt.legend(loc='upper right')\n", + " plt.xlabel('x1')\n", + " plt.ylabel('x2')\n", + " plt.xlim(-2, 2)\n", + " plt.ylim(-2, 2)\n", + " plt.title(title)\n", + " plt.show()\n", + " \n", + "plot_dataset(X, y, title=\"Dataset\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we import the necessary libraries and classes." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "from torch.nn import BCELoss" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then, we create the network as below. It will have two hidden layers. Since the data seems easily seperable, we can have a small network (2 hidden layers) with 10 units at each layer." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# Use GPU resource if available, otherwise wil use CPU\n", + "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n", + "\n", + "net = nn.Sequential(nn.Linear(in_features=2, out_features=10),\n", + " nn.ReLU(),\n", + " nn.Linear(10, 10),\n", + " nn.ReLU(),\n", + " nn.Linear(10, 1),\n", + " nn.Sigmoid())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's prepare the training set and validation set, and load each of them to a `DataLoader`, respectively." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "# Split the dataset into two parts: 80%-20% split\n", + "X_train, X_val = X[0:int(len(X)*0.8), :], X[int(len(X)*0.8):, :]\n", + "y_train, y_val = y[:int(len(X)*0.8)], y[int(len(X)*0.8):]\n", + "\n", + "# Use PyTorch DataLoaders to load the data in batches\n", + "batch_size = 4 # How many samples to use for each weight update \n", + "train_dataset = torch.utils.data.TensorDataset(torch.tensor(X_train, dtype=torch.float32),\n", + " torch.tensor(y_train, dtype=torch.float32))\n", + "train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size)\n", + "\n", + "# Move validation dataset on CPU/GPU device\n", + "X_val = torch.tensor(X_val, dtype=torch.float32).to(device)\n", + "y_val = torch.tensor(y_val, dtype=torch.float32).to(device)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Before the training, one last thing is to define the hyper-parameters for training." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "epochs = 50 # Total number of iterations\n", + "lr = 0.01 # Learning rate\n", + "\n", + "# Define the loss. As we used sigmoid in the last layer, we use `nn.BCELoss`.\n", + "# Otherwise we could have made use of `nn.BCEWithLogitsLoss`.\n", + "loss = BCELoss(reduction='none')\n", + "\n", + "# Define the optimizer, SGD with learning rate\n", + "optimizer = torch.optim.SGD(net.parameters(), lr=lr)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, it is the time for training! We will run through the training set 50 times (i.e., epochs) and print training and validation losses at each epoch." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 0. Train_loss 0.691067 Validation_loss 0.680190 Seconds 0.074899\n", + "Epoch 9. Train_loss 0.046372 Validation_loss 0.038246 Seconds 0.058565\n", + "Epoch 19. Train_loss 0.005466 Validation_loss 0.005107 Seconds 0.057234\n", + "Epoch 29. Train_loss 0.002521 Validation_loss 0.002381 Seconds 0.066555\n", + "Epoch 39. Train_loss 0.001571 Validation_loss 0.001485 Seconds 0.057680\n", + "Epoch 49. Train_loss 0.001119 Validation_loss 0.001055 Seconds 0.058494\n" + ] + } + ], + "source": [ + "train_losses = []\n", + "val_losses = []\n", + "for epoch in range(epochs):\n", + " start = time.time()\n", + " training_loss = 0\n", + " # Build a training loop, to train the network\n", + " for idx, (data, target) in enumerate(train_loader):\n", + " # zero the parameter gradients\n", + " optimizer.zero_grad()\n", + " \n", + " data = data.to(device)\n", + " target = target.to(device).view(-1, 1)\n", + " \n", + " output = net(data)\n", + " L = loss(output, target).sum()\n", + " training_loss += L.item()\n", + " L.backward()\n", + " optimizer.step()\n", + " \n", + " # Get validation predictions\n", + " val_predictions = net(X_val)\n", + " # Calculate the validation loss\n", + " val_loss = torch.sum(loss(val_predictions, y_val.view(-1, 1))).item()\n", + " \n", + " # Take the average losses\n", + " training_loss = training_loss / len(y_train)\n", + " val_loss = val_loss / len(y_val)\n", + " \n", + " train_losses.append(training_loss)\n", + " val_losses.append(val_loss)\n", + " \n", + " end = time.time()\n", + " # Print the losses every 10 epochs\n", + " if (epoch == 0) or ((epoch+1)%10 == 0):\n", + " print(\"Epoch %s. Train_loss %f Validation_loss %f Seconds %f\" % \\\n", + " (epoch, training_loss, val_loss, end-start))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's see the training and validation loss plots below. Losses go down as the training process continues as expected." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEXCAYAAACtTzM+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1PElEQVR4nO3deXhU5dn48e85M5N9T4Yk7KD1RhZRwR1XtFZcqnUtda/aWmtf2+qv1hW11LpSbbWLilpbqhZffWulrqBVUSnIpsAjqBAICYSEkASyzcz5/TETGEKAJORkZjL357q4mLPO/SQw9zzn2SzHcVBKKaXsWAeglFIqPmhCUEopBWhCUEopFaEJQSmlFKAJQSmlVIQmBKWUUoAmBNVHichQEWmIdRx7IyKOiBTFOg6lQBOCUkqpCG+sA1Cqt4lILvAYcDDgAP8GbjHGBETkLuAcoAWoBi43xlTsbn+7e64FDjDGVEb2fQzcBXwZeb8soD+wCLjQGNMUdf3lwHnGmDPab4tICnAfcDzgARYCPzHG1InItcAPI3E1AT8wxizr0R+YShpaQ1DJ6FHCH+pjgPHAWOBGERkE3AAcZowZD7wJHLG7/dE3NMZsAV4GLgYQkQOBUuAN4GrgWWPMUcD+wDDg9C7EezMQAMYZY8YC64HfiIgH+C3wLWPMYcCfgQld+UEoFU0TgkpGpwG/N8Y4xphm4I+RfeXAYuBTEXkQWGSMeWUP+9t7Args8voK4GljTAj4BVAlIv8P+APhWkJWF+I9A/g2sFBEFgFnAyONMUHgH8BcEfk9sAV4qgv3VWonmhBUMmr/794GfJEP7+OBywnXIKaJyCO729/+psaYDwCviBwOTAamRw79HbgGWANMAz4FrHaXO+32pUS99gD/Y4w52BhzMHA4cF7kPS8GzgRWEU48/9upn4BSHdCEoJLRG8B1ImKJSCrhD+u3RGQs8Bmw3BhzL+EP77G727+bez8J/A5YYowpi+w7FbjbGPMC4Q/+Iwh/yEerAkaLSJqIeAl/yEfH+2MRSRERm3BN5F4RKRKRtUC1Mea3wG17iEupvdJGZdWXZXbQ9fQo4CeEP7SXEv4m/jow1RjTIiIvAvMj1zUSbrxd3NH+3bzns8Cvge9G7bsFeFlEaoBtwHuE2xKivRnZvwKoAOYAB0WO3QM8SLgx2UO4UfrnkUblXwHviEgj4XaGqzr1k1GqA5ZOf62UUgr0kZFSSqkITQhKKaUATQhKKaUiNCEopZQCEreXUSpwGOHeGMEYx6KUUonCQ3gE/X+B5vYHEzUhHAa8H+sglFIqQR0LfNB+Z6ImhAqAzZu3Egp1vdtsYWEW1dVxPzOyK5K17Fru5KLl7phtW+TnZ0LkM7S9RE0IQYBQyOlWQmi7Nlkla9m13MlFy71HHT5q10ZlpZRSgMs1BBGZTHh+lRRgmjHmsahjBwPPRJ3uBzYbY0a7GZNSSqmOuZYQRGQAMBUYR7g1e66IzGlbvMMYs4jwAiWISAYwj/BCH0qpGHMch82bq2hpaSI8H19i2bjRJhQKxTqMXtdWbo/HS1ZWHunpmV263s0awsnAbGNMDYCIzCQ8Ze/dHZz7S+C9yPTBSqkYa2jYgmVZFBcPxLIS78my12sTCCRfQvB6bVpbg7S2tlBbWwXQpaTgZkLoz84t2RWE53HfiYjkEZ5+eExX36CwsCtrjOzM78/u9rWJLlnLruXuvOrq9RQWFuP1Jmq/k/CHYzLy+Tz4fOl4PP2oq6th8OCSTl/r5m+7/QIgAB2l7O8BrxhjNnb1DaqrG7rVk8Dvz6aqqr7L1/UFyVp2LXfXtLa24jiJ+y07mWsIbeW2bR/Nzc07/f5t29rjF2k3U2g5EJ2aSgmvBdve2cDzLsaxk/JNW7nsrjd4+T9f0dQS6K23VSrhWFZH3+lUoujO78/NGsLbwBQR8QNbgXMJPxraTkQswo3OH7kYx0765aUxenghr85dzX+WrOc7xw3nmNGl2Lb+41cqHj300H0sXbqYQKCVdevWMnTocADOP/8iTj/9rE7d4/LLJ/PMMzN2e/yDD95jxYrlXHXVvvVrmTp1CoccMo5Jk87c+8lxyLWEYIwpF5FbCa/8lAI8aYyZJyKzgDuMMfMJdzVtMcY0uRVHez6vh5suGc+EMSW88M5Knp61gnfmr+PCid/gwCH5vRWGUqqTfv7zXwBQUbGe66//wR4/2Hdnb9dMmHA8EyYc3634+hJXW4yMMTOAGe32TYp6vZGdHyu5LrRlA+X/msqggQfzywtP5L9f1jPz3VU88PeFHPKNIi6a+A38eem9GZJSqpvOO+9MRo4czcqVhscff5IXX/w7Cxb8l/r6OnJz85g69X4KC4uYMGE8H3wwn6ee+hObNlWxdm0ZGzZUcsYZ3+ayy77PrFmvsnDhAm69dQrnnXcmp546iXnzPqKxsYnbbruLESMO5KuvVjF16l0Eg0HGjj2Yjz+eywsvvNKpOGtqqvnNb+5hw4ZKPB4P11xzHUceeTTz58/j8ccfxbIssrOzmTLl1/h8XqZMuZXq6moArrzy6l5LVonbhaCbrPRs7LQMGuf9g5ZF/+LgkRM5+OKTePvzOv710Rpuf/ITzj52OKccNhCPnZy9FJRq78OlFXywpMPpb/bZhINKOWZMabevP/LIo7n77ntZt24tZWWr+eMfp5OS4uXOO2/jzTdf57vfvXin81etWsnjjz9JQ0M9F1xwNt/5zgW73DM3N5cnnvgLM2c+z3PPTWfq1Af41a+mcPXVP+Sooybwwgt/Ixjs/ETL06Y9wKGHjueiiy6mvHwdP/rRVTz99N949tmnuOmmX3LggaP4xz+e54svVlBdvYmSkv488MAjrF79Na+99s9eSwhJ94lnpWRQ+t07yDjnTrwDRtGy6DVa/vELJgb/w9SL9mPk0AJenLOKXz27gDWVydcrRalEM3JkeHKDgQMH8eMf/5RXX32FRx55mM8/X0pj47Zdzj/00PH4fD7y8wvIyclh69ZdJ4M74oijARg+fH/q6uqoq9tCZWUFRx01AYDTT/92l2L89NP/csYZZwMwYMBARo4czbJlnzFhwnHccstNPPzwfQwdOpTDDz+S0aMP4v333+WXv/w5S5Ys4vLLv9+l99oXSVdDaOPxDyP9lB8Tqq2kZfEsWle8h2/5u/xg/HdYOuow/vb2Su55dj7fPGwQ354wjNQUT6xDVipmjhmzb9/i3ZSamgrAihXLmTLlVi66aDInnTQRy7JwnF27paekpGx/3ZlzHMfBtj0dntdZu3aPdwgGg1x44fc45pjjmDv3fR5//FFOOOFzLrvs+8yYMZOPP/6IDz/8D88//1f+9reZvdLrK+lqCO3ZeSWkHX8lmd99EO+w8bT8dyajKv/JPVceyoSDSnl9Xhm3P/UJq8q3xDpUpdQeLFq0gEMOGcfZZ5/HsGHDmTfvkx6bviIrK4uBAwfy0UcfAvDWW6936QN63Ljx/OtfrwBQXr6OpUsXM2rUQVx99WVs27aVCy6YzAUXTOaLL1bw0ksv8NRTf+Kkk07m5z+/mc2bN9PQ0DtTeSdtDaE9OzOftInX0pLfn5YFr+Cpr+LSb17PUaOKmT5rOY/8YzG3XTqe4oKMWIeqlOrAxInf5JZbbuKyyy7C6/Wy3377U1HR0dCn7rn11ru49967eeKJx9lvv29sr5m09+CD9zJt2v1R249yww03cf/9U5k161Usy+IXv7iNoqIifvCD65g69S48Hg+pqancdNMvKS4uYcqUW7n00gvxer1ceeU1ZGf3zih7a1+qQTE0FPjarZHKras+pum9J7Ey8kn/1g1UWwX86tn5ZKX7uO3ScWSk+bofeYzpiN3k0t1yV1auoaRkiAsR9Q43Rio//fQTnHnmORQVFfHee7N5881/M3XqAz36Hvuqfbnb/x6jRioPA1a3vz7pHxl1xLf/kWSccTMEmtn2yq8oqF/FdeeMpqq2kT+88hnBJJxFUalkV1xcwk9/+iOuuGIyL730Itddd0OsQ+px+shoNzzF+5Nxzp00vv5bGl+fxvATr+GSU4Vn/r2C599exfe+eUCsQ1RK9aJJk85M2BHInaU1hD2wswrJOOsW7H7DafrgL0w4IItTDx/EO5+uY/an62IdnlJK9ShNCHthpaSTdtwV0NpM8ycvcv4J+3PQfoXMeGsln6+uiXV4SinVYzQhdIInfwApB51K4IsPCG1cxQ/OGkVpYQZ/ePkzKmt2HfiilFKJSBNCJ6UcehZWZgHNH/yFNJ/FT847CNu2+PM/P9+nAStKKRUvNCF0kuVLI/Wo7xKqWUvr5+/gz0vnnGOHsbqynq8rkq87o1Kq79GE0AXeYePxDBxN8/z/JbStliNHlZDq8/DuwvJYh6ZUn/WjH13FW2+9vtO+xsZGJk2aSG1tbYfX3H33ncya9SqbNlVx440/6fCcCRPG7/F9168v5957w0vAr1ixjN/85p6uB9/OU0/9iaee+tM+38ctmhC6wLIs0o65GIIBmj9+nvRUL0eNKmbe8g1sbWqNdXhK9UmTJp3JW2+9sdO+996bzaGHjiMvL2+P1xYV+XnwwUe79b6VlRWUl4d7E44YMZKbb769W/dJJDoOoYvs3BJSxp5Gy8JXCYw4nuMPHsi7i9Yz97NKThk/KNbhKeWK1i8+pNX8x5V7++Q4fAccs9vjJ510Co899gh1dVvIyckF4I03ZnHBBZNZuHABf/7z4zQ3N1FfX8+11/6Ek046efu1bYvqzJz5KhUV67n77ttpbGxk1KjR28+pqtrIvffeQ0NDPdXVmzj55FO59trreeSRB1m/vpyHHrqPE0+cyPTpf+b3v/8zZWVruP/+qdTX15GWls4NN9zIgQeOYurUKWRmZmHMcqqqNnLFFVd3ekW3pqYm7rvvV6xa9QW2bXPRRRdz2mlnsGrVSu6/fyrBYJCUlBRuueVOSkv7c++9d/HVV18CcM4553PWWed050e/C60hdEPKIWdgZRfR/OFzDO6XzvD+Oby7sFwbl5VyQUZGBsceezyzZ78NwKZNVZSVreGII47ipZde4Oabb2f69L9x882388wzT+z2PtOm3c+kSWfyzDMzGDNm7Pb9b731Bqeccip//vMzPPvs87z88kxqa2v5n/+5EZEDt6/Y1uaee27n/PMv4tlnn+f663/Gbbf9gpaWFgA2btzA448/yX33TeOxxx7pdBmnT/8Tubm5PPfcizzyyB+ZPv0JVq1ayYsvzuCiiy7mqaee47zzLuTzz5eydOli6urqePrpGfz2t4+zdOnirvw490hrCN1geVNJO+p7NL75CK1L3+SEgw9h+qzlfLG2Fhmsy3Cqvsd3wDF7/BbvttNPP4snnvgDZ599Lm+++W9OPXUStm1z++33MHfu+8yZ83Zk/YPG3d5j4cIFTJkyFYBvfvO07W0CkydfwqefzmfGjOf4+usvCQRaaWrq+D7btm1j3bp1HH/8SQCMHj2GnJwcysrWAHD44UdgWRbDh+9HXV3nZ0hesGD+9kdSeXl5HHvscSxcuICjjjqGhx++n08+mcvRRx/LCSdMpKGhnrKyNfzsZz/myCOP4dprr+/0++yN1hC6yTv0EDwDR9Oy9E3Gj/CTkepljjYuK+WKsWMPobp6Exs2VPLGG//e/ijmuuuuZvnyzxEZwaWXXrmXWrq1fTJMy7KwIysi/u530/jHP56npKSUyy77Prm5ebu9j+OEdjnmOGxfPS0lJXX7/bvCcULttiEYDHDiiSczffpfIyuq/Z0HH7yX3Nw8nnvuRc4990LKytZw5ZUXU1/fMz0dXU0IIjJZRJaJyCoRua6D4yIi74rIYhF5Q0QS6uu1b78jcLbV4qsr5+gxJSwwVdRtbYl1WEr1SaeddgbPPvsUOTk5DBgwkLq6Laxdu4bvfz+8rOW8eR/vcf2D8eMP5403ZgHhRum2xzzz53/C5MmXcNJJJ7Nx4waqqjYSCoXweLy7LJOZmZnFgAEDee+92QB89tlSamqqGT58v30q26GHHsZrr/0fALW1tbz//rsccsh47rjjlyxb9jlnn30uV131Q4xZwQcfvMfdd9/O0UdP4IYbbiQ9PZ2NGzfs0/u3ce2RkYgMAKYC44BmYK6IzDHGLIsct4B/Av9jjHldRH4D3Az8Ynf3jDeeQQcBEChbxImHnMzb89fxwdIKJh2ZuNMGKxWvvvWt0zn//LP45S/vACAnJ5czzjibSy65gMzMTEaNOoimpqbdPjb62c/+H/fccwf//Of/MmLESDIyMgG4+OLLueeeO8jKyqagoIARI0ayfn05BxwgNDTUc889t++0ZOYdd9zDAw/8mqee+hM+XwpTp96Pz9f5KfGfe+5pnn/+r9u3b7rpFq644ioeeug+Lr30QkKhEJdeeiUiI7jkkiu4775f8eyzT+LxeLj++p8yZsxY5sx5h0suuYCUlBSOP/4k9ttv/+78SHfh2noIInIZcJwx5vuR7dsByxhzd2R7HPCEMebQyHYOkGeMKevE7Yfi4noIXbH15bvAssg8+w7un/Epm7Y08ZsfHoXdC8vddYeuC5BcdD2E5BLP6yH0ByqitiuAgVHb+wOVIvKsiCwB/gD0zjpxPcg7eCyhjV8TaqzjhEMGsGlLE8u+1knvlFKJx81eRh19RY5O2V7gBMK1iPkicg/wMHB5Z98gkum6xe/vmSXpmsceTfmCV8ioXck3jz6Ov7+zkrnLNnDiEUN75P5u6KmyJxotd+dt3Gjj9SZ2n5NEj7+7ostt23aXfv9uJoRy4Nio7VIgeoHTSmClMWZ+ZPvvwMyuvEE8PDJyPEVY6bls/vwT0kvHcczoUl7/pAzzZRUFOWk98h49SR+dJJfuljsUCtHaGuxyb5l4oY+MwHEcQqHQTr//qEdGHXIzhb4NTBQRv4hkAOcC0ROSzAX8ItI2QuRMYIGL8bjCsmy8gw8isHYpTijA8Qf3x3Ec3l9SsfeLlYpTtu0hGAzEOgy1D1pbW/B4uvad37WEYIwpB24F5gCLgBnGmHkiMktExhtjGoFzgCdE5HPgJODnbsXjJs/gsdDaSLByJf68dEYPL+Q/i9cT0pHLKkGlp2dRX1+7S/94Ff8cx6GlpZna2iqysvK6dK2rI5WNMTOAGe32TYp6/QlwuJsx9AbvgFFgewiULcbb/0DGiZ+lX1VTtbmR4oKMWIenVJdlZeWyeXMVGzasAxLvi41t23sck9BXtZXb4/GSnZ1Penpml67XqSt6gJWSjqd0BMGyJXDkRQwtCTfirNlQrwlBJSTLsigo6BfrMLpN24y6Jzmb4V3gHXwQodr1hOo20r8oE6/HYk1l8v2DVEolLk0IPcQ7ONw2HihbjNdjM8CfxZoNmhCUUolDE0IPsXNLsHJLCJSFp6IdUpzNmsp6nRJbKZUwNCH0IO/gsQQrVuC0NjO0JJutTQGqtzTFOiyllOoUTQg9yDt4LAQDBMuXMSSqYVkppRKBJoQe5Ck5AHxpBMoWMdCfiW1ZmhCUUglDE0IPsjxevANHE1i7BK/Hpn9RJqu1p5FSKkFoQuhh3sFjcbZuJlRdxtASbVhWSiUOTQg9zDNoDBDufjqkJJv6ba3UNugqakqp+KcJoYfZGXnY/mEE1i5hSHGkYVkfGymlEoAmBBd4B40htOFLBub7sIDVlXWxDkkppfZKE4IL7IJBgIOvsYqSwgzKNiTcQnBKqSSkCcEFdl4pAKHainDDsnY9VUolAE0ILrBz+oFlEaqtYEhxNpvrm9myVRuWlVLxTROCCyxvCla2P5wQIiOWy7SWoJSKc5oQXGLnlRLaUsGgfuGEoAPUlFLxThOCS+y8UkK1laSn2hTnp1OmCUEpFec0IbjEziuFYCtOQzVDtGFZKZUANCG4JLqn0ZDibDZtaaKhsTXGUSml1O65uqayiEwGbgNSgGnGmMfaHb8D+D6wObLrifbnJKrohDC4ZCAQngp71NCCWIallFK75VpCEJEBwFRgHNAMzBWROcaYZVGnHQZcZIz5yK04YsVOy8ZKzQrXEPaP9DSq1ISglIpfbj4yOhmYbYypMcZsBWYC57U7ZzzwCxFZIiK/F5E0F+PpdeGG5Qqy0n0U5aZpO4JSKq65+cioP1ARtV0BHN62ISJZwELgRmA18AxwO3BrZ9+gsDCr28H5/dndvrazqkoGs23VAvz+bL4xOJ81FXW98r57Ew8xxIKWO7loubvOzYRgdbAv1PbCGNMATGrbFpGHgOl0ISFUVzcQCnV9rQG/P5uqKve/rbekFRHcWsvGdZWU5Kfz0dIK1qzdTEaaq003e9RbZY83Wu7kouXumG1be/wi7eYjo3KgJGq7FFjftiEig0XkyqjjFtCnuuG0n9MIYO3G5PtHqpRKDG4mhLeBiSLiF5EM4Fzg9ajjjcD9IjJMRCzgOuBlF+PpdTv1NNK1EZRScc61hGCMKSf8+GcOsAiYYYyZJyKzRGS8MaYK+AHwKmAI1xAeciueWLCyi8D2EqqtIDczhfzsVG1YVkrFLVcfZhtjZgAz2u2bFPX6JeAlN2OIJcv2YOcWE6oNt60PKc5mja6NoJSKUzpS2WVtXU8BhpRkU1G9leaWYIyjUkqpXWlCcJmdV0qorgonFGBwcRaOA+uqtJaglIo/mhBcZueVghMkVLeRfvkZAFTVNsY4KqWU2pUmBJdF9zQqyg0PxN60pSmWISmlVIc0IbjMzg0PxQjVVpDq85CTmcKmLVpDUErFH00ILrNS0rEy87c3LBflplFVqzUEpVT80YTQC6J7GhXlplGtj4yUUnFIE0IvsHPDCcFxHPx56VTXNXVrDiallHKTJoReYOeVQksjTuMWCnPTCIYcNtc3xzospZTaiSaEXhDd08ifmw6gDctKqbijCaEX7NT1NE+7niql4pMmhF5gZeaDN5VQbQUF2WlY6OA0pVT80YTQCyzL2t7TyOe1yctO1RqCUiruaELoJdFdT/25aZoQlFJxRxNCL7HzSnEaqnECzRTlpWujslIq7mhC6CU7GpYrKcpNY3NdM4FgaC9XKaVU79GE0Et2nuQuHQeoqdPHRkqp+KEJoZfYOf3AssJjESJdT6u0HUEpFUc0IfQSy5uCle0nVFtBYds02Nr1VCkVR1xNCCIyWUSWicgqEbluD+edLiJfuxlLPGjraVSQnYbHtrSnkVIqrriWEERkADAVmACMBa4RkZEdnFcMPAhYbsUSL+y8UkJbKrFwKMjRsQhKqfjiZg3hZGC2MabGGLMVmAmc18F5TwJ3uRhH3LDzSiHYitNQTVFuuj4yUkrFFTcTQn+gImq7AhgYfYKI/AT4FPjYxTjixvbV0+o2hBfK0RqCUiqOeF28d0ePgLZ3vBeR0cC5wETaJYrOKizM6l5kgN+f3e1ruyuQOpQyINOpZ8iAIby/pIKcvAxSfZ5ejSMWZY8HWu7kouXuOjcTQjlwbNR2KbA+avv8yL75QArQX0TeN8ZEX7NH1dUN3Vpoxu/PpqqqvsvX7Ssn5APbQ13FOtIzhwGwYlUV/Ysyey2GWJU91rTcyUXL3THbtvb4RdrNhPA2MEVE/MBWwrWBa9oOGmPuBO4EEJGhwLtdSQaJyLJtrKxCQnVV+PvvWBehNxOCUkrtjmttCMaYcuBWYA6wCJhhjJknIrNEZLxb7xvv7Gw/ofpNui6CUiruuFlDwBgzA5jRbt+kDs5bDQx1M5Z4YWcXEVizkNzMFHxem021mhCUUvFBRyr3Mivbj9NYB4EWCnPSqNJZT5VScaJTCUFEikXkrMjr34rIbBEZ625ofZOdXQSw/bGRPjJSSsWLztYQngH2E5GTgBOBvwCPuhVUX9aWEJz6Kvw6OE0pFUc6mxAKjTHTgNMINw4/A2S4FlUfZmX7gUgNITeNrU0BGpsDMY5KKaU6nxBSRMRHOCG8LSIZQPdHhSUxKz0HPCmE6qsoymvreqqPjZRSsdfZhPB/QBWwyRizAJhHu95DqnMsy8LOLsKJ1BBAp8FWSsWHTiWEyCCy0YTbDwAmG2PucS2qPs7KLgrXEHJ1oRylVPzodC8j4FBjjCMivwV+KyIHuRpZH9Y2OC0r3UdqikdrCEqpuLAvvYx+51ZQfZ2dXQQt26BlG/5c7XqqlIoP2ssoBqzosQi56WzSwWlKqTigvYxiwM5p63patX1dBMfp+qytSinVk7SXUQzYWW2D0zZRlJdOc0uQrU06FkEpFVtd6mVkjDkhskt7Ge2L1EzwpROqr8Lf1tNIG5aVUjHW2V5GNjBZROaIyAfA2SLi6kypfZllWdg5RYTqN1GYq9NgK6XiQ2cfGd0LnAQ8AjwMHA084FZQycDO9ofnM8rbsVCOUkrFUme/5X8LGG+MaQUQkdeAxcBP3Qqsr7Oyigit+4yMFA+ZaV5dF0EpFXOdrSHYbckAwBjTDLTu4Xy1F3aOHwItOE31FOWm67oISqmY62wNYZGITAN+H9m+DljiTkjJIXoa7KK8NNZv2hrjiJRSya6zNYTrgHxgLvAR0A/4o1tBJYPoabD9uels0rEISqkY61QNwRhTB1wevU9E6oAcF2JKCjtWTquiMHcArYEQW7a2kJeVGuPIlFLJal+6jlp7O0FEJgO3ASnANGPMY+2OnwPcBXiA/wLXGGNa9iGmhGH50rDSsnHqNuEfsKPrqSYEpVSsdPaRUUf2+HxDRAYAU4EJwFjgGhEZGXU8k3CbxCnGmFFAGu1qIX3djmmwI11PdXCaUiqG9iUh7M3JwGxjTI0xZiswEziv7WBk31BjzIZIcugHbHYxnrhjZxcRatixUM5GTQhKqRja4yMjEamn45qAxd5nO+0PVERtVwCHR59gjGkVkdOAvwLlwJt7C7gvsbP9BFYvxOe1KMhJZUONJgSlVOzsrQ1h9D7cu6M2hlD7HcaYfwOFIvJr4A/A5M6+QWFh9ydc9fuzu31tT6krHcimxQEK0gIM7JdNTUNzr8QVD2WPBS13ctFyd90eE4IxZk237xz+xn9s1HYpsL5tQ0QKCI9+bqsV/A14oStvUF3dQCjU9a6afn82VVX1Xb6upwWs8C+uavXXFGSl8N8VG12PK17K3tu03MlFy90x27b2+EXazTaEt4GJIuKPrJ9wLvB61HEL+KuIDI5sXwB84GI8cWfH4LRNFBdksLUpQEOjDgBXSsWGawnBGFMO3ArMARYRXmltnojMEpHxxphq4BrgXyKyGDgA+IVb8cQjK6sQCA9OK84PN8ls2LwtliEppZKYq1NYG2Nm0G4hHWPMpKjXrwCvuBlDPLO8KVgZeYTqqygeGu56uqFmG/v1z41xZEqpZOTmIyPVCVZ20fZpsC0L7WmklIoZTQgxZmf7CdVvwuuxKcpN00dGSqmY0YQQY3Z2Ec7WGpxQgOL8DDZs1hqCUio2NCHEmJ3tB8fBaagJJ4SabTrrqVIqJjQhxJi1fdbTTRQXpNPUEqRum3Y9VUr1Pk0IMWZvXxehiuKCSNfTGm1HUEr1Pk0IMWZlFYBlhwen5e/oeqqUUr1NE0KMWbYHK6sgslBOGh7b0oZlpVRMaEKIA3ZWEaH6TXhsG39eunY9VUrFhCaEOGBl+3HqNwFQnJ+uj4yUUjGhCSEO2DlFONtqcQItFBdksHFzIyHteqqU6mWaEOKAnRXpetoQnvW0JRCitr45xlEppZKNJoQ4YOWEu55qTyOlVCxpQogD28ci1FVFTYOtPY2UUr1LE0IcsDLywJdOaHM5+Tmp+Ly29jRSSvU6TQhxwLIsPIWDCFaXYVsW/fLTdRpspVSv04QQJ+zCQYSq1+I4ocisp1pDUEr1Lk0IccIuHAyBZpy6KooL0qmqbSQU0q6nSqneowkhTngKBgEQrC6jOD+DQNChuq4pxlEppZKJJoQ4YRcMBMsiVLNWu54qpWLC6+bNRWQycBuQAkwzxjzW7vi3gbsAC/gauMIYs9nNmOKV5U3Bzi0lVL2WkhE7up6OjnFcSqnk4VoNQUQGAFOBCcBY4BoRGRl1PAf4A3C6MWYssASY4lY8icCO9DTKyUwhNcWjNQSlVK9y85HRycBsY0yNMWYrMBM4L+q4D/iRMaY8sr0EGOxiPHHPLhyE01ANLdsozk+nUnsaKaV6kZuPjPoDFVHbFcDhbRvGmGrgFQARSQduBn7XlTcoLMzqdnB+f3a3r3XLtmFC5TzIDlYzuCSHL9dtcSXOeCx7b9ByJxctd9e5mRCsDvaF2u8QkVzCiWGxMebZrrxBdXVDt7pm+v3ZVFXVd/k6t4V84Sksar4y5GXux4aabVRUbsHr6bmKXLyW3W1a7uSi5e6YbVt7/CLt5iOjcqAkarsUWB99goiUAu8Di4GrXIwlIVjpuVhp2YSqyyjOTyfkOFTV6ohlpVTvcLOG8DYwRUT8wFbgXOCatoMi4gH+BbxojPmVi3EkDMuysAsHh8ci7L+jp1FpYWaMI1NKJQPXEoIxplxEbgXmEO52+qQxZp6IzALuAAYBhwAeEWlrbJ5vjEnqmoJdOIjWz9+mOC8VgI3a00gp1UtcHYdgjJkBzGi3b1Lk5Xx0YNwuPAWDaA0GyGiuJjPNS6VOg62U6iX6gRxn7KJwz9tQTRn98jN0LIJSqtdoQogzdl4p2N7wiOWCdDbqWASlVC/RhBBnLNuLnd9/+yR3NXXNtLQGYx2WUioJaEKIQ+G1EcroV5COA2zUrqdKqV6gCSEOeQoH4zTWUZoRANDV05RSvUITQhyyC8MNy0WhagBtR1BK9QpNCHGobbEcX305ORk+Kqo1ISil3KcJIQ5ZaVlYmQUEq9cyvH8uK8o24zi6nKZSyl2aEOJUW8PymOEFbNrSxAYdoKaUcpkmhDjlKRxMqLaCUUNyAFj6VXWMI1JK9XWaEOKUXTgInBCFTg3FBRl89lVNrENSSvVxmhDilKcgMoVF9VrGDCvAlG3WAWpKKVdpQohTVk4/8KYQrC5j9PBCWgIhvlhXG+uwlFJ9mCaEOGXZNnbBQELVZcjgPLweWx8bKaVcpQkhjnkKBxOsXkuK10YG52nDslLKVZoQ4phdOBhatuFsrWHMsAIqqrexaYt2P1VKuUMTQhxrG7EcirQjAHz2tT42Ukq5QxNCHLMLBgIQrF5LaWEGhTmp2o6glHKNJoQ4ZqWkY+eWEKxYgWVZjB5eyLLVNQSCoViHppTqg1xNCCIyWUSWicgqEbluD+c9KyKXuxlLovJ+42iC5csIbdnA6GGFNLUE+bJ8S6zDUkr1Qa4lBBEZAEwFJgBjgWtEZGS7c/qLyKvA+W7Fkeh8cixYNq0r3uPAIfl4bEvbEZRSrnCzhnAyMNsYU2OM2QrMBM5rd873gP8DXnQxjoRmZ+bjHXIIreZ90n0O+w3I1e6nSilXuJkQ+gMVUdsVwMDoE4wxDxhjnnQxhj7BN/JEnKZ6Al8vYMzwAso2NLCloTnWYSml+hivi/e2OtjXo62hhYVZ3b7W78/uwUjc5RQdwdq5xbDqfY498We89N5XlFVv46RhRd26XyKVvSdpuZOLlrvr3EwI5cCxUdulwPqefIPq6gZCoa4vHOP3Z1NVVd+TobjOPuA4mub9g4ytleRkpjB38XrGDMnv8n0Ssew9QcudXLTcHbNta49fpN18ZPQ2MFFE/CKSAZwLvO7i+/VpPjkWbA+BFe8xelgBn39d061kqJRSu+NaQjDGlAO3AnOARcAMY8w8EZklIuPdet++yk7PwTt0HK1ffMCYIdk0NLayujL5vgEppdzj5iMjjDEzgBnt9k3q4LzL3Yyjr/CNPJHAV/MYaX2FBXz2VTXD++fEOiylVB+hI5UTiKd0BFZuCfaX7zO0NIdPV1bhOPrYSCnVMzQhJBDLskg58ARCG1Zx6gEWZRsaeH9Jxd4vVEqpTtCEkGB8B0wAj5cxwc8ZMTiPF2avZHO9jklQSu07TQgJxkrLwjvsMAIr53L5KUMJBh2ee8PooyOl1D7ThJCAfCNPgtYm8qqXcs5xw1m0ahOfLN8Q67CUUglOE0IC8hTvj50/gNYlb3DyWD/D++cw462V1G1riXVoSqkEpgkhAVmWReqRFxLaUknzO49xxanfoLE5wIy3voh1aEqpBKYJIUF5Bx1E6rGXEVz3GQXLXuTMo4cwb/lGFq6sinVoSqkEpQkhgaWMOJ6UcWcTWPkhp/gWMNCfxV/eMGxrao11aEqpBKQJIcGlHPptfCOOJ7D4Na4dWUX91laen70q1mEppRKQJoQEZ1kWqRMuxTP4YLKWzuTSkdv4YEkFT7z6OY3NgViHp5RKIJoQ+gDL9pB+8rXY/YZzyMaXuexQDx8v28Bdz/yX1ZV1sQ5PKZUgNCH0EZY3lfRv3YCVVcSh5X/nrqPqCAVamfqXBbw5r0wHriml9koTQh9ip2WTcfpNeIr3J3fFK9xe8C9OH7iZ52ev5JGZS3ScglJqjzQh9DF2ViHpp/2c9G/9FNvjYWLDP7lnyAdsKVvJndPn8fK7q6jXxKCU6oCr6yGo2LAsC+/gsXgGjqJ1+Xvkzn+Zn2a/xnLPCF6fVc5z+DnkgH4cP7Y/MiQf2+po+WulVLLRhNCHWbaXlFET8e1/JM0LX2XkZ28zMnc5jZ4sFq0bxCsrB1KXNZQJYwdw6AF+SgoysDQ5KJW0NCEkASs1k7QjLyL1kDNJ32zYvORDjlq7lKN8y2kijcXzB/DKRyXU+PqR338o+w8u4IBBuQzql4XH1qeKSiULTQhJxErNJHvM8TSVHIoTaCaw9jO8X8/n8DWLOKL1SwBaN3lYV1nA5x8V8A5+yBtARmEJhf5C+hdlUlKYQUFOmj5mUqoP0oSQpCxvKr5h4/ANG0daKERoSwWhqtX4Nq1myIavGVzzNZ6ggSagHLat9VETymJFKJvNZBNIzcfKyMWbmUdqTgGZ+YXk5ueQn5VKdoaPzDQftq1JQ6lE4mpCEJHJwG1ACjDNGPNYu+MHA08AucB/gB8aY3R4bS+zbBtP/gA8+QPwHXAMaYATChGqqyRUW0mobiNOzQaKNldSWL+JlOZyPE4QthL+szF8n2bHS30ojc1OKtucFFqsNALedILeDEjJwPKlY6Wm4UlJx5uWgS8tA196Jr70NFLTMkhJTyc11Ueaz0NqiocUr61tGkr1ItcSgogMAKYC44BmYK6IzDHGLIs67a/AVcaYj0XkKeBq4A9uxaQ6z7JtPHn98eT1ByA16pjjhHAa63Ea63AatxCo38y2LTW01m3Gu7WWnOat5LZuw27dgi9YSUqgCTvQuYFxrY7NNsdLreOlFQ8BvAQtL0E8BG0fIcuLY3lxbM+Ov20veHxge7BsL3i8YHuxPB4sjxfL9pKemUZzq4Pl8WLbHiyPB9vjxfZ4sD1t2z5se8e2Z/vfduQ6G4/Hi2V7sD02tm2Hr7dtbNvCtixsG2zL0kSmEpKbNYSTgdnGmBoAEZkJnAfcHdkeAqQbYz6OnP8McBeaEOKeZdlYGbmQkQsMwguk7eF8x3GgtQmnpRGntYlQcyMt2xpo3raNlsatBFqaCLU0EWxpJtTajNPaghNogkArdiiAJ9iCFQpgOwHsUGP472AQ2wniIfLHCeKxen80dggIOOBgEcIihI0DOE74dQgrcl5kPxZO22vLjmyH94MV3raiXrft374PwAIr6hhA1DVEkpGDhe2xCYWInGNv38/2hGURDjGybdk79sP2991xTvvzrXav2+2Lfq9drou6Z/Rxq+P9VvT1u7z3zsdSU300twSwtt8Pdhp2ZbW/D9vPdbbfLjrG6L/al6H9Pa2oXe2/GOx8r53eu90pHd7TsrBtD0PHHUl6RiY9zc2E0B+oiNquAA7fy/GBLsajYsSyLEhJx0pJB8AD+ICe/ufsOCEIBSEYgFCQUDBAsLWVQGsLOdmpVFdtIRgMEAwGCAUChAJBQsEgTjBAKBTECQbDf4eCOIFAuCYUCoW3I38T2YcT3hf+tHXC773TH6fd6/DHPqEQ4GA54b+jj4X3sfOxtrTgRFKH05YSdpwT3s/241ZU6rGCRM5jp/3he+z8d9s527d3TjlR+3fdZ0Wdv+Pv3V0bPkebmLrvq6YGRp3y7R6/r5sJoaNfd6gLx/eqsDCrSwFF8/uzu31tokvWsucPinUEqiM75tnakfDYaV/4nFAkeTqR423XOc6O65y24xBJ3M5Ot3OcnfdtPzsU2nGdEzkv8vbRMe6yP/LCCUXF265cOy7Zyz3b/1wi93TY+WdiezwcN2w49m66hO/L/283E0I5cGzUdimwvt3xkj0c36vq6gZCIWfvJ7bj92dTVVXf5ev6gmQtu5Y7uey+3J69Xxz9xGsf7PoAqedUV2/tcP/eft+2be3xi7Sbo47eBiaKiF9EMoBzgdfbDhpj1gBNInJMZNelwL9djEcppdQeuJYQjDHlwK3AHGARMMMYM09EZonI+Mhp3wOmichywo+UH3UrHqWUUnvm6jgEY8wMYEa7fZOiXi9m54ZmpZRSMaIT1SillAI0ISillIrQhKCUUgpI3MntPMA+TZ6WzBOvJWvZtdzJRcu9x2Md9r+1EnTx9QnA+7EOQimlEtSxwAftdyZqQkgFDiM83UUwxrEopVSi8BAeBPxfwpOO7iRRE4JSSqkepo3KSimlAE0ISimlIjQhKKWUAjQhKKWUitCEoJRSCtCEoJRSKkITglJKKSBxp67oNhGZDNwGpADTjDGPxTgkV4lIDjAXOMMYs1pETgYeBtKBF4wxt8U0QBeIyJ3ABZHN14wx/y9Jyn03cB7hRRqfMsY8nAzlbiMiDwB+Y8zlInIw8ASQC/wH+KExJhDL+HqaiMwGioHWyK4fAPuxD59vSVVDEJEBwFTCU1+MBa4RkZGxjco9InIE4eHpB0S204HpwLeBA4HDROS02EXY8yIfgN8EDgEOBsaJyHfp++U+HjgJOAgYD1wvImPp4+VuIyITgcujdv0VuN4YcwDhVSyvjkVcbhERCxgBjDXGHGyMORhYxz5+viVVQgBOBmYbY2qMMVuBmYS/UfVVVwPXsWOt6sOBlcaYryPflv4KnB+r4FxSAfzcGNNijGkFlhNOiH263MaY94ATI+XrR7j2n0cfLzeAiBQQ/iD8dWR7CJBujPk4csoz9L1yC+Ga4L9FZLGI/Jge+HxLtoTQn/AHRpsKYGCMYnGdMeYqY0z0JIB9vvzGmM/bPghE5BvAhUCIPl5uAGNMq4jcBSwD3iEJft8RfyK8XO/myHYylDuf8O/4bGAi8ENgMPtY7mRLCB3NCxvq9ShiJ2nKLyKjgLeAG4EvOzilT5bbGHMn4AcGAd/o4JQ+VW4RuQpYa4x5J2p3n/93boz5yBhzqTFmqzFmE/AUcHcHp3ap3MmWEMqBkqjtUnY8TkkGSVF+ETmG8Lenm40xz5IE5RaREZGGVIwx24D/BU6kj5ebcA3wmyKyiPAH4lmEH5X26XKLyIRIu0kbC1jNPpY72XoZvQ1MERE/sBU4F7gmtiH1qk8AEZH9ga+ByYQbHfsMERkEvAJcaIyZHdnd58sNDAfuEpEJhJ8tf5vwo5QH+nK5jTGntL0WkcuBE4wxV4jIZyJyjDHmQ+BS4N+xitElecDdInI04AMuAy4G/rovn29JVUMwxpQTftY4B1gEzDDGzItpUL3IGNNEuCfGS4SfM68g3PDUl9wIpAEPi8iiyDfHy+nj5TbGzAJmAQuBBcBcY8zz9PFy78H3gGkishzIBB6NcTw9yhjzL+A1dvy+p0eS3z59vul6CEoppYAkqyEopZTaPU0ISimlAE0ISimlIjQhKKWUAjQhKKWUiki2cQhKdYqIOMBnQLDdobONMatdeC9/ZMSpUjGjCUGp3TtRP6RVMtGEoFQXicgJwAOEp8QYDjQClxtjlotILvAY4am3HcIjZG8xxgQi05E/SnigVAtwY9Ro6rtE5EigEHigr6/ToeKTtiEotXtz2kY7R/68HHXsUOAhY8xBwNPAc5H9jwLVwBjC6xKMBW4UER/hKTXuNsaMJjzfziMi0vZ/8CtjzDjgHOChyPlK9SqtISi1e3t6ZLQ4amrx6cBjIlIInAYcY4xxgGYR+SNwA/AmEDTGvAZgjFlAOGkgIgAzIvdaBKQCOYQTi1K9RmsISnVP9HKMVuRPkF3/T9mEJx8LEH6EtJ2IjBaRti9lrQCRRNJ2T6V6lSYEpbrnYBE5KPL6GuBDY0wt8AZwnYhYIpIaOfYWYABHRE4BEJFDgdno/0EVR/SRkVK7N0dE2nc7vQXYBlQCU0VkKLARuCRy/CfA74ClhBc6fx2YaoxpEZHvAL+NLAbfAnwnst/9kijVCTrbqVJdFOll9PtI47BSfYZWV5VSSgFaQ1BKKRWhNQSllFKAJgSllFIRmhCUUkoBmhCUUkpFaEJQSikFaEJQSikV8f8BuK5ZlFx8AXQAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "import seaborn as sns\n", + "\n", + "plt.plot(train_losses, label=\"Training Loss\")\n", + "plt.plot(val_losses, label=\"Validation Loss\")\n", + "plt.title(\"Loss values\")\n", + "plt.xlabel(\"Epoch\")\n", + "plt.ylabel(\"Loss\")\n", + "plt.legend()\n", + "plt.show()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 3310c2b2e020c55d9d5fda54a037195c93188a97 Mon Sep 17 00:00:00 2001 From: Anirudh Dagar Date: Thu, 22 Oct 2020 23:56:57 +0530 Subject: [PATCH 2/2] Fix mlu logo --- notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb b/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb index 3a8a0df..94dc318 100644 --- a/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb +++ b/notebooks/pytorch/MLA-CV-Lecture1-Neural-Networks.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "![MLU Logo](../data/MLU_Logo.png)" + "![MLU Logo](../../data/MLU_Logo.png)" ] }, {