main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reproduction of the paper [Visualizing and understanding RNNs](https://arxiv.org/abs/1506.02078)\n",
    "Project for the subject [344.063 Special Topics Natural Language Processing with Deep Learning ](https://kusss.jku.at/kusss/sz-lvadetail-overview.action?courseId=344063%2C2020S)\n",
    "\n",
    "Authors: M. Vogl, F. Shalaby\n",
    "\n",
    "Limitation: We only worked with 3mb warandpeace.txt, as the linux kernel dataset grew from 400mb during the writing of the paper to 800mb, which wouldn't fit into my GPU as a one-hot-able LongTensor.\n",
    "\n",
    "Training for the 3x4x6 grid (with early stopping and parameters seen in config.txt) for 3mb warandpeace.txt took ~4h on a GTX2070Super.\n",
    "\n",
    "**Improvements to other implementations**\n",
    "* Using pytorch and lightning instead of lua-torch unlike the [original kapathy charrnn](https://github.com/karpathy/char-rnn)\n",
    "* Still working with a current pytorch version unlike the great [huango implementation](https://github.com/huanghao-code/VisRNN_ICLR_2016_Text) \n",
    "* Shoter code - easier to understand: Old implementation. 1500+ LOC. New: 400 LOC (Counted with <br>`projects find proj -name \"*.py\" -print -exec cat {} \\; | wc -l`)\n",
    "* This version is extending the torch builtin [LSTM](https://pytorch.org/docs/master/generated/torch.nn.LSTM.html)/[GRU](https://pytorch.org/docs/master/generated/torch.nn.GRU.html)/[RNN](https://pytorch.org/docs/master/generated/torch.nn.RNN.html) with activation gate extraction, which is 10-100x faster than the other implementations. Includes support for apex 16 bit precision, cuda and optimized dataloading.\n",
    "\n",
    "This file assumes you have downloaded the models (too big for github) from [http://öä.eu/models.zip](http://öä.eu/models.zip).\n",
    "\n",
    "Hyperparameters from config:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import torch\n",
      "\n",
      "# linux = https://cs.stanford.edu/people/karpathy/char-rnn/\n",
      "_FILE_NAMES = [\"warandpeace\", \"shakespeare\"]\n",
      "FILE_NAME = _FILE_NAMES[0]\n",
      "FILE_PATH = f\"data/{FILE_NAME}.txt\"\n",
      "DEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
      "N_LAYERS = 3\n",
      "SEQ_LEN = 70\n",
      "HIDDEN_SIZE = 32\n",
      "BATCH_SIZE = 128\n",
      "PREDICT_SEQ_LEN = SEQ_LEN\n",
      "LR = 0.01\n",
      "CLIP = 5\n",
      "DROPOUT = 0.0\n",
      "MAX_EPOCHS = 100\n",
      "EMBEDDING_DIM = 0  # 0 = one-hot encoding\n",
      "MODEL_NAME = \"lstm\"\n",
      "PRECISION = 32  # 16 if DEVICE == \"cuda\" else 32\n",
      "SPLITS = (0, 90, 95, 100)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(open(\"config.py\").read())  # hyperparameters used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.display import display, IFrame\n",
    "from matplotlib_venn import venn3\n",
    "\n",
    "import dataloader\n",
    "from config import *\n",
    "from visualize import visualize_gate, visualize_cell\n",
    "from net.charrnn import CharRNN\n",
    "%matplotlib inline\n",
    "\n",
    "(trl, tel, val), vocab = dataloader.load(FILE_PATH, DEVICE, SPLITS, BATCH_SIZE, SEQ_LEN, unique=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rand\tHello !yHxxQ?O?viEGKECB:7OfMVOSNQ()4B*vYHMMMK)fJVg)RA7oAYO*giGPwM6L;x9RHxJ=g\n",
      "max\tHello the said her and the said her and the said her and the said her and th\n",
      "softrand\tHello ghens the ce rrasha rracke's. his by ce whot sicce of thestood reary a\n"
     ]
    }
   ],
   "source": [
    "net = CharRNN.load_from_file(\"rnn\", 1, 32).eval()\n",
    "for method in \"rand\", \"max\", \"softrand\":\n",
    "    print(method, net.predict(\"Hello \", PREDICT_SEQ_LEN, vocab, method=method), sep=\"\\t\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model accuracy comparison via venn diagram"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPEAAADtCAYAAABu35PpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO29d3hcV7X3/1nTNeq9uciW5RJ3Oy5JSOJAmgMp9J4Q+v3dC4EL5IVcwBGEfmkvcC+/S7ghQCghEIhDGiE97o4t965mW1bv0+fs948zcmRZZSRNOSOdz/PMI82cfc7eR5rv2XuvvdbaopTCxMQkdbEkuwEmJiaTwxSxiUmKY4rYxCTFMUVsYpLimCI2MUlxTBGbmKQ4pohNTFIcU8QmJimOKWITkxTHFLGJSYpjitjEJMUxRWxikuKYIjYxSXFMEZuYpDimiE1MRkBEHCLyiIjUiYgSkQ2TKS8iXxCRAyLSKyK1IvKFIcdXiMjLItItIqdF5KvRtNMUscm0QERsEzz1FeADwLkYlBfgdiAXuBH4NxF5z6DjvwNeAvKAq4F/EZFbxqrQFLFJSiMiq0RkT6R3+5OI/FFE7hORDZHe7P+IyDngARH5kIi8MuR8JSLzhru2UiqglPqRUuoVIDxWW8Yqr5T6rlLqNaVUSCl1FPgbcMWgIhXAQ0qpsFLqJPoDYfFY9ZoiNklZRMQBPAr8Cr33+j3w1kFFSiKfzwY+nuj2jYaICHAlcHDQxz8CbhcRu4gsAC4Dnh3rWqaITVKZ9YAN+L9KqaBS6i/AjkHHNWCTUsqvlPImpYUjcy+6/h4Y9NnjwDsAL3AE+KVSaudYFzJFbJLKlAFn1IWJ4hoH/d6qlPJFcyERmSUifQOvWJcfcu6/oc+N36yU8kc+ywOeAr4GuICZwA0i8v+NdT1TxCapTBNQHhmaDjBz0O9Ds0D2A+6BNyJScr6gUg1KqYyB11gVj7f8oDo/DHwReJNS6vSgQ3OBsFLq15E582ngD8BNY13TFLFJKrMV3YD0byJiE5FbgbWjlK8BFkeWclzoQ9pRERFnpCyAQ0RcQx4aUZcXkfcD3wSuU0qdGnLqMb2IvE9ELJEHzLsjbR69jWbK2umBVIsFfZjmApxDfg4sv6hhXiHAD/giLw/gVZuM8cURkUuB+4F5wJOAFdgDvAz8Vik1Y0j5/wA+iz7v/BLwG6BKKXVihOvXoRvGBjNHKVU33vIiUgvMQP97DvBbpdQnI+e+EfgOMD/Svs3AXUopzwi3r9dpinhqIdWSAeQM83KPdt44Uehi7gE6gY7Iq1NtUv7RTow3IrId+LlS6oExC08RTBGnMFItNqAIKEVfTikC7EltlC7uDnRnh7NAi9qktHhVJiJXA0eBNuD9wM+BuUqppnjVaTQm6sUSE0RkCfB9YDWQr5Qaca4RKX8H8GmgCr0X+B1wj1IqFDn+AvqyQyhyyhml1IJB57uB/wTehf5lr1FKXRXLe4onkSFxObpVthQowHh2DXfkNTCMDUm1DAj6LNAa46H4AuBhIAM4CbxjOgkYxtkTi4htQDAxqVxf0H4D+lP0r1GI+F+AA8B2oBB4DPiTUurbkeMvoM8x7h/h/N+iP7g+hd5brFBK7Y7N3cQHqRYrusV1Dvpcy5HcFk0aP1CLLrizRplbpzJjijgyUf9v9KHKAvQv0YeAr6M/cX+olPpGpOy9wCXoBpC3Ag3AHUqpXWPUMQ84PpaIhznv34FrlFI3R96/wAgijjwwdgIzlFI946kn0USEOxtduLNI/hA5XniICFptUtH6JpsMIdqh2HuBNwMrIu/fgC7oNwFfFZFFg8regr6+lYPeU/40Nk0dlqu40G0N4Fsi0iYirw6JIlkH1APVkeP7ReTtcWzbuJFqyZBqWYv+wLwWqGTqChj0TmAxcItUy/ukWlZJ9fnlGZMoiXZO/H+VUo0iUhF5Xx1xY6sRkRpgOXA4cuwVpdQTACLyG+AzMWzveUTkTuBS4KODPv4/wCEgALwH2CwiKyLO5DOAJcCf0eeUlwF/F5FDSqnDJBGplhnoX+ZZ6JEu05EM9P/nSqmWE8B+tUl1JLlNKUG0Im4c8n7w0MeD/g8Y6ZgrEgb2buD/j3z+slJq42gVRhbGhy0vIrcB3wauVUq1DXyulNo+6BIPish70T1efoK+7hYE7ovM618UkeeB63n9AZQwIkaqhcBSIDvR9RsYK/oob4FUy1l0G0i9OXcemWhFPOk/oFLqIeChyZYXkRuBX6D7ne4f6zK83rPti7bueCLVIuiL+auAzCQ3x+iURV6dUi071abhHSymO0ldnhAdFxGLa8RFzTlK+TeiC/vtSqkdQ47liMgNkWvYIj35VcDTkSIvoRvavhQ5fgWwYdDxuCPVUom+vHU1poDHQy5wvVTLrVL9ur+ziU5S14nRLbC1g9570Y1PFSOU/wr60POJQe6rA0NtO3Af+hA1jB7KdVsk+BqlVDDiW3s/ugN6PXC7UupILG9oOKRaZqH79OZd8LlC5YYJFwYJFwZRhSG0giDKFRn3DHhIaAI+Aa8F8VgQrwXVY0VOO7GdtWNTMm3m0cXoRrAGYIc5Z9YxPbbiiFRLpkVx2Ww/ZUu8hKp8UBjEkhtCssNY3RpWyyQNWWFQ7TaCTQ7CjQ60OidS58Ra78TusxjOESSWKHRPrW1qkwokuzHJxBRxrBGxBRwU//MmyuqymV8SIM2epGlLr4XQKRfB3emobRnYmxxTcrnKA7yqNqnaMUtOUUwRxwKRAvQpQFlnPll7LsfqyWDEuX288YdcYV8oPewLpWu+kFvzhdJVSLPTYnGogy6bOp6ZT316GWGxW/XniwbWXoWtXbC1g73Ngq3Diq3disWXKr15LbqYR434mYqYIp4oukGuCn05JE8T1IFL8TRU4iYBc9Rg2BHuDeSGun0F4R5/vuoN5Fj7g9nWQNhpA8uY9YcR7Vxanv9o9ixOZZa5NBnpnKCGtS+MtTeMrV3DdQLSDlpIO2zH6rHG+r4mSQB9eB13O4eRMEU8HnRr2kx04c4mMkz2uAnu2IDWlx2/3rfXnxNs9ZSH2jzlqstXZPeH3TEbGvst9tCJzDL/vtxKR68jPcrragpbewjXySBphyDtgBXXKQcSNoKRrR54frrMlU0RR4NIDrpwqxgSl9s0A+/ey3CEbcS0V/KF0kJneysD5/pmS7e/0B7SHAlZSWhx5XgP5MwZo3ceiaCGoymA+0CYrGdtpB1P2pQC6AX+oTa97gw0VTFFPBoihehLQ+VDDylQB1fjqZtPeqyq6wtkB073VAWb+iqsfYFcJ4ycBibe+C320L7cuYF9uZWusMU6sXmxtTNI5qsBsp+x46pNRvRVGNiiNiXXrTbemCIeDpFsYA168rKL8LsIbb+aUE8ek3bW94dc4bquxb7Gnvl2byjTcGGGPos9tCe/KnAwZ07a+HvmQdjaAmS+EiT7GTvOxkTf53HgZbUpdmG0RsIU8WD0pAGr0B1Ghu19OvPxb9+ANeSYuKOMUqKa+2f5aruW0OYpdUVjiEo2HqszuCt/QfBo9qw0NUqiuKiwNQfIejlI7mYnto5EORx1AE+pTWpc6WVTAVPEACJ29EisZYzixdZSim/XlTg068TWfYNhR/hk5zJfffciRyCclpJrtn02V+DVoqXh+oyStMlfLazI2OGl4CEbzvpE9M796EJuT0BdCWN6i1jEgp7EYBV61kdegtwPwp19ununug5e+gM89718Cr7Vz0c1hV2E8PuX8rsb51HnDWL94rN8sM3DLA2siwvZ+uWreKqlH+e/P83dAJpCeYKSW5heuO2WBbf9KXk3HDsa0ou8LxSvcPhsztgY9FxHvBT8TkjfE+944iDwjNqkzsS5noQxfUUskgm8Ed0f9zzbIfsQZN8JDfXgXAFf/mop3/9OD3e8cQ7Pvm8pBx/az5JnT3HDA7fy/Z/vYu3+Fpb/7CZ+0enF8YnHufeeK/n+ihLaPcGM4OG2tcGm3rmuX9c89JXlJcsfXla87Hhybjj2BMQa3lq0JHA0e1YMeuUIzlM+Cn9FnMWsAf+cKl5eyQ6ASA4ileibWV00hFsH3eugG2A2+DMdnH6tkFJ6oD9AGkBfgLR0O136pVAhDYc/hKU3gN0ihLMc9v6a5vX9jd0L0hQWe31XfVFQC2YuLVo6ZQQM4FBh69XNNWnzexq9z5Wssvfb0yb/ffLPdXH6a/EWswW4VqrlJbVJD5BJZaZXT6wnJ7gCfc13TH66gLQvnOSrP91I9clOcn64jbsAQSFfuZrvLCumwxvE+vl/8OFzfSwMaziWFef8dv2M27aEleP8MPPJ40++OaSF0m5ecPMjcbqzpBMSS/jVoqX+o9mzYpnfGtL2eyn9gR17W7w6nBfUJnUsTtdOCNOnJ9b9m99ElFk0nl5C+Csn+Peb5vFwcQa+72/l6o3zePjDK9nzi9dY/ePt3PHLW/jhs6eosAja966b9aktjZdkPnrkuf8oyWg+OTN75nkng5b+ljXrZqz733jdmhGwKc16dXONu9Tb7nmpePnklqMG412aRu3Pw+T/3kPLnyvYzCfOH/NRwCIeI5MuargZLyXcwrdYRX3kuJUH+QDdzAYUV/BHrkAX7BOs4QAbga/KN+UkAd4+OEtMKpEqzu2TQ2QZcBtRCnjvfPx31PLviwvZfudK9gAc7+CyD63Qf//ISnZ3ePWY5+fqLJdmO2fuOdh6oyPbNcuf5cw62djTeH4bj+Ptx2colHVhwcKGmN+XAZnfc9r91oaX/WkhX+zWZJXTStuH3Lj+q5FPz/kKd/N1/p37sBLgUvYwizPcyn+TzYXTlb9zJQB38zU+yI/YwjsJIwSx8Brv5g5+wN18jTK8ZHBPzNqbYKa2iEUciGxETygf1b3Wz8bzthY+me+m6YtveH2DZ5eN7r8eYT7Anw+zMMNBy6nOJZ6QNqPjdI9nqVKK/kC/o9ffO6fQXXg+z9jR9qNritKLdgxX11Ql39/jemfdCxR7O2K7pUtgppP6H9pp/pd+XrEuwEUrc+jgEs6xkOaLyndSygz0YIhSerHhYQ+z0RBA8OBAA4KkMZe8VM0aMnWH07rjxk0MyaYxGq0l+O7LZUltPeuznJz5wF/4CsDGKh597xJ+86dDvPvPh7FYkODaGUvvP9i63n35zL6Xnjj+xB2/rvn1vQClmaVbqvKrzi9ftPa3XrqhYsNPYnx3hselBW03N75q3Va42HMgd24M58lWoeumdGr+vobS3i36VlAjUMBp6lhOkJ00kEsfs+kgFyd1rOEhfsMmrPhJo4U7+R1wo1TL46nmbz01DVt6wMJNXJiFc1S6cvFvuRabNkYgQ1NvhXfPuQ2OwYYrk9Gpya30bC+8JHZCDvRbefSD3+W6797LrOZmSr/vwhK08EM+x9U8cn5OHMTCb3gHbSwgjXY0rCzmJa7kAP/Fp3kLv6WSVv6X9+Kmm/fxBHqSgb+kUlzy1BtOixShJ7CPWsAeN8Ftb8Q6moA1ZVH7mq/o39V0fZop4PGxvPOk+/KW/f0xu+ChR5aQltdA3rxe+q5wU//jIKG8i+fgdjQ+zMPczdf5FP9FCDeltHAgsk9UFa1YgMXsopXKyFlu9OWnlNFGyjQ0KvSNmd8M0QcmhC1o296INpovtDeYHnqp/m3++u7FMYtYmm4s6apLv/rc3tgI+eyOtZQs33n+fWCmk9qfQdh24e6LfTjojfgCvMQihDCLaaKILjyUci7yoD/BIrIYvAlbCXB5TNqaAKbOcFoX8EDWy6jZfTmeptkj793b2l/m23n2ervZ+8aGkxllnn+WrZ740Nrb6eCxj3ybm372H2SWegHY+8AKjj/xXkL+DKziIVM7zWf4MafI52HuQlA46eJWHmQOeobMR7mKY7wJIYyLdt7Dryhi6EPmpVTIEjI1RCxSCtzIOAVcNw/PgTUjC7ihe4GnpvnKtFSIMkol6tKLPf8oWzP5aKiRyHzRQ+kP02KQZSQMbFabVEssmhUvUn84rQfuj7sH7s0icHDVyMPuQ63r+muar3abAo49Ff3N7iub98XPcNR7tZv6H/gJ5Ux2rdoKXCfVI29oYARSW8T6MtL1jHOpTBPUzqtQapiQQk1Z1M6z13lOdi43579xZGFPQ/qyjhPxE7J/rou6nyp8cyabZysdg8+PU1fEehjhdTD+9Dj71+DxZF6c1C6sWbUtjTf7zvXNia3/r8mwrGs7nDa775w3bhWEs+00fNuKf/ZkhVwl1VIRgxbFhdQVsb5HcvGYpYbQVoSvsfJi4Yc1q7bl9M3+Tl9x7MLqTEZFQN7UtNuZ7+uOX1ZK5bbS8G0L/pmTreNKo+6dnJoiFrkEPYXOuNAEVbPu4nseEHCXr8gUcIKxKc1y05lt4o6lr/VQtAwbDd+1ECgNTuIqaegdh+FIPRHrlugJzVGOL8bjzbgwhtgUcPJJCwfsbz69NWxRWvyWSrQMGw3fg0DxZIQ8N7KzpaFILRGLZKDPg8fdbm8aoZOXcIFQNWVRpoCNQW6gz7m29XB8XR3D2XYa/hOChZMR8uVSLYbKj5Y6ItYD+q9nHN5Yg6lZT2BogrtdZ6/1mgI2Dku6TrljHvk0lHCOnfrvQ7BgosP3NGBlLJs0WVJHxHApUDCRE5vL8LaVXOjUcaDlMk9zf4VphTYQFpDrzu6y2LWQNnbpSRDOtVP/n9ok1pGXSrVE7Zsfb1JDxCJ5wJKJnBq2oO1be+E6cm3nYk9t11JTwAbEHfbbrz631xf3isL5Dhq/EUJZJzIPtwLrYt2kiZIaItatghNqa+0CfP601725mvtn+g60XmYOoQ3M3L4md2XPmfitHw8QmOWi6XMTradSqqUopu2ZIMYXscgC9KiScaMJ6uSi163RnmBGcPfZa+2mK6XxubJln90RDobjXlHvlW46bp2oQe2ymLZlghhbxPoewBMettTNxxt06kNpTVnU9jMbtbCym9FIKYBDC9kuaz0Y/2E1QOudaXgWT6SuYqmWWTFvzzgxtoj1HQknZI3WBHXikteH0TXNV3n1nQZNUoWqnkZ3rr8nAXsMW4WzX7ISzphIz7885s0ZJ8YVsUgxE/DKGqB+Ht6ASxdxY0+V53TPfNOQlWJYQK5uron/kBr0NeQz90zkgVGa7LmxMUWsx5lO2MVNgTqxWBdwfyAzuK/5SrMHTlGKfF1ps/qa42/kAj3Hdfs7JzI/XhHztowDY4oYFgH5Ez25cS7eAYv0rqbrwpqymfPgFOay1gOJ+562vT8Nb9V4HU5mS7VEldM8HhhVxEsnc/LJRXrCu5OdSz09/gJDRp6YRE920OOc392QoOyTVqHpc6BkPOvHQhLnxsYTschsotypYTi6cwj0Z+H0BtNDR9rWmMPoKcLKjuOJG00Fy5103jLeIXxVsjKAGE/Ek+yF6+YTBNjd9KagOYyeOmQHPc6y/tbELDkBtL3PSShrPO6fViApEU7GErFIPlA20dM1QTXNxNnQvcDT6YvFTvYmRmJVx/HEZXVUbistnxzvQ2N+XNoyBsYS8SR74XMz8PlsNsuh1nUX7TtskvqUettdWYG+yYQRjo/eK8Zr5CpKhoHLOCLWk97Nm8wl6qrgWPtqb1BzTd09pqYxArK6/VjiRIxFOPfZ8Z5UFY+WjIZxRAyLmUR7/E7CZ3PdttquxeYwegozt/esKyE+1QMEZjrpvHk8lvGED6mNIWI94H/RZC7RUIn/QNvlQU3ZjHFPJnHBirIs7K6Pb+KAobR+wDkOl8wMqZbSuLZnCEb5ws9lgj7SAxwtKAg39c0xe+FpwLzeM4n93iq3ldY7x2Pkmj12kdhhJBFPmJANbWt4nQXitC2IiaHI9/c40+KZHXM4ejakEXZH2xvPiGtbhpB8EYvYgfLJXOJ4fp6vzVdu7tgwTRCQqp7TCYhuGoRyWOi8NdreOE+qJWHfx6RZcUXECuzKhM4e+APA++Cap+AaAW0Z7H8e/gxwK9z4IrzBAtpn4A9fhUP14FwDdwN4WnvDvpZf5hemF26/ZcEtDyfrnkwSx7zeM5Z9eZNazBg/XW92kP+winKjthnA0Xg3CZLbE98FHM7QswfyXViwFVachK+1w70/gGcA/gSlW2BNLdz7B/jxD+B9PpDZ4G+Br9dTsOnDSz52r8Pq6KjIqdiTxPsxSSBJGVKHs+10vzFad8yZcW3LIJIiYhGZAbzZAvenRUT8IFz9MXgyF0IAK6EX4Lew/HLYmQuh66E9D1p/CXMGrrXXudF7rP/kjKAWzFxatPR4Mu7HJPEkZUgN0PGOaF15y6U6MTaaZPXEPwLuXgVZlkgbWqH4Jagqgy/Ogs//ImLha4XccugcODEPOk9BDkAIl/Za1hI51HpoTaG7cFe8trs1MSYz+1sS/w8PljnpXxnN3NjJBFMsj5eEi1hE3gK0KKV2Lxp0kxpYesHdCN/+D3jk8/CJMDCcs6xEPq7lZl9bhtfS0t+ypiq/akeCbsHEIBT6u5OzE0Pbe6L14U5Ixo9k9MRXALeISN3D8L06WLAcPpwNnbfBHivwCagT0A5ARhF0noHcgZM7IHcOdAM0cq11i3/LbIWyLixY2JCEezFJIg4tZEuoL/UAvkvSotxlsTDubSEJIlZKfUkpNUPBvK/B/RVwtAb+9wrY+89ITq3HoCgMtiXQ936o2QJrOsH2DOS3Q9FHoLaf0mA/pY4DPQeuKEovMnvhaUqZpz3xIgboeGc0RrWpKeJB5ImeEQGAn8CrTVBQAJs+Bh/7MjxgBd4FTeth1xy49z1w12fhdy5QDVwf6HP1BVs9rWsWFSzamcT7MEkiZd625FTctyaaSLkcqZa4L+OKUokL0bywZqkCrpno6f/k/sDRfH+opqLGzGI5jem1pQV+P/fa5ISezrzbj/vwWNk8HlWbVGs8m5HMnjhnoif2MjPgpdjRld4Vy/aYpCAZIW9idooYjp43RjOkzot3M1JSxA3cEALoSesx15SmOQKSG+hNrNPHAH1rorGOT/h7Hi3JFPGEMyA0cYUNwGf3mTm0TMgO9CWnJw7nO/DPHstKHXcf6uSIWPfKmJCIu6j0+yhwAARsATODhwnZgf7kVd6zYSzreNz3MU5WT5yJnh1w3DRyfQggZAlpYWs4+VFYJkknO5hEEfdeNlZHMkV74knME9pYagfw2RPs/G5iWDKDnuQ9zIPlToKFo/XGcV89SdbNT2goHcau9VNmB/A4PePJCWwyhUkPJdk2MvqQ2irVEteMM8kScdZETupifgCsAuBxeJK0wG1iNFxhvy15Dg9A/8qxVkniOqROlogn5LjexrLzVki/3W/2xCaAvgWqMxxInoj9c8b6Psd1P7BkiXhC9Xay6PwTL2xJzqqCiTGxKi15ItYybATzR/tCxnW4n1Ii7mbuefc6TcyO2OR1rCqc3C+E95LR1otNEQN4KQgFyT5vztcsmumtZXIem5bkkZl3yWgPkbjqLGVE3MHiCyyA5nDaZDA2pSW3J/bNGa23jatTUsqIuI1lF/yTzOG0yWCsKpzc1Ypg2WgiNntigB4ufNKZIjYZjD3Zw+lwth3NNdKXMq49cbJ8j8ctYj+5F5wjmFPiAR7Y88A3LWLxiYgSJPzB5R/85p8O/ulj3pC3BCCshdOsFqv39uW3fz0YDlofO/rYB/sCfbMAa0lGydYb5t3wVK+/1/nokUfvHrhmMBzMGZrHe2vj1lX7W/Z/4qrZV31zYcHC+iTc6ogoI3wdfHOCI8QXx7XHSZaIx22tC5B5wTm2sM109hjEzQtu/kFeWl7fwPt3Ln7nLwZ+33x08ztsFpsXYMeZHas1pdnvWHHH1zxBj+OPB/5475meMzvLs8rbb19++9cHzvlNzW/+Y3Ae715/r/NU56k3uu3u2kTd03jwWxzJl7F/Xhj34eGOxDWFULKG0+MSYAiXpuEcKuLYtmiKopSi1dN66SWFlwykMFJhFXaEtJDFF/LZRSTstrsvSIhe31VfNDSP9/N1z986P3/+04IkJ6fVGPis9uQHwwRHTKkV179ZspQwrrATH/kh4IIULPZwcrKVGpXHjz1+lyDMzJ750oaKDS8PfL6/ZX+V3WLvmZ0zuwVgbfna1872nl3x4N4Hv6cpzVGVX/VwblruBfvvDs3jfbTt6ExfyJe3pnzN/mPtx65P6I1FScBiABGHR/QmjmuS+5QQsZeii+YUtrAt+cMng3BT1U3fKc4o7m7ztGU+cfyJz+xr3nduWfGy4wCnOk+tKckoOZ9I8Gj70QoR0W5ffvvd3f5u9+PHHv9CY3fj4ZnZM89nnGvpb1mzbsa6/wXQlCa7zu5614aKDb9K+I2Ng4AReuJw5khHpmRP3Dd2kdfxUnCRiM2e+HWKM4q7AQrcBb2F7sK9zX3NFRRzPKSFLO2e9lXrqtbdN1D2RMeJtcXpxQfsVnu4wF3Qm+XMOtnY0zh7QMTH24/PGJzHuy/Q5/KFfGXPnHzmcwAhLZS9pXHLvwI/M4pxKyQWpYkl+Q/1cMZIbZiSIh5vT3zRHNoetif/n2YA+gP9Dk1pkunM9PcH+h3t3vZLFhUsehyg5lzNIpfNda40s/R8RkG33d3R0t+yUCm13RP0OHr9vXOWFi19duD40fajawbn8c5yZnk/suojnxt4/9C+hz63umz1I0YRMEBIrGGSuMPnebQRRWwOp70UXPTHSQukmfm1gA5vR9bzdc//C4BSylqcUbx9ddnqgwB1XXVryjLLLkisf/nMy1944vgTd/y65tf3ApRmlm6pyq86M3C8tb/10g0VG36SwFuYNH6r3SAidg83pFeAZ5jPY0ZywjBFMoD3RVt8G1/ztLHyggwJQWsw/NSKp0whm9DgLvI8NWNd8vOPW/rDVL1n6HeyT21Sv4trtfG8+Ch4GMcyUxjnRT2xPWy3WsNW023LhE7niAalxKK5htNTb7yrTY6IldKAaDdrRo3QTFfQZcg1S5PE0u7MMoh9xCqE3UM7lp5415pMs/w45sXDN9MduOgPZjINaXNmG2dapaUPdeKeoj2xTufYRXQUw+8e7vaZIp7uaKB6HOlGXm+Me0+cTIteCzA/moIjDacz/COa9E1SiKbeptzn656/MxgOZgOqPKv8pWvnXo/iXoMAABmkSURBVPvc5qOb397h7VgmImGn1dl647wbf5XtyvYODuJQYNHO1Wzjqi8/BcDmj32aoCcbpaxklh7nTd/6HVaHbn957RerqX3uZhBIy2vkpp/+MuY3I/6ho4IpLeJz0Ra0MHysaF5fnnGGUSYTxmqxaqtLVz+yoGBBQ6+/1/nnw3/+8qnOU4fLs8oPb6za+KjNYtMeP/b4215ueHnjW+a/5S+DgzhOWJ2h5167/7uc27uTkhXtXHPf/5BR7ENp8MS/fpK9v1rN6o/v4syOIupe3Mh13/suWTM8dJ6MjzXM4h3csWhAR1zqGVxlvCsYhU6i9GSxEBi2x832ZDvEDCxOeYrSi7oXFCxoAMh0ZvrTbGlNXb6unFWlqw7ZLDYNoNBdeMoX8uVGTjkfxHHa5nQgljBp+bqhNKPYB0DIb0Vp1vNbYB/565WUr3mBrBn6mm1uZRzmqmGFJTj4u9qhNqm4b3KQvJ5YKYVIC1A+VlELwWGFKohk+DP8vWm9cU0JapI4zvScyfcEPTOr8qouCHms7aq9YkbWjF1wYRBHGOWg4pqHyZ75ukPFYx+9C297BZnlB1hxx24AvB1FAPz1Q3ejNAvzNm5m6XsPxrTxEtS4MMy2JabXH4FkO42fjaaQdYSeGCC3P9dMtjVF6Av0OZ+rfe6TiwoXPZzpzPQNfP7Uiac2CqJdMfOK7fB6EMd7V3zoc9zwg3s4ve06ml4rOH+hW+7/Mbc9+AVU2M6RRxcCoDQr3o4i3vzf32f9Z3/Bkb/cTu/Z2O7MYLkoF/q0EPGZsYuAFd+IjiF5vXmmcWsKEAwHrZuPbv5kSUbJ9vUz1p9PRvBy/cuXtXnalt284OZfDoRGDgRxdGQUe8it7CWj5CRNu2dfcEFnVojCJTWc3bVCf5/ZSdGSvdjTwpSubMeZdY7m/UUxvQm5yEV6Woi4FfCPVchF24jH8vvyjby8YBIFSikeO/rY7emO9KbrKq87H4yx++zuxac6T92wcd7Gn7nt7vMKGQjiqHMXCp52B/3Nc8irOoenzUnbUX2fr5DfQtuhJaQXNwFQeule2o8vAKCrLgN/TzGFi0b+Yk0EywWdjV9tUl0jlo0hSd3CRm+BXAvMHa1IPdd79/OpEYc+Ty97OhiwB0wxpyj7m/fP23p66xdcNtcZIu64lxRe8uiBlgPv0ZRms1ls/QCZjsxTb1301of6An3OJ44/cUeXFioFhKKlW7jynmfoOJHJi9WfQmk2lLKQVX6Ea+57GJtTQ2nwz3veSXf9YrBoVFz9BKs/viumN+I86aPiMwP2mQa1ST0V0+uPgBFEvBC4arQibSz1beObIxqvds/Z7Tmbdzb5DvAmCaPXlhb4/dxrHWOXTCBpB73M+uJAZ/OK2qQOJaLaZA+nAU4Bo5rh3Zwb1Ype3lFuzounGceyZhrPb95+bnCPmLB46+SLWKkAupBHJI02KyM4fAAU9hQ6LZrFXC+eJihQh3NmD5caNrk4z6+KtalNalwx85Mh+SLWOTLaQUGJk54Re2urslpy+3PHNJCZTA1aXLk+j82V/CQAQ3GdGtBTQyKrNYaIlToHjGrJc9Ix6npwaWepmYd6mnA4e5Yx/9fO2gHjakJTFxlDxDqj9sZuWkYdLpd2ljpQ48tnbZJ6hMQSPplZbjwPPUt/CGufFfCoTao1kVUbaUhyDFjLkAdLG9iWwhe8fN+i4bBXUbW7murNAN/je9fsZvc1FixaRahi/42eGx+qtdTmP3XiqU/0B/orCtMLt9628LbfD1zr78f+fltzf/P6sBZ2f2z1xz6d2NsziQX16cWBsMUaW0+rWGA/F0LX08lEV20cESvlQ6SOIWvGuRDaAT+wsL5rC/+a8Qk+8YUneOKAF6/jCEdW3M/9X8sgI3SSk5m2Nps6W3o2uLRo6WNtnraybn/3BX7ZFTkVNWvK1zz/tyN/+zomKcm+vEojjR5fx9kwMFIcdh+XeGK0P8hFQ2orMBP8Bexx+PBZNTSrIDzHc1ffwA1PZpARAqiksre8o9yZacv0Li1eesJqsV60BLG4aHFtUXpRdwLuwyQONLtyfK2uXONZpWHAMt2cKC+twRhLxEqdZpggah/ITHruvZMP/edsZh/eyMbabrqLD3Kw6nZu/+Kd3Pn5p3l6tk2zWUq6Skwr9RRlR8GiZDdhZJynLCShFwajiVjntaEfuEC1wNf/yIfvaqKp4iVeKtPQLF687l/xq2+/m3c/8kt++QkNjcrmSjNRwBSkzZnlb3IXGM+gNYCzTjGGv0O8MKKIj8PwEQ+X0uCZxaxjr/Lq4nTSO9ezfo8FCzdyY50gWj31GTmeHGeGN8PsjacYOwsWGnflwdYaxNZdm4gEAMNhPBHrztzbBt7WQMZxSANQ7E6ro27RDGacW8SivXvZuxBgO9uLNDTbbGb3AVQ2V5oxxlOITkeGvzG92Li9sHtfkCQNpcEIARAjIXIjMOshKP8s3KnAokAqWLbjXr7xZD/91i/yxTvaaZ9pwRK6jdseeQfvOArwLt71zYAEXJrSbFaL1buhYsOP5ubObdp8dPPbWz2ta0NaKNtmsXWXZpS+srFq4+ak3qfJmDxTeqm3LrPUeMtKAxT94pT60d+eHbtgfDCyiHOBd3A+SZLODr7iaWHtmBFLp4pOeQ7OPGhGNqU4za5c799mvcG4AiasmP35zepbx6NO/BhrjDecHkCpTuDo0I8LqIkqYqmitSLNEXQYL9LFJGo0RHu+ZIVxfBmGQ2ktyRQwGFnEOrsYkhGzmJ1R/VMtyiKVzZWmiFOY/blzvT2ODGMne6idk3APraEYW8RKeYCawR+l02TPoDEq6/Ocljkue8ieFIuhyeTotzoDu/IXGHgYDXRl+ti+4nSym2FsEevsY4gDSDnPRWV9tiqrpbK5Mq4bPJvEh5eLl4fDFquxv5/7FvQoNXr0XSIw9h8JQKkQ8Cx6Nn0AZvGMc7QkAYOpbK5MS/OnmUJOIRrdhZ6GjGJj98LtOV7OlCQ0bngkjC9iAKXaGLR27KTHmstR3yhnnMeiLLK0YamZ9SNF8FodwedKVxnTP3oATTS2rrCgR94lndQQMYBSB4C6gbezeSrqtbHinmJXfk9+1PshmyQHDdQ/Si8N+60OY7vOHp3jpTejXSn6kt0USCUR67wI+h+ulFdcFvxRe2atqF9hM/dtMjZ78qq859z5xvXMAvC4AtQsdGOQXhhSTcRK+YF/ApqVoKWIXVH7SLsDbvvclrlmb2xQzrlyvbsLFhrfOWfbcg1lCQO1Y5ZNEKklYgClmtHXj5nNk+Nq/4KzC9KcAadp5DIYXqsj+HT5WmPlkB6OxhIvzYUu4JRS0e3omQhST8QASu0FThdS43LSGfUf06qsljWn1igzF5dx0BAtJebBQWuYHcsGHjT7k9qWIaSmiHWeA7rn8ui4noi5/bnOyuZKc1htABSo50tW+g0/DwbYc0mAgMMKnFaK9mQ3ZzCpK2KlfMDfK3g8bKNvXF5Zi84sSsv0Zka1RGUSP7YWLvaezCo39nowwLl8LydnD7Rzb1LbMgypK2IApfqsBJ+cxT/Gteu7ILLmxBqruWtE8tibW9l/IHeu8Q1ZHleAl9cMrFu3KhXdntqJJLVFDKBUVzHbn4fwuAxW6YF0++LGxWZvnASOZc3w7Ci8JD3Z7RiTsEXj+XUQsg3opGbU8kki9UUM5KsDbb9n4Z4A4+tZK9oq3GUdZZ54tcvkYhrcRZ4XSlYavwcG2LHMR0/mgDGrGwMtKw1mSogY4HcsOHQfa3rHK+SVdSvTsjxZZo+cAE5mlHmeLl9r/DkwwMmZ/dTNGPyw2aEMuqoxZUSsFME9FB65h8uC/diiNnRZlEXWH19vdwadhln3m4rsz5nT/8+y1W4lYvxtaDuyfOxYNljA55QyZi8MU0jEEQ4cJdf/Wa5UnUQvSmfIaV1/bL2yaBYzwV6MUaC2FSzybC1aYvw5MIDfHuLFdTa44GGzbcTyBmBKiVgpwsCWJtLtn+YqOUN61G6ZWb4sx6raVQHTESR26Ol1Vvr25c1LjTlwyBLmhXVhfM7B2WNOKkVL0toUBVNKxABK0QDUd+G03cWV9hqij14q7SpNW3RmkekIEgOCYg0/Vb42cCJrRmrMgcMWjRfWBenIGRwGGQZ2JKtJ0WLcbJeTQIQs4J3oWznxXo72v5vjbuuQzJkjcbzkuOdI+ZHU6D0MSIcj0/dU+Vprn91t7PxYA4RF48W1gYhf9GB2K8XupLRpHExJEQOIcCmwauD9Utp897DLlkEoqkR7R8qO9B8vPZ4a8zgDcTB7dv/WoiVuTSzGN2CBHuD/8qV+zl6USaQD+ItSGN4haCqL2Aa8DcgZ+CwPX+jL7AxV0R2Vr+6h8kP9J0tOmkKOgoDFFnquZFXQ8Gl1BqOheHW1j9MXJabXgL8qNfx2QkZjyooYQIQC4DaGzP1v5ZTndo64HGhj2gQOzDjQX1tcawp5FNqcWb6nytfZPDaXsXNED0ZDsW2lj/phfbf3KmX8ufAAU1rEACKsBNYM/bwQT/BL7A5H0ysfLjvcf6L0hCnkIYTEEt6Vv8C/P7cyLSXWfwfQUOxc5uXUrOHsHl3AnyMrHSnBdBCxADcDJcMdv5lazwc44nQTHjWetSG/wVMzuyYNic44NtVpdBd6XixZ4Uip3hf0ZaRXLg3SVDTcw1sDHjP6ktJQpryIAUTIRN/XaVhraQaB8Mc56L+aM2mWUSzYrZmtvp2VO+1h6+iCn8r02tICLxcv006nDysCY+NzBHl+naIre6QsIluU4kBC2xQDpoWIAUSYB7xxtDIz6Q38K/vDi+kY0TjT6+oNbJm/RQL2QGosn8SIgMUWei2vKpByQ+cBetL9PHeZFe+II4eTSvHPhLYpRkwbEQOIcBmwdKxyi2n33c4RdQmdw4rZZ/OFts7fGu5L6zN2fuQY4LU6gvtyK4MHcua4DL8jw0icKfLwymoX2ojt7wIeNVLerPEw3UQswEZgRjTlK+ny384RbQVtrqHDbE00tX/Wfk9DQcOUNHj12VyBPXlV4aPZs1wps+Y7FIVi/3wPB+eP9j8KoQu4M1HNijXTSsQAIjiAtwLZ0Z5TTl/g3RwPXcY5p2uIAawpp8m7p2KPY6rMkzsdGf7X8uZrKZE2ZzT89hBbVoU4d5EX1lCeVYpTCWlTnJh2IgYQIQd9/XhcaVLthLVrOO1/M/Uyl57zXw6v3RvaXrU91JvWm3rGHvQh88nMssCh7Ap7lzPT+Kljx+J0sZdtKxwE7WM9WLcpxb6EtCmOTEsRA4gwA7iRCQaBlNMXuIGG0DrO2crwOBRKHZxx0FNbVOtOhWWokFjCDenF/kPZsy1n08fsrVKDgC3E9uXBYTywhmOfUsYOMYyWaStiABHmANcSZWDESBTiCW7gTPAKmizZ7gZq5u4Wj9NjOKNXry0t0OTOD9WnF0tDerEzZQ1Vw3G20MPWlc5IWtmxOKEUz8W9TQliWosYQIQq4JpYXc9FSFssrf7MspeDeTN223PCPU6bGtu9Mx702VyBc2l5ocb0IhrdRU6fzTkl5u0XELCF2Lk0SEPUc/gzwJOpENgQLdNexAAiXAK8IeYXdnuCsnZPMD+31lro6wrnBPpUdqBfMkMea3rIZ3NooUmLSoHyWp2hHrs71OXI1DqcmdLuzLJ0OjLtU1K0A2goTpd42bXUiT/q+zyHLuCUXEoaCVPEEURYBqyPy8XLmr2s2W/F7bvAaGTXQlp2oC+UEfSGHVpQ2VRYbFpY2bWwsqkwNi2EVSmCFisBix2/1S5+qx2/xSF+q93iszqkz5Zmm1LD4mg4V+Bh1xIbvRnjMcKdAZ5WinFtNJAKmCIehAhLgMvjc3WlqGzwseT4RWI2iZL2HC+7lliGZN+IhgbgH6kU1DAeTBEPQYRK9DlynHo3pag442XpMSsZxjN+GZLuDD+vLVZRrPkOxynguak0Bx6KKeJhEKEcuJ4RAiZixsyzXpYdtZDVb4p5OLoz/Bys0kaI+Y2GY8CLRs0XHStMEY9AJKHARiD+nktlzV6WHRVye6bGeu1kCFs0Gkt8HJlrpXPcw+bB7FKK12LWLgNjingUIgn3rgfyElJhRn+QqroAs886SPNPqygpet1+js0Jc2qma9DeRxMhhN77noxV04yOKeIxiOTqugqYl9CKC9t9zKvXmNHsxDY1/LIvImzROFPk40iljfbcWBj7+oBnUiU3VqwwRRwlkbXky0l0rm7RFDObfFQ2QGGHE6tK7eWkgC1EU1GAhlILZ4sco4QHjpez6MEM025fLVPE40CEInQ3zYzkNEBTFHQGKG0NUdJmIafb+KLWRKMrM0BTUZjTJbYJLA+NWQOwC6iZ6gaskTBFPE5EcAFXAnOS3RZEU+R3BShrCVHcZiG3x4k1OS6egB6/63UF6ckI0ZmlaMm30JzvJDypOe5odKEvH02r4fNQTBFPEBHmortqGsiirBQZnhA5PSFyejSye4VMj4U0nw1HwEr/WTcv3Xc7vs4yAJZ+4EEaX1lJV90yxBLGkdnK1V/9FZllXs7tzeeFTdU4MpsByCg+xfXffwiAXf+zmvoXb0JTgqtyDyWP/pGOHCtdmfYYDo/H4hB6KOGU88AaL6aIJ0GkV34DMDfZbRkbpZBld+BcdJS8n7yApd2KanbgfbSSgo8dxOEIU/u+9yAKFj74B9q2FtBw9+eZdeBL+B0W/A7Bb7fiP55J6MqvwivfgOV9sPJD8K5t8KUjCbqRPuCVyJ5bJkBqpRs1GBEjyrORkMYrAAPv31SXhmpYiG/vrzlrTYNi4BLgmpN0DIwm3tQIT61m19J0aHaDzTJko21gZxEUtOgCBlh/GB5blQARh4Ea9MTu0773HYyxjSIpQmQD6j+if8kM6t73cgG4e2HNh6Doy3DpB+HskGWdv14BbxiUsrW7QC876/Pww8gS21Wt0FoCz+aDxwIvrIT23Dg3vgH4k1LsMgV8MaaIY4RSBJViO/AwGDFnU8AKzbPg4y9Cy33gCsCdN75+/OaNYNHgJ9v196u7Ye8X9bKbHoavfhRqXbDAA3c9BLd/DKq+AEVt+nlxoQd93fcppeiJUx0pjyniGKMUPUrxLPBXoCnZ7Xmd5Z2Q0QmfrNXfv303nJql//6Jy+C1ZfDCLyO7wQK5IVjcr//+kQbIbYV/Fuvvv7EPzn4bznwH5jRDaax3TOgBXgQeVoq6GF97ymGKOE4oRYtSbAY2gxGMMGt6ILsT/hoR4jOLYEYTVC+GR26AJ34GJYHXy9dkgC+StujpAugogita9fd7MvWfR93wxAb47MsxauRg8R6dypFHscS0TicIEfKA5UAlSXt4/noG3H07hG2Q3wp/exDW3aO/T4v0unNOwfaH4PMr4YFbwRLWh8sf3az3wABLPwpnI7m7P/A4/HjXJBvWCewHjpnCHT+miBOMCBnou1BUYag15oSjAbXAIaWMNO1IPUwRJwkRLEAFMB+YySQzbqYQfcBh4IhSeJPdmKmAKWIDIIIbPUpqPokKe0ws/ei9bi1wbrr6OMcLU8QGIxLDPCvyKuV1c3Gq0Ysu2lOptt9vqmGK2MBEYpnL0QVdAuRg3GF3H/qSWhPQpBTdSW7PtMEUcQoR2QyuACga9EqGq6cP3aLcCbSgi7Y3Ce0wwRRxyiOCE8ga5pUBONGT/Y2399bQhdo/6NVFRLimQcpYmCKeBkR68MEvABV5aZGXQs9P5VeKwHDXMTEmpohNTFIc0+3SxCTFMUVsYpLimCI2MUlxTBGbmKQ4pohNTFIcU8QmJimOKWITkxTHFLGBEJE6Ebk22e0wSS1MEacQIlIhIkpEYpZqWETWi8g/RKRDRFpF5E8iUjrGOW4R+S8RaRORbhF5aZgyDhE5IiKnY9VWk+ExRWySC/wPeoKC2eghhA+Mcc7/oMc9L4r8/OwwZb4AZghiIjBFbEBEZK2I7BKRHhFpFpEfRA4N9HhdItInIpeJyIdE5FUR+aGIdInIKRG5PPJ5o4i0iMgdI9WllHpSKfUnpVSPUsoD/BQ9Ef5IbVsA3AJ8XCnVqpQKK6V2DykzB/gA8K3J/B1MosMUsTH5MfBjpVQWemK9hyOfXxX5maOUylBKbY28XwfsA/KB3wF/ANagZwv5APBTEYl2J8ergIOjHF8H1APVkeH0fhF5+5AyPwHuATPaKRGYIjYmQWCeiBQopfqUUtvGKF+rlHpAKRVG34liJvA1pZRfKfUMECCKTdJFZBnwVfSh8EjMAJYA3UAZ8G/AgyKyKHKNtwI2pdSjY9VnEhtMERuTj6Dn2zoiIjtF5C1jlG8e9LsXQCk19LMMEZkVGYb3iUjf4AuIyDzgSeAupdTLkc+GK+9Ff8jcp5QKKKVeBJ4HrheRdOC7wKcmdNcmE8LcUM2AKKWOA+8VEQvwNuAREcmHySWYU0o1MMwG6SIyG3gW+LpS6jdjlN83ShVV6Aayl0UE9NjlbBE5B6xXStVNpv0mw2P2xAZERD4gIoVKKQ09owbouwK2ogfwx2wrVREpB54DfqaU+nkUp7yEvqPFl0TEJiJXABuAp4ED6EP5FZHXR9FHCSuAxli12eRCTBEbkxuBg5Eh7I+B9yilfBHr8TeAVyOW6PUxqOuj6A+FTSMNtQejlAoCtwI3oc+LfwHcrpQ6opQKKaXODbyADkCLvA/HoK0mw2Bm9jAxSXHMntjEJMUxRWxikuKYIjYxSXFMEZuYpDimiE1MUhxTxCYmKY4pYhOTFMcUsYlJimOK2MQkxfl/GpgyD8biX8sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sets = {}\n",
    "for params in [(\"rnn\", 1, 256), (\"gru\", 1, 128), (\"lstm\", 2, 64)]:\n",
    "    net = CharRNN.load_from_file(*params).eval()\n",
    "    val = set()\n",
    "    offset = 0\n",
    "    for x, y in tel:\n",
    "        correct = net.forward(x).argmax(-1) == y.flatten()\n",
    "        val |= {n + offset for n, b in enumerate(correct) if b}\n",
    "        offset += len(y.flatten())\n",
    "    sets[net.name()] = val\n",
    "venn3(sets.values(), sets.keys())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model performance table - Training | Test - Loss | Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'train data'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">lstm</th>\n",
       "      <th colspan=\"4\" halign=\"left\">rnn</th>\n",
       "      <th colspan=\"4\" halign=\"left\">gru</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.510</td>\n",
       "      <td>0.554</td>\n",
       "      <td>0.596</td>\n",
       "      <td>0.639</td>\n",
       "      <td>0.660</td>\n",
       "      <td>0.662</td>\n",
       "      <td>0.533</td>\n",
       "      <td>0.580</td>\n",
       "      <td>0.620</td>\n",
       "      <td>0.635</td>\n",
       "      <td>0.670</td>\n",
       "      <td>0.640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>0.537</td>\n",
       "      <td>0.568</td>\n",
       "      <td>0.590</td>\n",
       "      <td>0.625</td>\n",
       "      <td>0.540</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.542</td>\n",
       "      <td>0.583</td>\n",
       "      <td>0.580</td>\n",
       "      <td>0.619</td>\n",
       "      <td>0.380</td>\n",
       "      <td>0.288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>0.455</td>\n",
       "      <td>0.502</td>\n",
       "      <td>0.532</td>\n",
       "      <td>0.435</td>\n",
       "      <td>0.303</td>\n",
       "      <td>0.126</td>\n",
       "      <td>0.484</td>\n",
       "      <td>0.519</td>\n",
       "      <td>0.557</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.142</td>\n",
       "      <td>0.131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>0.494</td>\n",
       "      <td>0.544</td>\n",
       "      <td>0.482</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.098</td>\n",
       "      <td>0.494</td>\n",
       "      <td>0.530</td>\n",
       "      <td>0.221</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.161</td>\n",
       "      <td>0.039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>512</th>\n",
       "      <td>0.497</td>\n",
       "      <td>0.550</td>\n",
       "      <td>0.581</td>\n",
       "      <td>0.609</td>\n",
       "      <td>0.592</td>\n",
       "      <td>0.579</td>\n",
       "      <td>0.528</td>\n",
       "      <td>0.577</td>\n",
       "      <td>0.598</td>\n",
       "      <td>0.586</td>\n",
       "      <td>0.498</td>\n",
       "      <td>0.219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1024</th>\n",
       "      <td>0.529</td>\n",
       "      <td>0.561</td>\n",
       "      <td>0.597</td>\n",
       "      <td>0.400</td>\n",
       "      <td>0.292</td>\n",
       "      <td>0.160</td>\n",
       "      <td>0.537</td>\n",
       "      <td>0.540</td>\n",
       "      <td>0.571</td>\n",
       "      <td>0.527</td>\n",
       "      <td>0.161</td>\n",
       "      <td>0.160</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       lstm                         rnn                         gru         \\\n",
       "          1      2      3      4      1      2      3      4      1      2   \n",
       "32    0.510  0.554  0.596  0.639  0.660  0.662  0.533  0.580  0.620  0.635   \n",
       "64    0.537  0.568  0.590  0.625  0.540  0.162  0.542  0.583  0.580  0.619   \n",
       "128   0.455  0.502  0.532  0.435  0.303  0.126  0.484  0.519  0.557  0.162   \n",
       "256   0.494  0.544  0.482  0.162  0.162  0.098  0.494  0.530  0.221  0.162   \n",
       "512   0.497  0.550  0.581  0.609  0.592  0.579  0.528  0.577  0.598  0.586   \n",
       "1024  0.529  0.561  0.597  0.400  0.292  0.160  0.537  0.540  0.571  0.527   \n",
       "\n",
       "                    \n",
       "          3      4  \n",
       "32    0.670  0.640  \n",
       "64    0.380  0.288  \n",
       "128   0.142  0.131  \n",
       "256   0.161  0.039  \n",
       "512   0.498  0.219  \n",
       "1024  0.161  0.160  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">lstm</th>\n",
       "      <th colspan=\"4\" halign=\"left\">rnn</th>\n",
       "      <th colspan=\"4\" halign=\"left\">gru</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>1.669</td>\n",
       "      <td>1.493</td>\n",
       "      <td>1.327</td>\n",
       "      <td>1.166</td>\n",
       "      <td>1.080</td>\n",
       "      <td>1.075</td>\n",
       "      <td>1.580</td>\n",
       "      <td>1.398</td>\n",
       "      <td>1.237</td>\n",
       "      <td>1.182</td>\n",
       "      <td>1.041</td>\n",
       "      <td>1.156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>1.576</td>\n",
       "      <td>1.442</td>\n",
       "      <td>1.370</td>\n",
       "      <td>1.212</td>\n",
       "      <td>1.547</td>\n",
       "      <td>3.089</td>\n",
       "      <td>1.566</td>\n",
       "      <td>1.392</td>\n",
       "      <td>1.418</td>\n",
       "      <td>1.255</td>\n",
       "      <td>2.119</td>\n",
       "      <td>2.558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>1.874</td>\n",
       "      <td>1.703</td>\n",
       "      <td>1.571</td>\n",
       "      <td>1.956</td>\n",
       "      <td>2.453</td>\n",
       "      <td>3.192</td>\n",
       "      <td>1.775</td>\n",
       "      <td>1.636</td>\n",
       "      <td>1.478</td>\n",
       "      <td>3.130</td>\n",
       "      <td>3.161</td>\n",
       "      <td>3.214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>1.733</td>\n",
       "      <td>1.541</td>\n",
       "      <td>1.763</td>\n",
       "      <td>3.119</td>\n",
       "      <td>3.148</td>\n",
       "      <td>3.233</td>\n",
       "      <td>1.746</td>\n",
       "      <td>1.585</td>\n",
       "      <td>2.792</td>\n",
       "      <td>3.125</td>\n",
       "      <td>3.156</td>\n",
       "      <td>7.282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>512</th>\n",
       "      <td>1.711</td>\n",
       "      <td>1.502</td>\n",
       "      <td>1.379</td>\n",
       "      <td>1.266</td>\n",
       "      <td>1.333</td>\n",
       "      <td>1.382</td>\n",
       "      <td>1.617</td>\n",
       "      <td>1.387</td>\n",
       "      <td>1.301</td>\n",
       "      <td>1.347</td>\n",
       "      <td>1.708</td>\n",
       "      <td>2.840</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1024</th>\n",
       "      <td>1.603</td>\n",
       "      <td>1.482</td>\n",
       "      <td>1.329</td>\n",
       "      <td>2.117</td>\n",
       "      <td>2.615</td>\n",
       "      <td>3.140</td>\n",
       "      <td>1.580</td>\n",
       "      <td>1.584</td>\n",
       "      <td>1.426</td>\n",
       "      <td>1.601</td>\n",
       "      <td>3.145</td>\n",
       "      <td>3.141</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       lstm                         rnn                         gru         \\\n",
       "          1      2      3      4      1      2      3      4      1      2   \n",
       "32    1.669  1.493  1.327  1.166  1.080  1.075  1.580  1.398  1.237  1.182   \n",
       "64    1.576  1.442  1.370  1.212  1.547  3.089  1.566  1.392  1.418  1.255   \n",
       "128   1.874  1.703  1.571  1.956  2.453  3.192  1.775  1.636  1.478  3.130   \n",
       "256   1.733  1.541  1.763  3.119  3.148  3.233  1.746  1.585  2.792  3.125   \n",
       "512   1.711  1.502  1.379  1.266  1.333  1.382  1.617  1.387  1.301  1.347   \n",
       "1024  1.603  1.482  1.329  2.117  2.615  3.140  1.580  1.584  1.426  1.601   \n",
       "\n",
       "                    \n",
       "          3      4  \n",
       "32    1.041  1.156  \n",
       "64    2.119  2.558  \n",
       "128   3.161  3.214  \n",
       "256   3.156  7.282  \n",
       "512   1.708  2.840  \n",
       "1024  3.145  3.141  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'test data'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">lstm</th>\n",
       "      <th colspan=\"4\" halign=\"left\">rnn</th>\n",
       "      <th colspan=\"4\" halign=\"left\">gru</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.510</td>\n",
       "      <td>0.551</td>\n",
       "      <td>0.588</td>\n",
       "      <td>0.615</td>\n",
       "      <td>0.621</td>\n",
       "      <td>0.624</td>\n",
       "      <td>0.533</td>\n",
       "      <td>0.574</td>\n",
       "      <td>0.604</td>\n",
       "      <td>0.609</td>\n",
       "      <td>0.627</td>\n",
       "      <td>0.594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>0.537</td>\n",
       "      <td>0.564</td>\n",
       "      <td>0.583</td>\n",
       "      <td>0.607</td>\n",
       "      <td>0.540</td>\n",
       "      <td>0.163</td>\n",
       "      <td>0.540</td>\n",
       "      <td>0.578</td>\n",
       "      <td>0.572</td>\n",
       "      <td>0.601</td>\n",
       "      <td>0.378</td>\n",
       "      <td>0.288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>0.457</td>\n",
       "      <td>0.503</td>\n",
       "      <td>0.532</td>\n",
       "      <td>0.437</td>\n",
       "      <td>0.303</td>\n",
       "      <td>0.125</td>\n",
       "      <td>0.484</td>\n",
       "      <td>0.518</td>\n",
       "      <td>0.554</td>\n",
       "      <td>0.163</td>\n",
       "      <td>0.143</td>\n",
       "      <td>0.131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>0.495</td>\n",
       "      <td>0.542</td>\n",
       "      <td>0.485</td>\n",
       "      <td>0.163</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.102</td>\n",
       "      <td>0.499</td>\n",
       "      <td>0.531</td>\n",
       "      <td>0.224</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>512</th>\n",
       "      <td>0.500</td>\n",
       "      <td>0.548</td>\n",
       "      <td>0.575</td>\n",
       "      <td>0.598</td>\n",
       "      <td>0.584</td>\n",
       "      <td>0.570</td>\n",
       "      <td>0.526</td>\n",
       "      <td>0.573</td>\n",
       "      <td>0.589</td>\n",
       "      <td>0.578</td>\n",
       "      <td>0.497</td>\n",
       "      <td>0.221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1024</th>\n",
       "      <td>0.529</td>\n",
       "      <td>0.557</td>\n",
       "      <td>0.588</td>\n",
       "      <td>0.401</td>\n",
       "      <td>0.292</td>\n",
       "      <td>0.161</td>\n",
       "      <td>0.534</td>\n",
       "      <td>0.537</td>\n",
       "      <td>0.567</td>\n",
       "      <td>0.529</td>\n",
       "      <td>0.162</td>\n",
       "      <td>0.161</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       lstm                         rnn                         gru         \\\n",
       "          1      2      3      4      1      2      3      4      1      2   \n",
       "32    0.510  0.551  0.588  0.615  0.621  0.624  0.533  0.574  0.604  0.609   \n",
       "64    0.537  0.564  0.583  0.607  0.540  0.163  0.540  0.578  0.572  0.601   \n",
       "128   0.457  0.503  0.532  0.437  0.303  0.125  0.484  0.518  0.554  0.163   \n",
       "256   0.495  0.542  0.485  0.163  0.162  0.102  0.499  0.531  0.224  0.162   \n",
       "512   0.500  0.548  0.575  0.598  0.584  0.570  0.526  0.573  0.589  0.578   \n",
       "1024  0.529  0.557  0.588  0.401  0.292  0.161  0.534  0.537  0.567  0.529   \n",
       "\n",
       "                    \n",
       "          3      4  \n",
       "32    0.627  0.594  \n",
       "64    0.378  0.288  \n",
       "128   0.143  0.131  \n",
       "256   0.162  0.037  \n",
       "512   0.497  0.221  \n",
       "1024  0.162  0.161  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"4\" halign=\"left\">lstm</th>\n",
       "      <th colspan=\"4\" halign=\"left\">rnn</th>\n",
       "      <th colspan=\"4\" halign=\"left\">gru</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>1.657</td>\n",
       "      <td>1.490</td>\n",
       "      <td>1.356</td>\n",
       "      <td>1.256</td>\n",
       "      <td>1.224</td>\n",
       "      <td>1.218</td>\n",
       "      <td>1.572</td>\n",
       "      <td>1.411</td>\n",
       "      <td>1.296</td>\n",
       "      <td>1.274</td>\n",
       "      <td>1.207</td>\n",
       "      <td>1.334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>1.566</td>\n",
       "      <td>1.449</td>\n",
       "      <td>1.395</td>\n",
       "      <td>1.275</td>\n",
       "      <td>1.548</td>\n",
       "      <td>3.066</td>\n",
       "      <td>1.558</td>\n",
       "      <td>1.409</td>\n",
       "      <td>1.440</td>\n",
       "      <td>1.315</td>\n",
       "      <td>2.114</td>\n",
       "      <td>2.554</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>1.849</td>\n",
       "      <td>1.685</td>\n",
       "      <td>1.562</td>\n",
       "      <td>1.936</td>\n",
       "      <td>2.439</td>\n",
       "      <td>3.170</td>\n",
       "      <td>1.756</td>\n",
       "      <td>1.624</td>\n",
       "      <td>1.482</td>\n",
       "      <td>3.099</td>\n",
       "      <td>3.139</td>\n",
       "      <td>3.172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256</th>\n",
       "      <td>1.714</td>\n",
       "      <td>1.535</td>\n",
       "      <td>1.740</td>\n",
       "      <td>3.098</td>\n",
       "      <td>3.120</td>\n",
       "      <td>3.220</td>\n",
       "      <td>1.720</td>\n",
       "      <td>1.571</td>\n",
       "      <td>2.765</td>\n",
       "      <td>3.098</td>\n",
       "      <td>3.130</td>\n",
       "      <td>7.255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>512</th>\n",
       "      <td>1.690</td>\n",
       "      <td>1.497</td>\n",
       "      <td>1.392</td>\n",
       "      <td>1.304</td>\n",
       "      <td>1.366</td>\n",
       "      <td>1.409</td>\n",
       "      <td>1.604</td>\n",
       "      <td>1.396</td>\n",
       "      <td>1.331</td>\n",
       "      <td>1.374</td>\n",
       "      <td>1.701</td>\n",
       "      <td>2.818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1024</th>\n",
       "      <td>1.593</td>\n",
       "      <td>1.486</td>\n",
       "      <td>1.357</td>\n",
       "      <td>2.097</td>\n",
       "      <td>2.595</td>\n",
       "      <td>3.120</td>\n",
       "      <td>1.576</td>\n",
       "      <td>1.582</td>\n",
       "      <td>1.436</td>\n",
       "      <td>1.588</td>\n",
       "      <td>3.117</td>\n",
       "      <td>3.118</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       lstm                         rnn                         gru         \\\n",
       "          1      2      3      4      1      2      3      4      1      2   \n",
       "32    1.657  1.490  1.356  1.256  1.224  1.218  1.572  1.411  1.296  1.274   \n",
       "64    1.566  1.449  1.395  1.275  1.548  3.066  1.558  1.409  1.440  1.315   \n",
       "128   1.849  1.685  1.562  1.936  2.439  3.170  1.756  1.624  1.482  3.099   \n",
       "256   1.714  1.535  1.740  3.098  3.120  3.220  1.720  1.571  2.765  3.098   \n",
       "512   1.690  1.497  1.392  1.304  1.366  1.409  1.604  1.396  1.331  1.374   \n",
       "1024  1.593  1.486  1.357  2.097  2.595  3.120  1.576  1.582  1.436  1.588   \n",
       "\n",
       "                    \n",
       "          3      4  \n",
       "32    1.207  1.334  \n",
       "64    2.114  2.554  \n",
       "128   3.139  3.172  \n",
       "256   3.130  7.255  \n",
       "512   1.701  2.818  \n",
       "1024  3.117  3.118  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_names = [\"lstm\", \"rnn\", \"gru\"]\n",
    "n_layers = [1, 2, 3, 4]\n",
    "hidden_sizes = [32, 64, 128, 256, 512, 1024]\n",
    "\n",
    "for loader, loadername in (trl, \"train data\"), (tel, \"test data\"):\n",
    "    corrects, losses = [], []\n",
    "    display(loadername)\n",
    "    for MODEL_NAME in model_names:\n",
    "        for N_LAYERS in n_layers:\n",
    "            for HIDDEN_SIZE in hidden_sizes:\n",
    "                net = CharRNN.load_from_file(MODEL_NAME, N_LAYERS, HIDDEN_SIZE).eval()\n",
    "                correct, loss = [], []\n",
    "                for x, y in loader:\n",
    "                    out = net.forward(x)\n",
    "                    correct += (out.argmax(-1) == y.flatten()).tolist()\n",
    "                    loss += [(torch.nn.functional.cross_entropy(out, y.flatten())).tolist()]\n",
    "                corrects.append(np.mean(correct))\n",
    "                losses.append(np.mean(loss))\n",
    "\n",
    "    for title, arr in ((\"accuracy\", corrects), (\"loss\", losses)):\n",
    "        arr = np.array(arr).reshape((len(hidden_sizes), len(model_names) * len(n_layers)))\n",
    "        columns = pd.MultiIndex.from_product([model_names, n_layers])\n",
    "        df = pd.DataFrame(arr, columns=columns, index=hidden_sizes)\n",
    "        df.columns.name = title\n",
    "        display(df.round(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gate Visualization (test set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAADoCAYAAAAe98jLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOydd3xc1ZX4v2f6qIy6rOYiY2OMG250DARjigFDKCEEAiEkIWRD2CWbsj/SgF3SdpOQEJIQWqiBhFBMscH0YuOCe7flJqv3GWk07f7+uDPSSJY0sq3u+/185qOn+9qZd9688+45554rSikMBoPBYOgJy2ALYDAYDIahjzEWBoPBYEiIMRYGg8FgSIgxFgaDwWBIiDEWBoPBYEiIMRYGg8FgSIgxFoY+R0QuF5H3RaRSRFpEZK+IvCgiFx7hsf6jP+Q8DBnOEZGfioilU/s4EVEictMAy3NT9LyxT5OIrBORfxMRW9x2e0TksSM8/s19KrRh2GOMhaFPEZHbgX8BO4CvAguBe6OrP3cEh7wcGFRjAZwD/IRDfy9lwGnAqwMtUJSro+e/EvgU+D3w4z447k2AMRaGDtgSb2IwHBbfBV5USn01ru1t4KHOb+aDhYhYAVFKhY7mOEqpVmB530h1RKxVSu2MLi8VkQnAHfSNwTAYOjAkfryGEUUmUN7VCqVUJLYsIjki8mcR2S4izSKyX0SeFpHCuG0eA24ECuNcLnui62KumHHx54i6i1SnNiUi/y0iPxCREiAATBMRl4j8RkQ2iohXRMpF5BUROSH+eOheBUAwJkd0XZduKBG5PuoW8otItYg8ISL5nbbZIyJPisi1IrJFRHwiskpEzkx4hbtnJZAqIrndbSAiJ4vIW9Hv6xORZSJyctz6d4GzgTPirvm7RyGTYYRgehaGvuZT4EYR2Q28pJTa3s12mYAf+CFQBRQAdwIficgJSik/cA+QA8wFLovu13qEct0E7Eb3fHzAQcAJpKLdZGVRmW4DlkdlKAf+ChShXWpnAuGeTiIiXwf+DPw9+t0KgP8BThGRWUopb9zmZwGTgB9Fr8U9wGIRGaeUqj+C71gclc/b1UoRmQ68B2xGXw8F/AB4T0ROVUqtQ3//JwEr8I3oro1HIIthpKGUMh/z6bMPcDywHv0gUkA18AywIMF+VmB0dJ8r4tofAw50sf1N0W3HdWr/qb6tO7QptHFw90KGJKAJ+PfOxwRsnbYfF22/KW7/CuCdTtudGd3u9ri2PUAdkBHXNie63XUJ5Ix990noF74M9IM9jHYBxp/jsbj//wHUA+lxbR6gFnghru1d4MPBvpfMZ2h9jBvK0Kco3ZOYiXZl/DewFrgCWCIid8VvKyLfjLprvEAI2BddNakfRHtDKdXSuVFErhGRFSJSH5XBB6QcoQyTgFzgqfhGpdSHwF70NYnnE6VUXdz/G6J/x/TyfFuBIPph/8foeXsKTM8DFqu4XotSqhF4uQvZDIYOGDeUoc9RSoWB96MfRKQAeAP4iYg8oJSqE5FvA/cD/wf8J/ot24IOGLv6Qayyzg0icinaXfQ48DN0LygCvHaEMmR2dy50HCezU1tt/D9KqVYR4TDOfQVwAN0T2qu06y6RfN3JltHLcxqOUYyxMPQ7SqmDIvJX4HfARHRc41pgmVLqzth2IlJ8GIeNPRgdndqzuhOji7ZrgZ1KqZviZLBz6EO9t8Qe/nldrMsDVh3hcbtjo2rPhuoNtXQvW20X7QZDG8YNZehTRGR0N6tiGUaxTKkktAslnq90sV8r4O6ifW/079S4c9uABb2TtE2GzumzN6BjD51loBs54tmGjllcG98oIqcDY9HB5cHkPWChiKTGGqLLl9JRtu6uueEYxvQsDH3NRhF5Bz0wrwQdQL0YuBV4TikVi0u8AXxfRP4L3dP4HHBVF8fbDGSKyDfRb+Z+pdQGdJroLuBX0fEbrehMHudhyPoGcLmI/AZYDMwGbkcHgTvLAHCniLwOhJVSh/QSlFJhEfkx8GcReRKdVVSIjt3sAB49DNn6g3uAS4BlIvILdG/r+2ijeXfcdpuB20TkC+hr3KSU2jbQwhqGGIMdYTefkfVBG4WX0W/+fnTA+DPge4Ajbjs38CA6bbYJ/bAuRj/Afhq3XTI6m6ouum5P3Lop6MwdLzo4/h90nw11bxeyWtBpsweBZvTb9UwOzSKyAg8AleiYhoq2jyMuGypu++uBdWgDVgM8AeR32mYP8GQXMnX4/t1c45ui201IsF2H7xFtOwV4K3rNfMAy4ORO2+Sh4zZN0fO8O9j3lfkM/keUMtOqGgwGg6FnTMzCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJMcbCYDAYDAkxxsJgMBgMCTHGwmAwGAwJOWaNhYjsEZH5A3Cen4rIk73Y7loRWSEiPhGpjC7fJiLSi33HiYgSEVvfSD28iOqyRUS8cZ+CAZbB6LmPEZGbRGSDiDSLSLmIPCgi6Yexf5/+xntzPBFJFZH/i27rE5F9IvIPETm5l+fo1X00GByzxmIoISJ3Ar8DfgXkAaOAW4EzAMcgijacuFQplRL3OXg4Ow/EA9joufdEr9UvgP8E0oBTgbHAmyIyJK+ViDiBt4FpwCWAB5gMPAtcPIii9Q1KqWPyA+wB5keXbwI+BH4N1AElwEVx274L3Ad8CjQALwGZ0XXnAAe6OjZwIRAAgoAXWNeFHGmAD7gygbwLgc+ARmA/8NO4dfsAFT2HFzgt2n4zsCX6nZYAYwf7uve3Lju1O4HfAgejn98Czni9Ad8HyoEnADfwePR6bQG+F69boAD4J1AVvUduj7YbPfetPj3R73dNp/YUoBK4Ofr/Y8C9cevbfotRfUaAluixvgeMi16/r0fvhzLgzrj9D+t4Xch9S/SYyQm+3++ium0EVgNn9XQfRe+dh6PHLgXuBawDrpfBvjEG8YZse8CgjUUQ+BpgBb4ZvZkkuv7dqJKmAsnRB8aTnW+obo7909i23chxIRACbAnkPQf9xmIBpgMVwOXRdbEfgS1u+8uBneg3GxtwF/DxYF/3/tZlp/a7geVALpADfAzcE3c9Q+i3VyfaUPwceA/IAIqA9XEPC0v0h/1jdC9gPLAbuMDouc/12e21QhvzZ6LLj9HNw72r+yLu+j0T/R1PQxv++UdyvC5kexZ4rBff73ogK6qvO9EvK67u7iPgReDPUZlz0S+t3xhovRg3VDt7lVIPKaXC6BsyH+0miPGEUmqjUsoH/Ai4RkSsfXDebKBaKRWKNYjIxyJSH/XDzwNQSr2rlNqglIoopdajb/izezjuN4D7lFJbosf+H+AkERnbBzIPRV6MXrN6EXkx2vYl4G6lVKVSqgr4GXBD3D4R4CdKqValVAtwDfA/Sqk6pdQB4P64becCOUqpu5VSAaXUbuAh4Npeymf03HsOuVZxlEXXHw0/U0r5lFIbgEeBLx7l8WJkox/8AIjISVH9NorItli7UupJpVSNUiqklPpf9MvKpK4OKCKjgIuAO6IyVwK/off3XZ9hjEU7bUpWSjVHF1Pi1u+PW94L2Dn6mxagBsiO95krpU5XSqVH11kAROQUEXlHRKpEpAHt6+7p/GOB38UeoEAtIEBhH8g8FLlcKZUe/VwebStA6yrG3mhbjCqllD/u/wI66jl+eSxQEGeQ6oH/ouMLRU8YPfeeajpdqzjyo+uPhs6/5b5KhqhByweAUmptVL+fRxsEQMdjRGSLiDREdZZG9zoei37WlMXp+M/oHsaAYoxF7xkdtzwG7baqRvuhk2Iror2NnLhtVYLjfgK0AosSbPc08DIwWimVBvwJ/VDo7hz70V3V9LiPWyn1cYLzjCQOon9sMcZE22J0vm5laPdTjHid7wdKOl3PVKVULHBp9Nx3xK7V5+MbRSQZ/Za9LNrU4beHThqIpzuddP4tx+6JIz1ejGXAgqicXSIiZ6HjZNcAGVFj0kD3Ot6PvhbZcfr1KKWmJJClzzHGovdcLyInikgS2hf+j6jLajvgEpGFImJH+4ydcftVAONEpMtrrZSqR7tH/igiV4lIiohYROQktI8yRipQq5TyR9PwrotbV4V2qYyPa/sT8EMRmQIgImkicvVRfP/hyDPAXSKSIyLZ6HhDT2mJz6GvWYaIFAL/FrfuU6BRRL4vIm4RsYrIVBGZG11v9NxHKKUa0Nfq9yJyoYjYRWQc8Dw6KeGJ6KZrgYtFJFNE8oA7Oh2qgo7XKsaPRCQpes2+Avz9KI8X42/oF45/Re8Nq4i4gDlx26Si4zFVgE1EfowO6Mefo+0+UkqVAUuB/xURT/SeOU5EenJN9g8DHSQZKh+6yIbqtF4BE6LL79KeDdUIvIK29MTtX4bO1Phup2NnoTOt6oA1Pcjzpejxm9E30gp01oYjuv4qdJe5CVgM/IG4QBjagFUB9cCp0bYbgA20Z9Y8MtjXvb912andhY47lEU/99MeSDyHQxMTktEPonp0dtFdwK649QVoA1Qe1edyo+d+1etXgY3oDKQKtPslo5N+/x793uuBf6djQHoROoOsPvq7HEfHbKhy4rKaDvd43cichs6624vuqexFJ8ScHF1vRWc2NUbvye+R4HkRPeaDaEPZgM6Wu3ag9RHL9jH0gIi8i/7B/nWwZTEMHCLyTfSPcuDf4gx9TrR3UgLYVdfBc0MPGDeUwRBFRPJF5IxoV38SOq3xX4Mtl8EwFOh21KqI/EdPOyql/q/vxTH0N0avPeJAuzqK0a6GZ4E/DqpEh4HRraE/6anEQWr07yR0jvnL0f8vBd7vT6GGGkqpcwZbhj7E6LUblFJ70QMvhytGtz2glNpDe9aR4TBJGLMQkaXoEgVN0f9TgeeVUhcOgHyGfsLodeRidGvoD3oTsxiDrlcSI4DOKjAMb4xeRy5Gt4Y+pzc9i/+HHkDyL3Ta2RXAc0qp/+l/8Q4lOztbjRs3bjBOPaIoKyujrq6O9HRd8bm+vp6MjAzy8/MT7NnO6tWrq5VSOYm3TIzRa99xtLrtS72C0e1Q4Wj1mrAss1Lqv0XkdeCsaNNXlFKfJdpPRB5Bl+mtVEod4gcWEUFXX7wYnXN+k1JqTaLjjhs3jlWrViXazNAL1qxZwwcffADAvHnzmDlzZsJ9br75ZhYvXkxubi50LKUBGL0OFY5Gt3Qsc9OG0e3wRkQO+b0eDr1NnU0CGpVSvwMOiEhxL/Z5DF09sjsuAiZGP19HDzoxDCDNzc14PB6+853vUFRURElJScJ9brrpJt54442eNjF6HQIY3Rr6moTGQkR+gq5l8sNok52eSyYAoJR6H13UrDsWAX9TmuVAuoj03gcSpaE5eLi7GICf/exn/OIXv+C+++4DIBgMcv311yfcb968eWRmZva0Sd/otSVIJGIGjB4JQ1m3/mAYfzB8OLsYhgi96VlcAVyGHrqO0jOQpfa4R+8opGP1xwMcZqXMfTXNnPd/7/LIh4nfmgwd+de//sXLL79McrIuS1RQUEBTU1NfHPqo9eptDXHVgx/zo5c2GoNxBAxV3SqluPXJ1Xztb6uMwRiG9MZYBJSOgitoq/zYF3SV79zlk0FEvi4iq0RkVVVVVVt7frqLOWMzuXvxZmMwDhOHw4GIINGpn30+X18d+qj1muywMv/EUTy1Yp8xGEfAUNWtiHDJ9AI+3FltDMYwpDfG4jkR+TO6y/k14C2gL2okHaBjqeAiOpaPbkMp9Rel1Byl1JycnPZgvt1q4ffXzeTCKXnGYBwm11xzDd/4xjeor6/noYceYv78+dxyyy19ceij1quI8L0LJvHNc47jqRX7uMsYjMNiKOv2qtlF/PLK6cZgDEN6kw31axE5H10lcRLwY6XUm31w7peBfxORZ4FTgAaly/EeFjGD8e2nP+PuxZsBuPnM3sTfj22++93v8uabb+LxeNi2bRt33303559/fl8cuk/0GjMYAA++uwsB7lk0FYvFDMBNxFDX7dVztL353j/X87W/reKhL8/BZe+LSScN/UovygT/ojdtXWzzDLoEbxD9RvJV9Kxft0bXC/AAsAtdXnlOb8rkzp49W3VFIBRWtz6xSo39/mL18Ae7u9zG0M73vve9XrV15tprr1V5eXnKZrMp9GCvftVrJBJRv3h9ixr7/cXqv15Yr8LhyBF+42OHo9Utes6Mfv/NPrdynxr3g8Xq+r8uVy2B0BF+W0NvAVapoykZn3CDLmrzA+uP5qRH8+nuxlNKG4xv/E0bjL8ag9EjM2fOPKRt2rRph3WMo735VC/1Gm8wfmgMRkKOVrd9qVeVQLfGYAwcR6vXnqrOfhO4DRgvIuvjVqUCHx12F2YAiHdJ3RN1SX3VuKQ68OCDD/LHP/6R3bt3M3369Lb2pqYmzjjjjEGUrHtEhP+8YBIK7ZICuNe4pA5hOOr26jmjtX7/sY5bHl/FX280LqmhSk8xi6eB19EzxP0grr1JKdXT+IlBxRiMnrnuuuu46KKL+OEPf8jPf/7ztvbU1NREOfaDSucYBhiD0ZnhqturZhehlOJ7/1xvDMYQpltjofQ8uA3AFwFEJBc97WCKiKQopfYNjIiHT2eDIZigd4y0tDTS0tJ45plnAKisrMTv9+P1evF6vYwZM2aQJeweE/TumeGs2/igtzEYQ5PejOC+VER2oKcjfA89X+zr/SzXUWPSanvmlVdeYeLEiRQXF3P22Wczbtw4LrroosEWKyGd02rNOIxDGa66vXrOaH555XQ+2lXNLY+btNqhRm/GWdwLnApsV0oVA+cxRGMWnYkZjIumGoPRmbvuuovly5dz/PHHU1JSwrJly4asX7szMYNxmzEYXTKcdXv1nNH86qoZxmAMQXpjLIJKqRrAIiIWpdQ7wEn9LFefYbdauP+LpofRGbvdTlZWFpFIhEgkwrnnnsvatWsHW6xeEwt6G4NxKMNdt1fNLjIGYwiScFAeUC8iKehpGZ8SkUog1L9i9S1m4N6hpKen4/V6mTdvHl/60pfIzc3FZuvN7TB06CpLysQwRoZur5pdBGCypIYQvelZLELXrv934A30gJxL+1Oo/qBzDOPhY7yH8dJLL5GUlMRvfvMbLrzwQo477jheeeWVwRbrsDGlQQ5lpOjW9DCGFj2+boiIFXhJKTUfParz8QGRqp8wWVKacDjMokWLeOutt7BYLNx4442DLdJRYbKk2hlpujU9jKFDjz0LpVQYaBaRtAGSp98xQW+wWq0kJSXR0NAw2KL0GSborRmJujU9jKFBbxyZfmCDiLxJdE4LAKXU7f0mVT8TC3ofyzEMl8vFtGnTOP/889vmPQC4//77B1GqbqivB48HLBYIh8Ha9ZtlLIYB8MdjOIYxrHTbS0wPY/DpjbF4NfoZURzrQe+FCxeycOHCwRYjMU1N8MwzMH8+5ObC6tVw1lnQ0gLNzZCX12FzE/QeRro9TIzBGFx6U6J8WMcpeqKzwVAcO6VBho0vOyUFLroI8vPB4YCZM8FuB5sNnM4ud+kcw1AcW6VBho1ujwBTGmTwSGgsRKSELmbDUkqN7xeJBphjNehdXFzcNpNaPLt37x4EaXogEgGfDwIB2L4dVqyA887TPYqGhkN6FjGO5VpSw0a3R4gpDTI49MYNNSdu2QVcDQzdqmRHwLHoklq1alXbst/v5/nnn6e2dgjWh6ythVdfhQsugPXr4eBB2LxZu6E2bIAvfKHbXY/VLKlho9ujwBiMgSfhOAulVE3cp1Qp9VvgcwMg24ByrNWSysrKavsUFhZyxx138Pbbbw+2WIeSkwM33ggTJ2r3U309iMD48bqHkYBjsZbUsNHtURJfS8pM0dr/9MYNNSvuXwu6p5HabxINIsdSD2PNmjVty5FIhFWrVtHU1DSIEnXi4EFIToZgUPcskpNh/34ds1i/HrZu1dtdcQUU96yjmMEQjo0sqSGv2z7ETNE6cPTGDfW/ccshdPXZa/pHnMEnZjBuf2ZkG4w777yzbdlms1FcXMxzzz03iBJ1oqJC9ypaW6G0VBsLl0t/YuvT0qCuLqGxgGMrS2rI67aPiZ9AyRiM/qM3xuKrSqkOkTERGXlPzzg6j8MYiVlSDz/8MOPHd8xRKCkZQq63mTP130gEPv95Pc5i8WLYu1en0M6dC5WVcNxxvT7ksZIlNeR12w+YtNr+pze1of7Ry7YRRXwM454RGMO46qqretU2aLz1lnY1bdoEr7wCa9bAnDng9cKJJ8Jnn2m31Ec9VMsvLdU9kzjiYxhPj9BaUkNet/2EGendv/Q0B/cJwBQgTUQ+H7fKg86KGvGMxBjG1q1b2bRpEw0NDbzwwgtt7Y2Njfj9/kGUrBM2mx6xbbHogLbVCklJ2vXkirv9LD2873SzbqRmSQ0b3fYjpofRf/TkhpoEXAKk07HKbBPwtf4Uaigx0gzGtm3bWLx4MfX19R0qkaampvLQQw8NomSdOOcc/Vcp7YLKy9NjLS68UGdCeb2wZw+cdlr3x8jP73ZVV+MwhrvBGDa67WeMwegfepqD+yXgJRE5TSn1yQDKNOQYSUHvRYsWsWjRIj755BNO6+lBO9isWKFjEx4P/OMfOk12715d7uPMM/UYC78ftm3TI7uDQR3HqKnRI7uTkmDlSpg0CdLTuzzFSDMYw0a3A4AxGH1PbwLcn4nIt9Auqbb+v1Lq5n6Taggy0oLeM2fO5IEHHmDTpk0dXBSPPPLIIEoVR1ERpKZqQzB9uu5ZhMOwaxeMHq2zoWIZU8nJeh2A263HY1gsumcRV0ivK0Zi0HvI63aAMAajb+lNgPsJIA+4AHgPKEK7oo45Oge9h/MESjfccAPl5eUsWbKEs88+mwMHDpCaOoSGzxQW6l6Fy6WNRWam7iVceKFOlT3/fP0ZN04P1IuV5I7FOUAbm1iF2k6B7nhGWtB7yOt2ADFB776jN8ZiglLqR4AvWlRwITCtf8UauowUg7Fz507uuecekpOTufHGG3n11VfZsGHDYIt1KPX18NRTsHOnHoz3+OOweze8/z68+KJ2Tb30Erz8MjQ2wrJlOoOqqQmeew5KSrRr6q23jhmDMWx0O0AYg9E39MYNFYz+rReRqUA5MK7fJBoGjITig3a7HdDzNW/cuJG8vDz27NkzuEJ1RVoaXHYZFBToGIVS2g2VkgJjxmi3k8UCoZCuF3XaaTpe4XTqarV5edotdeqp3VapjTFSsqSGjW4HEFOt9ujpjbH4i4hkAHcBLwMpwI/7VaphwHDPkvr6179OXV0d9957L5dddhler5e77757sMU6lEAAysu1S0opXYE2FNJjMPbs0eMsXC5tMF58Ec49V8cx7Hbdu8jM1PvV1upAt4g+pqvr7O+REPQeNrodYExpkKOjN/NZ/DW6+D4wIsqS9xXDOUvqlltuAWDevHlDu3R1ZaV2IUUi+v9lyyA7W7uk6uqgulpnTcV47TWYNUsP3HvjDR3jSErSy9ddp41IS0u3xgKGfy2pYaPbQcAYjCMnYcxCRL4jIh7R/FVE1ojIgoEQbjgQy5IabtVqf/e739HY2IhSiltuuYVZs2axdOnSwRbrUAoLddXZOXNg9my4/nqYMkXPnFdcrHsP69Zp47FrF3zuc9rllJ2t9zvhBB0Ev+EGyMrSPZRRoxKeNlZLajjO6T1sdDtIxKrVfrjTVKs9HHoT4L5ZKdUILAByga8AP+9XqYYZncubD4eg9yOPPILH42Hp0qVUVlby6KOP8oMf/GCwxTqUgwe1e2ntWj3G4vXX9RiLVavgvfe0wZg3T0+1Ons2/P73sHSpdju99po2IBs3wpNPwr59un3LFn1sr1cbG2ifYCmOmMGIlTcfLkHvYaPbPsAX8KGUIhIM4DsQ7UU1NOhSMKB7nrG5PHw+PR4HbTB+ddUMYzAOg94Yi1jf+2LgUaXUurg2Q5ThVktKKf3Qe+211/jKV77CjBkz2tqGFPX1+oFeUQFVVdqFVF6uHwiBQHvAe/RoXSKksRHKyvT6Awf0fpWV+oFRV6c/+/bpY/v9+nix5WDwkNN3zpIaDj2MYaPbPiAYDhKKhAi3NBOsr9ONTU36HgF9/9TXRzcO6nhXlKtmF7X1MEyWVGJ6E+BeLSJLgWLghyKSCkT6V6zhyXAKes+ePZsFCxZQUlLCfffdR1NTE5ae6iwNFscfr4PWxx+vH/wNDXq5qUkHsn0++PhjbTTS02HhQr19bq52VU2YoDOmRo3SLimrVbulQLuqYmRldSvCcAt6Dxvd9gHp7ujofE866VNn6+WiIv0Brf8YaWk6wQF0GrXTaWbcOwx6VaIcOAnYrZRqFpEstCvK0AXDJej98MMPs3btWsaPH09SUhI1NTU8+uijgy3WoSxfro3B+vX67d/hgEcf1amzkye3B7+tVv3X5dJVakeP1uMt1q3TxmH1av1WWV+v3VA33KCPGwrBxRfrXkhqqn6gHDigDVFcqu1wCnoPG90eIfUt9ThtTtx2N9UHdpCWU0SkoY7KFW8z+oJraKkspbXiIOmzz6Bl7UrEasM14QTtipwyRadhP/OMTn6YOdPMh9FLepMNFQHWxP1fA9T0p1DDnVjQey2FS2AAACAASURBVCgbDIvFwqxZ7ZMgxqbhHHJYrfpt0GbTf0MhvayUNg6FhdoYWK364e716mW7XZcAsVg6VrC12drLgTgc7aO9Y9VtOy/HEYthwNA2GMNGt0eIxWJBovoRqxWxWBCLFavdCSKIxYrFpseaSERBqFVn0S1bpl86UlL0PO5eL2RkwLhxHUqDGIPRNb3pWRiOgJFWS2rQOPVU/SaYl6ff+Fev1lOplpbq2MTFF8MHH2gjcsklep6L3Fy9rqVFP/QdDj3ewunUfmuPR/cwHA5tOPz+9nIhHo9+gEQHtnXmWJpxb6jicXralrPyirWOs0dRcOHVAIhS+qVgyxZc736oe6dNTVqvy5fr++CKK7Qb8sUXddZcRoYxGAnoV0emiFwoIttEZKeIHJKOISLniEiDiKyNfkbUYL+RUhqkM2+88QaTJk0CmNrvevX5dFZTaanObNq8WQcvw+H2ebpranQAe9MmPUnSK6/AJ59oV9SuXboUiIh2Pbz/vi4B8vjj8PzzuiTI66/DCy/o5fJy+MtfdHmRqir45z+10fF6dWYNHYPewylLKhEDqtdu8If8lHt1cLol2EKjvxGA/Q372d+gM5xeX/4E20tW01JfzT///B2qSjbStH8XG576DaqpiSVP38MjL/wI36N/ZuNnS9iycwU0NlLlseFNskFdHQf/+RiBdWt08sMLL7QFvk3Qu3sS9ixE5Aml1A2J2rrYzwo8AJwPHABWisjLSqnNnTb9QCl1yWHKPWwYqqVBbrjhBp544omEbZ0Jh8N861vf4s033+S4447bBHyxX/WalKRLj2dn66Bkfr4OVscKBKal6Z5HMKj/trToH/6WLdp4+Hx6u82bdW8hVhbE79e9CLtdGxaLBcaO1S6K8eN1/MLj0YP7nE5tnOJcU10FvYdKtdoj0e2A67UbnFYnSfYkAOxWe5u7Kc2VhkSTMCcVTCcrvQBnUhqTp56LJ7sIm9VOzolzkORkJhROw72xCefm7eQd3IulMQSVe0kKNePcdxDESnLqKOwrVkJGJpx8cvv9BCbo3Q29cUNNif8nagRm92K/k4Gdsfm7ReRZYBHQ+eYb8QzFLKlNmzZ1+D8cDrN69eqE+3366adMmDAhNsezAvpHr83N2k0UCukfcjis02JBu4wqKnRvIy9PP/hDId2rsNl0UHvKFG0U1qzRPYScHP0Wefzx2sDk5Oi/p52mXVtbt2qD0dSkz11Zqef+TknR54yvZhsdTS4Wy5A0GEei2wHTawJEpM3NZLPY2gxEks3dtpw/agIOuwuLxcoJE0/DkpSKCgZJ9eRAJELIYcNSUUXI6sYfbsVCCLDj2H8QazhCJBKivqkKt6RiHzUK//bNuP0XEGz20tpYS0rxJGMwuqBbN5SI/FBEmoDpItIY/TQBlcBLvTh2IbA/7v8D0bbOnCYi60TkdRGZ0sX6EUHMYFw0dXBHet93332kpqayfv16PB4PHo+H1NRUcnNzWbRoUcL9S0tLGT16dHxT/+h16VL90C8p0e6hd97R2UtOJyxZogfpNTTobKjKSu0iWrGiveDgxx/rT0mJNiAOh+4ZrF2rDUjMrfXJJ/Dhh9qVtXo1PPywrmS7ZYt2WT3xhI5/bNumzwWwYwds3w609zBuGwLjMI5GtwOm126o9FYSCAdoDbVS6a3UMjWWsqVaD6DcsHYp69fpUehvfvQ427Z9hKqqYs1Tv8a7YxMH17zHa0//lIb1K1n62T/4yLuZXXvX8vzoJp6faiGS5GZFlp89BckcyE/mieImtocrKWs4wJJVz+Ldsp6XX/4VDz3zXXbsX8fSNf8g076Kn102kY92mYF70HPPYqdSKlVEnlNKXXMEx+7q9arzr2gNMFYp5RWRi4EXgYmHHEjk68DXAcaMGXMEogwNhkKW1IQJE2hqauKaa67hueeeO+z9uxnc1fd6nTdPj48Ih/W8FSeeqF1DGzfqkdoffqgf2ldeqR/k4bAeZ/H889r9FKtIm5amg9nBoI475Obq/0tKdFtFhc6sstm0mysUgnff1fuHwzruUV2teySZmVq2wkK9T/v3GBJZUkej277UKxz+bzbDnYHdam9bBshLySPTra/5CZPPatv2zJmXk5KSiTR4OXHi6SS503BPHcvZdieegmJO2Rum4mCQsV4HF9RkYLXbsSQ7mVFRixs/9UU5TA8KLluIbZEqSl1uflv6PIGMEDuCAf726o1ku7Jw292kuDx86Ywv8tRH1cd80LsnY/FD4HlgQg/b9MQBIP5VpQg4GL9BtIxIbPk1EfmjiGQrpao7bfcX4C8Ac+bMGdaRxME2GPfddx9XX301O3fuPKL9i4qK2L9/f4cm+kOvNpt2P7ndMG2ajllkZuqeQ2mpjmXMnKl7A1arftAHAtpgTJig57woKNA9Coej3RVlsWgj4PXqMRktLdoIxdJvp03TPYnMTG2M0tPh1Vd1z0YEzj5b91gAvvAF3YtJS0NSUjpkSQ3GjHtHo9u+1Gt0/WH9ZgPhQJuxiP0NqzCt4VbcdneHbS3NLUiwHv7xAnXNFTi37yZUmE/TgU1k+d6hRUI0F40itOMgfpcVq9VOrc/KSxMC5DY0sz/ZS1lrPkvHV5ATtNCoCvjos+cJW6Ax3ILV46FFpTIl7USqI3VsaPor86ZdwPsbOKYNRk/GokZE3gGKReTlziuVUpclOPZKYKKIFAOlwLXAdfEbiEgeUKGUUiJyMtotNuLHcAxmWm1WVhbnnnsuJSUlXHbZoSp8+eVDVN2BuXPnsmPHDkpKSkD3HvtHr3v26FTHggLtNpo1Sz/Y33lHr8/J0QZj7179EI9EdK/A49F/d+zQsYeGBm14Yj2HWHwjHNb/x8Zg7NypjUYsLlFerg1VbFrSPXt0cH3dOh07EdHur3Xr9Ax+M2cOetD7aHQ7YHrthjp/HRaxdDAM5d5yKr2VzCmcQ1nFLrBYGD92BiWr3yInZRTZLU18mtHMGTXgW7+LtzxVfH5HhA1pflqamilNt/JBQQhLazNne63st/vZlys4gnYCZTPYk99MGV42pvipDdRQaPPgttgIBAIcaKikOFiBf1SEPfV7GJe+hGtPv5lnP64+ZmMYPRmLhcAs9LSq/3u4B1ZKhUTk34AlgBV4RCm1SURuja7/E3AV8E0RCQEtwLVqpBax6cRgZUm9+uqrrFmzhhtuuIE777zzsPe32Wz84Q9/4IILLgCd/HBPv+h12rT27KMFC/Rbv1Lw5S/rWMPbb7fPs11bq3sVkYiOSZSVaaPR1KQNQ3OzzopSSh+zuVkbmkik3YjE6kotX66D2mlpOjW3vl4bjZwcvZyXp7cX0SVCLrlEr48ymAbjaHQ7YHrthiJP0SFtY9LGMNqjnRPji2e2tZ908Vf1WIrt5VyypgR7YQaRi67jqi2ryTo/g/lP/i/lgRATI+nYQ6lYaiopbA1zUrmfAp/w2jRYX/wG5emKpJYgyuElaLHhd1hJDzvwOoSUZDs7HftoqfOT5c7CYXWwP/A8P7r0v7jnlZJj0mBIIl2LSI5SqkpEkpVSvgGSq1vmzJmjVq1aNdhi9BnBcIRvP/0Zb2wq58eXnDhgLqmqqipycnLw+XwkJycf0TFEZLVSak5fyHOIXktK9AM7I0MHk8eM0Q/+det0ufI339Tt+fm6F+D369G5ra06s6mpST/8U1K0UWhu1mmygYDuoSTp9My2QoSZmTrI3dKiDUdWljYICxdqV9eECTqF99VXtcsqVu7c59O9nyuu6JBaq5Til0u28eC7u/jSKWMGNIZxtLrtS71C736zvoAPt92NRdpzblpqyvHXVJJx/HRa9u8GEdwFY9m/5HnSCorxL1vC4ua1zJNx7Hf4WRPYw4Kic1jxyp9oCDRwZlMGH6c1IVYrMw5GWJVUT4sN8HioswXZnRykmQgRyygOpFfidTlxRVyEUiMEDk7Fq+oZf0KQgvqF5I0Osje8isk5k5ngvp77l9YyqQCeueVsMpJS+upS9StHq9dezcEtIpuBLdETzhCRPx7pCQ0d6VzefKCypHbu3MmJJ57I5MmTAVi3bh233XbbgJy7V1RWaheSUnq5pUU/mHfu1A/82IPZ59Puo+RkbRRycrSBaG5un1kvEtEfEd1msej/k5L0vvX12sBUV2tDkZyst01K0scJBnW12tJSHetwu7WhqKpqT7Pt9NLVeeDeQGZJDXnddkEwHCQc6ZhtFGisw1eh4yiB6koCVRUQClG1ewO+A7tpCjdTlmalmmaqvBW0WCJU1h6gMd1Fa4aHqlQbrcEWWsOtVEkzEZsVb5IVi1I4/EHcYRBHGlZHJmFLBAlb8YUVdiUk10/A1TCextYmPtvawNrd+6j0VVLXUsdJxX7uuCCbrQcV33xy7TGTJdWbcRa/BS5AT6mKUmqdiMzrV6mOMQaj+OAdd9zBkiVL2nzbM2bM4P333+/Xcx4Wp5zSvnxWeyYMN92k/55/vt4mGIS//13/HTNGxzR27dLLwaCOTVRW6l6J3d4+4tvj0a6tYFAbDKtVGwqvV8+RETMgDoeOi1RV6U8wCJ9+qg3GHXfonk9GRnusI47BqlY75HXbBW3VY+NIK55MWrE2eGkzT21rn3X17ZCSgsocy63vv0H6rJOZmp7CzM8+YNznrmDM3loqD+5gzpSzcKx5AVtpNbOqbCwtDDGnysHGzGY2FkSoykpGLLDVsoGw24HVFiZTWXGl5lB/wmoI1GOzWsg4ZRkRi4PG5sa2EeZfPPk8RITfLKk6ZoLevSr3oZTa36np2DClA0jnGfcGojRIp7x6rNYhdLNXVmpXUiQCK1fqXobPp8dShEJ64qNnntE9gFjNp7Q0mDFDu4iam/V21dX6rd/t1tvEeg+x3onNpg1KrLhgYaE+j9tN8ymzWFvsJjzpeHzFU6k+7VJIT6c0/3hKZ58LK1dS8fd3aVr6iZb5wAHtBotjsEqDDGnd9kAoEmor8bH64GqeWv8UoUiIqn1bqD6wndC+PTz7yH+w7slfU+tSPBtay75QLXuXL+H1+pXU/usZPkxvZEWmny2fvsb6E7JYW2Tj4yLFrhwb6wqttFgtZNedjzOUjrfBib/kc/jDQrguA1V2AsFqJ7Xvz0etuIoMy1gC22dSW+4k0zWK4L6Z7K6o4IFPH8BnW8bNZzv4cEc1l/3xNVoCoQTfbnjTG2OxX0ROB5SIOETku0RdUoa+ZSAnUBo9ejQff/wxIkIgEODXv/51m9tiSBCrNiuiH+RWa3ul2FhxQIejvYqsw6G3SUnRn9i4iqQknSIbS8N1u7XxcDrbjzlpkh5/EXNRWa3Q3Iyltg67Mwmx27Eku7HmZILNho0INsLgcGB1WLE4o0UH40d5x9F5AqX+NhhDXrc9IEhbiQ+H1UGyIxlBsFodWK12xGojyeLE4UjCaneQKi7sTjcOmxO3smGzO3HY3VgLC7E7k/A31dFiFxzJaYSzMxGLFTsWVKSZMfVhprQ4Ge1MosCeRcgSpMXSRFACuG0WHHYhMymd7JRkUtxOijyFWGxhkAgOqwOrWDlnsot/vzCbHWUWbhnhA/d644a6FfgdejTnAWAp8K3+FOpYZqBKg/zpT3/iO9/5DqWlpRQVFbFgwQIeeOCBPj/PERNfUvukkzoui+igsoh2Ky1Y0B78fucdnQ0Vq+1ks7UVACQ1VY+bqKjQAfFIRBuNzZt1r8Tv1z2aefOguhrXZ58xJagf6u6URtyvHICiIkZFvLB/K9x+O9lKaTcUaJm6YSCzpIa8bnvAarGS6kwF4MScE5mQOQGrxUpGwXhAX8fTbrqL5JRMkqwuLjn/W6RPnEYo7OKSta1knnMRJ33QQlXVHqrnZhOpWoWkpbIn0Io/KRVHUEE4gC1vCy6xUlpkI8InZKWNwkkVle5tSHIuctK/EIlQGXATKYpwWt4s7FY7tqwqTho9lzPHnElToIn8lHzOGmslP2X/iC8N0pv5LKqBLw2ALMcefr92h3g8HZoHwmBkZ2fz1FNP9ekx+5TVq/VAvLFjdWZUTo52Lb35Jpx3nq4sG5smc9063StITdUP+qoqnfYKul2p9ulT09J0T6O6uj0ltrqaRhfYxxbiVtE4RlISrdPnUps6lvy0ZraGc9ibPpELWlexxDoXFbJy4Zo1/HNpOceNdTPtu1/mhT+/wVmXziavuGuj0dlgCP0Twxjyuu0BX8DH3oa9TM6ezPt732dz1WZumXULH+77EBFhxqgZPPrWrxlfOI2Tw6N46r0/cNHoc2msLuUd10EWvVTFsvAO/CE/db5G5kw9h0DFQV5I/YhLrNPYmbqbxiwL5x5/AW/41pK1ycpMuZXNBb/C2jQVtzeX+vH/wr3um7htIULZr1O/Zg77JgW4/nMX8sYSIWXcWPw+B3v2uiiapY3CsVBLqltjISK/59Dh/m0opW7vF4mOJbqZMwH6L+j97W9/u62b3xX333//UZ+jTygsbC/il5amH+x2u+5ZpKdrt1FVlZ4+s7FRG42UFD2/dnKyNhzNzdoYOxz67T8Q0Gm2SulU2dxcbagjERw52dhq6/UxsrNBKRyB/SSn2SA9nTyxYUtqhtTxTGps0r2asVOYfGILoyYWYHXaOX76GLLys3v8Wv05496w0W0PuO1uspOyERGOyzgOt82Nw+rg+KzjEYQURwonjJvD2PzJZIRTmZE8noKJs0hvDHBixQEKKhqwW2uoj3jJrPOyoWUpESJ4QmG2FTbgtQSoC7TwRt1K9qgaIjngDryF023HQQ0ZjhAh11jqCnZgtYVZdMLVNDidnDxlMudPO4/Wus84Lj+XzFQnBbkdYxTxM+6NRIPRU89i5AxmGKpYrR1KI3emP0Z6z5nTZ+nz/UteXvtyenp7LGH8eN0zWLhQP/gnTtTZTaGQLl2+bJkOWBcX61Hc4bDuKTgc2qhE515um22vtRXCYZyp6cjeUiivpKYom4xaH7L/AEFSYPJM7KOKce+pgi9fQ9JH2/W+c+fituRhL0zXMfLcSYSUAzvQ4o/gdun4RSiksNk6ljePryXVV6VBho1ue8AiFjJc2q2X5UzHEQwjIqTiQMSCw+pg6ujZZKTlwao1eJQDdu9mZ2s5B5vK+NjqxD9lEu7qCkrte2kJ1aM8qVgawFe6hXpXEEdIYSuvIehuJN2TS05LBeH0TPx1uWQ1FhCe7CbHm4/bZmdUUjHN9S58VcnsrirlQFUZOys9hCIh1pTuISdnOi5LCq2tipQU6TCB0kgzGN0aC6XU4wMpyDFPY6P2oad3TCHsPNIbjs5g3HjjjUcl5qCwdavuQYRCui7T/Pl6pPX69bq38cEH7WMsMjJ0vOP443UvIy1NG4bYPN2xYHdra4eAdJn4cBdlYau18lCrl0XpydA4gWfTL+HOuhJe31jJqsps7jmrhMeeqkUpuP2Uan75s+XMmZ3KOdfP5+f3rufGm4qYMM3D6++X87WrJrBlTy0HK5tZcFoRjSW6MoanOKvLGfeO1mAMS912oq6ljvWV6zlrzFl89Mnf2bx3Fbfe8DteXfoHLFi4cOF3+PuL93JC8VzyN+3j1aQD7N+2h1JXkGxrEm+49jOnyoO7KsByT4hL64oIpOazlLWcUDufBvteHGoHnpazyHSuwluRTGXrxUTsf6ds62mEQvmE0l4ma+NsLBYnS9Ramt6dwcaVlWysK2Pts2OprtxA0ZgDvP2WDc+/bcfWMIE9+0LcfF0mdXVw/oQifnXVyDMYZlrVoUJK96NA+9pgDDtivQml4Iwz9PLJJ+uR1OPG6YKCoZAOMP/973qf2lptPMrKtCGOZT6lpbUPxFNK13wKhcjJz8WWlAlb93Btfh5Fyoe01nNj1at4InD+3POY3uTFNf14rrnRhooo3EVZ3HDbBEaPz2XUaBu33jaeqdOzSE0VLj2nEKsVJo/LYEye1m1KUccXgcGuJTUUSXelMzt/NhaxMHf2ZUw87mRcNhfnn30zAGnudBbO/xajcsdh973DaZ+VMsM2mk/2rmBT635Oqmpl5aQttFr9NDfW8YqlgZC3jKpALR/YX6ci3IjVBoWpH7PSvZ88i5N0xyt85i6Dce9hb5pMacZOKjO3kG53UqDyOUAuBXYrrSVjaXRUEK49mezRNopH7yddRhOUVLJSwzQ0aK+miJ5xTyk1omIYxlj0BZGIDlbHSkhECVQ3Yk12YXU79HqXS6+ILUci2kXidNLa3EQkEsbtySQQDmCz2DqUPhiqM+4NCLHrFpsAKT29fQKkCRP0+lCofYBdMKiNRVaWrj5rt2uDY7HobKfkZDjhBPxl+1FVQdwZmfjX78OV5sSWnIyvqoFwOEzV3DP5wF1ETp6w6bXP2JY/k4IV63ns04OgYMEkO39evJr8YuFLC6/ktVfLiWCnocnFBx9W88UvFOKL1LK1pIEbLhnP9tJqLCJMHptLU5MOw9hswrfO7P+g93BBlNKxo7wUHBY7LosdwmH8Kz7CYrESODWJ/R+9hho1EUvVPna5m3HuWEPD6HRSDtaxO7mJrLJ6vDaF1RVk/IFC6pOF5owWHCXnkZq1i1BOCWXeWeS6HbTW29lUO4M8FzTUnUR9SzYpu75MqNmGMzCO9ftcnDpmC/vLZvJedTPTZ+1jf5mND94eRf6EZO78oZfcLD/K4WP7jmJmn+TE7dZzap2eP5p7L4W7XhkZBsMYi77A6+1YbyhK864y7Fkekifk62Ar6Dfcpib9gItlQzmd1JaXEImEKPRk0hJswW61t00vGWMozrg3oNTW6smKsrLgs8/0PBbjx+tBe+GwrhOVlaX1MH26NiCtrdpgtLbqp7PDoTOhGhqodfiJRLwUtbipTJ1MakEaqTUbWGqNYE9JY1N9mKf3ZjLFWcYb9QWst3iY/EkJn67WeR+u4s1UbXZRXx7iw+wDrFy7meRkGy3hNHZUbGTtxgy2VZWyfquXC04v5J9v78JqFf7fjblUVLRP2FdfL9x+9sCP9B6KBOqq8W3dgCu3gN07VrBz3zquSLuNTza8hoiF07OzWVX6KRVlO/DYU4ikJLPavxZHRTpj9zeyckyES0sdlCdH2JfspKn8ZLxJQVyO9dRWz6CFbLKtAQ7umkOWNFPVkE5d7TQ8aT7CtWm0IjQ15ZKauYZgs5PWpgmo1joiTT6q/alUrBjFlvJsAq1uPCl5bF6TgSvFT/ooP2VbImzeAFOn6vJlH30Ec2aM5pdXjowsqd7MwZ0DfA0YF7+9Uurm/hNrmOHxHJL+CpB+yqT2f2IT54B+QoB+qEUNTP6E9rEEaa60bk/VV1lSVVVVPPTQQ+zZs4dQqD2r45FHHjnsYw0Yubl6qlOHQ5f7mDVLl/W48kptGNxu6h5/GddxU3GLtNeAstu1Ya6p0dfe5wMRCqq9EHZDupvjmjZA61ioaeCb1bW4IqWMra5l4qRGpmyqx+NMZb81wMlnzePKSAOg+NyFs6ko+YhxEzKYf+YJ1O71cN75+dTUCS2hU5k5NZkxvjGMz/eS6XFxy8I5bSWtJsTNElNUBNB3WVLDUrdRHFm5ZJ1zEQBTZ17ICdPPA5uDy666C0RwjjuOK3w15OYfR2j5xzQu+wtz6nJYbfGzvEAxo9rJ4qJmGu1hamwhLNNew+tUlKS3MEY9jTe9nIbkOnKdT7M5fzsZzlRGOSrYN2olIq2keY/joG07DZsuJUXSaAx7eXfbmfirx2GxWVjV4kQ1pyDBJDasdGO3OhC/m0BNCr5kJ8uW6SS9uXPhggu0d3mStT1LajiXBulNz+Il4APgLUyZjy4J+ZsJNntxZ+ZSs7mCcCBM7kkF7Nu0j5T0ZNJyM/j06Z2ceH4hAUeAxYtX8oVrz8Zb6aV0dxUzzzmBkk/eJxwMMmHeeZTv3Uhyahapmfna556d3SHNtnOWFBy+wVi0aBFnnXUW8+fPHzalICgv18Hs88/Xf0tK4PLL2zKgdo+fz4vLJ5OSMpkvp72Eq/oA1NcTGTee2o0Hyba3tGdDNTTouIXdDvv34011Yt+xlUprMQ8lXcXF4RVkRfy8uu94PI5N/KP1DA7ax1D1f7tZFUknYrHQ9P1dHKgJ410fouTDdeyuWMW29ScSykmnuXUrv31kDtUt5UTwUVEONXUKi8C3bhjDEy+WMv2EFBbOT2PZh42cNjuFJLeFa088+hn3hqVuo0R8Xlq2bCB51ilUVZZQWrmLmdMXsLtiK2KxkGYP8fbq58lJLySpvJZqW4B3i10EGlsYE3CxNifItHInfiV8XNRAcosdWoNku2BUS5CQs5XGxklU770IZ2sKvuZimgOT4KALVTmGqrCHpFAB9sKVuFqzCVfOJduzi0Z/Ni2WCFI3BuzgDYZwWK2keFoIBi3UNwoqFMZqj7B5s/CDH1jJL2xl4UJhwXwHVZ8W8d2z4Nfvr+PK367i2W/NITVpeOmmN8YiSSn1/X6XZBgTDrQS8uvq7U0HGwk2B8k9qYD68npEhCSPh31bmxkz00+z28fmnbX4fC3UVzVRtreBmUDtgRKCra1M4Dzqaw5isdi0sWhq0r2WTmMyjtYl1dzczC9+8Ys+uwYDQn29znDy+dor0dbX6ziGUnjLGmn1axdRYNIYXCdPh8WLCU2ZSUvdetSB5UiFrlxKIKAD3C4X2Gy0WoNIRFEv6VT5wpQFhIDTyS5HBtXODGoCiuoWLzX1FupTfSgRKqpSabG0EPG78df4ieCnodFPqzWAVbVQ2eqnoVVwJQkHysLUN3uxiFBdDbv3tpLktBEIpFGyJ8KUSSGsFgdNTcJ3F3TMkjpcgzEsdRsl7PMSLCuFUIiauoMcrNzJTBZwsHQLIoItNY2WgI/6gztp3l2Ks9FHfagSu9VOvi0dn6olt85CowgpuYrxFW7q3U6C6QFSmwpIcTfjb80m4Hfiqc2kOeAhrEJk1GbT0OoigJVI/SjC/iRaWz34m9y0uuz4GpPxBZ1YQyEkZCESdGD3gMURLTqdeQAAIABJREFUINjiJOC3IykR3G4LKMW+fVYqKoXSUsHt1O85F88q4nvnwC/eXcc3nlzFI18ZXj2M3sxncS/wsVLqtYERqWeG03wWKhxBoj9yVV2DpKfph34opNM5t2/XrpHp04kIqEgEa0rqYZ0jfj6MH11yYq+zpO666y5OP/10Lr744sP+XjH6dT6L7mhp0bGH2HwVBQU6oykSIZSSztYH38EzOo0xY4CXX9bbl5bqa93crMdvlJdrPcRmw3O52iZPijjdVERyyApVYHNaqZx8NtbaKh4sX0RN8limp+/j44aJIBYmTxI+2mMnPdlO6vTj2bC9jlF52fjFSnltAzmebFojDfiCzZx0XBH+YAsOuzDl+DTq6hQul3DxxTqMlZenYxix0Ff8fBjXnTLmsLKkjla3gzGfRQdiAykbGnQ8sKBA6xuIOB3sevkx0tds5oArwOJ1zzOrKZl1SV42ZgQZ1+Jiq7WOgCVMk8tKSksYv8NCZTKMrVfsyFSgXIw/OJb1hXux10zAUzmX0rGfYN9yGZHGSTSoFJLEhd2fS019Bs7kRlprRhNR4HIKoZZkVMSC3S6kuG20tlpwOCO4nbY2HTmdOnYR84YWFGjX1FlnwQuf7ef+5es5Y0L2gMYwjlavvTEWTUAyEACC0WallDrUST8ADBVjEQr4qd23jZzx0yjfs48DO3czd8Hn+PCZl2hpbuXcL17O7259i0nTUjn9rGx+/t/ruOaiNNJOzOPhv6ziP26dyfrXXue9auEnCyfzq4830RqM8P9+ejuP/Pm/OWHiVE5Z8EUe/OUrLLpmLunZyXyw5GMu/sKF2C2if1DRdNsjmUApNTUVn8+Hw+HAHu21iAiNjY0J9mxnwI1FY6OeYvW00/TYi/XrdQzjqad0gNvj0cFrn08/dQsL4fXX9cC9Tz+Figq2RirZnOLn8s1hfNYwym7Dg6ttfgu/uHm9/lTOsCxn1XFn8HT4JGaHrNQlR2gcrShZcRoZJ68n7Eqh7KP/T96bh9l1lWe+v7X2dOap5lkllUZbtiQPsjzIlm0IZrBjMzSEBBMCdHITQjqEBHK5ffumm0DDzSV06BC4JMYmBoPNDLbjCc+2ZEvWZE2loVTzeKrOvOfVf+ySJTsGZIItOf0+z3lq1977nLP3+vZZ3/reb+qn9ZptlCf7qEz20r7yQUbG34CWccnruxmeu5Z42zz51iKje84nnjuOYXjUJy4gt/QIzckCutOC3jbIhoFezjlX8b17pvnz3+8jlq4zW6px98HSK26g9G+V7ZlUFv7OHdQevJfsO3+bIz/6OkOVUa55yx/x+AP/hBCC9edcy60P/A3LRmscuHAJ1vMHGPSmSC3UWTXh8pNlipWTHglf8FA/bDiSohRXHOyucv6xJHvbXVwRxz58I6XlDzM19ZuEKoZt59Gy4wTzqwlnVmClJgkXBlB6gFdLI0ITp9qOFmoEoYE0GqAEkhhhINAMH00Y6JYbFQzIxLHSFRKmSdyyWL8+ir+Ym4vcl5WWEW4f3M1ly147hfFvlevp1IZ6ZUvd/00gpYZmmJFpbOjEk1FrzUQ6haZLhBTkChqpnIluGeQtm0SyGSsRI5sWGHGLtKmR03xELEYurtPQA6RukM1kiSfTSE2SyViYcQPTMkkmY0hNnriAF67lV3F6VxZXaq8rnEiqk/JkA6ITlWh9P/rrOCfbp87MRDTV3r2R9WFZWOikkjlIOUi3BpoJRix6X6mEzMdI5gz0kkdiZoxEdjXJuTnmS7EoC7tcpjQYEMZDXFsyMdhGvVSgVG2hcvhCJqd70JM20zGPqYlWsmWTqaEc00cLGHGdMJDgmfRqTYgmi8khiZ5pI+npLO0PIDQRCAxNI24av5LT+3Up20UIK4YWi0q7WEachLTAsogbCaSQSNMiLSySvo0+PoVbnEHTHXxNMBsD4Xl4UhHoGoqAQAb4EgJN4hlQMxVVMyDIVJjNeuhTHsqxcALwx9cRTl1A4OuIwmFwU3ilXEQ5mTYojUSiClqcmg2ha5DO+XiuxHUkugXJdEC1ZDIfgFE3cZKS46XIqO3pidYvjQZc3tnDuvNfX07v0wqdFUJcD5xoePSwUuonr94lvT4gdYOmvqjsc0tPNy1dUfG4dZesjxxdocf73izQchI5OcgfrhsjHSpqeifXXtROsr2FRP9aljsT1DpXkvDqmL7CrfisWnkdze0FqhVBKr+Mhp2iYKTp7LuQQBmggR1oJOGFDm2/itP7Rz/60QtNca666ire+ta3vjqD9W9AEERuitZWohahlyw2wVm+PAqb1XW44YbIMjh6FB5//GTnvMHBk725F3tY9NuSflsHLSARGigZh9ZW5iZcEsY88WycyxaeJKFX2FjZS4tdIhV6fKb6J4xXO2gLpjg+3ocdb8J2DRpjLVTr7aQTPo6fQDd9fDRqfgdWss78dArfy5JIOtiegfIFlhFwdF8TU0lFWwuYIk4qIfnGbQEtSz0e+JnDm65Ngp1EKV5UGgROT2G8HmT7ctBWryG1eg0A3Td/hM56DQpNXND5yeiERII3CpfUV24hM7nAjzWfi6d0Hmt3eKJfkKmbHGj1cXSFp0t299ZwTMm0FTLYVuV4M2Qdj9jSe0mbNRY6n4LxTQjbQtpxfOUTaiAWlqAUBIQoBK5vEqoAO5Ao5UIYw4wpkgmdqi9RuiIIJHbNwm5IAg9ioY5dFy9UzM/nYWQkWu+USnD9ld04b4NP/fj1oTBOJ3T2s8BFwIkylh8VQlyulPrEq3plZymKx0okCjGUX2fwkadY+9Zref7hPRx8/CBv/9Q7+MH/9QB1G377/W38xdefZ12qyjXNLh873s8HyxMcve+nfGuqi4/s+zHfPqgz5xoMTj3Mg2MCBci//SnffrZBT95lwwUb+e792+m6exXpTDc/e6LIe9/l0JQdY+fOcb7wtes5/OAPCDyf8298J43hQT73xg6AX6owPvGJT/DMM8/w3vdGBYW/+MUv8vjjj/PZz372NRnH08WhQ/DNb0YJ22972ykH9uyJci3e+174zndOJuVdfHFEUY2NwerVJ5smrVgRlSJ3nGiZZ9t8N341nqvzlmPP8XX3Zvq1YTaPPMrn9XfwH8L7ed7s5ZvJS3m7MwqFUazkAY5PvpHWJd9iqraKBf18lvcOMViLU50LyAQ5nHAKzU7hVZLUNEVgS2LSw6GBcgrEEy56ZhphW9SrMWaKIfVxm/kFgRO4LBVTzMz67Nnfgq0q/OUf9jJXLfOG3jbUVafn9H69yPbl4AYuNbdGPp5n74FHGRzazo2/+Um23vXFqLjgm9/LLY9/iaXpMtW4TnbY5JHuOkvmNS6bMLirp8zGMY2UE/LT5fCmoxrTScWONkHPcCdWtcpkXGdk5D1keu7Fm7yCwC3gl/vQm4cgZaMVO6kXl0BgIVXUu0T5JmGo49UNQkJQOjJ0mZ6W+C4oJdD1AE95+IGGAEpVH8MQFCyNhYrLbFHHbhj090d9su66C8rlbj57I3zi+2e/wjgdy+LNwDqlVAgghLgVeA7431JZxDImVtoEpdOyvB9hGPSc0xs5sg2D865owa0H0NXFW3q20ZNL04TNWzPDrBIeum6xrDrLQOiwxNapF5Msj4/w4EQfoRKsij1NX5CjryE4ulNjdPgSpiY62Xy5yUCvjRPmeWJnmrZCnskpQUnfgNJUlNuXa0KPJ/m732r/pcUH7777bnbu3IlcpLNuvvlm1q9ff9ZNKC0tJ42IF6G7O7IWdD3qjud5UTvVbduinArDiDK85+YiJ+nsbOR1jMUiiioWY31wHN/1SVgBa909dHnDZLQFNotnWCLmwQnZYHWwKlfjQG059WqKRsNnaPIaSm4LpUoX+0KfycpSTFElUEWq5SWgaYgwxFMmSoR4hoNA4HqS0A3JIXEdRaDA9wSGFVCaS+L6cYb2dkJ7jsJAjLYuSSYDsZSFFII/X30yce8XFR98vcj25WBIA0u3AOjuOYdELGLB+9ddhRCSVKaFc5dfSo89zHRG47nKJL22QS1W5pgIMD3FUCbARKcY83miO6BsCY5nFaFTYbC1gRYIPLGVam4SszaBPZvDDH3c6QH8+XORwkPqVaTS8JUiDFMYZgUt0FFOFtN08H2J65kY5iLliUIpidcwo/uIgyEhcDXCQMN1DcZGNEZHo3XNkiWRszsMYeNF3UgZlQY5mxXG6WZw54Di4vbPzxj7dw4/9Bn2j9Lp9ZAx07S2N4MQpOIGXW0pCAKEJpAW1AQccHyqyqJsJ3j8qEZdTTNunofbHnLbQzVGswKR97hj/xpkV4D0bb40sopyq8mOueW0zD5OKrEcX5TZuqedVHqIB59oYnpYx9QFTz7ToLWQJJfx2TcC155Xp6lTJ9sb53+841z+GH5haZCFhQUKi8mCpVLptR3M00RzM7zvd078IHmBdmPfPtToGKK9nWr3KsJag8yJwoJS4syU2TeTZMl4g3x6sadFEBB4DnZHK4mFKtXUJIHv4ekDeA0fz5PoqRir7RKGqrJraY26uYOhRg29eh5B0ETCq2E7JhUvjh6CHabQpQ1IbC+LlD6+BwpJEGhoMsSrJ/GqJlKGuLUEM66ODHW0uE2pIjBME4RHvrmG7+ikm8rsOeAz589x389MzMwcuiZ561Wd/O6Fywh8xVcfPwr8fIXxepDty0EIQVyP/H+ZfDvxZFRPK9+xNPIPhoLzY0vIDrRS+PF3OWaXWX28xk8vaeaYPYtVhaquKKdBCUWowNMkoQgJzQq+oUBILGsGobloTTuQro4qtxF6GZSbAM1GWS5Ks9FDHT800YwivptAaC5IBxVqoBJ4bhhduJL4isXnUxCqkMCWhCpkYgKkrqiUQ1pbNYQIaW+XPPJIZPDW63BVXw//5Tr4L/ecvZnep6MsPgM8J4T4GSCIfBeffFWv6izFaHmU2x/+H2xZ8xY2Js/l6W//iKs//Ds89p1H2PrEBH/xqWv4wj+P0vAFvz1zjO/t6yK5v8zypM1EupU7DmTJpaeJ50d5orwBS5Vpits8WV5Pd/wIWmCxbeZCllVmOFZayqhmoged1AMDXwQ0xZYy19AwZA2pwcRMyGGjQSE/xehkgTv/5zRvufIIH/3bq5i7bxef37IMVPiylNQnP/lJ1q9fz5YtW1BK8eijj/KZz3zmTA3tL8bISFTGYzGBjtZWSj/bQa0h6ezZz4PfD3Fmy7xrnQcbN8LMDEPP7+cO7b1clY9zXfMzkSM8CDikz3MoO8fVRcUPmyWhFkftlXwreQkdXoOMfTe3u+/gEuvHbBMZxkavY7awDbLzFOMLuGOdtPbcSXnmCsJEE4aWIZY6QqmcRzntSKOC5uTwwhRBEIKQoDSUCEBAqABfEYYQ1CzCUKFQKCGYmRMUmhRPPyXxG5LpiqA82cBLj5FIe1yyto2//8YEGy8o8AdXiZ9bfPB1JduXoOE1mKxO0p/v59jBp5mcOMxlV9/M3rv+HjTJisuu59vPfYO1dobaOVmKpU7uz47T7Eg2zGS5o1DljYMhC8mApzoFV0yaHM6HpB2IT3di1WqMWmnGht5LU+/3KZcuwZ9fRWgE6J1P4hseotKG9JMoo4KsdgMKp9oatcINDcLQACUQIkThAzoCUDKEUEcIRaB8fKWBEiTSPp6o4NeyuL5i34GQXM5kYSFydi9fDo8+Crnc2V0a5JeGzgIIITqI/BYC2KqUmny1L+zn4UyGzrqBy57RHfS3rqBgZqkPj5BY0oc7NcvMkTG6LlrDnq/fh2P7DFy9jC9+/sf0FgRNQ1X+/mA/51X2c0zfyAGZxxyPU03ZNGQce74Lq+kojpchrKURmVnqtR40UcIN8+iaje9nkUaFhBGjXkniBTpxSyMRC4CQWCqB4ddYu9Lj/X+Q4MqOw6RXdxMk4nzk9u3ce3iB//yWVXzgoq4XCvNNTEzwzDPPoJRi48aNtJ/aQ+I08JqFznreyaREz4uopwcewD8+hn7DW5hfEISVGk1b745s/EqF+sgcO9R6Vg7fT0vej3Iq5ufxpsapaSHZhQZPtYWErsu68Tg/Ma+lR19gY7CDw24vbfky31oSst/rY62qMDZyAyNOilK4goWsw2S5g5LXikhPUK73I/BQho+Hge/GUaFBoPmIIIUwy4SaQjUyCBkisqOEpQ4IDDCr4MdBKGRiARlaJDIN/GqeVFOFc5bmKS4oli13+PM/aqO106a1ySQRFy/kYbxcWO2/RbZnOs/CCzwMzcD3XZx6hWSmCfv4ERCCWL6FIz/5BoVUM6UDu3ho8imWHSvxXN6hXpxkNqiSaHjUEgY7mzy6yiEzWZPjcZf+WZ1jBR9P6hgzyxC548x7SwnGLsCrdCECg8r8uYhqE9S6CNEQeoOw0RZFXLkxcNJgeOBLEBpChqgwWnNreohSUQ/xVBJsWxEEGvmsBM2jVDRoadGwYgG5rEa1CmvWwAc/GDGpphkFctz5bNSi9fKB5l8rJfWqhc4KIVYppQ4IITYs7hpd/NsphOhUSu34Vb/0dQfXBU3D1Exy8V7SZgbVsKkcqZDodCk/P8T4rhG6lrYxPz6HY4eMPTfF4dkuZkeqtOlraG8ZY6ixmYo1R16UmcqsJBWbwqn2YuQHcSpdhECYmCMo9SI1G9tvQflpPECpEN1LUBE2rmdh6AGanMX1dGw3gfTKOLrLth2SsT+d5R8KNr9zwUO866Jp/tNMEQrn8Vc/PcDQt+/kr/70XexYWACgOypMxPj4OOPj42zYsOHnj8OZwqnZ6ye2r7kGfbE4oOEsEBppuO46+Md/hLY2Ev/xzZzz7fvILT2X+gNPcGQwxqpwHuOCi8nt3Qu9rfTNWoSlCqnEFDcGd2PIGLN+niflei61xrn++QMkUlneLOe51a5ihknSQYWaayJ8DYSLKYIoMEEL0aWGp0CERmRR+BZhKJBuIjIwfBOl+YhaGwQmhICfAHwIDQh0fM/Ek7PQPEUslWe8WiKeASdd4pt3WQTS4aJ1Md55Q4bVVooPbOrnn546hm3D752vs2bNanbsiH6arwvZvgwMLZJxaW6cuekhVqy9ikZpFoRAT6YZXRhGpZLQXKA+4RIrVSnk00zFNS44qvFMS8B8SmL5i/2tQg9dgd4oEIQldOEidJ9KLCAVzmMnRognB5nd+59gbhVC+JAZRZb7UV4SRAhGGRGCCmJIs4IihfKNF6K0hZIEfkSjIRSVcqRIwkAwvwBKaWgSZmZCMhnFwjysXBkph127osjuJUvgTW+CS9pOVqs9m3wYv4iG+lPgw8DfvMwxBVz9qlzR2YhGA3SdIB7j2aF9SCFpmahz6Ht7KAwU2HH/AR7brXP+uiN8f1dI1ZNcWx1mWsBk2cDWfWaTQ8z5PXhhgsBuYrLWRpMH9XonVQMML3KGuppO4KXRjTqhclEqRCYn8GudBKFEGj5St1FCEIgGSsXxQ4EKG4h4HbeeZHguztFGB8VSki53H6Mddb7gbcOVa/nC4w+z5MIL+Matf/2vblMIwUMPPXQGBvhXwInEu3ic4ceH8d2A8y7LRE7sxRLl46Ukml9k5sK38UAtQ9vc39HqOFG+Rr3OVn8TvgU93IltCAIZMhx2sYc1JGc9moMKjzkbWVlbYDShMZG0aZlu0NR8H3PaVUhSZAkg41Fy2wmdVpSMnNkqNFGeAUIRKg2kBygIBaGnQxDRU7gmyACED14clMS2QTddpmd9pBTETJOF+SaGVIAduuzepWHpNt99dJaPvK8LdYnklqeP8NAtt7D93jv52Mc+9q+G63Ul20Xs2/UAew8/ydJzLmf/Y99HSI0Vl72NJ6e3M1CeIC0s5rub2FVcoNKZo2eoxtFu6CmW6amFbG2XvG1I56l2n6kEpI6eg7SHIDHL9PDlGKvuo1ZfSmN8M2bXk5gd24nlDlKb3ERo2ARCIFEIs0IYxlBuBoEktHMgFOCD0lFKoAhBRJn3CFChQiqJpgUgJKEPhuGTy2noRkC9qjM5GcVgnHdeFOk9PQ2XXx7197pofQ+ff8fZlYdxOhncMaWU/cv2vVY40xncoQqjPhNhGCV85fMEkzPUR+dIr+1n8FtP4No+TSvb+X//n+dors4wNLWU3W6e+WIzIl1jIsiBnUHEx3DDLEqGuHYTGA2UZqNq7SijigpjCC+O0hoQxqNlkuaDdJHKROpVNBGighi6gnh8Ds/JYClJS2qIrPTp7drNilXf5pKuHFtaL+U/LXRyb1jgE29Yxu9fs+pF92bbNrETvSNOA2ek3MfLoVxG+QEik46yuvP5aLl3zz3Q34/avYfSwUlyh5+Nst5HR0FKbCuLOnyEeHzxc+Jx7KrPoUYPvel5EqUJJmtpOs1ZHnY3clguYb+2nu1mL7NuK/UwSyx3iKnyOfhagDIC/NCIFAASz6hAtQ80F6wq2FlQEgwH3BggwSpHVoZRh/g8NJogVkGg0DRBaCfQ4nUSCdC8LO29VTrbdGrTHcRignPWaPze7ynuGT/IVx6NSoN86jcGSCTiLxqiVyLbM01DnYDvu9iNCql0E+F8EYRAJlOMPvUvZDuWEIwMs/u+b7DCzzJpz3LgyFbOG/W4tz9gUrfB86ngMJ2WjCcCehYEQwVFKAWtUzmGOxZQxDAWmpjPFAmHbqBe68azM1i1HOXS+fheHOGnwI8RBBZaGMkqsDOgLRa0CA0kBmEQKQqBhgolmh6iSYmpa0gZApJCQUOpKEivtTWingwjyuq++GJ44xth8+ZonxBw1/ZRPn7Xrl9Lpve/Va7yl5/Ck6e57/UP141KSpzYdpxou1oFx6HqVvnyD+7kwMggvu0ztH0OFYTcct82/vSWp9i9c5Kv/bDB1+62ueefxplOJHhmoZ9ye0C2ZztW6wjSnKPZmMRKDdOoLUM5aex6O9I3Ce0CQbWHMBQopwncRLRiCZIRZUFEbeAlCQMT387juBlCJWgIn5pTAOHjJ4dJ5ieZSR5neMFn9/Pv59nxUb785Hdprj3HNckaH33vW/nydx550e1feumlr+lw/1pQLMLXv4647daoQu+hQ3D8eKQoPA927UJ4LrlmHWIxFnIZjg704xfL3DF3Bd8Sb8etODRUDE/pxIwAPZwmJuocyA7wD81bmEx18WTLEh5u6WA662O17MHKHEaaVTRClPSIE5D2BVooEEiEFmJgRhOKVY0UhhIgQzAXIuUgXYjNRTQHRLOD5oAvUL6F70tCq0RQz1Kfy9MIyowej/Hc9hiHjrocn6gyOmHzF38Rsu2WJbzr3GV8c+sw/eduiJyxp+D1KFtVLqPNzIHrUvze7RS//02ChXn27Lqf8eefRu3dy2iziee7xOsutZgE0yL0XHKhhbQsNCSJUCPla+RcRUsVysXzmBz8AOWJK0hVs8QnziE0fNLmMZKxIfSmw/hhVOJDkyEyM4ZQBpqQYNTQ4jUkBpppoxkKoaIJXNNB1wN0TaDrkUwDHxxX4fg+jQaUygEzcx6xWIhh+mRzLpYFfX1Rtf0dO6K0ocVyZ2zId/PfbzyfJ47M8sFbn8X2zlzh71/ks2gHuoC4EGI9kXMbIAMkft77Xtc4UY0UotThMIxojcV9DWEzNFFkqlhkSbyD8miZ0AsYHqkwNaaYnSpRtB3cMGBuTqMRr1OVGTAUQunURYxqownXzeIZDrqsITUHXasSNtpQmoMQAcpLgF6DQI+oCkIQUdYoaKA74OugTAggJECpJLZeQWk+QZBlxC7i6oqYVkZUUngqwVi9zvSBGX5rxf18Vzn813t2MVkpc8P6LsrlMvUTDZpeT7Dt6KVUpOinpqJJ17aj5Vq1GimNtjbIZKhfdSVFu0anozM/3U9g1PBjKwgrNUToosfizBt5Gt4UU47kmNXCfENnMpliTouT9X26E/vAEQQiRl/THvxYBc3OEjpNaOkpSqVV+EKPwitFAE4mkpkWhdjiJiA0QfpgNwEhBAbCj6GUABnRVdSzEJ9DSRtMheNIHM8g1TKLIUJKJZOR44pGA7whjdDLcslam++WqnzoC3fyB1cuQ0rxupVtWK/iF2ehrYtKZRaAdGWB4eoY6ak0KUcyEXNZAGpdTcykmpgPQsKwTKxcZyYdcMGMzlgtJGhWvOGoYE8rlEWaljkLK9/EylKS6sgSWpsTFOc70eqddMdK2IVdFM2A0IlRyywgwxy1eg4fD7wEUg8J3QQqNBFKQ0hF6OsoZYKKfFjIEBUKlIJQSYQIqdclQoZIKaNGmYEi9KPp5sSUc/w4rFoVpRItLMD153UjzoI8jJ9LQwkhbgbeD1wIPMNJZVEGblVKfe+1uMCX4kzTULZvY2lW5MhaRM2xmZkr0dfRxoNffRyn4bGuL8ktf/08iWqZR+sreN5OMFdcTi4xzIzfgik9yk4TemyewGwQLPQT6vUoncCPoYSKomWifJ/FlwQW+e9QAyQIO1Ia0kfoDRQ6ItCw4jPEjQph816WDvyQlU2PcUlV4/GROsPTMfYVq2R6VzFXc1nSlGRlTys333wzN91002mPxVlBQykVlfkIgigGsV6PbPtDh2Dr1iibr7c3+r9cjjrtNRpQKFCadlAHD5HbtJrqd+7GsksY6RhhGCD9AK+0wHGlsyxhMC1bmLItdlobuD3VQ91JkSoPMBtboFhbSsXQ8A2Xql1AUxq+HhJ4CYSvg+4Q+gkIYpGCECpSIkEM4nNg2pECUXqkUOwUmPXoHDcdyTZeRdWbkDIg3VImqOaQUmKqFL9xrYVuBRzadwdjY19nYmobsm2AtozF0uYkmUzmFcn2bKGhXoT5+ehvPo8/N4OWTCP27aP+yAPEB1aj9uymcu8PSVdctq/OMn9sP11luKepSGiZaFJnQlRRhkkNQbHRicrN0CEV1Bocb5KouQ1MqDTx+CGqIzcyl8jiFJupNLoIan3omTKa14ynXBpT3YCEcLH9sW9FLqlQACJK0lXRWpNQEgRREGJbm0alEq1L8/kosM9i9qbCAAAgAElEQVS2Izfa6tWwZQvceGO0tnlJ480XKKlfNUrqVYuGUkrdCtwqhHi7Uuq7v+oXvG5h29EElExGjXba2mD5cpzH9qGtXUZRLHDvk7dxw5Y/4Lu37uDBR6f4zx9fx1fv2Y0fCN7RodjamocwRsMYZUlc4qmAulsgg8OCECSUjXA17CCDj4v0Y4RKRJO/YtGJJkEF0URyQnOEixFBIgAVi/ZLB9DA11EInHo7MiEwKwOkpjZSshVPdqRJp8/hnVfcy9BIiY1XbOCJ/Ie4d980v3F5Fze9dd3PG42zD43GC5O+3bUsKhoriGipVAqWLGHiof0UegawQjtSFpdeSvmp5ymJFnqyGpUgjlqTJ3xuD7fV30OnO8TVrT/jn1akecuucbb2ruL73mb+JPgRo0Y3j8plvFt7AFV+N7FYHeSzxLw8WDni0iQtygz6fcQTM7hhDDfQ8TQXXUpIzhI6mUimZhUaOdAbkYWhLYA0wDdA2pC0oZGH3CggFhcPDhKBMBrUywZG0yHwcggtxXDdRrl1mjtvwkq+k7Yld7D59y7lrn1HWLGsly//3r+DFq35/Aub9tQYRksbwpBsqw2yPuyjpts8shyuP5zAdAIqhqLHT2JaNpppsbQI03kbK5DkXZ8eNYJThQUjyby3jJ7xkO3TW0Ap5lrTGKGB9F0aiQrZ1H6KyQXCyiqSrsIu96I1HUZWuxCGS1AtoCfnUbqLV2pBMzwSyZBaKYFueTTnYkxOKayYJB73mF8IMIwYCo+FMjQXDDo7oxShahXuvDMqWX/99fCTn0Qd97JZeMNAN597+5mzME4nKe8CIcSDSqkFACFEHviYUupTr+6lnWEIEal7iCafRS+oTMYRho6BRT7RhK6b5JriNBd04ikLywzRAkEqYWIoha4BpoZKNYg1fISYJ6b52LqgVj8PTYV4gQXohIG5WADfj+gnJSNfhTjhWjphYmgRzy0UqBAwAIUCUBYs0lGONU/dTfF8pYe23Ha65RS2laMibQ5OzdPjVfnc21fiVOt85oGD3H3n1/jJrV96TYf5V4aU0bKMkz0DgMiq0HXQNIxCGhkzo0y4XA4SCbSWAlZDQt7CLDdQLVn0rk1Yo4JEqgtj41UUmCRez5CWTSSmPJJanpwvabdLpJRL2hoHoZGMaehpD7ueQJcN1ogdzJkaGRzqhk5ddlIzfLRGD26oo+ILUXKek4gioKxZ0F3IjIKbAbMCMkDoPqrUDm0HMOo9hPPdBK270Yoh+Ca+rEFiDGYyKBEwND9EIgGm1oUmC9QW9vDOlTcCcNe+I3zs9jKJA3fz6U9/+gwJ69cLGU8gdRMRi5M0EuiJFKYRIyviaMt6SfQVyB4K0DMZYrWn0AydjOvTZAtivg8IAkMn5gSkHUXSdSn4cCwwCD1FuuTiONCIxZlxK3SpCjtSHiIJ7akKzuGr8U2PuK/jBklkvoHfiKFpMYRJVBnaM1CBIAw06nWBChUqhFJJoElBoQDxBPT0+bQ2GWQyJzv+KhVZFboeKYnFogRoGrzzwh6AM6IwTica6jml1PqX7NuhlDojQdtnmoZ6Ab4flcBuawPfJyhV0VoKFLfvIHBcmpJJtv31D2jKCB46kOT2oWUsTLcT5gImHQvbbSaQNkJ3cFWCsJ5HWZUo6QcQRhXl5BfJv4DIwtAg1BeVhEYUehGVTkb60TnCA91DxGdQSofEDFpiBrPrKcylP6DdmoXmkLHbfN76/72VrJWlOd7Gs/vX8rNPf4Iv3XkvH7pi4Off9yk4K2ioX4QTTaYgskJiMfj+96Ms8He9K6I2whAaDYK/+jQy8BBXb+FEAR/11JM0fJtE3WOhr43yyGG6GjrPN5koqdPq5Lmtuo4OUWZQa+WpsIua9JiKxZijFb/SjIxPEdgduE4aqTsEuk1Y6wSrBF4K9AYiP4Kys1CLIqHwYpCcjiKjZADVDohPQ60j8mMZDYQM0Y0Aam2kWoqYhkY2rbP2fJdt993E4cHd6FLj8/9ykC8/chT7O3/G+OF9p2VhnJU01KkIw5Org/n5aCFw9GgU1DA+HtUB27kTPA/XqYProkudw6pIIZ4nXJhna7vPWreANEwO2CNc4rWyVW9hwldcUxvmkd4kZj5HMJfgO/J8nF6HtHaI484GUqbBVOlaZo+uZnlfgkze5dBBA8PLMzQSQCAwDR0lAmIxSb0qQYSkkgLD0NC0aP2pVGQwrVwZKYSODvjQh6LHtlCIHtdTb/VU/CqJe69FNJQmhLBO+cI4YP2C8//9oVyOuHBgobGA4ztUtj7LY5/7B8JDg+y67Um+/tGHCccm+Nbfb+eWv99L5eFtfMFexu0zaRrpBIlzR9AyNqn0IEkEoXTpUEVML47hB0gl0HERMuppoAIrWjIrDZQR+SiUTiQyL1IKKECPrAx1SuJaqEc+j9DE8OIIvQG6R3jwPzB95APMKwPP9dh78CB3D97ND56/A0f/ElLV+fRPD/LnP7iHhtc4AwP9q2F6OkrQfhFqNbj11shbODQEX/saHDhAdd8wMyUzyoR69ll4+ml49lmmrruUUs6KJpypKdi5k1pnC4Pv2ELY18ttPTn+ZmAVUwWLR/yNPOpuQFYUe7R+xrQuLtcP4WoxNlnP0aeXWGpUsTqfIU4cctNo8TIyUSJmhsjsBFrfY8S794DuobRGVKeoZRCW3Q/Ne5F6EFmWdgriU+DkwLPAaESRVb6F0huo3FFqc3nsuiT0DI7UnmO+7LD14AEePv4wl66Z5YObuiiW63zqh3v/VZTU6wYzM9EL4Ac/gOefj+JPb7st6sV+4ED0EDQ3UylOMFuIQaXC8JouRk2bsF7jnvYk+/GYj8MzTQ5jOcmTusP38ss53myy3wh5Kl4g09rKQrCJ8kgPV63oxEinuDIpiEkfVRog3zZOv1ajNbsfQwiOjjj4WoVyNaCpZwYtZpPKBnSfPwiajRX3SHWMocVsbNunb8UClgXJtMuKcyoUCpEz2zCihoDf+Q4888zJ7bm5iBU/4baByML43NvP4/HDs3zottcmSup0aKh/Bh4UQtxCNDt9ALj1Vb2qsw2W9QIlZekWpmZiLOun94K1yPY2ei/U0Q2JbC6w+eImHNsjtbLAG3YPMmDYzJUSxMf76bUapHsmSaemsWc3YdQ8rKCCJ3Q010f5MZRnEfHUi94tEUYhlyxGRckTikFFoZcyiI6bC+DHorIRsQUEEEgXr1FAjl9CIELUsp+SapnH0x2yF+YY/Np+mi4tUJF1jj65jWXXDKBSg3zn6eUs2N/nr9/2RpqTzWdmzF8B0uloBfYixONR74uWlmgJt2kTdHQQa04hi4utOru6Ir/U0BDpg7NY56yDNTKyLAoFEi3NtLWkkbkpNvS0k1LHyOkr2TI5h8rnyQwMcPn+cVZaRZYsaWHTyCgXarOUm8aYt9uJN+8nFDCWkrhd29DcFkqhhwhMVGIGN+YgrTlkZgzlpgnDECFD6NhDCGh+GgKLcHolaj6G1ECkpglmliDCBFbmKG7gEXpJguZDpNdP07Z8lHDuAv7kA7/Lb73vPWhS497b7+LaG97JN7cOI3jlPb3PCix2hQRg/fpo6R2Pw6WXRjJ23WhWPf98YvOzaOUFaF1CS3secXwW3amxqaLoc0IMR2dd0aJvep50zGAmFdA267CJKm0Zh0RzO5uNAN0OyaYKXOOOs7apmbZhiHcPsSZp4q7LsHUUbroefngvzJZ8VmwKKZmzHNwhKVgJzEKKHLBQFDT1QQpBtSRYs1LHrUNTsySeNFg5ELnaLCsiKi66KIrLSCSiW81kounnpc/4qZTUa1FL6nQ65X1OCLEHuIaIFPmvSql/OZ0PF0K8Cfgi0Uz3NaXUZ19yXCwefzNQB95/xsuInMjfPxUnSEMgrkclrkV7G93vvgl0Sdo7Rk+sBhMTFBbq+I6PtqSHK/oOEWsbYM/uZbQe3ouVNdl2+HxmQ0WmFlJKuDheBb++BDNzhIbTBthIXER8nqDaGTlAzXLEaYtgMSZfAiqKyT/hDNfcyAoJEoSaEykVOw1Gg1B6eH4cNbeOSjBJom0S49IKhVYTOa4wRYLWNxlcft2l+N5B9h3Nct/OVsYqX+fjb7iQq5Zc9aLhuPfee/noRz8KcK4Q4hNnSq4nTPR4/GUOShnFH2pa1NNCCLAstPe9F312AXo7Tobcrl5NuqcnIoj7+6O+F0uWoB57BPn0z+Dmm+m+/07M6VHMWoblCJgexipN86G+PjTbhn0z/HE6QftciYY5xn1t8/yxOs6tLYqa24J1yVP4k5txamNY3btwD21BpPfgFnbB+IWYHQepU4TiAOneI9i+h+W2E2+epT60Glc1SDkD1AcvxmnbQVLPoOwsRmyK5DX/QCqjSPWE6Cmd9/zlZYxsb+X+rfejCY2Pf+LjbL5mM//wswm+8ugx4OUVxtki15f9DZ6aUJhMRr6pev3Ffbo3bADXxRA6RjwD5QbZPYdgrgqtrVzc1hZ10nIFN1TyaI0Fuow0y2brJOuC1pjBBYkUcmycc+wjET+05wBXej20tObYsOpa1hsNll3wTuJXb6FU8cgV4px3YY2J+QrntMb4l/tXcXG/pFQ0KBa7Mbuj6KdkqoclfZJly2CuGGfjxQAalapgzerFkiEiYk0vu+zkra5ceXL7pdFR8Nr6ME6HhkIpdY9S6s+UUh97BYpCA/4ncB2wBniPEGLNS067Dli++Pow8OXTvvJXA7VatDp5KUZGTprAMzNg29SrNk8+uB/lehz6zk7uvquOevwJvvV0E/+8o5vGg0/w37Ym+P/vrzI2McnkqmNMpReg1eaNFw8Szw2xMv08V6S3kUoeY016hIRwMY0yUoRoSiANG6SPjM2D1gD0RR+3DqEFQTwqERGYkTLxY5FDvNEK1TbwkpCYQLbvxCgMI+eXoWkh/vgaKgdWI4KNJN6SgIH1NFItPDP2DLsPlNESd5LJHuH5I6v5x8ePEoQnTdwgCPjDP/xD7rnnHoDnOUNyLRaj3/0vxIEDJ/t233EHHDnC4Z3jPPXw4hu3bYtoKMuK6KpqNTr/4YdhcpLh+eN8L9zLvFvm3vYqd1/Wynizxfb8NWxvrAal2NWxnKFkE/Mxnb/t3sTeiy5hW3ozo9PvZmf6PLTS5fSHm4kVzyM/uo7s9MX0JVcTq/STCJrpSHVhzK4nKzuJqxbMmYvJGc2oqVW4hy/HVzZhtYmwWsBb9mO83EH0NT9F/41Pkb7sG6Su+SK9A1VaUwWSSUhbaboz3STXJLn4dy9m84c2s/KSlfzk0E+4br3L/3HVMm7fOsz/+YPdL6Kkzha5YtuRZfdSFIsRD+N5UbjQ9u1Ru9yHHoq6Iz73XNQl68knowy39euj91xzTWRVdnfDuefC0qWoTJo9WYd6NsFkW5IHLijgN+WZb0kx1ZoE22YmbzHXnKQY7+Kr7pu4e7Yf56YbuX/cZTadYmR4lJ/dG02F337kPu647zFUYpoxnoH2Hfzmb8LkQpl0c5k/+iNFyVng4k0enf0lpmrj9PXBZGWKwakRTDNys5xKM70SvFaU1Ol0yrsE+DtgNWASWQk1pVTml7z1YuCwUuro4ufcAdwA7DvlnBuA21TkZX9aCJETQnQopSZe+a38GpBMvngFcwJdXScjo5qaQNNIABddsRxhGgz85rm0HxpHbLqEtx99As8LiW/ZxJ/ZzxLPNXH30dVUdtfJtBbQ7UkOV5u5oGOWsqeoOdAZNPDjBnGzgq4JZD2GE+ioQAMCVLl70XcBSHPR6a0i+ulE9q9/IoQ2iFZm1jxC6ah6O6GbJ3AyiNwonp0hbHsQ5qeZvLuIuiOAYBAVKmZMwZrPh8SsNOXYF7G83+exvWv4x8eP8uHNywHYtm0bAwMDLI26ESngjMg1n4/op1+I5cujFej550cKob+fZVaczpVt0fELLzzZH+PqqyMTxTRfCHTvKV/M24olcudeyLW1GSamj9J+aTctfgxqJjwzwZrGHKbmIVes4ffPb2fJJNwUr5JN7uL65iaCzDizCyXWXtxGuT9Any/SvarAkniRIB4yrjyCCx5Ey45h+QvYq3/IHOPIDhPRfIyyU8LsewwND181MM/5CbGkg6WbpLuHmbYXKGOQyTrM1DyOl44zNzjHnq/vYW5kjsAL+MvwL0kkE8wWZzn/N0yUUnz5kaNIIV+wMM4WuRKLRZ7el+JE6KyUkXwymajCwsJCJN90OqKqfB8eeSSaeQcGIuK/VIoUxbp1MDCA+NGPWO3WsdKK2PVv48rqHLo1SC7wCS6/DLoP0jw5AUvPgd/ZzPsm03SsbiW9qpv3vP+3aF9zHiKRIN8UXdO7r7mSyYU5WhIt3HxdDEPTyCfhI/8xRW+3TkuL4I8+lGFZv04slqW1EEfX4aJzWmkM+EBU+uOlxtQrwWthYZyOz+JLwLuBO4kS9N4HnE64TBcwcsr/o8DG0zinCzgzygJOKoVT4S3WgNF1KnWNVAqE3SB2NDJVtVQcPWFCRwf9v/+m6IFtbaVp7ySZvhbedkUTX/7bdja/oYfRe02ee26MLWsH2Hd0D89VTS7Xa+wWFq2xeYqlXmqGi+MkMWJFDBnSCBLgJhDKQCYmCRotEGroqQn8RlMUVZMaj6JlpAPJKQhNNKuI7yYRfgIVn0cPdALhoTlp6j8cRbvyEsTTz9D84RVUd4+hV2C+XsUyJU2JHBsHhjlwtJu/vvsQtm/zx1evZWxsjJ6enlNH54zIVYgXF6N9OdihSQwifnux3IUEkk2L/Pepdn0u9+LtUgn9sSfoCix4+GEGrn0bA7OzkQL6ylcgDnzwg9Qe+iki3o1lxMnsP4xsb+bKtg76jo2z7KpraUspho8fZN0bP8JXHv4bmtttqokCQ+mdTNVHaUk1cdH5GXZNHCNh6GSSDrafRJkKXfOBVsp2GUsatKX6KFpFAhXQkuqk4lRIGAkaQYNs3CeuJYgZMXZ9cRdb/mwLD3/+YT76jx/F3m4zOTxJw28gheRN610C1cdXHz2OG7j895s2nDVyBU5GsJ2KU0OCOqN+96RScKK3uFIRhZjPR/vn5iLr4uDByBFQKkVBDfE4rF6NFYaQzSIXSuRm5sEwENksenEBzj0X5003IvI5YpvWs2LnAczeDBSLdAweg+Z2lKah79kBm6+h206RKTloUqMpE4+aNGmwaePJB3TtOSfvqa056qZnGhLTiLZfbtp5pXjnhT0I8eoVHzxdGuowoCmlAqXULcCW03jby+nJl4ZinM45CCE+LIR4Vgjx7MwJOui1RLUaUVScEhg1PQ0PPgjz85S37mf+6QPRA3rsGBw+DJUKu+47ztGH9jM9Os6OfcfZt+coEyWF1jXF4ITOvlQ7c53tuNJmoO1+zhu4m6WxWVb2PEdWb5CKz5OJT6MpiYZ6IYlLygbSKiMCA3QbNBehJGgNpFlFBIkoqlZJjMwkWtd2tK5d6HYWZdRRM8tQbgytchlhoGjMtdO5/kKcww7NWi85o411beswdMm7Lq2ysqvO1x4boVT3+Dmh1melXBuNk+W9XhHK5Ui+QQA9PZH8t2+PKI7FCsRoGszPc2x1G1OFKHJtcvNVlJvbmHYCdncuQ9XrHNz1NFuP76IxNY4aOkZidIq1bWvZ1LGFdS0bSZkp0maapJlkeWE5WStLTI+xNL8UXehYmoUmNVJWCiEEpmaSsTI4vkPVqZKxMnSlurhu2XX0Znvpz/VHQRgtUQn0bDzLhus2cP+D93O4eJh5e56Hhx7mPZuSfPCKHn7w3BSD09Vfq1zhDPxmq9UoGkqISG7FYmRtNDVFpetvuCGyNgqFiJoSIrJINm6MrMmPfxx++7ejBhO/+7ssVDRKkw3wPBr3P469ZzCiKnfvhoMHcQ8+j731CZiZYfjpvRy+exsoRc2tUffOXGmVd1zQzeffcT5PHJ7l0UO/3nE/HcuiLoQwgZ1CiM8RrSKSp/G+UeDUpUo3MP4rnINS6qvAVyGK2T6N7/71oqXlhc2ursWNRC984AOQTpN/57Xk5+ejgydOkJLr/u83ItNJgnSOz/03neXndPL8k3V2/ayZy7bEedP2owxOV7EvvILByQEIQ5asWIIzUWaVWaQqHKa1BWLHptBiNmUkNWpolTZcAUFsHiO0+F/tnXtwXFUdxz+/5p00TdKktIUk0JYmUC2kUBCwKDoICIMjDpaHYAujjCDDlBlGZVSQcRjRGRU6DC8dhLGIg0MHAQuIWEQqD0tpCy225VX6SqFN0jTbNJvd/PzjnITbsNlHss/w+8zs5Obuued+93zP7rn3nHt+Z6C/B615B93fRHFZFwOVISaUd1GuysH+MijtozhaTMWstWh1mPJj1hN9qZeak55lYH8t5W9vgj2VlA2UMW9WE42TGlm3ex31VfV0hzv59cIFVBc3U1NZQmNjI9u2BS8u89fXwKTf1Dj8cPejMm2a6z9vbXXdVIPxpi5wE96oqGD+ihVwbAPU1zN39Wo4fi41zc1csHkznHoqX+ZLnLZnN9Wt8/hG1U2ICJPqD6ek6B+cXn4cH4Y+ZFfPLhonNVJaXEooHOLV7a/S2tBK0YQiDkQOUFteS3e4G1XlsKrDKJpQRGQgwhFTj6CypJK68jpm1c+iP9rPxj0bmVAygWPrjmVjy0aWL13OgjkLqKSSE6afwASZwBXzrqC6tJofnwuXnjyTmVMm0pFGXyEH39nqavdDDzB37sfbMwLLCd92m7uziEadx42N7u5y/vyPK4sfXJ++8PShibl137/UHReJuPM0NVFWVETZ7GOhqYk5TU3uaSwRJldOzvhHTcSFJzZyQnMtM6dMTJw4BZKZlHcksBs3XnE9bg3uu/zdRrzjioHNuKeoduDiS12qqhsCac4DrsU9XfE5YKmqnhwv37yZlJcO+vqgr4+DpZPYvNnV05YWCB8cQATWLH+fLa910Lm1h119ITZ2TGJHlzK98XVC/bV8sGcK5UQpqdnB3v2tSPUBopFKwtF9VDW9SJVOJLSnlf76Dcw9poPW05VdfTspKyqjo72D3rJe6kvrCa8K0763ndoFtcz/7HymTpxKOBKmbXobLfUtzK6fPSQ5EonQ0tLCc889x8yZM9fgLjjGp6/RqLuTqKpKrkN5pBlUceiP9tN5sJPq0mrae9rp6O1gW/c23u18l87eTrZ2baW1oZX3ut6jfX87c6fOBeCdzndorW+lt7+XipIK6irqKJIi2qa1EemMMO/oeRCFO26/g/3d+7nmmms4+uiRe48z5SvkqbefQsY6KS9uY+GfaHpQVS8bpbhzgdtxg+L3q+qtIvI9AFW9xz+KdydwDu5RvCtUNW6t+rRWvJ7397B3W4i3O2tY9cZBQj1hJtXUUV6pTKxWaqd1s6l7NRt3b6G0bzrbtvdzoHQ7U2ZtZ8q0frr7uglHwzTXNHNE1REsv205991/H+FomHe73qW8uJxpVdMoKy6jtryW5prmoRXLhrNixQqWLFnCli1b+nCPUpuvaeZA/wEiA27wc0f3DgZ0AEVp72mnqqSKlvoWKkoqKCsqozfSSygccg0GRSxatIhly5alfM5M+Armbb6Q0cbCn+AZ4HxVDY/2JOnEKp67AxkYiD0otu/gPjbt3cTO7p2EwiF6o70USzEVJRU0VDYwuWIyh1UdxpXfvJInnniC0tLSUevI+3Afn1LOPvvsMXmb9+E+jFGRjcbiXuAE4HEgNLhfVX8z2pOOBRH5CNg6bHcDsCcHckZLPug9ErcuSRd+aSXP7hHSx9J8pKpOiZU4VcaJr5AfmlPxNqO+Qkxv86GMUmU8aB6Tr8kMcO/0rwlAoifbM06sDysiq9N5JZRp8kGviNwca7+q3jJC+oxqHg++Qn5oTsXbbOgd7m0+lFGqmOb4K+X9UVUvB7pU9Y50ndDILebr+MW8NTJJvEc3TvRPQl0pInUiMjn4ypZAI+2Yr+MX89bIGPG6oe4BngZmAq9x6IQc9fvzhftyLSBFcql3tL7mQnOh+QqF5635mhyfes3JDHDfrapXp/OkRu4xX8cv5q2RCRI2FoZhGIaR2nTTHCIi54jIJhF5W0R+FON9EZGl/v31IpKTZV+HaUqk+QwR2Scia/3rplzoDOi5X0Q+FJE3R3g/7WVsvmaeXPjq8y0obwvNV68pe96qat6/cDPA38H1uZYC64A5w9KcCzyF66c9BXilADSfATyZ6/IN6PkCbk7NmyO8n9YyNl/Hp6+F6G0h+pptbwvlzmJobQx1M8kHY+0HGYq1r6ovA7UiEiMwftZIRnNeoaovAB1xkqS7jM3XLJADX6HwvC04XyG73hZKYzFSHP1U02STZPWcKiLrROQpEflMdqSNmnSXsfmaH2SijAvN2/HoK6SxjJOZwZ0PpG1tjCySjJ41uCn4PeKCLj6GW7IyX0l3GZuv+UEmyrjQvB2PvkIay7hQ7izStjZGFkmoR1W7VbXHb68ASkSkIXsSUybdZWy+5geZKONC83Y8+gppLONCaSz+C8wWkRniFmK6GBfYMMjjwLf96P8pwD7N1VrejoSaRWSaiFsoQUROxvmxN+tKkyfdZWy+5geZKONC83Y8+gppLOOC6IZS1YiIXAs8w8drY2yQQKx9YAVu5P9tfKz9XOn1mpLRfCFwtYhEgF7gYvWPMOQCEXkY98RHg4hsB24GSiAzZWy+Zods++rzLShvC9FXyK63NinPMAzDSEihdEMZhmEYOcQaC8MwDCMh1lgYhmEYCbHGwjAMw0iINRaGYRhGQvKusRCR60TkLRF5KA15LRaRwwP//15E5ow13xjnmS8iSxOkOSpOZMhDdI5BR5ufWZo2RGSJiFSmeMwZIvLksH3m6+h1mK9pxHz9xDGf8DUWeddYANcA56rqt4I7RWQ0c0IWA0Omqup3VHXj2OQdiogUq+pqVb1uDNksJqBzDLThnqlOGj9ZJ149WAKkVPlGwHwdPeZrmjBfxwSubB4AAAQWSURBVMBow9Vm4oVbFjIMvAFcD/wMtzTg34E/AUcB/8bFaFkDnBY49gf+uHXAbbgJND3AJmAtUAE8D8z36S/x6d8EfhnIpwe41efzMjA1hs7hus7Ahy4GpgDPen33AluBBq/9LeB3wAZ/bEUsncPOdR2wEVgP/NnvOxn4D/C6/9uKC6v8AfCRz+cir/OGQF5veh2DWu7yeRwJ3A2s9tpuCZx70I+Vft9ZwEv+8/0FmOj3nwP8D3gRWEoglLP5ar6ar4Xj64i/z7luIGIY+z7QEDD5tUFDcC1mud+eDaz221/1JlT6/yf7v0OVLfg/7qrgA19RioF/Al/3aRQ432//CvjJCJUvqCtY+e4EbgwYooHKFwHa/HuPAJfF0jnsXDuBMr9d6/9OAor99pnAo357MXDnMJ0jVb4B4JTAe4NlVuT1HBfDjwbgBaDK//9D4CagHBfZcjYucNkjwyuf+Wq+mq+F42usVyGE+3hcVXv9dglwp4i0AVGgxe8/E/iDqh4AUNV48d0BTgKeV9WPAHx/6xdwUSTDwGD/3WvAV5LQFWQBcIHX8bSIdAbee09V1wbyPiqBTnBXKA+JyGNeH0AN8KCIzMZV7pIk8hnOVnXx7QdZKCJX4b6M04E5/txBTvH7V/kQOaW4q5ZjcJ9tC4CILAOuSnB+89V8NV+TJ9e+FkRjEQpsXw/sBo7Hjbcc9PuF1MLuxgrbO0i/+mYYV8FHKqPQCPvj5d0X2I7ibmsTcR7ui/E14KfiYuj/HHebeYGIHIW7sohFhEPHpcoD20P6RWQGcANwkqp2isgDw9IOJQWeVdVLDtnpfgxSKf9Dzo/5ar7Gx3zNva95OcAdjxpgl6oOAJfjbsHA9SdeOfgUgIhM9vv3A9Ux8nkF+KKINIhIEa4/9F9p0vgisNDrOAuoS+KYmDr9QFaTqq7E9fHWAhNx5bDDJ1scJ5/3cUsuIm7t3RkjnH8SrjLuE5GpuG6CWHm+DHxeRI72eVaKSAuu73OGiMzy6Q6pnElgvpqvsT7nIObroeTE10JrLO4CFonIy7hb2hC420dcKN7VIrIW1+oCPADcI25x9aGrAnUhem8EVuIGxtao6l/TpPEW4CwRWYMzcRfOwHjE1In7ci0TkTdwA1u/VdUuXN/sL0RkFR9/AcF9njk+n4uAR4HJvkyuBjbHOrmqrvP5bwDuB1YF3r4PeEpEVvpugMXAwyKyHlcZj1HVg7jb2L+JyIu4QcJUMF/NVzBf89pXizqbZkSkDIiqC3l8KnC3qrblWpcxNszX8Yn5mjyFMGZRaDQDj/hb0jDw3RzrMdKD+To+MV+TxO4sDMMwjIQU2piFYRiGkQOssTAMwzASYo2FYRiGkRBrLAzDMIyEWGNhGIZhJMQaC8MwDCMh/we1k7/FV8+OFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "net = CharRNN.load_from_file(\"lstm\", N_LAYERS, HIDDEN_SIZE).eval()\n",
    "input_gates, forget_gates, cell_gates, output_gates, cell_states = net.extract_from_loader(tel)\n",
    "visualize_gate(input_gates, forget_gates, output_gates)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cell visualization (test set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "net = CharRNN.load_from_file(\"gru\", N_LAYERS, HIDDEN_SIZE).eval()\n",
    "reset_gates, update_gates = net.extract_from_loader(tel)\n",
    "visualize_cell(reset_gates[:, 50:1000], dataloader.decode(tel.dataset.data[50:1000], vocab))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Showcase of the rnn cell visualization\n",
    "![screenshot](visualization/screenshot.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}