homework_module_3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oMohh_6CwC4W"
   },
   "source": [
    "### Задача определения частей речи, Part-Of-Speech Tagger (POS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2Aad2tmBwC4Y"
   },
   "source": [
    "Мы будем решать задачу определения частей речи (POS-теггинга)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gYYV0mdmwC4f",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import nltk\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from nltk.corpus import brown\n",
    "import matplotlib.pyplot as plt\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FPgI52lRwC4n"
   },
   "source": [
    "Вам в помощь http://www.nltk.org/book/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hxdJxMEAwC4o"
   },
   "source": [
    "Загрузим brown корпус"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZvhXAL_9wC4q",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package brown to\n",
      "[nltk_data]     /Users/vladimir_yankovskiy/nltk_data...\n",
      "[nltk_data]   Package brown is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('brown')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wto8PSC6wC4v"
   },
   "source": [
    "<b>Существует не одна система тегирования, поэтому будьте внимательны, когда прогнозируете тег слов в тексте и вычисляете качество прогноза. Можете получить несправедливо низкое качество вашего решения."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "eJ6tuHA_wC4z"
   },
   "source": [
    "Cейчас будем использовать универсальную систему тегирования universal_tagset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Cht7dImWwC42"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package universal_tagset to\n",
      "[nltk_data]     /Users/vladimir_yankovskiy/nltk_data...\n",
      "[nltk_data]   Package universal_tagset is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('universal_tagset')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IiTimRRywC47"
   },
   "source": [
    "<img src=\"https://4.bp.blogspot.com/-IcFli2wljs0/WrVCw3umY_I/AAAAAAAACYM/UJ_neoUAs3wF95dj2Ouf3BzxXzB_b2TbQCLcBGAs/s1600/postags.png\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iyDBMcBSwC48"
   },
   "source": [
    "Мы имеем массив предложений пар (слово-тег)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BobflewQwC4-",
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[('The', 'DET'), ('Fulton', 'NOUN'), ('County', 'NOUN'), ('Grand', 'ADJ'), ('Jury', 'NOUN'), ('said', 'VERB'), ('Friday', 'NOUN'), ('an', 'DET'), ('investigation', 'NOUN'), ('of', 'ADP'), (\"Atlanta's\", 'NOUN'), ('recent', 'ADJ'), ('primary', 'NOUN'), ('election', 'NOUN'), ('produced', 'VERB'), ('``', '.'), ('no', 'DET'), ('evidence', 'NOUN'), (\"''\", '.'), ('that', 'ADP'), ('any', 'DET'), ('irregularities', 'NOUN'), ('took', 'VERB'), ('place', 'NOUN'), ('.', '.')], [('The', 'DET'), ('jury', 'NOUN'), ('further', 'ADV'), ('said', 'VERB'), ('in', 'ADP'), ('term-end', 'NOUN'), ('presentments', 'NOUN'), ('that', 'ADP'), ('the', 'DET'), ('City', 'NOUN'), ('Executive', 'ADJ'), ('Committee', 'NOUN'), (',', '.'), ('which', 'DET'), ('had', 'VERB'), ('over-all', 'ADJ'), ('charge', 'NOUN'), ('of', 'ADP'), ('the', 'DET'), ('election', 'NOUN'), (',', '.'), ('``', '.'), ('deserves', 'VERB'), ('the', 'DET'), ('praise', 'NOUN'), ('and', 'CONJ'), ('thanks', 'NOUN'), ('of', 'ADP'), ('the', 'DET'), ('City', 'NOUN'), ('of', 'ADP'), ('Atlanta', 'NOUN'), (\"''\", '.'), ('for', 'ADP'), ('the', 'DET'), ('manner', 'NOUN'), ('in', 'ADP'), ('which', 'DET'), ('the', 'DET'), ('election', 'NOUN'), ('was', 'VERB'), ('conducted', 'VERB'), ('.', '.')], ...]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "brown_tagged_sents = brown.tagged_sents(tagset=\"universal\")\n",
    "brown_tagged_sents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jSu1KqRrwC5L"
   },
   "source": [
    "Первое предложение"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zCHCZPlkwC5N"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('The', 'DET'),\n",
       " ('Fulton', 'NOUN'),\n",
       " ('County', 'NOUN'),\n",
       " ('Grand', 'ADJ'),\n",
       " ('Jury', 'NOUN'),\n",
       " ('said', 'VERB'),\n",
       " ('Friday', 'NOUN'),\n",
       " ('an', 'DET'),\n",
       " ('investigation', 'NOUN'),\n",
       " ('of', 'ADP'),\n",
       " (\"Atlanta's\", 'NOUN'),\n",
       " ('recent', 'ADJ'),\n",
       " ('primary', 'NOUN'),\n",
       " ('election', 'NOUN'),\n",
       " ('produced', 'VERB'),\n",
       " ('``', '.'),\n",
       " ('no', 'DET'),\n",
       " ('evidence', 'NOUN'),\n",
       " (\"''\", '.'),\n",
       " ('that', 'ADP'),\n",
       " ('any', 'DET'),\n",
       " ('irregularities', 'NOUN'),\n",
       " ('took', 'VERB'),\n",
       " ('place', 'NOUN'),\n",
       " ('.', '.')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "brown_tagged_sents[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SIV2MiRxwC5Q"
   },
   "source": [
    "Все пары (слово-тег)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dVx9e9HcwC5R"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('The', 'DET'), ('Fulton', 'NOUN'), ...]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "brown_tagged_words = brown.tagged_words(tagset='universal')\n",
    "brown_tagged_words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "y-ADby6LwC5V"
   },
   "source": [
    "Проанализируйте данные, с которыми Вы работаете. Используйте `nltk.FreqDist()` для подсчета частоты встречаемости тега и слова в нашем корпусе. Под частой элемента подразумевается кол-во этого элемента в корпусе."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JzRoXuKFcMZK",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Приведем слова к нижнему регистру\n",
    "brown_tagged_words = list(map(lambda x: (x[0].lower(), x[1]), brown_tagged_words))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4giWaqXjwC5W"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Кол-во предложений:  57340\n"
     ]
    }
   ],
   "source": [
    "print('Кол-во предложений: ', len(brown_tagged_sents))\n",
    "tags = [tag for (word, tag) in brown_tagged_words] # наши теги\n",
    "words = [word for (word, tag) in brown_tagged_words] # наши слова\n",
    "\n",
    "tag_num = pd.Series(Counter(tags)).sort_values(ascending=False) # тег - кол-во тега в корпусе\n",
    "word_num = pd.Series(Counter(words)).sort_values(ascending=False) # слово - кол-во слова в корпусе"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yfiPpCcLwC5Z",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NOUN    275558\n",
       "VERB    182750\n",
       ".       147565\n",
       "ADP     144766\n",
       "DET     137019\n",
       "ADJ      83721\n",
       "ADV      56239\n",
       "PRON     49334\n",
       "CONJ     38151\n",
       "PRT      29829\n",
       "NUM      14874\n",
       "X         1386\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tag_num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8Y1huw7TwC5b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAAE/CAYAAACTuN+cAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAgQUlEQVR4nO3df5SkVX3n8fdHRjlkDQRkJMggYwSjwCYYWSTRJBoSQE0WzMJxSBTcJcEY3IT8OgE3G1xdEswGyRIDLgYC+AsI/mJFohPRqFlEBiXyS8IoBEYmgg5BNEgc/O4fdVufaXpmerpvd9W079c5dbrq+zz3qXupoubTt+/zVKoKSZIkSf08btwdkCRJkpYaQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJWmJSPI/k3wlyT+Puy+S9L0uXidbkhZGkq8PHn4f8AjwaHv8qqp6R8fn2hv4R2Cfqrqv13ElSXOzbNwdkKSlqqqeOHU/yV3Ar1TV3y7Q0+0DfHVzATvJsqrauEDPLUmaxuUikrTIkhyS5Nok/5JkfZI3J3nCYPvhSW5P8mCSc5P8XZJf2cLxfhZYDTwlydeTXJRkZZJKcmKSu4Fr2r7/JcltSR5I8qEk+wyO83NJPt+e983D503yuiRvH+w7dfxl7fEuSS5o4/lSW7qyQ9v2yiSfTPKn7XnvTPKiwbF2S/JXSe5t29/X6jcn+YXBfo9vy2EOmudLIEkLzpAtSYvvUeC3gN2BHwcOA34dIMnuwBXAacCTgNuBn9jSwdrs+IuAe6vqiVX1ysHmnwaeBRyR5GjgtcAvAsuBTwDvGjzvu4E/aP36AvC8bRjTxcBGYF/g2cDhwPAXg+e2sewO/AlwQZK0bW9jtJzmAODJwNmtfgnw8sExXgysr6obt6FfkjQWhmxJWmRVdUNVfaqqNlbVXcD/YRSGYRQkb6mq97TlHecA8zmR8XVV9Y2qehh4FfDHVXVbO/YfAQe12ewXA7dW1RVV9S3gz2b7vEn2YBTyT2nPdR+joLxqsNs/VdVbq+pRRoF8T2CPJHu2tr9WVQ9U1beq6u9am7cDL06yc3v8CkaBXJImnmuyJWmRJXkG8CbgYEYzuMuAG9rmpwD3TO1bVZVk3Tye7p7B/X2A/53krGF3gL0287zDtluyD/B4YP13J6d53LTn/k5gr6p/bfs9EdgN2FBVD0w/aFXdm+Tvgf+U5L2MwvhvzrJPkjRWhmxJWnznAZ8Fjquqh5KcAhzTtq0HVkzt2JZUrHjMEWZveAmpe4AzZrqqSZL9gL2nPe/eg12+wegXgik/OO24jwC7z+HkynuA3ZL8QFX9ywzbL2a07GQZcG1VfWkbjy9JY+FyEUlafN8PfA34epJnAq8ebLsK+PdJjm4nFZ7MpoF2Pt4CnJbkAPjOyYrHDp73gCS/2J73N6Y9743ATyV5apJdGK0ZB6Cq1gMfBs5KsnOSxyV5epKfZita26uBc5Ps2k5u/KnBLu8DfozRDPYlcxu2JC0+Q7YkLb7fBX4JeAh4K3DZ1Iaq+gpwLKOTA78K7A+sYTRTPC9V9V7gjcClSb4G3MxoCcbwec9sz7sf8PeDtqtbPz/HaGnLB6Yd/njgCcCtwAOMTt7cc5ZdewXwLeDzwH3AKYPnfZjRCZlPA94z27FK0rj5ZTSSNMGSPA5YB/xyVX10kZ/7Y8Dbq+ovF/N5Z+jHHwLPqKqXb3VnSZoQzmRL0oRJckSSH0iyI6NL7gX41Ji7NRZJdgNOBM4fd18kaVsYsiVp8vw4o+tUfwX4BeDoqno4yVval81Mv71lvN1dGEl+ldGJkVdX1cfH3R9J2hYuF5EkSZI6cyZbkiRJ6syQLUmSJHW25L6MZvfdd6+VK1eOuxuSJEla4m644YavVNXymbYtuZC9cuVK1qxZM+5uSJIkaYlL8k+b2+ZyEUmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSeps2bg7sJSsPPWqcXdhq+468yXj7oIkSdKS50y2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHW21ZCdZO8kH01yW5Jbkvxmq78uyZeS3NhuLx60OS3J2iS3JzliUH9OkpvatnOSpNV3THJZq1+XZOWgzQlJ7mi3E7qOXpIkSVoAy2axz0bgd6rqM0m+H7ghyeq27eyq+tPhzkn2B1YBBwBPAf42yTOq6lHgPOAk4FPAB4EjgauBE4EHqmrfJKuANwIvS7IbcDpwMFDtua+sqgfmN2xJkiRp4Wx1Jruq1lfVZ9r9h4DbgL220OQo4NKqeqSq7gTWAock2RPYuaquraoCLgGOHrS5uN2/AjiszXIfAayuqg0tWK9mFMwlSZKkibVNa7LbMo5nA9e10muSfC7JhUl2bbW9gHsGzda12l7t/vT6Jm2qaiPwIPCkLRxLkiRJmlizDtlJngi8Gzilqr7GaOnH04GDgPXAWVO7ztC8tlCfa5th305KsibJmvvvv39Lw5AkSZIW3KxCdpLHMwrY76iq9wBU1Zer6tGq+jbwVuCQtvs6YO9B8xXAva2+Yob6Jm2SLAN2ATZs4VibqKrzq+rgqjp4+fLlsxmSJEmStGBmc3WRABcAt1XVmwb1PQe7vRS4ud2/EljVrhjyNGA/4NNVtR54KMmh7ZjHA+8ftJm6csgxwDVt3faHgMOT7NqWoxzeapIkSdLEms3VRZ4HvAK4KcmNrfZa4LgkBzFavnEX8CqAqrolyeXArYyuTHJyu7IIwKuBi4CdGF1V5OpWvwB4W5K1jGawV7VjbUjyBuD6tt/rq2rDXAYqSZIkLZathuyq+iQzr43+4BbanAGcMUN9DXDgDPVvAsdu5lgXAhdurZ+SJEnSpPAbHyVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHW21ZCdZO8kH01yW5Jbkvxmq++WZHWSO9rPXQdtTkuyNsntSY4Y1J+T5Ka27ZwkafUdk1zW6tclWTloc0J7jjuSnNB19JIkSdICmM1M9kbgd6rqWcChwMlJ9gdOBT5SVfsBH2mPadtWAQcARwLnJtmhHes84CRgv3Y7stVPBB6oqn2Bs4E3tmPtBpwOPBc4BDh9GOYlSZKkSbTVkF1V66vqM+3+Q8BtwF7AUcDFbbeLgaPb/aOAS6vqkaq6E1gLHJJkT2Dnqrq2qgq4ZFqbqWNdARzWZrmPAFZX1YaqegBYzXeDuSRJkjSRtmlNdlvG8WzgOmCPqloPoyAOPLntthdwz6DZulbbq92fXt+kTVVtBB4EnrSFY0mSJEkTa9YhO8kTgXcDp1TV17a06wy12kJ9rm2GfTspyZoka+6///4tdE2SJElaeLMK2Ukezyhgv6Oq3tPKX25LQGg/72v1dcDeg+YrgHtbfcUM9U3aJFkG7AJs2MKxNlFV51fVwVV18PLly2czJEmSJGnBzObqIgEuAG6rqjcNNl0JTF3t4wTg/YP6qnbFkKcxOsHx021JyUNJDm3HPH5am6ljHQNc09Ztfwg4PMmu7YTHw1tNkiRJmljLZrHP84BXADclubHVXgucCVye5ETgbuBYgKq6JcnlwK2MrkxyclU92tq9GrgI2Am4ut1gFOLflmQtoxnsVe1YG5K8Abi+7ff6qtowt6FKkiRJi2OrIbuqPsnMa6MBDttMmzOAM2aorwEOnKH+TVpIn2HbhcCFW+unJEmSNCn8xkdJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqbNm4O6DJtPLUq8bdhVm568yXjLsLkiRJj+FMtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqbKshO8mFSe5LcvOg9rokX0pyY7u9eLDttCRrk9ye5IhB/TlJbmrbzkmSVt8xyWWtfl2SlYM2JyS5o91O6DZqSZIkaQHNZib7IuDIGepnV9VB7fZBgCT7A6uAA1qbc5Ps0PY/DzgJ2K/dpo55IvBAVe0LnA28sR1rN+B04LnAIcDpSXbd5hFKkiRJi2yrIbuqPg5smOXxjgIurapHqupOYC1wSJI9gZ2r6tqqKuAS4OhBm4vb/SuAw9os9xHA6qraUFUPAKuZOexLkiRJE2U+a7Jfk+RzbTnJ1AzzXsA9g33Wtdpe7f70+iZtqmoj8CDwpC0cS5IkSZpocw3Z5wFPBw4C1gNntXpm2Le2UJ9rm00kOSnJmiRr7r///i10W5IkSVp4cwrZVfXlqnq0qr4NvJXRmmkYzTbvPdh1BXBvq6+Yob5JmyTLgF0YLU/Z3LFm6s/5VXVwVR28fPnyuQxJkiRJ6mbZXBol2bOq1reHLwWmrjxyJfDOJG8CnsLoBMdPV9WjSR5KcihwHXA88OeDNicA1wLHANdUVSX5EPBHg6UohwOnzaW/0spTrxp3F7bqrjNfMu4uSJKkTrYaspO8C3gBsHuSdYyu+PGCJAcxWr5xF/AqgKq6JcnlwK3ARuDkqnq0HerVjK5UshNwdbsBXAC8LclaRjPYq9qxNiR5A3B92+/1VTXbEzAlSZKksdlqyK6q42YoX7CF/c8AzpihvgY4cIb6N4FjN3OsC4ELt9ZHSZIkaZLMabmIpPFy+YskSZPNr1WXJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZl/CTNFbbw+UIwUsSSpK2jTPZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI622rITnJhkvuS3Dyo7ZZkdZI72s9dB9tOS7I2ye1JjhjUn5PkprbtnCRp9R2TXNbq1yVZOWhzQnuOO5Kc0G3UkiRJ0gKazUz2RcCR02qnAh+pqv2Aj7THJNkfWAUc0Nqcm2SH1uY84CRgv3abOuaJwANVtS9wNvDGdqzdgNOB5wKHAKcPw7wkSZI0qbYasqvq48CGaeWjgIvb/YuBowf1S6vqkaq6E1gLHJJkT2Dnqrq2qgq4ZFqbqWNdARzWZrmPAFZX1YaqegBYzWPDviRJkjRx5rome4+qWg/Qfj651fcC7hnst67V9mr3p9c3aVNVG4EHgSdt4ViSJEnSROt94mNmqNUW6nNts+mTJiclWZNkzf333z+rjkqSJEkLZa4h+8ttCQjt532tvg7Ye7DfCuDeVl8xQ32TNkmWAbswWp6yuWM9RlWdX1UHV9XBy5cvn+OQJEmSpD7mGrKvBKau9nEC8P5BfVW7YsjTGJ3g+Om2pOShJIe29dbHT2szdaxjgGvauu0PAYcn2bWd8Hh4q0mSJEkTbdnWdkjyLuAFwO5J1jG64seZwOVJTgTuBo4FqKpbklwO3ApsBE6uqkfboV7N6EolOwFXtxvABcDbkqxlNIO9qh1rQ5I3ANe3/V5fVdNPwJQkSZImzlZDdlUdt5lNh21m/zOAM2aorwEOnKH+TVpIn2HbhcCFW+ujJEmSNEn8xkdJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktTZVr9WXZI0eytPvWrcXdiqu858ybi7IElLnjPZkiRJUmeGbEmSJKkzQ7YkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjozZEuSJEmdGbIlSZKkzgzZkiRJUmfLxt0BSdJkWnnqVePuwqzcdeZLxt0FSXoMZ7IlSZKkzuYVspPcleSmJDcmWdNquyVZneSO9nPXwf6nJVmb5PYkRwzqz2nHWZvknCRp9R2TXNbq1yVZOZ/+SpIkSYuhx0z2C6vqoKo6uD0+FfhIVe0HfKQ9Jsn+wCrgAOBI4NwkO7Q25wEnAfu125GtfiLwQFXtC5wNvLFDfyVJkqQFtRDLRY4CLm73LwaOHtQvrapHqupOYC1wSJI9gZ2r6tqqKuCSaW2mjnUFcNjULLckSZI0qeYbsgv4cJIbkpzUantU1XqA9vPJrb4XcM+g7bpW26vdn17fpE1VbQQeBJ40zz5LkiRJC2q+Vxd5XlXdm+TJwOokn9/CvjPNQNcW6ltqs+mBRwH/JICnPvWpW+6xJOl70vZwtRSvlCItHfOaya6qe9vP+4D3AocAX25LQGg/72u7rwP2HjRfAdzb6itmqG/SJskyYBdgwwz9OL+qDq6qg5cvXz6fIUmSJEnzNueQneTfJfn+qfvA4cDNwJXACW23E4D3t/tXAqvaFUOexugEx0+3JSUPJTm0rbc+flqbqWMdA1zT1m1LkiRJE2s+y0X2AN7bzkNcBryzqv4myfXA5UlOBO4GjgWoqluSXA7cCmwETq6qR9uxXg1cBOwEXN1uABcAb0uyltEM9qp59FeSJElaFHMO2VX1ReBHZ6h/FThsM23OAM6Yob4GOHCG+jdpIV2SJEnaXviNj5IkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOpvv16pLkqQx8GvipcnmTLYkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnXidbkiSNldf81lLkTLYkSZLUmSFbkiRJ6syQLUmSJHVmyJYkSZI6M2RLkiRJnRmyJUmSpM4M2ZIkSVJnhmxJkiSpM0O2JEmS1JkhW5IkSerMkC1JkiR1ZsiWJEmSOjNkS5IkSZ0tG3cHJEmSlpKVp1417i5s1V1nvmTcXVjynMmWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktSZIVuSJEnqzJAtSZIkdWbIliRJkjrbLkJ2kiOT3J5kbZJTx90fSZIkaUsmPmQn2QH4C+BFwP7AcUn2H2+vJEmSpM1bNu4OzMIhwNqq+iJAkkuBo4Bbx9orSZKkJW7lqVeNuwuzcteZLxl3Fx5j4meygb2AewaP17WaJEmSNJFSVePuwxYlORY4oqp+pT1+BXBIVf3XwT4nASe1hz8M3L7oHV04uwNfGXcnOllKY4GlNZ6lNBZYWuNZSmMBxzPJltJYYGmNZymNBZbWePapquUzbdgelousA/YePF4B3DvcoarOB85fzE4tliRrqurgcfejh6U0Flha41lKY4GlNZ6lNBZwPJNsKY0FltZ4ltJYYOmNZ3O2h+Ui1wP7JXlakicAq4Arx9wnSZIkabMmfia7qjYmeQ3wIWAH4MKqumXM3ZIkSZI2a+JDNkBVfRD44Lj7MSZLaRnMUhoLLK3xLKWxwNIaz1IaCzieSbaUxgJLazxLaSyw9MYzo4k/8VGSJEna3mwPa7IlSZKk7YohexEkqSRnDR7/bpLXDR6flOTz7fbpJM8fbLsrye6Dxy9I8oF2/5VJvp3kRwbbb06ychHG9LEkR0yrnZLkg0keTnLj4Hb8YCw3Jflckr9Lss+g7aNt339I8pkkP7HQY/hekuSl7X34zPZ4ZXudPpvktva+O2Gw/yuT3N9ek1uT/Or4er+pwXvllvZ++e0kj2vbXpDkwWnvv5cN7v9zki8NHj9h3OOZMsfX6M3j6/FjbcsY2mt17bT2y5J8OcmeY+r/1Hvr5iR/neT7Zqj/3yQ/MGhzQJJrkvxjkjuS/PckadvG9hk9eL4fTHJpki+0/5c/mOQZ8+l3pv27NC7b8nolua7V7h58tt24mK/FZsaw2XyQ5KIkx0zb/+vt58rW9g2Dbbsn+dakfS5MSbJ3kjuT7NYe79oe77O1ttsrQ/bieAT4xZk+lJL8PPAq4PlV9Uzg14B3JvnBWR57HfDfuvV09t7F6EovQ6uAPwa+UFUHDW6XDPZ5YVX9CPAx4A8G9Yfbvj8KnNaOo36OAz7Jpq/ZF6rq2VX1rFb/rST/ebD9sqo6CHgB8EdJ9liszm7F1HvlAODngBcDpw+2f2La+++yqfvAW4CzB9v+bQz935y5vEaTZlvG8HFgxbSQ87PAzVW1frE6PM3Ue+tA4N8YfR5Pr28ATgZIshOjq12dWVXPAH4U+Ang1wfHHNdnNC00vxf4WFU9var2B14L7MEE93sbzPr1qqrnts+AP6R9trXbXWPp+XdtNh/MwheBnx88PhaY2AtDVNU9wHnAma10JnB+Vf3T+Hq1sAzZi2Mjo0X+vzXDtt8Hfq+qvgJQVZ8BLqZ9iM/CB4ADkvxwj45ugyuAn0+yI4x+qwaewuiDeTauZfPf3Lkz8MB8O6iRJE8EngecyGN/MQKgqr4I/DbwGzNsuw/4AjBxsw2tbycBr5mahdsezfc1mgTbOoaq+jbw18DLBrusYvQL/CT4BLDvDPXhZ9cvAX9fVR8GqKp/BV4DnDrYf1yf0QAvBL5VVW+ZKlTVjcAzmOx+z8VsXq9JtKV8sDUPA7clmbre9MuAy3t1bIGcDRya5BTg+cBZW959+2bIXjx/Afxykl2m1Q8AbphWW9Pqs/Ft4E8YzU4smqr6KvBp4MhWWgVcBhTw9Gz65/qfnOEQRwLvGzzeqe37eeAvgTfM0EZzczTwN1X1j8CGJD+2mf0+AzxzejHJDwE/BKxdsB7OQwtujwOe3Eo/Oe399/Qxdm+2jmYer9GEOJptH8N3/iLWfmF/MfDuBe7nViVZBrwIuGlafQfgML77XQ2P+fyuqi8AT0yycyuN5TO6OZDH/vsCk9/vbbINr9ek2lw+mI1LgVVJVgCPMu3L+iZNVX0L+D1GYfuUCftrYneG7EVSVV8DLmF2s1BhFFYZ/NzkcNMev5PRb4ZPm3sP52S4ZGQ4AzV9ucgnBm0+muQ+Rn8WfuegPvXnvWcyCuCXbM8zkxPmOEYfxLSfx21mv+n/vV+W5EZGr+urqmrDwnSvi2Hfpy8X+cLYejV7c32NJsk2j6GqrmcU7H6YUUj6VFWN869YO7X3/BrgbuCCafWvArsBq1t9+Fk93bA+rs/ozdle+z3dtr5eE2kL+WA2//7/DaNlc8cxmujaHrwIWM/ol8Albbu4TvYS8meMZnH+alC7FXgOcM2g9mOtDqMPiV2Br7THuw3uA9/5wp6zGC09WUzvA97UZqx2qqrPzOIkkhcC3wAuAl7P6E/Hm6iqa9v6tOXAfT07/L0myZOAnwEOTFKMvtCpgHNn2P3ZwG2Dx5dV1WsWvpfz02baH2X0XnnWmLuzzeb5Gk2EeY7hUka/pD+L8S8Vebit252x3mYaP8BoOd85jNa//tRwx/Z+/HpVPTQ1TzDGz+hbgGM2U5/kfs/Wtr5ek+zPeGw+mPr3H4B2wuD0f///LckNwO8w+gvFLyx4T+chyUGMfik4FPhkkkvHeA7GgnMmexG1mcDLGa1ZnPInwBvbP1JTb8BX8t1/nD4GvKJt2wF4OfDRGQ5/EaPZ4eXdO74ZVfV1Rv27kG34x7GqHgZOAY6fOst4KKMrE+zA6ANG83MMcElV7VNVK6tqb+BOYMVwp/bL0Z8Cf774XZy7JMsZncz45tp+L/q/FF6j+YzhXYw+136GCf+zflU9yGi28XeTPB54B/D8JD8L3zkR8hxGn+vTXcQif0YzmrzZMYOrAyX5D8AdTHa/u5jh9ZpYm8kHH2P0F8WpqyC9kpn//T8L+P22jHNitb9On8domcjdwP9i9HmwZBmyF99ZwHfOIq6qKxmF1P/X1iO/FXj54De7NwD7JvkH4LOM1sW+ffpB27qmc/juutTF8i5GZ6ZfOqhNX5M908l061vbqRM8p9Zk38joT14nVNWjC9z3OcvoMlhPGXc/ZuE4RlcXGHo3o3WWT0+7tBqjD/c/r6q/mn6ACTT1XrkF+Fvgw8D/GGyfviZ7ppm8STLX12gZoysTTII5v8+q6lbgX4Frquobi9XhuaqqzwL/AKxqEwZHAX+Q5HZGa4KvBx5zCbVxfEa3XzxfCvxcRpfwuwV4HaN1u/Pp9yS997Zo+HqNuy+zMD0ffIDRCZ03tH8bn8cMf1Woqluq6uLF6uQ8/Cpwd1VNLd85F3hmkp8eY58WlN/4KEnboSRnA3dU1UxLMqQF0f56dGNVTfIVO6SJ4Ey2JG1nklwN/Aij5QrSokjyHxnNrJ427r5I2wNnsiVJkqTOnMmWJEmSOjNkS5IkSZ0ZsiVJkqTODNmSJElSZ4ZsSZIkqTNDtiRJktTZ/wf2vM3LslX1QgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 5))\n",
    "plt.bar(tag_num.index, tag_num.values)\n",
    "plt.title(\"Tag_frequency\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gBbhnJsmwC5f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "the    69971\n",
       ",      58334\n",
       ".      49346\n",
       "of     36412\n",
       "and    28853\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_num[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1WmEOBMkwC5i"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtIAAAE/CAYAAABrWCRrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAev0lEQVR4nO3df5Td9V3n8eerSUuxld8DSxNoUHKqgLaVyFJbu9W4S5TV4Fk4pqsSNW7citqurhq0q1U3u3TdtcoqeLC4BPqD5lBroy0qJ5VSLYUGCwVKWaKlJIKQFkpDW6ih7/3jfmZ7M51kZj6ZmTtJno9zvud+7/t+P9/7+TAzmdd8+Hy/N1WFJEmSpJl5zqg7IEmSJB2MDNKSJElSB4O0JEmS1MEgLUmSJHUwSEuSJEkdDNKSJElSB4O0JC1QSd6U5G3TOO4Hk+xI8lSSl89H3yRJBmlJmpEklyZ5/4TaA/uorZmnbv1P4Geq6oVV9bF5ek9JOuwZpCVpZm4BXplkEUCSfwE8F/i2CbXT27HTkmTxAfTpxcC9c3BeSdJ+GKQlaWY+yiA4v6w9fzXw18D9E2p/D5BkS5LHk2xP8h/GT9KWbdyQ5G1JPg/8WJLTknwwye4kNwEn7K8jSY5I8hSwCLgryfh7Ppjkl5N8HPhCksVJzk3y4SSfS3JXktcMnWev903y++NLSpK8JsnOCe/7YJLvafvPSbIhyd8n+WySzUmOa68tS1JJ1iZ5KMlnkvzq0HkWJfmV1nZ3kjuSnJLkD5L8rwnv+WdJ3rD/L40kzS+DtCTNQFV9GbiNQVimPX4I+JsJtVuAdwI7gRcBFwL/LcnKodOtBm4AjgHeDrwDuINBgP4tYO0UfXmmql7Ynr60qr5x6OXXAue3c58EvA/4r8BxwH8G3p1krB07o/ed4OeAC4B/1cb5BPAHE455FfASYCXwa0m+udV/vvXz+4CjgJ8AvghsAl6b5DkASU5obd85g35J0pwzSEvSzH2Qr4bm72QQpD80ofZBBgHyl6vq6aq6E3gr8KND57m1qv60qr4CjAHfDvyXFpBvAf7sAPp4eVXtqKovAT8CvL+q3l9VX6mqm4BtwPclOfUA3/engF+tqp1V9QzwJuDCCUtKfqOqvlRVdwF3AS9t9Z8E3lhV99fAXVX12aq6HXiSQXgGWAPcXFWPdv2XkKQ5YpCWpJm7BXhVkmOBsap6APgw8B2tdhbwSeDxqto91O7TwJKh5zuG9l8EPFFVX5hwfK/hc78YuKgt6/hcks8xCPknz8L7vhh4z9B57wOeZTALPu6fhva/CIzPop9CWwIziU0M/gCgPV43gz5J0rwwSEvSzN0KHA2sB/4WoKo+Dzzcag+37bgkXz/U7lTgH4ee19D+I8CxSV4w4fhew+feAVxXVccMbS+oqsum8b5fAL5u/Em7oHJs6PUdwPdOOPfzq2p4nPuyA/jGfbz2NmB1kpcC3wz86TTOJ0nzyiAtSTPUlktsY7DG90NDL/1Nq91SVTsYzFL/9yTPT/KtwDoGa6EnO+en2zl/I8nzkrwK+P5Z6vLbgO9Pcl67wO/57SLCpdN43/8LPD/J+UmeC7wROGLo9T8ENiZ5MUCSsSSrp9mvtwK/lWR5Br41yfEAVbWTwYWd1wHvbv/NJWlBMUhLUp8PAicyCM/jPtRq47e9ey2wjMHs9HuAX2/rk/fl3wP/Engc+HXg2tnoaAv1q4FfAXYxmAn+Rb76O2Cf71tVTwI/zSD0/iODGerhu3j8HrAF+Ksku4GPtHNNx+8Am4G/Aj4PXA0cOfT6JuBbcFmHpAUqVTX1UZKkw0aSNwGnV9WPTHXsHPfj1Qxm05e1CzIlaUFxRlqStOC0ZSSvB95qiJa0UBmkJWmBS/LDSZ6aZJv00wwPdu0+059jcFeR3x1pZyRpP1zaIUmSJHVwRlqSJEnqYJCWJEmSOiye+pCF6YQTTqhly5aNuhuSJEk6xN1xxx2fqaqxifWDNkgvW7aMbdu2jbobkiRJOsQl+fRkdZd2SJIkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUYcogneQlSe4c2j6f5A1JjktyU5IH2uOxQ20uTbI9yf1Jzhuqn53k7vba5UnS6kckeVer35Zk2ZyMVpIkSZolUwbpqrq/ql5WVS8Dzga+CLwH2ABsrarlwNb2nCRnAGuAM4FVwBVJFrXTXQmsB5a3bVWrrwOeqKrTgbcAb56V0UmSJElzZKZLO1YCf19VnwZWA5tafRNwQdtfDVxfVc9U1aeA7cA5SU4GjqqqW6uqgGsntBk/1w3AyvHZakmSJGkhmmmQXgO8s+2fVFWPALTHE1t9CbBjqM3OVlvS9ifW92pTVXuAJ4HjZ9g3SZIkad4snu6BSZ4H/ABw6VSHTlKr/dT312ZiH9YzWBrCqaeeOkU35s6yDe8b2XvPlgcvO3/UXZAkSTqozWRG+nuBv6uqR9vzR9tyDdrjY62+EzhlqN1S4OFWXzpJfa82SRYDRwOPT+xAVV1VVSuqasXY2NgMui5JkiTNrpkE6dfy1WUdAFuAtW1/LfDeofqadieO0xhcVHh7W/6xO8m5bf3zxRPajJ/rQuADbR21JEmStCBNa2lHkq8D/jXwU0Ply4DNSdYBDwEXAVTVvUk2A58A9gCXVNWzrc3rgGuAI4Eb2wZwNXBdku0MZqLXHMCYJEmSpDk3rSBdVV9kwsV/VfVZBnfxmOz4jcDGSerbgLMmqT9NC+KSJEnSwcBPNpQkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSeowrSCd5JgkNyT5ZJL7krwiyXFJbkryQHs8duj4S5NsT3J/kvOG6mcnubu9dnmStPoRSd7V6rclWTbrI5UkSZJm0XRnpH8P+Iuq+ibgpcB9wAZga1UtB7a25yQ5A1gDnAmsAq5Isqid50pgPbC8batafR3wRFWdDrwFePMBjkuSJEmaU1MG6SRHAa8Grgaoqi9X1eeA1cCmdtgm4IK2vxq4vqqeqapPAduBc5KcDBxVVbdWVQHXTmgzfq4bgJXjs9WSJEnSQjSdGelvAHYB/yfJx5K8NckLgJOq6hGA9nhiO34JsGOo/c5WW9L2J9b3alNVe4AngeO7RiRJkiTNg+kE6cXAtwFXVtXLgS/QlnHsw2QzybWf+v7a7H3iZH2SbUm27dq1a/+9liRJkubQdIL0TmBnVd3Wnt/AIFg/2pZr0B4fGzr+lKH2S4GHW33pJPW92iRZDBwNPD6xI1V1VVWtqKoVY2Nj0+i6JEmSNDemDNJV9U/AjiQvaaWVwCeALcDaVlsLvLftbwHWtDtxnMbgosLb2/KP3UnObeufL57QZvxcFwIfaOuoJUmSpAVp8TSP+1ng7UmeB/wD8OMMQvjmJOuAh4CLAKrq3iSbGYTtPcAlVfVsO8/rgGuAI4Eb2waDCxmvS7KdwUz0mgMclyRJkjSnphWkq+pOYMUkL63cx/EbgY2T1LcBZ01Sf5oWxCVJkqSDgZ9sKEmSJHWY7tIOiWUb3jfqLhywBy87f9RdkCRJhwhnpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpg0FakiRJ6mCQliRJkjoYpCVJkqQOBmlJkiSpw7SCdJIHk9yd5M4k21rtuCQ3JXmgPR47dPylSbYnuT/JeUP1s9t5tie5PEla/Ygk72r125Ism+VxSpIkSbNqJjPS31VVL6uqFe35BmBrVS0HtrbnJDkDWAOcCawCrkiyqLW5ElgPLG/bqlZfBzxRVacDbwHe3D8kSZIkae4dyNKO1cCmtr8JuGCofn1VPVNVnwK2A+ckORk4qqpuraoCrp3QZvxcNwArx2erJUmSpIVoukG6gL9KckeS9a12UlU9AtAeT2z1JcCOobY7W21J259Y36tNVe0BngSOn9lQJEmSpPmzeJrHvbKqHk5yInBTkk/u59jJZpJrP/X9tdn7xIMQvx7g1FNP3X+PJUmSpDk0rRnpqnq4PT4GvAc4B3i0LdegPT7WDt8JnDLUfCnwcKsvnaS+V5ski4Gjgccn6cdVVbWiqlaMjY1Np+uSJEnSnJhyRjrJC4DnVNXutv9vgN8EtgBrgcva43tbky3AO5L8DvAiBhcV3l5VzybZneRc4DbgYuB/D7VZC9wKXAh8oK2jlkZu2Yb3jboLB+zBy84fdRckSTrkTGdpx0nAe9q1f4uBd1TVXyT5KLA5yTrgIeAigKq6N8lm4BPAHuCSqnq2net1wDXAkcCNbQO4GrguyXYGM9FrZmFskiRJ0pyZMkhX1T8AL52k/llg5T7abAQ2TlLfBpw1Sf1pWhCXJEmSDgZ+sqEkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVIHg7QkSZLUwSAtSZIkdTBIS5IkSR0M0pIkSVKHaQfpJIuSfCzJn7fnxyW5KckD7fHYoWMvTbI9yf1Jzhuqn53k7vba5UnS6kckeVer35Zk2SyOUZIkSZp1M5mRfj1w39DzDcDWqloObG3PSXIGsAY4E1gFXJFkUWtzJbAeWN62Va2+Dniiqk4H3gK8uWs0kiRJ0jyZVpBOshQ4H3jrUHk1sKntbwIuGKpfX1XPVNWngO3AOUlOBo6qqlurqoBrJ7QZP9cNwMrx2WpJkiRpIZrujPTvAr8EfGWodlJVPQLQHk9s9SXAjqHjdrbakrY/sb5Xm6raAzwJHD/dQUiSJEnzbcogneTfAo9V1R3TPOdkM8m1n/r+2kzsy/ok25Js27Vr1zS7I0mSJM2+6cxIvxL4gSQPAtcD353kbcCjbbkG7fGxdvxO4JSh9kuBh1t96ST1vdokWQwcDTw+sSNVdVVVraiqFWNjY9MaoCRJkjQXpgzSVXVpVS2tqmUMLiL8QFX9CLAFWNsOWwu8t+1vAda0O3GcxuCiwtvb8o/dSc5t658vntBm/FwXtvf4mhlpSZIkaaFYfABtLwM2J1kHPARcBFBV9ybZDHwC2ANcUlXPtjavA64BjgRubBvA1cB1SbYzmIlecwD9kiRJkubcjIJ0Vd0M3Nz2Pwus3MdxG4GNk9S3AWdNUn+aFsQlSZKkg4GfbChJkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktThQD7ZUNIhbNmG9426C7PiwcvOH3UXJEmHKGekJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqMGWQTvL8JLcnuSvJvUl+o9WPS3JTkgfa47FDbS5Nsj3J/UnOG6qfneTu9trlSdLqRyR5V6vflmTZHIxVkiRJmjXTmZF+Bvjuqnop8DJgVZJzgQ3A1qpaDmxtz0lyBrAGOBNYBVyRZFE715XAemB521a1+jrgiao6HXgL8OYDH5okSZI0d6YM0jXwVHv63LYVsBrY1OqbgAva/mrg+qp6pqo+BWwHzklyMnBUVd1aVQVcO6HN+LluAFaOz1ZLkiRJC9G01kgnWZTkTuAx4Kaqug04qaoeAWiPJ7bDlwA7hprvbLUlbX9ifa82VbUHeBI4vmM8kiRJ0ryYVpCuqmer6mXAUgazy2ft5/DJZpJrP/X9tdn7xMn6JNuSbNu1a9cUvZYkSZLmzozu2lFVnwNuZrC2+dG2XIP2+Fg7bCdwylCzpcDDrb50kvpebZIsBo4GHp/k/a+qqhVVtWJsbGwmXZckSZJm1XTu2jGW5Ji2fyTwPcAngS3A2nbYWuC9bX8LsKbdieM0BhcV3t6Wf+xOcm5b/3zxhDbj57oQ+EBbRy1JkiQtSIuncczJwKZ2543nAJur6s+T3ApsTrIOeAi4CKCq7k2yGfgEsAe4pKqebed6HXANcCRwY9sArgauS7KdwUz0mtkYnCRJkjRXpgzSVfVx4OWT1D8LrNxHm43Axknq24CvWV9dVU/TgrgkjdKyDe8bdRdmxYOXnT/qLkjSIc9PNpQkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6GKQlSZKkDgZpSZIkqYNBWpIkSepgkJYkSZI6TOcjwiVJhzg/0VGSZs4ZaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnq4EeES5IOW340uqQD4Yy0JEmS1MEgLUmSJHUwSEuSJEkdDNKSJElSB4O0JEmS1MEgLUmSJHXw9neSJB1mvO2fNDumnJFOckqSv05yX5J7k7y+1Y9LclOSB9rjsUNtLk2yPcn9Sc4bqp+d5O722uVJ0upHJHlXq9+WZNkcjFWSJEmaNdNZ2rEH+IWq+mbgXOCSJGcAG4CtVbUc2Nqe015bA5wJrAKuSLKonetKYD2wvG2rWn0d8ERVnQ68BXjzLIxNkiRJmjNTBumqeqSq/q7t7wbuA5YAq4FN7bBNwAVtfzVwfVU9U1WfArYD5yQ5GTiqqm6tqgKundBm/Fw3ACvHZ6slSZKkhWhGFxu2JRcvB24DTqqqR2AQtoET22FLgB1DzXa22pK2P7G+V5uq2gM8CRw/k75JkiRJ82naQTrJC4F3A2+oqs/v79BJarWf+v7aTOzD+iTbkmzbtWvXVF2WJEmS5sy0gnSS5zII0W+vqj9p5Ufbcg3a42OtvhM4Zaj5UuDhVl86SX2vNkkWA0cDj0/sR1VdVVUrqmrF2NjYdLouSZIkzYnp3LUjwNXAfVX1O0MvbQHWtv21wHuH6mvanThOY3BR4e1t+cfuJOe2c148oc34uS4EPtDWUUuSJEkL0nTuI/1K4EeBu5Pc2Wq/AlwGbE6yDngIuAigqu5Nshn4BIM7flxSVc+2dq8DrgGOBG5sGwyC+nVJtjOYiV5zYMOSJEmS5taUQbqq/obJ1zADrNxHm43Axknq24CzJqk/TQvikiRJ0sHAjwiXJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqYJCWJEmSOhikJUmSpA4GaUmSJKmDQVqSJEnqMGWQTvLHSR5Lcs9Q7bgkNyV5oD0eO/TapUm2J7k/yXlD9bOT3N1euzxJWv2IJO9q9duSLJvlMUqSJEmzbjoz0tcAqybUNgBbq2o5sLU9J8kZwBrgzNbmiiSLWpsrgfXA8raNn3Md8ERVnQ68BXhz72AkSZKk+bJ4qgOq6pZJZolXA69p+5uAm4FfbvXrq+oZ4FNJtgPnJHkQOKqqbgVIci1wAXBja/Omdq4bgN9Pkqqq3kFJkiRNtGzD+0bdhVnx4GXnj7oLaqYM0vtwUlU9AlBVjyQ5sdWXAB8ZOm5nq/1z259YH2+zo51rT5IngeOBz3T2TZIkSUMOhT8iFuIfELN9sWEmqdV+6vtr87UnT9Yn2ZZk265duzq7KEmSJB243iD9aJKTAdrjY62+Ezhl6LilwMOtvnSS+l5tkiwGjgYen+xNq+qqqlpRVSvGxsY6uy5JkiQduN4gvQVY2/bXAu8dqq9pd+I4jcFFhbe3ZSC7k5zb7tZx8YQ24+e6EPiA66MlSZK00E25RjrJOxlcWHhCkp3ArwOXAZuTrAMeAi4CqKp7k2wGPgHsAS6pqmfbqV7H4A4gRzK4yPDGVr8auK5dmPg4g7t+SJIkSQvadO7a8dp9vLRyH8dvBDZOUt8GnDVJ/WlaEJckSZIOFn6yoSRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUocFE6STrEpyf5LtSTaMuj+SJEnS/iyIIJ1kEfAHwPcCZwCvTXLGaHslSZIk7duCCNLAOcD2qvqHqvoycD2wesR9kiRJkvZpoQTpJcCOoec7W02SJElakFJVo+4DSS4Czquqn2zPfxQ4p6p+dsJx64H17elLgPvntaPz5wTgM6PuxAgcruOGw3fsjvvw4rgPL4778HKoj/vFVTU2sbh4FD2ZxE7glKHnS4GHJx5UVVcBV81Xp0YlybaqWjHqfsy3w3XccPiO3XEfXhz34cVxH14O13EvlKUdHwWWJzktyfOANcCWEfdJkiRJ2qcFMSNdVXuS/Azwl8Ai4I+r6t4Rd0uSJEnapwURpAGq6v3A+0fdjwXikF++sg+H67jh8B274z68OO7Di+M+vByW414QFxtKkiRJB5uFskZakiRJOqgYpEcgyTFJfrrtvybJn4+6T9KoJPm5JPclefuo+7IQJHlq1H2YDcP/zunwkeTDo+7DXDnQ391JfizJi+amd6NxKH+9p8sgPRrHAP6CkQZ+Gvi+qvrhUXdEs+oY/HfusFNV3zHqPsyhYziw7+kfAw6pIH2If72nxSA9GpcB35jkTuC3gRcmuSHJJ5O8PUkAkpyd5INJ7kjyl0lOHmWnpQOV5OeT3NO2NyT5Q+AbgC1J/tOo+zdbkvxp+7m9t32QFEmeSrIxyV1JPpLkpFY/LcmtST6a5LdG2/NZ9f//nUvy2227J8ndSX5o1J2bL5N9LxzKxv+PSpuxvXmy320Hsen+7v619vN8T5KrMnAhsAJ4e/uZOHJ0w5g9Q1/vk5Pc0sZ2T5LvHHXf5k1Vuc3zBiwD7mn7rwGeZPAhNM8BbgVeBTwX+DAw1o77IQa3BRx5/93cejbgbOBu4AXAC4F7gZcDDwInjLp/szzW49rjkcA9wPFAAd/f6v8DeGPb3wJc3PYvAZ4adf9n6b/B8L9z/w64icHtTU8CHgJOHnUfR/W9MOo+zfF4n2qPk/5uG3X/DnBsU/7uHv6at/3rhn7ubwZWjHocc/T1/gXgV9v+IuDrR923+dqckV4Ybq+qnVX1FeBOBj+sLwHOAm5qf/2+kcEPrHSwehXwnqr6QlU9BfwJcKjOWvxckruAjzD41NblwJeB8TWVdzD4OQd4JfDOtn/dPPZxPr0KeGdVPVtVjwIfBL59xH2aL5N9LxwuJvvddijZ1/i+K8ltSe4Gvhs4c0T9m08fBX48yZuAb6mq3SPuz7xZMPeRPsw9M7T/LIOvS4B7q+oVo+mSNOsO9v+tOy1JXgN8D/CKqvpikpuB5wP/XG26hq/+nI871O9Delh87Sfaz/fC4WKy322Hkq8ZX5LnA1cwmHne0YLlIf81r6pbkrwaOB+4LslvV9W1o+7XfHBGejR2A18/xTH3A2NJXgGQ5LlJDvm/apNsTbJk1P3QnLgFuCDJ1yV5AfCDwIdG3Ke5cDTwRAtO3wScO8XxfwusafuH0gWXw//O3QL8UJJFScaAVwO3j6xn82em3wta2Kbzu3s8NH8myQuBC2fY/qCU5MXAY1X1R8DVwLeNuEvz5lD76/CgUFWfTfK3Se4BvgQ8OskxX24XJ1ye5GgGX6vfZbCu9JCU5DnA6cDjo+7LfEryfuAnq+rhUfdlLlXV3yW5hq8GqLdW1ccO/uuPvsZfAP8xyccZ/EH8kSmOfz3wjiSvB949152bLxP+nbsR+DhwF4PZ91+qqn8aaQfnx0y/F7SATfN39+eS/BGD60EeZLDkYdw1wB8m+RKD/0vxpbnv9bx5DfCLSf4ZeAq4eLTdmT9+sqEWjCRnAT9RVT8/6r5IkiRNxSAtSZIkdXCNtCRJktTBIC1JkiR1MEhLkiRJHQzSkiRJUgeDtCRJktTBIC1JkiR1MEhLkiRJHf4fiwDtvUwPiLcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12, 5))\n",
    "plt.bar(word_num.index[:10], word_num.values[:10])\n",
    "plt.title(\"Word_frequency\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n08z2PjMwC5o"
   },
   "source": [
    "### Вопрос 1:\n",
    "* Кол-во слова `cat` в корпусе? **(0.5 балл)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jhB7di3YwC5p"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "23"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_num['cat']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UsCfVLsewC5s"
   },
   "source": [
    "### Вопрос 2:\n",
    "* Самое популярное слово с самым популярным тегом? **(0.5 балл)**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oio-XBYkwC5t"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'time'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Выбираем сначала слова с самым популярным тегом, а затем среди них выбираем самое популярное слово.\n",
    "tg = tag_num.index[tag_num.argmax()]\n",
    "w = [word for (word, tag) in brown_tagged_words if tag == tg]\n",
    "c = Counter(w)\n",
    "max(w, key = lambda x: c[x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_f1rl5x0wC55"
   },
   "source": [
    "Cделайте разбиение выборки на обучение и контроль в отношении 9:1. **(0.5 балл)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GX9t-1qowC58"
   },
   "outputs": [],
   "source": [
    "brown_tagged_sents = brown.tagged_sents(tagset=\"universal\")\n",
    "# Приведем слова к нижнему регистру\n",
    "my_brown_tagged_sents = []\n",
    "for sent in brown_tagged_sents:\n",
    "    my_brown_tagged_sents.append(list(map(lambda x: (x[0].lower(), x[1]), sent)))\n",
    "my_brown_tagged_sents = np.array(my_brown_tagged_sents, dtype=object)\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "train_sents, test_sents = train_test_split(my_brown_tagged_sents, random_state=0, train_size=0.9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pXkVwUjYwC5-"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(51606, 5734)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_sents), len(test_sents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FpAgfZRTwC6h"
   },
   "source": [
    "## DefaultTagger"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9b4cPKyiwC6j"
   },
   "source": [
    "### Вопрос 3:\n",
    "* Какое качество вы бы получили, если бы предсказывали любой тег, как самый популярный тег на выборке train(округлите до одного знака после запятой)? **(0.5 балл)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Td-0Pe0vwC6k"
   },
   "source": [
    "Вы можете использовать DefaultTagger(метод tag для предсказания частей речи предложения)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NfZYlMxJwC6m"
   },
   "outputs": [],
   "source": [
    "from nltk.tag import DefaultTagger\n",
    "default_tagger = DefaultTagger(tag_num.index[tag_num.argmax()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9CXKibo_cMcB"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 23.47521651004238 %\n"
     ]
    }
   ],
   "source": [
    "true_pred = 0\n",
    "num_pred = 0\n",
    "\n",
    "for sent in test_sents:\n",
    "    tags = np.array([tag for (word, tag) in sent])\n",
    "    words = np.array([word for (word, tag) in sent])\n",
    "    \n",
    "    tagged_sent = default_tagger.tag(words)\n",
    "    outputs = [tag for token, tag in tagged_sent]\n",
    "    \n",
    "    true_pred += len(tags[tags==outputs])\n",
    "    num_pred += len(words)\n",
    "    \n",
    "print(\"Accuracy:\", true_pred / num_pred * 100, '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5w1W5hSkcMcV"
   },
   "source": [
    "## LSTMTagger"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mm1-S3t2cMcW"
   },
   "source": [
    "### Подготовка данных"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GayTl7mUcMcX"
   },
   "source": [
    "Изменим структуру данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CnXcI64fxoj4",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('The', 'Fulton', 'County', 'Grand', 'Jury', 'said', 'Friday', 'an', 'investigation', 'of', \"Atlanta's\", 'recent', 'primary', 'election', 'produced', '``', 'no', 'evidence', \"''\", 'that', 'any', 'irregularities', 'took', 'place', '.'), ('DET', 'NOUN', 'NOUN', 'ADJ', 'NOUN', 'VERB', 'NOUN', 'DET', 'NOUN', 'ADP', 'NOUN', 'ADJ', 'NOUN', 'NOUN', 'VERB', '.', 'DET', 'NOUN', '.', 'ADP', 'DET', 'NOUN', 'VERB', 'NOUN', '.')]\n"
     ]
    }
   ],
   "source": [
    "pos_data = [list(zip(*sent)) for sent in brown_tagged_sents]\n",
    "print(pos_data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DpRE3c-3cMcc"
   },
   "source": [
    "Пора эксплуатировать pytorch!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gvFlzrYnxokE"
   },
   "outputs": [],
   "source": [
    "from torchtext.legacy.data import Field, BucketIterator\n",
    "import torchtext\n",
    "\n",
    "# наши поля\n",
    "WORD = Field(lower=True)\n",
    "TAG = Field(unk_token=None) # все токены нам извсетны\n",
    "\n",
    "# создаем примеры\n",
    "examples = []\n",
    "for words, tags in pos_data:\n",
    "    examples.append(torchtext.legacy.data.Example.fromlist([list(words), list(tags)], fields=[('words', WORD), ('tags', TAG)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tjl6u6cpOc1u"
   },
   "source": [
    "Вот один наш пример:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dnrzktytN9rL"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'words': ['the', 'fulton', 'county', 'grand', 'jury', 'said', 'friday', 'an', 'investigation', 'of', \"atlanta's\", 'recent', 'primary', 'election', 'produced', '``', 'no', 'evidence', \"''\", 'that', 'any', 'irregularities', 'took', 'place', '.'], 'tags': ['DET', 'NOUN', 'NOUN', 'ADJ', 'NOUN', 'VERB', 'NOUN', 'DET', 'NOUN', 'ADP', 'NOUN', 'ADJ', 'NOUN', 'NOUN', 'VERB', '.', 'DET', 'NOUN', '.', 'ADP', 'DET', 'NOUN', 'VERB', 'NOUN', '.']}\n"
     ]
    }
   ],
   "source": [
    "print(vars(examples[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nUhTrWCWcMcj"
   },
   "source": [
    "Теперь формируем наш датасет"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LGKkbZUIxokO",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples: 45872\n",
      "Number of validation examples: 5734\n",
      "Number of testing examples: 5734\n"
     ]
    }
   ],
   "source": [
    "# кладем примеры в наш датасет\n",
    "dataset = torchtext.legacy.data.Dataset(examples, fields=[('words', WORD), ('tags', TAG)])\n",
    "\n",
    "train_data, valid_data, test_data = dataset.split(split_ratio=[0.8, 0.1, 0.1])\n",
    "\n",
    "print(f\"Number of training examples: {len(train_data.examples)}\")\n",
    "print(f\"Number of validation examples: {len(valid_data.examples)}\")\n",
    "print(f\"Number of testing examples: {len(test_data.examples)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "T89unpppcMcp"
   },
   "source": [
    "Построим словари. Параметр `min_freq` выберете сами. При построении словаря испольузем только **train** **(0.5 балл)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tZwkwhlrxoka",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique tokens in source (ru) vocabulary: 12305\n",
      "Unique tokens in target (en) vocabulary: 13\n",
      "['<unk>', '2', 'individual', 'received', 'spring', 'exactly', 'attempt', 'turning', 'nobody', 'declared', 'parties', '22', 'chamber', 'express', 'serves', 'returning', 'senior', 'shorts', 'preceding', 'curiosity', 'recommend', 'ultimately', 'behalf', 'breathing', 'anniversary', 'tactics', 'partisan', 'eliminating', 'wanting', 'objections', 'bottles', 'oils', 'candy', 'litigation', 'adolescent', 'interpretations', 'suburb', 'counts', 'mercenaries', 'tightly', 'coincidence', 'harness', 'quantitative', 'workable', 'comparing', 'gravel', 'outgoing', 'staircase', 'airborne', 'coupling', \"georgia's\", 'misplaced', 'recipient', 'thankful', 'allusions', 'cease-fire', 'drives', 'hanged', 'martha', 'posterity', 'shutters', 'ultracentrifugation']\n",
      "['<pad>', 'NOUN', 'VERB', '.', 'ADP', 'DET', 'ADJ', 'ADV', 'PRON', 'CONJ', 'PRT', 'NUM', 'X']\n"
     ]
    }
   ],
   "source": [
    "WORD.build_vocab(train_data, min_freq=5)\n",
    "TAG.build_vocab(train_data)\n",
    "\n",
    "print(f\"Unique tokens in source (ru) vocabulary: {len(WORD.vocab)}\")\n",
    "print(f\"Unique tokens in target (en) vocabulary: {len(TAG.vocab)}\")\n",
    "\n",
    "print(WORD.vocab.itos[::200])\n",
    "print(TAG.vocab.itos)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ESwY5AZ9QKiv"
   },
   "source": [
    "Здесь вы увидете токен `unk` и `pad`. Первый служит для обозначения слов, которых у нас нет в словаре. Второй служит для того, что объекты в одном батче были одинакового размера."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vjn07NP-xokl",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'words': ['if', 'death', 'receives', 'more', 'than', 'its', 'share', 'of', 'attention', 'from', 'the', 'theologian', 'and', 'if', 'sin', 'receives', 'less', 'than', 'its', 'share', ',', 'the', 'gift', 'of', 'the', 'life', 'eternal', 'through', 'christ', 'begins', 'to', 'look', 'like', 'the', 'divinely', 'appointed', 'means', 'of', 'rescue', 'from', 'temporal', ',', 'i.e.', ',', 'created', ',', 'existence', '.'], 'tags': ['ADP', 'NOUN', 'VERB', 'ADJ', 'ADP', 'DET', 'NOUN', 'ADP', 'NOUN', 'ADP', 'DET', 'NOUN', 'CONJ', 'ADP', 'NOUN', 'VERB', 'ADJ', 'ADP', 'DET', 'NOUN', '.', 'DET', 'NOUN', 'ADP', 'DET', 'NOUN', 'ADJ', 'ADP', 'NOUN', 'VERB', 'PRT', 'VERB', 'ADP', 'DET', 'ADV', 'VERB', 'NOUN', 'ADP', 'NOUN', 'ADP', 'ADJ', '.', 'ADV', '.', 'VERB', '.', 'NOUN', '.']}\n"
     ]
    }
   ],
   "source": [
    "print(vars(train_data.examples[9]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LxgkU4cZcMcz"
   },
   "source": [
    "Посмотрим с насколько большими предложениями мы имеем дело"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dVpMi1_0xoku",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAEICAYAAABLWh2RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAcKElEQVR4nO3df7Rd5V3n8fdHYim0pfwKCEna0DbtCCy1koU4VcdZVElpbZgRnDC1RMtMlKGjVbvaUEZB16RSf1VZFhRLJVQEMthKphUHpCrjGgoGCkKgSFoopEkhLbSE/sCGfueP89x6cnNu7s29B+7d575fa5119vnuH+d5zj73fs5+zr77pqqQJEnd9R2z3QBJkjQzhrkkSR1nmEuS1HGGuSRJHWeYS5LUcYa5JEkdZ5hL05RkaZJKsmCa6/9Mkn/oe/x0klcMqW3vSfLBYbRzwLZf1tq63zC2N27bQ3sN5kIbkjyc5PXD2Ja0N4a5Omk2fkk+189ZVS+uqs9O0oYfTbJ1Ctt6b1X9l2G0a3y/q+qR1tZnh7H9flN5DQa074dbAD+d5Kvtg8vTfbeXPddtGIbW7lc938+r0TCUT+qS5o4kC6pq12y34/lSVf8XeDH0RiGAh4CDB70G8+210fzhkblGSpLvSLI2yWeSfCnJhiSHtnljw82rkzyS5ItJzu9b94Ak65M8meT+JO8aOwpO8mHgZcD/bkd77+p72rcM2t6Ath2WZGOSp5LcDrxy3PxvH5klOTXJfUl2Jvl8kncmeRFwA3B031Hn0UkuTHJdkj9L8hTwM632Z+Oa8LYk25JsT/Irfc97RZL/2ff420f/g/o9fti+tWFjkieSbEnyX/u2dWHbB1e2vmxOsnwvr1H/a3BFkg8k+Xhb97Ykr5xo3Qm2N+i1OTHJrUm+3F6LP0zygmG0Iclbk3yuvffOHzdvwudNcktb7O72Ov+nJIck+ViSHe09+bEki/el/5o/DHONml8ATgP+HXA08CTwgXHL/BDwGuBk4NeSfHerXwAsBV4B/Bjw02MrVNVbgUeAn2jDsL81he2N9wHgG8BRwNvabSKXAz9XVS8Bjgc+UVVfBd4AbGtteHFVbWvLrwSuAw4Grppgm/8eWAb8OLA2U/jKYJJ+j7ka2Erv9T4deG+Sk/vmvxm4prVtI/CHkz1vnzOBXwcOAbYA6/Zh3THjX5tngV8CDgd+kN5++28zbUOSY4FLgbfSey0OA/rDd8Lnraofact8b3udr6X3+/lPgZfT+0D1dfbttdM8Yphr1PwccH5Vba2qZ4ALgdOz+8lfv15VX6+qu4G7ge9t9Z8C3ltVT1bVVuDiKT7nRNv7tvROFvtJ4Neq6qtVdS+wfi/b/CZwbJKDWnvunKQNt1bVX1bVt6rq63tp51er6h56IXHmpD2bRJIl9D7MvLuqvlFVdwEfpBdoY/6hqv6qfcf+YQa8Pnvxkaq6vQ2NXwV83zSaudtrU1V3VNUnq2pXVT0M/DG9D38zbcPpwMeq6pb23vtV4FtjM/f1eavqS1X1F1X1taraSe9DxN7aqXnMMNeoeTnw0TaU+WXgfnpHREf2LfOFvumv0b5vpXc09WjfvP7pvZloe/0W0jtHpX+bn9vLNn8SOBX4XJK/T/KDk7RhKm0d/9xHT2GdyRwNPNHCpn/bi/oej399Xpipn1k/ldd2Mru9Nkle3Yasv9CG3t9L72h5pm3Y7f3TRlK+NN3nTXJgkj9uw/ZPAbcAB+c5+CsCdZ9hrlHzKPCGqjq47/bCqvr8FNbdzu7DokvGzZ/JvxjcAewat80Jz7Kuqn+sqpXAEcBfAhsmacNU2jb+uceG6L8KHNg377v2YdvbgEOTvGTctqfyej9fxrf/UuDTwLKqOgh4D5AhPM92+l7jJAfSG2qf7vP+Cr2vb36gLT82FD+MtmrEGObqsu9M8sK+2wLgj4B1SV4OkGRhkpVT3N4G4Lx24tEi4O3j5j9G7/v0fdaGmD8CXNiOuI4FVg9aNskLkrwlyUur6pvAU/RGF8bacFiSl06jGb/anvs44GeBa1v9LuDUJIcm+S7gHePWm7DfVfUo8P+A32z74HuAs5n4e/u54CX0XtOnk/wb4Jwhbfc64E1Jfqid2PYb7P47drLnHf86v4Te9+RfTu8kzguG1E6NIMNcXfZX9H7Zjd0uBP6A3klWNybZCXwS+IEpbu836J3I9RDwN/R+OT/TN/83gf/RhvDfOY32vp3eEO0XgCvofW89kbcCD7fh1Z+nnYxXVZ+md8LZZ1s79mWo/O/pncB1M/A7VXVjq3+Y3nf9DwM38q8hP2ayfp9J78TBbcBHgQuq6qZ9aNfz7Z3AfwZ2An/Cnv2dlqraDJwL/Dm9o/Qn6b2fpvq8FwLr2+v8U8DvAwcAX6T3Pv7rYbRToylVMxk5lEZXknOAVVXlSUeS5jSPzKUmyVFJXpfe36q/ht53lh+d7XZJ0mS8Apz0r15A78+FjgG+TO9voy+ZzQZJ0lQ4zC5JUsc5zC5JUsd1dpj98MMPr6VLl852MyRJel7ccccdX6yqhYPmdTbMly5dyqZNm2a7GZIkPS+STHjVSIfZJUnqOMNckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeq4zl4BrguWrv34ULf38EVvHOr2JEmjwSNzSZI6zjCXJKnjJg3zJB9K8niSewfMe2eSSnJ4X+28JFuSPJDklL76CUnuafMuTpJW3z/Jta1+W5KlQ+qbJEnzwlSOzK8AVowvJlkC/BjwSF/tWGAVcFxb55Ik+7XZlwJrgGXtNrbNs4Enq+pVwPuB902nI5IkzVeThnlV3QI8MWDW+4F3AdVXWwlcU1XPVNVDwBbgxCRHAQdV1a1VVcCVwGl966xv09cBJ48dtUuSpMlN6zvzJG8GPl9Vd4+btQh4tO/x1lZb1KbH13dbp6p2AV8BDpvgedck2ZRk044dO6bTdEmSRs4+h3mSA4HzgV8bNHtArfZS39s6exarLquq5VW1fOHChVNpriRJI286R+avBI4B7k7yMLAYuDPJd9E74l7St+xiYFurLx5Qp3+dJAuAlzJ4WF+SJA2wz2FeVfdU1RFVtbSqltIL4++vqi8AG4FV7Qz1Y+id6HZ7VW0HdiY5qX0ffhZwfdvkRmB1mz4d+ET7Xl2SJE3BVP407WrgVuA1SbYmOXuiZatqM7ABuA/4a+Dcqnq2zT4H+CC9k+I+A9zQ6pcDhyXZAvwysHaafZEkaV6a9HKuVXXmJPOXjnu8Dlg3YLlNwPED6t8AzpisHZIkaTCvACdJUscZ5pIkdZxhLklSxxnmkiR1nGEuSVLHGeaSJHWcYS5JUscZ5pIkdZxhLklSxxnmkiR1nGEuSVLHGeaSJHWcYS5JUscZ5pIkdZxhLklSxxnmkiR1nGEuSVLHGeaSJHWcYS5JUscZ5pIkddykYZ7kQ0keT3JvX+23k3w6yT8l+WiSg/vmnZdkS5IHkpzSVz8hyT1t3sVJ0ur7J7m21W9LsnS4XZQkabRN5cj8CmDFuNpNwPFV9T3APwPnASQ5FlgFHNfWuSTJfm2dS4E1wLJ2G9vm2cCTVfUq4P3A+6bbGUmS5qNJw7yqbgGeGFe7sap2tYefBBa36ZXANVX1TFU9BGwBTkxyFHBQVd1aVQVcCZzWt876Nn0dcPLYUbskSZrcML4zfxtwQ5teBDzaN29rqy1q0+Pru63TPiB8BThs0BMlWZNkU5JNO3bsGELTJUnqvhmFeZLzgV3AVWOlAYvVXup7W2fPYtVlVbW8qpYvXLhwX5srSdJImnaYJ1kNvAl4Sxs6h94R95K+xRYD21p98YD6buskWQC8lHHD+pIkaWLTCvMkK4B3A2+uqq/1zdoIrGpnqB9D70S326tqO7AzyUnt+/CzgOv71lndpk8HPtH34UCSJE1iwWQLJLka+FHg8CRbgQvonb2+P3BTO1ftk1X181W1OckG4D56w+/nVtWzbVPn0Dsz/gB637GPfc9+OfDhJFvoHZGvGk7XJEmaHyYN86o6c0D58r0svw5YN6C+CTh+QP0bwBmTtUOSJA3mFeAkSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOm/QfrWjuWLr240Pd3sMXvXGo25MkzQ6PzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeo4w1ySpI6bNMyTfCjJ40nu7asdmuSmJA+2+0P65p2XZEuSB5Kc0lc/Ick9bd7FSdLq+ye5ttVvS7J0yH2UJGmkTeXI/ApgxbjaWuDmqloG3Nwek+RYYBVwXFvnkiT7tXUuBdYAy9ptbJtnA09W1auA9wPvm25nJEmajyYN86q6BXhiXHklsL5NrwdO66tfU1XPVNVDwBbgxCRHAQdV1a1VVcCV49YZ29Z1wMljR+2SJGly0/3O/Miq2g7Q7o9o9UXAo33LbW21RW16fH23dapqF/AV4LBBT5pkTZJNSTbt2LFjmk2XJGm0DPsEuEFH1LWX+t7W2bNYdVlVLa+q5QsXLpxmEyVJGi3TDfPH2tA57f7xVt8KLOlbbjGwrdUXD6jvtk6SBcBL2XNYX5IkTWC6Yb4RWN2mVwPX99VXtTPUj6F3otvtbSh+Z5KT2vfhZ41bZ2xbpwOfaN+rS5KkKZj0v6YluRr4UeDwJFuBC4CLgA1JzgYeAc4AqKrNSTYA9wG7gHOr6tm2qXPonRl/AHBDuwFcDnw4yRZ6R+SrhtIzSZLmiUnDvKrOnGDWyRMsvw5YN6C+CTh+QP0btA8Ds2nY/15UkqTni1eAkySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6bkZhnuSXkmxOcm+Sq5O8MMmhSW5K8mC7P6Rv+fOSbEnyQJJT+uonJLmnzbs4SWbSLkmS5pNph3mSRcAvAMur6nhgP2AVsBa4uaqWATe3xyQ5ts0/DlgBXJJkv7a5S4E1wLJ2WzHddkmSNN/MdJh9AXBAkgXAgcA2YCWwvs1fD5zWplcC11TVM1X1ELAFODHJUcBBVXVrVRVwZd86kiRpEtMO86r6PPA7wCPAduArVXUjcGRVbW/LbAeOaKssAh7t28TWVlvUpsfX95BkTZJNSTbt2LFjuk2XJGmkzGSY/RB6R9vHAEcDL0ry03tbZUCt9lLfs1h1WVUtr6rlCxcu3NcmS5I0kmYyzP564KGq2lFV3wQ+Avxb4LE2dE67f7wtvxVY0rf+YnrD8lvb9Pi6JEmagpmE+SPASUkObGefnwzcD2wEVrdlVgPXt+mNwKok+yc5ht6Jbre3ofidSU5q2zmrbx1JkjSJBdNdsapuS3IdcCewC/gUcBnwYmBDkrPpBf4ZbfnNSTYA97Xlz62qZ9vmzgGuAA4Abmg3SZI0BdMOc4CqugC4YFz5GXpH6YOWXwesG1DfBBw/k7ZIkjRfeQU4SZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjZhTmSQ5Ocl2STye5P8kPJjk0yU1JHmz3h/Qtf16SLUkeSHJKX/2EJPe0eRcnyUzaJUnSfLJghuv/AfDXVXV6khcABwLvAW6uqouSrAXWAu9OciywCjgOOBr4mySvrqpngUuBNcAngb8CVgA3zLBtmsTStR8f+jYfvuiNQ9+mJGnvpn1knuQg4EeAywGq6l+q6svASmB9W2w9cFqbXglcU1XPVNVDwBbgxCRHAQdV1a1VVcCVfetIkqRJzGSY/RXADuBPk3wqyQeTvAg4sqq2A7T7I9ryi4BH+9bf2mqL2vT4+h6SrEmyKcmmHTt2zKDpkiSNjpmE+QLg+4FLq+q1wFfpDalPZND34LWX+p7FqsuqanlVLV+4cOG+tleSpJE0kzDfCmytqtva4+vohftjbeicdv943/JL+tZfDGxr9cUD6pIkaQqmHeZV9QXg0SSvaaWTgfuAjcDqVlsNXN+mNwKrkuyf5BhgGXB7G4rfmeSkdhb7WX3rSJKkScz0bPb/DlzVzmT/LPCz9D4gbEhyNvAIcAZAVW1OsoFe4O8Czm1nsgOcA1wBHEDvLHbPZJckaYpmFOZVdRewfMCskydYfh2wbkB9E3D8TNoiSdJ85RXgJEnqOMNckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnquBmHeZL9knwqycfa40OT3JTkwXZ/SN+y5yXZkuSBJKf01U9Ick+bd3GSzLRdkiTNF8M4Mv9F4P6+x2uBm6tqGXBze0ySY4FVwHHACuCSJPu1dS4F1gDL2m3FENolSdK8MKMwT7IYeCPwwb7ySmB9m14PnNZXv6aqnqmqh4AtwIlJjgIOqqpbq6qAK/vWkSRJk5jpkfnvA+8CvtVXO7KqtgO0+yNafRHwaN9yW1ttUZseX99DkjVJNiXZtGPHjhk2XZKk0TDtME/yJuDxqrpjqqsMqNVe6nsWqy6rquVVtXzhwoVTfFpJkkbbghms+zrgzUlOBV4IHJTkz4DHkhxVVdvbEPrjbfmtwJK+9RcD21p98YC6JEmagmkfmVfVeVW1uKqW0jux7RNV9dPARmB1W2w1cH2b3gisSrJ/kmPoneh2exuK35nkpHYW+1l960iSpEnM5Mh8IhcBG5KcDTwCnAFQVZuTbADuA3YB51bVs22dc4ArgAOAG9pNkiRNwVDCvKr+Dvi7Nv0l4OQJllsHrBtQ3wQcP4y2SJI033gFOEmSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI6zjCXJKnjDHNJkjruufhHK5rHlq79+FC39/BFbxzq9iRpFHlkLklSxxnmkiR1nGEuSVLHGeaSJHWcYS5JUscZ5pIkdZxhLklSxxnmkiR13LTDPMmSJH+b5P4km5P8YqsfmuSmJA+2+0P61jkvyZYkDyQ5pa9+QpJ72ryLk2Rm3ZIkaf6YyZH5LuBXquq7gZOAc5McC6wFbq6qZcDN7TFt3irgOGAFcEmS/dq2LgXWAMvabcUM2iVJ0rwy7TCvqu1VdWeb3gncDywCVgLr22LrgdPa9Ergmqp6pqoeArYAJyY5Cjioqm6tqgKu7FtHkiRNYijfmSdZCrwWuA04sqq2Qy/wgSPaYouAR/tW29pqi9r0+Pqg51mTZFOSTTt27BhG0yVJ6rwZh3mSFwN/Abyjqp7a26IDarWX+p7FqsuqanlVLV+4cOG+N1aSpBE0ozBP8p30gvyqqvpIKz/Whs5p94+3+lZgSd/qi4Ftrb54QF2SJE3BTM5mD3A5cH9V/V7frI3A6ja9Gri+r74qyf5JjqF3otvtbSh+Z5KT2jbP6ltHkiRNYib/z/x1wFuBe5Lc1WrvAS4CNiQ5G3gEOAOgqjYn2QDcR+9M+HOr6tm23jnAFcABwA3tJkmSpmDaYV5V/8Dg77sBTp5gnXXAugH1TcDx022LJEnzmVeAkySp4wxzSZI6zjCXJKnjDHNJkjrOMJckqeMMc0mSOm4mf2cuPeeWrv34ULf38EVvHOr2JGku8MhckqSOM8wlSeo4w1ySpI4zzCVJ6jjDXJKkjjPMJUnqOMNckqSOM8wlSeo4LxqjecWL0EgaRR6ZS5LUcYa5JEkdZ5hLktRxhrkkSR03Z06AS7IC+ANgP+CDVXXRLDdJmtSwT6gDT6qTtO/mRJgn2Q/4APBjwFbgH5NsrKr7Zrdl0vPPM+4l7as5EebAicCWqvosQJJrgJWAYS7NkB8OpNE3V8J8EfBo3+OtwA+MXyjJGmBNe/h0kgeG8NyHA18cwnbmslHv46j3D+ZQH/O+52zTc6aPz5FR7x/Yx+fayyeaMVfCPANqtUeh6jLgsqE+cbKpqpYPc5tzzaj3cdT7B/ZxFIx6/8A+zqa5cjb7VmBJ3+PFwLZZaoskSZ0yV8L8H4FlSY5J8gJgFbBxltskSVInzIlh9qraleTtwP+h96dpH6qqzc/T0w912H6OGvU+jnr/wD6OglHvH9jHWZOqPb6aliRJHTJXhtklSdI0GeaSJHXcvA7zJCuSPJBkS5K1s92emUqyJMnfJrk/yeYkv9jqFyb5fJK72u3U2W7rTCR5OMk9rS+bWu3QJDclebDdHzLb7ZyuJK/p21d3JXkqyTu6vB+TfCjJ40nu7atNuM+SnNd+Lh9IcsrstHrfTNDH307y6ST/lOSjSQ5u9aVJvt63L/9o1hq+Dybo44Tvy67txwn6d21f3x5Oclerz6l9OG+/M2+XkP1n+i4hC5zZ5UvIJjkKOKqq7kzyEuAO4DTgp4Cnq+p3ZrN9w5LkYWB5VX2xr/ZbwBNVdVH7YHZIVb17tto4LO19+nl6F1H6WTq6H5P8CPA0cGVVHd9qA/dZkmOBq+ldGfJo4G+AV1fVs7PU/CmZoI8/DnyineT7PoDWx6XAx8aW64oJ+nghA96XXdyPg/o3bv7vAl+pqt+Ya/twPh+Zf/sSslX1L8DYJWQ7q6q2V9WdbXoncD+9q+vNByuB9W16Pb0PMaPgZOAzVfW52W7ITFTVLcAT48oT7bOVwDVV9UxVPQRsoffzOqcN6mNV3VhVu9rDT9K7hkZnTbAfJ9K5/bi3/iUJvQOjq5/XRk3RfA7zQZeQHZnga58aXwvc1kpvb0N9H+ryEHRTwI1J7miX+AU4sqq2Q+9DDXDErLVuuFax+y+PUdqPE+2zUf3ZfBtwQ9/jY5J8KsnfJ/nh2WrUkAx6X47afvxh4LGqerCvNmf24XwO8yldQraLkrwY+AvgHVX1FHAp8Erg+4DtwO/OXuuG4nVV9f3AG4Bz29DYyEnvAkpvBv5XK43afpzIyP1sJjkf2AVc1UrbgZdV1WuBXwb+PMlBs9W+GZrofTlq+/FMdv9gPaf24XwO85G8hGyS76QX5FdV1UcAquqxqnq2qr4F/AlzfKhrMlW1rd0/DnyUXn8ea+cMjJ078PjstXBo3gDcWVWPwejtRybeZyP1s5lkNfAm4C3VTlJqQ89fatN3AJ8BXj17rZy+vbwvR2Y/JlkA/Efg2rHaXNuH8znMR+4Ssu07ncuB+6vq9/rqR/Ut9h+Ae8ev2xVJXtRO7iPJi4Afp9efjcDqtthq4PrZaeFQ7XYkMEr7sZlon20EViXZP8kxwDLg9llo34wlWQG8G3hzVX2tr76wndxIklfQ6+NnZ6eVM7OX9+XI7Efg9cCnq2rrWGHO7cOqmrc34FR6Z7R/Bjh/ttszhP78EL1hrH8C7mq3U4EPA/e0+kZ6Z7zPenun2cdXAHe32+ax/QYcBtwMPNjuD53tts6wnwcCXwJe2lfr7H6k96FkO/BNekdsZ+9tnwHnt5/LB4A3zHb7Z9DHLfS+Nx77efyjtuxPtvfv3cCdwE/Mdvtn0McJ35dd24+D+tfqVwA/P27ZObUP5+2fpkmSNCrm8zC7JEkjwTCXJKnjDHNJkjrOMJckqeMMc0mSOs4wlySp4wxzSZI67v8DWgMT7J3YEJQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "length = map(len, [vars(x)['words'] for x in train_data.examples])\n",
    "\n",
    "plt.figure(figsize=[8, 4])\n",
    "plt.title(\"Length distribution in Train data\")\n",
    "plt.hist(list(length), bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yi28N2RBcMc5"
   },
   "source": [
    "Для обучения `LSTM` лучше использовать colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LAGSrqWsxok2",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2DSWm0UjcMc-"
   },
   "source": [
    "Для более быстрого и устойчивого обучения сгруппируем наши данные по батчам"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dmwAyhNgxok_"
   },
   "outputs": [],
   "source": [
    "# бьем нашу выборку на батч, не забывая сначала отсортировать выборку по длине\n",
    "def _len_sort_key(x):\n",
    "    return len(x.words)\n",
    "\n",
    "BATCH_SIZE = 32\n",
    "\n",
    "train_iterator, valid_iterator, test_iterator = BucketIterator.splits(\n",
    "    (train_data, valid_data, test_data), \n",
    "    batch_size = BATCH_SIZE, \n",
    "    device = device,\n",
    "    sort_key=_len_sort_key\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6aTjW00nxolI"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1434, 180, 180]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# посморим  на количество батчей\n",
    "list(map(len, [train_iterator, valid_iterator, test_iterator]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zyLQsizhcMdI"
   },
   "source": [
    "### Модель и её обучение"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-i9oHzcrcMdJ"
   },
   "source": [
    "Инициализируем нашу модель. Прочитайте про dropout [тут](https://habr.com/ru/company/wunderfund/blog/330814/). **(3 балла)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ff7BLWs_xolS",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LSTMTagger(\n",
       "  (embeddings): Embedding(12305, 10)\n",
       "  (dropout): Dropout(p=0.5, inplace=False)\n",
       "  (rnn): LSTM(10, 20, num_layers=2)\n",
       "  (tag): Linear(in_features=20, out_features=13, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class LSTMTagger(nn.Module):\n",
    "\n",
    "    def __init__(self, input_dim, emb_dim, hid_dim, output_dim, dropout, num_layers):\n",
    "        super().__init__()\n",
    "        \n",
    "  \n",
    "        self.embeddings = nn.Embedding(num_embeddings=input_dim, embedding_dim=emb_dim)\n",
    "        self.dropout = nn.Dropout(p=dropout)\n",
    "        \n",
    "        self.rnn = nn.LSTM(input_size=emb_dim, hidden_size=hid_dim, num_layers=num_layers)\n",
    "        self.tag = nn.Linear(hid_dim, output_dim)\n",
    "\n",
    "    def forward(self, sent):\n",
    "        \n",
    "        #sent = [sent len, batch size] \n",
    "        \n",
    "        # не забываем применить dropout к embedding\n",
    "        embedded = self.dropout(self.embeddings(sent))\n",
    "\n",
    "        output, _ = self.rnn(embedded)\n",
    "        #output = [sent len, batch size, hid dim * n directions]\n",
    "\n",
    "        prediction = self.tag(output)\n",
    "    \n",
    "        return prediction\n",
    "        \n",
    "# параметры модели\n",
    "INPUT_DIM = len(WORD.vocab)\n",
    "OUTPUT_DIM = len(TAG.vocab)\n",
    "EMB_DIM = 10\n",
    "HID_DIM = 20\n",
    "DROPOUT = 0.5\n",
    "NUM_LAYERS = 2\n",
    "\n",
    "model = LSTMTagger(INPUT_DIM, EMB_DIM, HID_DIM, OUTPUT_DIM, DROPOUT, NUM_LAYERS).to(device)\n",
    "\n",
    "# инициализируем веса\n",
    "def init_weights(m):\n",
    "    for name, param in m.named_parameters():\n",
    "        nn.init.uniform_(param, -0.08, 0.08)\n",
    "        \n",
    "model.apply(init_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EJLqq8IHcMdQ"
   },
   "source": [
    "Подсчитаем количество обучаемых параметров нашей модели. Используйте метод `numel()`. **(1 балл)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_Auu53Kdxolm"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The model has 129,243 trainable parameters\n"
     ]
    }
   ],
   "source": [
    "def count_parameters(model):\n",
    "    return torch.numel(model.tag.weight) + torch.numel(model.tag.bias) \\\n",
    "            + sum([torch.numel(model.rnn._parameters['weight_ih_l'+str(i)]) for i in range(NUM_LAYERS)]) \\\n",
    "            + sum([torch.numel(model.rnn._parameters['weight_hh_l'+str(i)]) for i in range(NUM_LAYERS)]) \\\n",
    "            + sum([torch.numel(model.rnn._parameters['bias_ih_l'+str(i)]) for i in range(NUM_LAYERS)]) \\\n",
    "            + sum([torch.numel(model.rnn._parameters['bias_ih_l'+str(i)]) for i in range(NUM_LAYERS)]) \\\n",
    "            + torch.numel(model.embeddings.weight)\n",
    "\n",
    "print(f'The model has {count_parameters(model):,} trainable parameters')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oSBfvf9HcMd9"
   },
   "source": [
    "Погнали обучать **(2 балла)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AjD1Y7Rmxolu",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "PAD_IDX = TAG.vocab.stoi['<pad>']\n",
    "optimizer = optim.Adam(model.parameters())\n",
    "criterion = nn.CrossEntropyLoss(ignore_index = PAD_IDX)\n",
    "\n",
    "def train(model, iterator, optimizer, criterion, clip, train_history=None, valid_history=None):\n",
    "    model.train()\n",
    "    \n",
    "    epoch_loss = 0\n",
    "    history = []\n",
    "    for i, batch in enumerate(iterator):\n",
    "        \n",
    "        words = batch.words\n",
    "        tags = batch.tags\n",
    "        \n",
    "        optimizer.zero_grad()\n",
    "        \n",
    "        output = model(words)\n",
    "        \n",
    "        #tags = [sent len, batch size]\n",
    "        #output = [sent len, batch size, output dim]\n",
    "        \n",
    "        output = torch.flatten(output, start_dim=0, end_dim=1)\n",
    "        tags = tags.view(-1)\n",
    "        \n",
    "        #tags = [sent len * batch size]\n",
    "        #output = [sent len * batch size, output dim]\n",
    "        \n",
    "        loss = criterion(output, tags)\n",
    "        \n",
    "        loss.backward()\n",
    "        \n",
    "        # Gradient clipping(решение проблемы взрыва граденты), clip - максимальная норма вектора\n",
    "        torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=clip)\n",
    "        \n",
    "        optimizer.step()\n",
    "        \n",
    "        epoch_loss += loss.item()\n",
    "        \n",
    "        history.append(loss.cpu().data.numpy())\n",
    "        if (i+1)%10==0:\n",
    "            fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(12, 8))\n",
    "\n",
    "            clear_output(True)\n",
    "            ax[0].plot(history, label='train loss')\n",
    "            ax[0].set_xlabel('Batch')\n",
    "            ax[0].set_title('Train loss')\n",
    "            \n",
    "            if train_history is not None:\n",
    "                ax[1].plot(train_history, label='general train history')\n",
    "                ax[1].set_xlabel('Epoch')\n",
    "            if valid_history is not None:\n",
    "                ax[1].plot(valid_history, label='general valid history')\n",
    "            plt.legend()\n",
    "            \n",
    "            plt.show()\n",
    "\n",
    "        \n",
    "    return epoch_loss / len(iterator)\n",
    "\n",
    "def evaluate(model, iterator, criterion):\n",
    "    model.eval()\n",
    "    \n",
    "    epoch_loss = 0\n",
    "    \n",
    "    history = []\n",
    "    \n",
    "    with torch.no_grad():\n",
    "    \n",
    "        for i, batch in enumerate(iterator):\n",
    "\n",
    "            words = batch.words\n",
    "            tags = batch.tags\n",
    "\n",
    "            output = model(words)\n",
    "\n",
    "            #tags = [sent len, batch size]\n",
    "            #output = [sent len, batch size, output dim]\n",
    "\n",
    "            output = torch.flatten(output, start_dim=0, end_dim=1)\n",
    "            tags = tags.view(-1)\n",
    "            \n",
    "\n",
    "            #tags = [sent len * batch size]\n",
    "            #output = [sent len * batch size, output dim]\n",
    "\n",
    "            loss = criterion(output, tags)\n",
    "            \n",
    "            epoch_loss += loss.item()\n",
    "        \n",
    "    return epoch_loss / len(iterator)\n",
    "\n",
    "def epoch_time(start_time, end_time):\n",
    "    elapsed_time = end_time - start_time\n",
    "    elapsed_mins = int(elapsed_time / 60)\n",
    "    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))\n",
    "    return elapsed_mins, elapsed_secs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TJdXIyTHxol2",
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsIAAAHwCAYAAACsSAniAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACOL0lEQVR4nO3dd5xU1fnH8c/ZxsJSlqXDArsgRaSXFWyAFRv2ronGBEssSewxlp+9txhDjFFjTLAbG3ZEJKIUKdK7stSld9hyfn/M7DI7O31n5k75vl8vXuzO3Ln3mbJznznznOcYay0iIiIiIukmw+kAREREREScoERYRERERNKSEmERERERSUtKhEVEREQkLSkRFhEREZG0pERYRERERNKSEmFJOsaYj40xv4zwtiuNMcdGOyYRERFJPllOByDpwRiz0+PXRsA+oNL9+xXW2n+Hui9r7YnRjE1ERETSkxJhiQtrbePqn40xK4FfW2u/8N7OGJNlra2IZ2wiIiKSnlQaIY4yxowwxpQaY24xxqwDXjLGNDfGfGiMKTPGbHH/XOhxm4nGmF+7f77UGDPZGPOYe9sVxpiQRoyNMQ2MMU8ZY9a4/z1ljGngvq6l+7hbjTGbjTHfGGMy3NfdYoxZbYzZYYxZZIw5JgYPjYiIiMSYEmFJBG2BAqAzMAbX6/Il9++dgD3AswFufyiwCGgJPAL8wxhjQjju7cBQoD/QDygB/uS+7gagFGgFtAH+CFhjTA/gGmCItbYJcAKwMrS7KSIiIolEibAkgirgLmvtPmvtHmvtJmvt29ba3dbaHcD9wPAAt//JWvt3a20l8E+gHa7kNZiLgHustRustWXA/wGXuK8rd++ns7W23Fr7jbXW4qprbgD0MsZkW2tXWmuXRXSvRURExFFKhCURlFlr91b/YoxpZIz5mzHmJ2PMdmASkG+MyfRz+3XVP1hrd7t/bOxnW0/tgZ88fv/JfRnAo8BS4DNjzHJjzK3u/S8FfgfcDWwwxrxmjGmPiIiIJB0lwpIIrNfvNwA9gEOttU2Bo9yXh1LuEI41uMovqnVyX4a1doe19gZrbRfgVOAP1bXA1tr/WGuPcN/WAg9HOS4RERGJAyXCkoia4KoL3mqMKQDuitFxxgF/Msa0Msa0BO4EXgUwxpxijDnIXWu8HVdJRKUxpocx5mj3pLq97jgr/exfREREEpgSYUlETwENgY3Ad8AnMTrOfcB0YA7wI/CD+zKAbsAXwE5gCvCctXYirvrgh9yxrQNa45pIJyIiIknGuOb/iIiIiIikF40Ii4iIiEhaUiIsIiIiImlJibCIiIiIpCUlwiIiIiKSlpQIi4iIiEhaynLqwC1btrRFRUVOHV5EpF5mzJix0Vrbyuk44kXv2SKSzPy9ZzuWCBcVFTF9+nSnDi8iUi/GmJ+Cb5U69J4tIsnM33u2SiNEREREJC0pERYRERGRtKREWERERETSkmM1wiIiIiKhKi8vp7S0lL179zodiiSw3NxcCgsLyc7ODml7JcIiIiKS8EpLS2nSpAlFRUUYY5wORxKQtZZNmzZRWlpKcXFxSLdRaYSISIoxxrxojNlgjJkbYJsRxphZxph5xpiv4xmfSCT27t1LixYtlASLX8YYWrRoEda3BkqERURSz8vAKH9XGmPygeeA0dbaQ4Bz4hOWSP0oCZZgwn2NKBEWEUkx1tpJwOYAm1wIvGOt/dm9/Ya4BCYiUTNixAifvb2feuopdu/eHfb+7rzzTr744ouQt3/55Ze55pprfF530kknsXXrVr+3jTTGWFAiLCKSfroDzY0xE40xM4wxv/C1kTFmjDFmujFmellZWZxDFElv1lqqqqrCvl2gJLOystLv7e655x6OPfbYsI/ny/jx48nPz/d7fSSJcKDY60OJsIhI+skCBgEnAycAdxhjuntvZK193lo72Fo7uFWrtFlNWsSve++9l549e3LcccdxwQUX8NhjjwGwbNkyRo0axaBBgzjyyCNZuHAhAJdeeinXXXcdhx12GF26dOGtt96q2dejjz7KkCFD6Nu3L3fddRcAK1eu5OCDD+bqq69m4MCBrFq1iquuuorBgwdzyCGH1GznzzPPPMOaNWsYOXIkI0eOBKBx48bceeedHHrooUyZMoV77rmHIUOG0Lt3b8aMGYO1tibW6viKioq46667GDhwIH369Km5P97WrFnDqFGj6NatGzfffHPN5UVFRWzcuJFdu3Zx8skn069fP3r37s3rr7/uM8Zx48bRp08fevfuzS233FKzH8/Y77vvPs4444ya6z7//HPOPPPMEJ61wNQ1QkQk/ZQCG621u4BdxphJQD9gsbNhiYTm/z6Yx/w126O6z17tm3LXqYf4vX769Om8/fbbzJw5k4qKCgYOHMigQYMAGDNmDGPHjqVbt258//33XH311UyYMAGAtWvXMnnyZBYuXMjo0aM5++yz+eyzz1iyZAlTp07FWsvo0aOZNGkSnTp1YtGiRbz00ks899xzANx///0UFBRQWVnJMcccw5w5c+jbt6/PGK+77jqeeOIJvvrqK1q2bAnArl276N27N/fcc4/rfvbqxZ133gnAJZdcwocffsipp55aZ18tW7bkhx9+4LnnnuOxxx7jhRdeqLPNrFmzmDlzJg0aNKBHjx5ce+21dOzYseb6Tz75hPbt2/PRRx8BsG3bNpo1a1YrxjVr1nDLLbcwY8YMmjdvzvHHH89///tfTj/99FqxW2s5+OCDKSsro1WrVrz00ktcdtllgZ/UEGhEWEQk/bwHHGmMyTLGNAIOBRY4HJNIQps8eTKnnXYaDRs2pEmTJjXJ486dO/n2228555xz6N+/P1dccQVr166tud3pp59ORkYGvXr1Yv369QB89tlnfPbZZwwYMICBAweycOFClixZAkDnzp0ZOnRoze3feOMNBg4cyIABA5g3bx7z588PK+7MzEzOOuusmt+/+uorDj30UPr06cOECROYN2+ez9tVj7YOGjSIlStX+tzmmGOOoVmzZuTm5tKrVy9++umnWtf36dOHL774gltuuYVvvvmGZs2a1dnHtGnTGDFiBK1atSIrK4uLLrqISZMm1YndGMMll1zCq6++ytatW5kyZQonnnhiWI+FLxoRFhFJMcaYccAIoKUxphS4C8gGsNaOtdYuMMZ8AswBqoAXrLV+W62JJJpAI7exUl1C4K2qqor8/HxmzZrl8/oGDRrU2Ye1lttuu40rrrii1rYrV64kLy+v5vcVK1bw2GOPMW3aNJo3b86ll14a9oIiubm5ZGZmAq4WdFdffTXTp0+nY8eO3H333X73Vx13ZmYmFRUVQe+br+26d+/OjBkzGD9+PLfddhvHH398zWh0NX+Pq3fsAJdddhmnnnoqubm5nHPOOWRl1T+N1YiwiEiKsdZeYK1tZ63NttYWWmv/4U6Ax3ps86i1tpe1tre19ikHwxVJCkcccQQffPABe/fuZefOnTVf9zdt2pTi4mLefPNNwJXYzZ49O+C+TjjhBF588UV27twJwOrVq9mwoW7zlu3bt5OXl0ezZs1Yv349H3/8cdA4mzRpwo4dO3xeV530tmzZkp07d9aqWY6FNWvW0KhRIy6++GJuvPFGfvjhhzoxHnrooXz99dds3LiRyspKxo0bx/Dhw33ur3379rRv35777ruPSy+9NCoxakRYREREJIghQ4YwevRo+vXrR+fOnRk8eHDNV/3//ve/ueqqq7jvvvsoLy/n/PPPp1+/fn73dfzxx7NgwQKGDRsGuCaFvfrqq7VGPwH69evHgAEDOOSQQ+jSpQuHH3540DjHjBnDiSeeSLt27fjqq69qXZefn89vfvMb+vTpQ1FREUOGDAn3YQjLjz/+yE033URGRgbZ2dn89a9/9Rnjgw8+yMiRI7HWctJJJ3Haaaf53edFF11EWVkZvXr1ikqMJtCQdCwNHjzY+up/JyKSDIwxM6y1g52OI170ni1OW7BgAQcffLCjMezcuZPGjRuze/dujjrqKJ5//nkGDhzoaEzp5pprrmHAgAFcfvnlfrfx9Vrx956tEWEREYkJay3rt++jbbNcp0MRiYoxY8Ywf/589u7dyy9/+UslwXE2aNAg8vLyePzxx6O2TyXCIpLQLnj+O+av3c7su453OhQJ05OfL2bspOXMuet4crMzg99AJMH95z//cTqEtDZjxoyo71OT5UQkoU1Zvolte8qdDkMi0Kcwn/0VVcwp3eZ0KCIiPikRFhGRmBhS1ByAqSs2ORyJiIhvSoRFRCQm8hvl0LNtE75fsdnpUEREfFIiLCIiMVNSXMCMn7ZQUVnldCgiInUoERYRkZgpKS5g9/5K5q3Z7nQoIillxIgRRKOloed+TjrpJLZu3Vpnm7vvvpvHHnuszuWXXnqpz0U51qxZw9lnn+33mFu3buW5556LPOgoUiIsIiIxU1JUAMBUlUeIhMVaS1VVfL9JGT9+PPn5+fXeT/v27QOuWhdJIlxZWVnfsHxSIiwiIjHTumkuxS3z+F4T5iQF3HvvvfTs2ZPjjjuOCy64oGaUdNmyZYwaNYpBgwZx5JFHsnDhQsA1Ynrddddx2GGH0aVLl1rJ4aOPPsqQIUPo27cvd911FwArV67k4IMP5uqrr2bgwIGsWrWKq666isGDB3PIIYfUbOfPxx9/zLnnnlvz+8SJEzn11FMBQtpPUVERGzduBOD++++nR48eHHvssSxatMjvMSdNmlTn/q1cuZLevXsDMG/ePEpKSujfvz99+/ZlyZIl3HrrrSxbtoz+/ftz0003Ya3lpptuonfv3vTp04fXX3+9Jv6RI0dy4YUX0qdPH+644w6efvrpmmPffvvtPPPMMwEfk2DUR1hERGKqpKiAj+euparKkpFhnA5HUsHHt8K6H6O7z7Z94MSH/F49ffp03n77bWbOnElFRQUDBw5k0KBBgGuhjbFjx9KtWze+//57rr76aiZMmADA2rVrmTx5MgsXLmT06NGcffbZfPbZZyxZsoSpU6dirWX06NFMmjSJTp06sWjRIl566aWaEdP777+fgoICKisrOeaYY5gzZw59+/b1GeNxxx3HFVdcwa5du8jLy+P111/nvPPOC3s/M2bM4LXXXvN5X735un+exo4dy/XXX89FF13E/v37qays5KGHHmLu3LnMmjULgLfffptZs2Yxe/ZsNm7cyJAhQzjqqKMAmDp1KnPnzqW4uJiVK1dy5plncv3111NVVcVrr73G1KlT/T5noVAiLCIiMXVolwJen76KRet3cHC7pk6HIxKRyZMnc9ppp9GwYUOAmpHWnTt38u2333LOOefUbLtv376an08//XQyMjLo1asX69evB+Czzz7js88+Y8CAATX7WLJkCZ06daJz584MHTq05vZvvPEGzz//PBUVFaxdu5b58+f7TWCzsrIYNWoUH3zwAWeffTYfffQRjzzySNj7+eabbzjjjDNo1KgRAKNHj/b7uPi6f56GDRvG/fffT2lpKWeeeSbdunWrs83kyZO54IILyMzMpE2bNgwfPpxp06bRtGlTSkpKKC4uBlwj1i1atGDmzJmsX7+eAQMG0KJFC7+xhUKJsIiIxFRJ8YE6YSXCEhUBRm5jxVrr8/Kqqiry8/NrRje9NWjQoM4+rLXcdtttXHHFFbW2XblyJXl5eTW/r1ixgscee4xp06bRvHlzLr30Uvbu3RswzvPOO4+//OUvFBQUMGTIEJo0aRLRfowJ7dsbX/fP04UXXsihhx7KRx99xAknnMALL7xAly5dam3j77EFaj0eAL/+9a95+eWXWbduHb/61a9CijEQ1QiLiEhMFTZvRIf8hpowJ0ntiCOO4IMPPmDv3r3s3LmTjz76CICmTZtSXFzMm2++CbiSutmzZwfc1wknnMCLL77Izp07AVi9ejUbNmyos9327dvJy8ujWbNmrF+/no8//jhonCNGjOCHH37g73//e01ZRLj7Oeqoo3j33XfZs2cPO3bs4IMPPgh6XH+WL19Oly5duO666xg9ejRz5syhSZMm7Nixo9bxXn/9dSorKykrK2PSpEmUlJT43N8ZZ5zBJ598wrRp0zjhhBMijquaRoRFRCTmSooL+GbJRqy1IY80iSSSIUOGMHr0aPr160fnzp0ZPHgwzZo1A+Df//43V111Fffddx/l5eWcf/759OvXz+++jj/+eBYsWMCwYcMAaNy4Ma+++iqZmZm1tuvXrx8DBgzgkEMOoUuXLhx++OFB48zMzOSUU07h5Zdf5p///GdE+xk4cCDnnXce/fv3p3Pnzhx55JFBj+vP66+/zquvvkp2djZt27blzjvvpKCggMMPP5zevXtz4okn8sgjjzBlyhT69euHMYZHHnmEtm3b1kw69JSTk8PIkSPJz8+v83hFwgQajo6lwYMH22j0vxOR1FZ0q2vUZeVDJzscSW3GmBnW2sFOxxEv9X3PHjf1Z25750cm3DCcLq0aRzEySRcLFizg4IMPdjSGnTt30rhxY3bv3s1RRx3F888/z8CBAx2NKd1UVVUxcOBA3nzzTZ/1xuD7teLvPVulESIiEnOedcIiyWrMmDH079+fgQMHctZZZykJjrP58+dz0EEHccwxx/hNgsOl0ggREYm5Li3zaNk4h6krNnN+SSenwxGJyH/+8x+nQ0hrvXr1Yvny5VHdp0aERUQk5owxlBQX8L1GhEUkgSgRFhGRuCgpKmD11j2UbtntdCiSpJya1yTJI9zXiBJhERGJi5JiV+P7aSs1Kizhy83NZdOmTUqGxS9rLZs2bSI3Nzfk26hGWERE4qJH2yY0zc1i6orNnDGg0OlwJMkUFhZSWlpKWVmZ06FIAsvNzaWwMPT3FyXCIiISF5kZhiFFqhOWyGRnZ9cstSsSLSqNEBGRuCkpLmB52S7KduxzOhQRESXCIiISP9X9hFUnLCKJIKRE2BgzyhizyBiz1Bhzq4/rbzLGzHL/m2uMqTTGFEQ/XBERSWa9OzSjYXamFtYQkYQQNBE2xmQCfwFOBHoBFxhjenluY6191Frb31rbH7gN+Npaq3c5ERGpJTszg0Gdm6tOWEQSQigjwiXAUmvtcmvtfuA14LQA218AjItGcCIiknpKigtYuG4723aXOx2KiKS5UBLhDsAqj99L3ZfVYYxpBIwC3q5/aCIikopKiguwVnXCIuK8UBJh4+Myf92sTwX+568swhgzxhgz3RgzXX0ARUTSU/+O+eRkZjBVibCIOCyURLgU6OjxeyGwxs+25xOgLMJa+7y1drC1dnCrVq1Cj1JERFJGbnYm/Tvmq05YRBwXSiI8DehmjCk2xuTgSnbf997IGNMMGA68F90QRUQk1ZQUFzB39TZ27atwOhQRSWNBE2FrbQVwDfApsAB4w1o7zxhzpTHmSo9NzwA+s9buik2oIiKSKkqKC6issvzw8xanQxGRNBbSEsvW2vHAeK/Lxnr9/jLwcrQCExGR1DWwc3MyMwxTV2zmyG4qlRMRZ2hlORERibvGDbLo3b6p6oRFxFFKhEVExBElxQXMWrWVveWVTociImlKibCIiDiipLgF+yuqmFO6zelQRCRNKREWERFHDClqDsDUFZscjkRE0pUSYRERcUR+oxx6tm2iOmERcYwSYRERcUxJcQEzftpCRWWV06GISBpSIiwiIo4pKS5g9/5K5q3Z7nQoIpKGlAiLiIhjSooKAJiq8ggRcYASYRERcUzrprkUt8xTnbCIOEKJsIiIOKqkqIBpKzdTVWWdDkVE0owSYRERcVRJcQHb9pSzeMMOp0MRkTSjRFhERBxVUqw6YRFxhhJhERFxVGHzhrRvlsv3y5UIi0h8KREWERFHGWMoKS7g+xWbsVZ1wiISP0qERUTEcYd2acHGnftYsXGX06GISBpRIiwiIo5TnbCIOEGJsIiIOK5LyzxaNs5RIiwicaVEWEQkxRhjXjTGbDDGzA2y3RBjTKUx5ux4xRYglpo6YRGReFEiLCKSel4GRgXawBiTCTwMfBqPgEJRUlTA6q17KN2y2+lQRCRNKBEWEUkx1tpJQLCh1WuBt4ENsY8oNCXFLQCYtlKjwiISH0qERUTSjDGmA3AGMNbpWDz1aNuEprlZqhMWkbhRIiwikn6eAm6x1lYG2sgYM8YYM90YM72srCzmQWVmGIYUqU5YROJHibCISPoZDLxmjFkJnA08Z4w53Xsja+3z1trB1trBrVq1iktgJcUFLC/bRdmOfXE5noikNyXCIiJpxlpbbK0tstYWAW8BV1tr/+tsVC7V/YRVJywi8aBEWEQkxRhjxgFTgB7GmFJjzOXGmCuNMVc6HVswvTs0o2F2puqERSQuspwOQEREostae0EY214aw1DClp2ZwaDOzVUnLCJxoRFhERFJKCXFBSxct51tu8udDkVEUpwSYRERSSglxQVYC9N/0qiwiMSWEmEREUko/Tvmk5OZoTphEYk5JcIiIpJQcrMz6dexmeqERSTmlAiLiEjCKSku4MfV29i1r8LpUEQkhSkRFhGRhFNS3ILKKssPP29xOhQRSWFKhEVEJOEM6tyczAyjOmERiSklwiIiknAaN8iid/umqhMWkZhSIiwiIgmppLiAWau2sre80ulQRCRFKREWEZGEVFLcgv0VVcwp3eZ0KCKSopQIi4hIQhpS1ByAqSs2ORyJiKQqJcIiIpKQ8hvl0LNtE9UJi0jMKBEWEZGEVVJcwIyftlBRWeV0KCKSgpQIi4hIwiopLmD3/krmrdnudCgikoKUCIuISMIqKSoAUD9hEYkJJcIiCa6qyrJm6x6nwxBxROumuRS3zFOdsIjEhBJhkQT35wlLOeyhCfy8abfToYg4oqSogGkrN1NVZZ0ORURSjBJhkQQ3eWkZAOu273U4EhFnlBQXsG1POYs37HA6FBFJMUqERUQkoZUUq05YRGJDibCIiCS0wuYNad8sV3XCIhJ1SoRFRCShGWMoKS5g6orNWKs6YRGJHiXCIiKS8EqKW1C2Yx8rNWlURKJIibCIiCS86jrh75dvcjgSEUklSoRFRCThdW2VR8vGOZowJyJRpURYREQSXnWdsCbMiUg0KREWEZGkUFJUwOqteyjdojphEYkOJcIiIpIUSopbADBtpUaFRSQ6lAiLiEhS6NG2CU1zs1QnLCJRo0RYJEmof6qku8wMw5Ai1QmLSPQoERYRkaRRUlzA8rJdlO3Y53QoIpIClAiLJAljjNMhiDiuup+w6oRFJBqUCIuISNLo3aEZDbMzVScsIlGhRFgkSahGWASyMzMY1Lm56oRFJCqUCIuISFIpKS5g4brtbNtd7nQoIpLklAiLJAnVCIu4lBQXYC1M/0mjwiJSP0qERUQkqfTvmE9OZobqhEWk3pQIiyQJ1QiLuORmZ9KvYzPVCYtIvSkRFhGRpFNSXMDc1dvYta/C6VBEJIkpERZJEqoRFjmgpLgFFVWWmT9vdToUEUliSoRFRCTpDOrcnAwDU1dscjoUEUliSoRFkoRqhEUOaNwgi94dVCcsIvWjRFhERJJSSVEBM1dtZW95pdOhiEiSUiIskiRUIyxSW0lxAfsrqphTus3pUEQkSSkRFhGRpFRSXACoTlhEIhdSImyMGWWMWWSMWWqMudXPNiOMMbOMMfOMMV9HN0wRUY2wSG35jXLo2baJ6oRFJGJBE2FjTCbwF+BEoBdwgTGml9c2+cBzwGhr7SHAOdEPVSQ9GVQSIeJPSXEBM37aQkVlldOhiEgSCmVEuARYaq1dbq3dD7wGnOa1zYXAO9banwGstRuiG6ZI+rJoJFjEn5LiAnbvr2Temu1OhyIiSSiURLgDsMrj91L3ZZ66A82NMRONMTOMMb+IVoAi4qLJciJ1lRRV1wmrPEJEwhdKIuzr7Os9RJUFDAJOBk4A7jDGdK+zI2PGGGOmG2Oml5WVhR2sSDpTjbBIXa2b5lLcMk91wiISkVAS4VKgo8fvhcAaH9t8Yq3dZa3dCEwC+nnvyFr7vLV2sLV2cKtWrSKNWSStqEZYJLCSogKmrdxMVZU+LIpIeEJJhKcB3YwxxcaYHOB84H2vbd4DjjTGZBljGgGHAguiG6pIelKNsEhgJcUFbNtTzuINO5wORUSSTNBE2FpbAVwDfIoruX3DWjvPGHOlMeZK9zYLgE+AOcBU4AVr7dzYhS2SflQjLOLbgX7CKo8QkfBkhbKRtXY8MN7rsrFevz8KPBq90ERERIIrbN6Q9s1y+X7FZn4xrMjpcEQkiWhlOZEkoclyEipjzIvGmA3GGJ/fzBljLjLGzHH/+9YYU2dORzIxxlBSXMDUFZv1dyIiYVEiLCKSel4GRgW4fgUw3FrbF7gXeD4eQcVSSXELynbsY+Wm3U6HIiJJRImwSJJQjbCEylo7CfBbMGut/dZau8X963e4ugEltQN1wpscjkREkokSYRGR9HY58LGvK5Kp93vXVnm0yMtRP2ERCYsSYZEkodpHiTZjzEhcifAtvq5Ppt7vnnXCIiKhUiIsIpKGjDF9gReA06y1KVFPUFJcQOmWPazeusfpUEQkSSgRFkkSqhGWaDHGdALeAS6x1i52Op5oUZ2wiIRLibCISIoxxowDpgA9jDGlxpjLPRdBAu4EWgDPGWNmGWOmOxZsFPVs25QmuVkqjxCRkIW0oIaIOE81whIqa+0FQa7/NfDrOIUTN5kZhpKiAk2YE5GQaURYJMEZVBIhEqqS4gKWl+2ibMc+p0MRkSSgRFgkwVk0EiwSquo64WkrNSosIsEpERZJEposJxJc7w7NaJidqTphEQmJEmGRJKEaYZHgsjMzGNS5ueqERSQkSoRFEpxqhEXCU1JcwMJ129m2u9zpUEQkwSkRFklwqhEWCU9JcQHWwvSfNCosIoEpERZJEqoRFglN/4755GRmqE5YRIJSIiySJFQjLBKa3OxM+nVspjphEQlKibBIglONsEj4SooLmLt6G7v2VTgdiogkMCXCIglONcIi4SspbkFFlWXmz1udDkVEEpgSYZEkoRphkdAN6tycDANTV2xyOhQRSWBKhEWShGqERULXuEEWvTuoTlhEAlMiLJLgVCMsEpmSogJmrtrKvopKp0MRkQSlRFgkwalGWCQyJcUF7K+oYk7pNqdDEZEEpURYJEmoRlgkPEOKCgDUT1hE/FIiLJIkVCMsEp7meTn0aNOE75ZrwpyI+KZEWCTBqUZYJHIlxQXM+GkLFZVVTociIglIibCIiKSsQ7sUsHt/JfPWbHc6FBFJQEqERRKcJsuJRK5EdcIiEoASYRERSVmtm+ZS3DJP/YRFxCclwiIJTjXCIvVTUlTAtJWbqarStysiUpsSYRERSWklxQVs21PO4g07nA5FRBKMEmGRBKcaYZH6KSlWnbCI+KZEWKJi2+5yHhi/gHK1KBKRBFPYvCHtm+WqTlhE6lAiLFFx//j5PD9pOeN/XOt0KClHNcIi9WOMoaS4gKkrNmthGhGpRYmwREV5pevkUqnJKCKSgEqKW1C2Yx8rN+12OhQRSSBKhEUSnGqERervQJ2wllsWkQOUCIuISMrr2iqPFnk5qhMWkVqUCIskONUIi9SfZ52wiEg1JcIiIpIWSooLKN2yh9Vb9zgdiogkCCXCIglONcIi0VFdJzxNo8Ii4qZEWKJCLYlEJNH1bNuUJrlZqhMWkRpKhEUSnGqERaIjM8MwpKhAnSNEpIYSYYkKY5SsiUjiKykuYFnZLjbu3Od0KCKSAJQIS1SoNEJEksGBfsIqjxARJcIiIpJG+nRoRsPsTCXCIgIoERYRkTSSnZnBoM7NNWFORAAlwhJlKhUWkURXUlzAwnXb2ba73OlQRMRhSoQlqlQqLCKJrqS4AGth+k8aFRZJd0qERUQkrfTvmE9OZobqhEVEibBEl0ojRCTR5WZn0q9jM9UJi4gSYRERST8lxQXMXb2NXfsqnA5FRBykRFhERNJOSXELKqosM3/e6nQoIuIgJcIiIpJ2BnVuToZByy2LpDklwiIiknYaN8iidwfVCYukOyXCIiKSlkqKCpi5aiv7KiqdDkVEHKJEWKJC7YNFJNmUFBewv6KKOaXbnA5FRByiRFhERNLSkKICAPUTFkljSoQlKtQ+WESSTfO8HHq0aaI6YZE0pkRYokKlESKSjEqKC5ixcjMVlVVOhyIiDlAiLCIiaaukuIBd+yuZv3a706GIiAOUCIuISNoqKVadsEg6UyIsUWVULSwevl26kQkL1zsdRtoxxrxojNlgjJnr53pjjHnGGLPUGDPHGDMw3jEmijZNcylq0YjvlisRFklHSoQlqqyqhcXDhS98z69enu50GOnoZWBUgOtPBLq5/40B/hqHmBJWSXEB01ZupqpK718i6UaJsIhIirHWTgICDXGeBrxiXb4D8o0x7eITXeI5tLgF2/aUs3jDDqdDEZE4UyIsUaXSiNjRWJVEUQdglcfvpe7L0pLqhEXSlxJhSRiL1+/gyc8XOx2GSDrw9Ym1zmctY8wYY8x0Y8z0srKyOITljMLmDWnfLFf9hEXSkBJhSRjn/m0KT3+5hB17y50OJSFprF2iqBTo6PF7IbDGeyNr7fPW2sHW2sGtWrWKW3DxZoyhpLiAqSs2Y62+exFJJ0qEJWGUV6ihvVMqKqt4buJS9uyvdDoUiY/3gV+4u0cMBbZZa9c6HZSTSopbULZjHys37XY6FBGJoyynAxCR0MRynOqdmat55JNFbNtTzm0nHhzDI0k8GGPGASOAlsaYUuAuIBvAWjsWGA+cBCwFdgOXORNp4jhQJ7yJ4pZ5DkcjIvGiRFhE2FvuGgnevU8jwqnAWntBkOst8Ns4hZMUurbKo0VeDt+v2Mx5Qzo5HY6IxElIpRHGmFHGmEXu5uu3+rh+hDFmmzFmlvvfndEPVVKdMaqCDUSPjkjseNYJi0j6CJoIG2Mygb/gasDeC7jAGNPLx6bfWGv7u//dE+U4RaQedu2r4NQ/T2bB2u1OhxITb0xbxVL1gJV6KikuoHTLHlZv3eN0KCISJ6GMCJcAS621y621+4HXcDVjF5Ek8d3yTfy4ehuPfLLQ5/XJPlH+5rfncOwTk5wOQ5JcdZ3wNI0Ki6SNUBLhUBuvDzPGzDbGfGyMOSQq0YlIWquuXRaJh55tm9IkN0v9hEXSSCiT5UJpvP4D0Nlau9MYcxLwX1xr2NfekTFjcK1rT6dOmowgtVW/0JJ8cDIhBSu/TtTy7Be+We50CJJGMjMMQ4oKmLpik9OhiEichDIiHLTxurV2u7V2p/vn8UC2Maal947SpTl7Okr2r9bTXaI+f3s0IixxVlJcwLKyXWzcuc/pUEQkDkJJhKcB3YwxxcaYHOB8XM3Yaxhj2hr3lH9jTIl7v/pILZIgEjXRDSZZ45bkpTphkfQSNBG21lYA1wCfAguAN6y184wxVxpjrnRvdjYw1xgzG3gGON9qncq0kqhfrUttalEnEljv9s1omJ2pOmGRNBHSghrucofxXpeN9fj5WeDZ6IYmySSaH3v0EUpEnJKTlcHAzvnqJyySJkJaUENEkps+XIiErqSoBQvWbWfb7nKnQxGRGFMiHGPfLCljxk8aWZDEEKwwItEqJ5S/ixNKiguwFqbrvVsk5SkRjrFL/jGVs/46xekw4qZeiVSCJWGJJh5JoUaOJarK98Lyr52OImwDOuWTk5mh8giRNKBEWKJKiZSI1Jj4ALx6Fmz5yelIwpKbnUm/js00YU4kDSgRlsSjZNqn+gyYh/qQJlpphCS5Q6+EjEz4+mGnIwlbSXEBc1dvY9e+CqdDEZEYUiIsUaVEKrE5+fws3bCTcLsq6huGJNe0PQz5NcweB2WLnY4mLCXFLaiossz8eavToYhIDCkRloRjNSTsUzQeFacSyynLNnHsE18zbuoqZwIQ5xzxe8hu5CqTSCKDOjcnw6DllkVSnBJhSRgaTI4dpx/bZWU7Afhx9TaHI5G4y2sJQ6+Cee/C2jlORxOyxg2y6N1BdcIiqU6JsCQcfR3uWzxqhEViYtg1kNsMvrrf6UjCUlJUwMxVW9lXUel0KCISI0qEJeEoaYudZKvhVplMimiYD4dfD4s/gVXTnI4mZCXFBeyvqGJOqb7JEElVSoQlYZhky9LiLBVSQj3FaezQKyGvFUy4x+lIQjakqABA/YRFUpgSYUk44XYWkMSnZ1TIyYMjb4AVk5JmkY3meTn0aNNEdcIiKUyJsEiSqFeNcJAPF/H68KEB4TQ36DJo2gEm3Jc0kwFKiguYsXIzFZVVTociIjGgRFgSTnKcHpOVUlFxUHYuHHUTlE6FJZ85HU1ISooL2LW/kvlrtzsdiojEgBJhiYpoJq9JMlAUd2n5sKTlnU5xAy6G5sUw4V6oSvxR1pJi1QmLpDIlwpIwNJEqhenTjVTLzIYRt8G6H2HBe05HE1SbprkUtWikOmGRFKVEWKIimjmsWmZFX7BHNF4dO/RhRwDocza06glfPQBVid+jt6S4gGkrN1NVpfcmkVSjRFiiQqeH5OAvEY31ZDm9PqSWjEwYeTtsXAxzXnc6mqBKiluwdXc5SzbsdDoUEYkyJcKSeJQ1+RSNXDXZKhSSLFwJx8GnQrv+MPFBqNjvdDQBHVpTJ7zJ4UhEJNqUCKeR0c9O5p/frnQ6DAmXygkkFRkDR98BW3+Gma84HU1Ahc0b0q5ZLt+pTlgk5SgRTiNzSrdx1/vznA4jKI0CeoniAxKsRjfWObeJwRG0AEsSO+gY6DQMJj0G5XucjsYvYwyHFhcwdcVmvd5EUowSYUkY1SmSzjMH7KuoZF9F4k8mEolI9ajwjrUw7QWnowmopLgFZTv2sXLTbqdDEZEoUiIMzF61ld+9NlMzgiXhDLr3C2aXbgPSs5uGRt/SQNHh0PVomPwk7NvhdDR+lahOWCQlKREGfv3KdP47aw0bd+5zOhQhPRM+f3buq4jKflI5n0zl+5Y2jv4T7N4E3/3V6Uj86toqjxZ5OeonLJJilAiLpBGn5t0pWZWAOgyCnqfAt3+G3YmZaBpjKHHXCYtI6lAiLAlHSVPi2rhzX71KiLSghvg18nZXacS3zzgdiV8lxQWUbtnD6q2JO7FPRMKTFonwsrKdaVn/u2nnPrbtKY/qPuet2cZzE5dGdZ/Vqlc3+80r02Oy/6Tn8Et47bY9DL7vC/48IfznP9Ja31Buln5/2SmqTS/XinPf/w12rHc6Gp+q64SnaVRYJGUkZSL8xvRVXDtuZkjbzl29jWMe/5q/f7M86LapdkIddN8XDL7v86ju8+RnJvPIJ4uiuk9v89Zsj+n+01P9X93rtu0FYMKiDfXel4hPI26Din0w+QmnI/GpZ9umNMnNUp2wSApJykT45rfm8MHsNSFtu2qzq9XNzJ+3xjCixFVemWrpvdSH3yWWQ7itXkkScy26woCLYPqLsHWV09HUkZlhGFJUoM4RIikkKRNhkXQ0c9VWp0MA6jfhLhYlwmqxlmKOutn1/6RHnI3Dj5LiApaV7VKXIZEUkfKJcKqcIisqq6hMwzpnOeDRT+tfkuIvZ4z1HDa9ciVk+R1h8K9g5r9h0zKno6lDdcIiqSXpEuG95em5ylbPOz7hqEe+qnN50a0f8eyEJQ5EFH1qKOCcSBLVrxZtYMZPW6IeiyeVbKSpI/4AWQ1g4kNOR1JH7/bNaJidqTphkRSRdInwpS9Njfo+o5GA3fTmbLrdPj4Ke/Ktosr6bdnz2GeLY3Zcf/R1dHKKZvuyy16axll//TZ6OxSp1qQNlIyBH9+E9fOdjqaWnKwMBnbOVz9hkRSRdInwd8vDe/Opztdi2b/068VlvDmjNK0mpikPTj/ReM6NGglLqA6/Hho0ga/udzqSOkqKWrBg3faot6cUkfhLukQ4EaVKaQLAnNKt3Pjm7KB9l72v1Qhx+ogkl/X38nhz+iqKbv2Irbv3RxxPJC+9N6avYsn6HREfU+KgUQEMuwYWfgirf3A6mlpKiguwFmb8pFFhkWSnRJj61xgar+KKmT9voejWj1i7LflWH/rVy9N4a0YpG3cFnhEdi8RXg4Wxk6ifU56b6JoMtXGn70Q4VnHf/NYcjntyUmx2LtEz9CpoWAAT7nM6kloGdMonO9OoTlgkBaRNIhzTJMtr3//67icAJi/ZGPVDFd36EeOm/hz1/R7gvjNBEhDvq/WVd3Lw/tBW5/qAz2P0s9LqFlSNcjKjvm9JAblN4Yjfw7Iv4afEqUfPzc6kX6HqhEVSQdokwoHUN4Xzvn2wZMOf9dv3smH73qDbPf5ZbFZ2W7lxV8jbeo/UqTQiNYTyPEbzI8++8qp678Oqb0RqK/kNNG4LX96bUF9tlBQX8GPpNnbvr3A6FBGpByXCUVCf1bo8HfrAl5Q88GXI20cz+Zy6YjMjHptYM0IXbM9KPiQcwV9PIn5kN4SjboSfv3WNDCeIQ7u0oKLKMmWZVpkTSWYpnwjHI2HzHgGuSYxjfOhoDo4sK9vp2LEl9kJ9umJd4uK9+2B/n/rAJQAM/CXkd3LVCifIm09JUQEd8hvyfx/MZ8dedY8QSVYpnwjHg/fJ/UAeHPkbtrWW75dvcqzkINLDRjORem/Wau58b27U9ifOTUgM9jquz+s8QfIiiaWsHBh+K6yZ6eoikQAa5mTy9Pn9Kd2ym9vfnavyMJEkpUTYj00794XcI7JOIlw936we74v/mfoz5z3/HZ/MXee3lZmTb7v+7lv1yeDbZRt5Y9qqeh3j+tdm8cqUn+q1D4meeJ/nF6/fwarNu+N7UElcfc+DFt1gwv1QlRgrjA4uKuB3x3bn/dlreGtGqdPhiEgEUj4RrllQI4QpPp4n+kH3fUH/ez4L6Rh1SiOiMJ1oRZlr4tqqLbv5fMF6n9tEcwTCO+L6fmV94d+/5+a359QzCt/e+aGUT+auC3PfEi2xKJ/wfil/Mncdxz85iS8WbIj6sSRJZWbByD9C2QKY+7bT0dT47ciDGNqlgDvfmxd2iZmIOC+pE+HyyvrPOA8k1DwzWpPl/O1zb7nv0Y9w9h/tr+387S4ebdT+8MZsrnx1RsyPk0ystVQGWQQlEYT6ITGWz6++wk5ivU6HNn3gqwegMjHqcjMzDE+dN4Dc7Ayu/c9M9lUkxmi1iIQmqRPhbrd/zLbdifFm6CledZjhnM/DPff7T3Td14e3O4mxhz9ZRNc/jmd/he8Ph8r9DkiHx8IYM8oYs8gYs9QYc6uP65sZYz4wxsw2xswzxlzmRJxhy8iAo/8EW1bArH87HU2Nts1yefTsfsxfu52HPl7odDgiEoakToQBynYGXgEtFNFOXKNRI1zNWv+jrLGcUR+03VU6ZBNJ5JUpKwHY7+dbkkTtvhCtv5Gwtq//IROaMSYT+AtwItALuMAY08trs98C8621/YARwOPGmJy4Bhqp7idA4RD4+hEoD953PV6O7dWGSw8r4qX/reRLP+VsIpJ4kj4RTgR1E1XX7/VJPgKVGFSf+MMaEY44ktri1BlOIuTvVVP9Wtm4cx9bdvlezjiYVHnO0+BDXAmw1Fq73Fq7H3gNOM1rGws0Ma43msbAZiA5VoYwBo6+A7avhhkvOR1NLbee2JOD2zXlxjdns25b4iTpIuJfUiXCT32xOOzbXDtupuuHEEZ9I01cgyUfyaBOf1c/wVcn6KHet799vYyKEGu5022V5hk/bWbphh1R3ae/p6X68mkrtzDg3s/rXh/Ga7U+T5Pfevo4/q0k0Z9lpDoAni1bSt2XeXoWOBhYA/wIXG+trfOHaowZY4yZboyZXlZWFqt4w9dlOBQfBd88DvtDXxEz1nKzM3n2wgHsLa/id6/PTIq6fZF0l1SJ8A8/b/Vxaf3faOp7EvbXPi2YrxeX8dL/VtS5fE7p1pqffYUWScLomdgOe/BLLn7h+yDbB9thaMd98OOFvP1DcrYV2rWvIuLR01Cc9dcpHPvEpJjt35PTo6B+2+1FY99h7qUqmT6hRsbXO4T3nT4BmAW0B/oDzxpjmta5kbXPW2sHW2sHt2rVKtpx1s/Rd8KuMvh+rNOR1NK1VWP+77RD+G75Zv46canT4YhIEEmVCMd6wDDiRSQ8fv7rxGWUbtnj2l+Q2/3yxan83wfz61z+j8krau0zGiPOnpuu3baXyUs3Bt7e32S5mv3V3iBQKHv2J+cs6hGPTfQ5egquxNLp5NKXWP6NROPu+n0tx3GcNgGftmgrBTp6/F6Ia+TX02XAO9ZlKbAC6Bmn+KKj4xDoPgr+9zTs2ep0NLWcM6iQU/u158kvljDjp81OhyMiASRXIuzjLBqVk3M9OyF41vM+/MlCJi12f4UYYXCedzPQLmKZPPgbNYvmRMA6+47+LuulbIfviZg79pZTfNt4xn69PM4R+Rfs+Yjk+dq+t5zt7qVjx369jHP/NgWITgnLuWOn8Levl9V/R+LLNKCbMabYPQHufOB9r21+Bo4BMMa0AXoAifOCDtXI22HvNpjyrNOR1GKM4f4zetM+P5frxs1KyO5GIuKSVImwL6Ge3yM9d28PYQ15/6NckTHGBAw4oslyQbb17u/qa/M3p6+ivNL6vD7RkthY2rLL9Zr4z9TEW/XOf0/rwC8AX7fre/dn9L3btajM2CgnrVNXbuZBjzZT9flgFWiVQ1+j9qk+ImytrQCuAT4FFgBvWGvnGWOuNMZc6d7sXuAwY8yPwJfALdbawF8TJaJ2fV29hb/7K+xKrPCb5mbz5wsGsn77Xm59Z05CfoMkIkmWCMe+NKLuG9VhD04IertIRshqJvEFsWtfRdAJUKGIRh3lHe/NPbA/r+vT6S2++vmuiu16LhHxftqmrdxM0a0fsXBt4El5wT8o1TMuP6+QWCYHxbeN5/rXZtW5PA1qhLHWjrfWdrfWdrXW3u++bKy1dqz75zXW2uOttX2stb2tta86G3E9jLwdynfD5CedjqSO/h3zufGEHnw8dx3jptZvyXkRiY3kSoQdaCuwc18oHYX89PkNcL79YLZ3yZ7vvT371VJe+Kb2N5bxeBh8xZ7hceBESCVCGa1PZR/MXsOKja4Z8/4SzY9/dC1F/c2S+o2WRetvz3s31ZPqY/V6et/H31kivHYlilp1h77nw7QXYPtap6OpY8yRXTiyW0v+74N5LF4f3S4xIlJ/yZUI+7gsmoM7EU+Wi3FiOqd0m8/LwxlNC39lubo3qJUIR/Fx/275Jo5/8mv2+VkVzZ+fN+2OXhBhyMhwPQ5OjyxeO24mJz39Ta3LvCNKtIU0YvGQhbtLfUWdgkbcAlWVMOlRpyOpIyPD8Pi5/WiSm8U1//mBveXJOXlYJFUlVyKcZIWosT7hxnLvwdq2RTPBuuu9eSxev5Nte8Ib4XXq9VDTOcPBfKq6P+ker5NqrF5ztbuYRP+Bj2dyqjQ4BTUvgoG/gB/+CVtWOh1NHa2b5PL4uf1ZvH4n935Yt1OQiDgnqRLh+ohlWUW0J8thAicbgSbLHf34RJ74vPbCI6GVd9Tma7Qzo3YmXEssFllIVNWPg5OjrfsqwhtVSpTkL5TnevaqrWHt874wE4u6y0ZISjjqJsjIgokPOx2JT8O7t2LMUV349/c/88ncxCvhEElXSZYI1z2LRjMZiXZphOeiQuEmLiHxEe/ysl088+WSWpdt3LEv4H3bW17JhIUbal3mayJYZob/GuGvFyfQqlMR2rJrP0W3fsTkIPW0NZPlwny9RLOf8t5y1xOUnRm82wfUf8TV8zU+deVmpq4Irzdq0PZuHj+f9pf/hbXv16aFNwkp0cpFJEqatoMhv4Y5r0HZIqej8enG43vQt7AZN781h9Vb9zgdjoiQZIlwrPoI15e/0VvP5OMvE0JfYSjYV88H+h6HduczggzDPTB+AZ/MW1frMl/79tffeOXGXezYG/6ocyL4+6Tl/OaV6QDMWe2qxf7bpMCtwg6URhx4EHbtqwhY+/f5/PUcfOcnYY92+lN9rNysTHcsvrcL9Pfx35mrmRVyPLVfQ9U9hcPlxIRXb4nwniExcsQfILsRfPWA05H4lJOVwTPnD6CyynL9uJkhLz8vIrGTXImwj8vqc1LbsH0vG3ceWDQh0pEi/yPCB/YXTv2rMaF9hRzqfTcm8H2r7jzgb9/llVU8O2EJ+z3etKf/tJkXJ7uWh46k9CIaolGrev/4BXw+f717fy7+Htc735tL9z99XPNIWgvXjZvJteNmcshdn3LEw/5b7VUvshJ64hlY9cTCnKzaf8LlFVUhN+//3euzOD3M0ddYiWdyqjw4heW1gKFXw/z/wtrZTkfjU1HLPO4/ow/Tf9rCM2EMkIhIbCRXIhwk79m8az9bd+/3fVuPn8dN/Zktu/ZT8sCXDL7vi5qEKtqlEZ77q/7x6S+W0OW2jwLvL7Iw/MrIMLw/y3+7Nl88Yx839Wce+2xxrVHfa/4zk3vctZme2zo/3he5YCPtr0z5if0VVTX3t8pa3p+9pqYV3sadvl97nvuOluqRJM9yFYArX51Bv3s+q7N9fRPNBBjIjRqnu31IjB12DeTmw4T7nY7Er9MHdOCsgYU8O2EJ3y3f5HQ4ImktqRLhYAbe+zn97/k84DaL1+/gtnd+5Hevz6q5rL41g/5GJn3VkD75xeKQaksD5R01k+WC7waADAO3vvOj3+t9JQael+0Oo7bVV0yxSjs+n78+aqUG4P95/M0r0/n39wdWkat+vTiZTlW6n58DE/dcpq3c4nP7+r/G6yf40ePYNUJ5cGrLbQaHXw9LPoVVU52Oxq97TjuEzi3y+N1rs9iyy/+HaBGJraRKhH0lKuGe4KuTOl8jx/Xp8uCLZzIZTiIR6uhbqBOggpUQ+NrNpl37KLr1o5qygYC3j3ES85evlvJ7jw8u1Z78YnHYE6tC8b+lm5i35kDv5s/nr+f2dz1X1XP9XxXubDnqP2lt2INf8um8dTXt09Zt3xvicSM/5tzV26I2IlwZwWNWX+/NWl3rd8/X68c/avZ+Sjr0CshrDV/e43QkfuU1yOLPFwxg06593PTWbPW3FnFIciXCUTgZ+/tKOZBg9b1+26d5vLFF8y3uwFf4tW3wkxQFS1R9vf8ucC/JW10HHHJsPi7btqe8Xm/yj366iHdnrg6+oR+hdmvwfH09N9H/hDnr9X9I+w5j20DWbtvLXe/Ni+uo5rY95azfvi/4hgF84f5A9Y/JK3z+PcXy/tRZZtnjWJ8vCP5BT5JQTh4ceQOs/AaWT3Q6Gr96d2jGrScezBcLNvDKlJ+C30BEoi5lEuF/frsypH1UuEeksjIO3PXqk7y/ZO24J74G4MK/f8e7M0tDOo5rfyFvWkeg++qrj/Cn89ZR8sCXEcURqGYylHrKYJs89cUS/u61TLSnWHYSWLJ+Bwff+Qn/DZJI3/Xe3Fot7gKXpvivTVkTp5ZI4Y6sOj3WNP2nAyUbnhNUneDAoLQ4YfBl0LQQvrw3oethfnV4EUf3bM394xcwf812p8MRSTvJlQj7Ko1wv7/d9f68kPZR/XW2rxFhC2zfW16T+FbbsMN14v522SZ+/3rdmcg/b/a91G+kJ9xQuyF4jvT+4JFo1N0u2H4iu86f16b+XOeyLxZs8LFldFzxr+lc8o/vfV43f+129/EDj/z9c8pP/MtjRCZQcu45Wc7b/iDLREfrdFwZ5ok90b92jXZ0ge6v+giniawGMPxmWD0dFn/idDR+GWN49Oy+5DfM5ppxP7B7f3K2ohRJVkmVCEdDefWIcKbvROfbpZtYsmFnncsD9XucU7rN5+X1mZ0eUjLsoyuFz82CxBEwaQhy2zVb99Q6tsX3xLxYNh34dN56vgmyCEYowv3g4mtzfz2boznqbbF165P99hEOfqfWb98b8PUTLPL3Zq0Oq156pY92faH4ZO5alpcd+Ntct21vTccOb4HudoJ/JpBo6n8hFHSBCff5XiUoQbRo3ICnzuvPio27uDvEQR0RiY7kSoTrkUtU5yGVVf5rhAOdIPdH0Pg80lG4SHKmQIlI0FW96pE0PPVF7eWc/d1nY1yLQNzzwXx27K1dI5oonbk8H/dQunb4XIY6Tn9R9Z105rmq1aEPfBmwFjzYka5/bRZvzQi9ZOjyf06ve4wQ7s6Vr/7A0Y8f+Lbmwr9/x7XjZvrcNtCHUOXBaSQzG0b8EdbPhfnvOh1NQIcd1JKrR3TljemlvO/nA56IRF9SJcLRWFCjorK6Rtjn3vzebl95BIlw2LdwCZYI+5osF3hEOPD+Al0dbFTbYGpPCvSzucHw2tSfefF/K/hzHJvIe47Elm7ZzStTVvrd1nM0N2CNdnX7NB/3NdgqftbCtt3ltUY2w2Vt/Uevz/NaGa6+y71ucqD9U6CYA76mVSScXnqfBa17uVabq0zssoPfHdudgZ3yuf2dH1nlp+RORKIruRLhKHy9XJ3YhdM1AmBpBIlL5KURJuzJct6H+r8PDny9FqwmMpqjZ/6OZcyBiYrVH0bC5d0GK7y44OIXvufO9+b5XXTFU6R9nIMlwnvKK+l3z2e1RjYj4f2c+Xvc/T3SG+rZBSLaol23q0UzpEZGBoy8HTYthTmvOR1NQNmZGTx9/gAwcO24mZRrCWaRmAspETbGjDLGLDLGLDXG3BpguyHGmEpjzNnRC9Fj//W4bZV11fmWu5OwnzbV/bRtrf+RwHPGTvF9RQCxPhd7Jg/eJ/6X/rfS47og+wlwfZUNfL1r+WaP7cN4394UZveAUEaSyyur/JZnbHW37arv81LTPs1XaUSQF+mjny6q38Hdx/csjQipBMdrk1RPFFUjLLX0PBnaD4CJD0NFYn0I9NaxoBEPndmXWau28vhni4PfQETqJWgibIzJBP4CnAj0Ai4wxvTys93DwKfRDvLAMepe5mskacHaui1oPpi9huOfmlSTQCxctyPq8XmLuGtEiBn/5BAniAWdLOfjsomLNlTfOLRg3PwlWN736a0ZpQy67wt+LA19sYZg96O8soput3/Mw5/4TjaD3ZVQvySojiOchybaHeI8u0aEUi9cn9wvHjXc0U5OAybCqhJOP8bA0X+CbT/DD684HU1QJ/dtxwUlHRn79TK+WVLmdDgiKS2UEeESYKm1drm1dj/wGnCaj+2uBd4GYtcnK0T+VhtbXrYrrpNo6tOyKpTk46a35oR0rKDt03zctnqp3mA5ljGBSzT8mex+c1+yIfQPJMF2Xb1M6VszVh2Iz8d2/pNSzxrhAO3TAuwnXimWZ51rKK3UvJ/jVEgFA9bFB7hWJcJpqusx0OkwmPQY7E/8+ts7TzmEg1o35g9vzHa897ZIKgslEe4ArPL4vdR9WQ1jTAfgDGBs9EKrK9SRqeq6Ks8FEqrF8ytTz6S7ylru/XB+VPYbbgJWn64RVdYGGUGrHYy/DxreF1cnI5kZhj3loa38trxsF0W3fuT3+uoVyzbu3M/3yzf5vd6fcLtGBDN95WZmr9oa2sZh8J4s5yueYImvE0sdB3Li099ErX9qRWVVTT26L4neU1lixBg45g7YuQ6mveB0NEE1zMnk2QsHsG1POTe8MVuTPEViJJREOJT2Ck8Bt1hrA2Y0xpgxxpjpxpjpZWXhf93ja5TO3znti/nr6fGnuk3Ub3iz7oIY1a5/zXcrpkh5xrZ0w07+EeJyxf/5/mcWrfc/UurrPgcc6Q5aGhHotgFvWrOHA3GEtp/qUUzjXWRcDzv3HUikrvFqq7XdIwn216PZ+P3Fmyvgch+T/jzv59ljp9R8OxFKX+g/vDGLols/CmmCjGciW1ll6zy+3hMqfdXE+xLDRf6C6nXnp/zopye3L75Crais4qDbP+bcADX9ng9VqIvXSIrofJhrZHjyk7A38Vdx69m2KXecfDBfLy4L+fwhIuEJJREuBTp6/F4IeDc5HAy8ZoxZCZwNPGeMOd17R9ba5621g621g1u1ahV2sKE2PLMWvloUfoXGwnU7eDuMfqjBBBu1C+TTeYFXQvMWuCYysEAT3MKdVOV3RNgriurkPDOqC0147t+VFN353lwAdu8PPupce0Q4+MpyvmOIPKt/5wdXV4xut3/M14sDfVC0tR5nX6URyTp29PXi0P9u9/lYxe/mt13lQoHmANQq5UnaR0oidvSfYM9m+O6vTkcSkouHduaEQ9rwyKcLmVO61elwRFJOKInwNKCbMabYGJMDnA+877mBtbbYWltkrS0C3gKuttb+N9rB+spN/I12Rnp6+2x+eAloIPGcmf/v7+sua1wtWBj1WgGvTo1waPuqHk3NiN6AcB3fLN3Ilt2ukeBapQB+clzP5DfgKHlUogtsYpAPcrW6RqjDUo3qDxOBqDQizXUYCD1PgSnPwu7NTkcTlDGGh8/qS8vGDbhu3Mxa33yJSP0FTYSttRXANbi6QSwA3rDWzjPGXGmMuTLWAQbj76v4N6at8n1FPbVsnBPytp4n3J8dbI5en0Q3/BFh35d77mby0jI+d3/g2L2/khURLrkb6BiuCw78GFpnhQPbBBoRDlyG4vvycAe+g9WyV3gM41fauml7TWeLMNN2p3PEaC5F7c9mj8U/VBqRpo7+E+zbAf972ulIQpLfKIenzx/Az5t3c8d/5zodjkhKCamPsLV2vLW2u7W2q7X2fvdlY621dSbHWWsvtda+Fe1AAbbv8fVJ2PeZO9Bkmfro0rJxyNt6hrB2216/241+dnJYs4L3V1axcF3o9W2xXGLZ4NVHOIQR+sXrDyxOUv1VdnR4TxA78Hs4jxfArFVb/F4XqJTE173/auGGmo4W0bJz34FSD19JvsY8/Tvv+e9qflZpRJpqfTD0OQe+/xvsiN63gLFUUlzAdcd0492Zq6NawieS7pJqZTlf/YHjPYLlfeIMNLkn1JPsnNJtvPND6G9sO/ZWMOqpb0Le3l8ca7ftYfaqrQFHOIMusew1oOb384efy6PZvWBZWe2RZc/QfU1s8+a5vfe+PAV7TD6bt65Wd4vLXp7GOzPDWxUv2Nf3uzy+HvW1baR/F5EOyMZzkl3g+mmREI24FSr3wzePOx1JyK49uhslxQXc8d7cei3TLiIHJFUinJNVN9x4j+dUWVd7tjVb9wDw9JdL/G776nf+63bjyV9SNOzBCZz2l/8FbjUV5v7912zH/pmq/UGpbieFYELNyQOPoFven+09lzR8gUKxwM69BxLhUPoIp5JfvjiVRVFaEEelEWmsRVcYcDHMeAm2xqaULtoyMwxPndef7MwMrnttps8WoSISnqRKhLN8LP0V796K5ZVV3PnePA57aELQvrTheD1GNc0QfHQwYI2uhUf8rNQG8MHstbVGdZ1cujfkQ/vdLrQdRFIjHHSfXq/jYPvZ79FizXdpRBSfhwTMFbfvjd7fnqSx4Te7/v/6YWfjCEP7/IY8enZf5q7eHvC9WURCk1SJcHZm3XAf/TS+bwRzSrcxbqprpHdXFGfvBvoqvr7qkxQFu+W2PeW1+lvuLfddQLu3vKrWJKVY8ByNttZ/7C/+bwVD7v8i4uPEYgT2xrdq97cO3NvZ1kqcq6p8LKBh4d2ZpVSEUBISDQ99vJAB93xW7/28MmVl/YMRCVWzQhh8Ocz6D2xa5nQ0ITv+kLb8Ylhn/jF5BV8tdHwxV5GkllSJcIaPaKf/5H9SU6w5ufhAOGI9SLthx4GJgE98vtjnNj+u3sZzE2N7opm7pnYNub8yjae/XELZjrqTE0N9nGLRfiuUtl81xyf4BMX3Z6/h96/P5rUYftPgrbpVXX2s3x7apNE0qwaRWDryD5DVAL56wOlIwvLHkw6mZ9sm3PDmbDZs9z8ZW0QCS6pEOIQFt+IqWeoL61OuEErSlyhJyQyPD0Wbdu0Pe5T9yxBHVgJV40TrsQin04evEepIR9+376lgyjKv5akT5Pmtj7Xb9rBjbznDH/2q1uVv/1DK4gCrOEoaaNwaDr0S5r4N6+c5HU3IcrNdSzDv3l/B79+YpSWYRSKUVImwGuFHJlat5Ko5WRccyMOfLIzJfqPZ6cKfYEfwLJ2oqvLVRziy4779QykX/P07Pp23ruaysjBa+70+7WeKbv0ormUwoRj24AT63P2Zz6Wm/xNgMRpJE4dfBw2awoT7nY4kLAe1bsLdpx7C/5Zu4q9fJ09ph0giSapEOB4JSDhcpRGJFZMv54ydEvFtw+0akQ4CJf4bduzlwzlr632McEaEff1Z1Lds54p/zaj5+frXZoV8u+pOKascXEAmXMlS4iQx1LA5HHYtLPoISmcE3z6BnDekIyf3bccTny/mh5+dKxUUSVZJlQj7S0AC9fKNJZ0/XdIsDw6YpAZqpxfmUQIe33NEtLKqbqs4pz+cWGBHDDs7RPPuJUuJk8TY0CuhUQuYcK/TkYTFGMMDZ/ShXbNcrhs3M6rdjETSQZIlwr4vP/XZyfENpJbUPomGklClW8nKpDgs6BCsV7Hn1RMX161tdnrFtPs/mk+fu+vfRUIkbho0gSP+AMu/gpVOnlPC16xhNk+fP4C12/byx3d/TLv3ZJH6SLJEOMH+uJOkNCLWEu1pibW/TVru9zonJstNX5k4X4fOdy9qMi3GMUXzNafSCKkx5HJo0g6+vDfp3tgGdW7OH47rzkdz1sa0L71IqlEiLAGFMrLo9OhjIonWYxGwj7DX9Yn0d5FodfwiYcluCEfdCKu+g6WR9xp3ylXDu3L4QS24+4N5LN2gbigioUiuRDjB2qcp/3NJoDwsLLH4+jBeI8Ke+aaSz/rRgLDUMuAXkN/JVSucZG9uGRmGJ8/tT6OcLK75z0z2lmsJZpFgkioRTrQTfmJFExsh1QjH4LhL4tDbNRbnuKglwmEcx9eIcJKdv8MWzW8hVBohtWTlwIjbYO1sWPCB09GErXXTXB4/px8L1+3ggfELnA5HJOElVSKcSF8BQ+onG0467slJMT9GLJ6++JWJ1F5iOVFkxCup1N+exFLf86Bld/jqfqhKvlHVkT1bc/kRxbwy5adaPcFFpK4kS4SdjqC2N6avYtaqrU6HEVOlW/YE3SbRPqCEauiDX0Z9n3EpjbC1lyL2tbKcU+L1NxrV9mkaEhZvGZkw8o9QthB+fMvpaCJy86ge9O7QlJvfmsOarcHfx0XSVVIlwonWEuaJzxezcWdsV9BKCon1tISsbEfoK6aFKpIPRhu2761z2Zbd/l9XO/ZVMMFjOWhffxcVlUn6pIRo7ba6j5lIVB18GrTtAxMfgMrk683bICuTP18wkPLKKn73+qyEKy0USRRJlQgn0siXHKBn5YB9FeHXKZQ8UHdk2jPRDcbXCe7JLxaHHUcyufHN2VHbl8aDxaeMDDj6DtiyEma+6nQ0ESlumce9p/Vm6orN/HlCtBb7EUktyZUI6xNtQkq0kfp0k+KDv7GnTFj86XY8FJbApEehPDm/hThrUCFnDOjAM18u4fvlm5wORyThJFUirHwrMelpcVaVPiCKxIYxcMwdsH01TH/R6Wgidu/pvelU0IjfvT6LrQHKrkTSUVIlwsk6KSvV6WmJjbvfnxfSdvqmpH6MhoQlkOKjoHg4fPM47NvpdDQRadwgiz9fMJCNO/dx81tz9C2eiIekSoR1wk9MP2/e7XQIKenlb1eGtJ3+LkRi7Jg7YfdG+H6s05FErE9hM24+oSefzV/Pq9/95HQ4IgkjqRJhjQiL1LW3Ivn6nCYSdU+ToAoHQ/cT4dtnYM9Wp6OJ2OVHFDO8eyvu/WgBC9ZudzockYSQZImw0xGIJJ595Qm0ooZIqjr6dti7Db79s9ORRCwjw/D4uf1o1jCba8fNZM9+fYgWSbJEWJlwfWXFbekviZd9GhGuF/1FSEja9oFDzoTv/go7y5yOJmItGzfgyXP7s6xsJ/d8GNo8BJFUllSJsPLg+stUIpxyIuldLAeoNEJCNvKPULEHJj/pdCT1ckS3llxxVFfGTV3FR3PWOh2OiKOSKhGW+tOIcOrZW64RYanNGDPKGLPIGLPUGHOrn21GGGNmGWPmGWO+jneMSallN+h3IUx7Abatdjqaernh+O7075jPre/MYZUmPEsaUyIchpaNc5wOod5yszOdDkGiTLXz9ZNq7dOMMZnAX4ATgV7ABcaYXl7b5APPAaOttYcA58Q7zqQ1/GawVa5FNpJYdmYGf75gAFi47rWZlFfqmyVJT0qE04wSYZGUVwIstdYut9buB14DTvPa5kLgHWvtzwDW2tDX9E53zTvDoEth5r9g8wqno6mXjgWNeODMPsz8eStPpfiy7CL+KBFOM6u37nE6BJGEkoI1wh2AVR6/l7ov89QdaG6MmWiMmWGM+UXcoksFR94AGVkw8SGnI6m3U/u159zBhTw3cRnfLt3odDgicZdUiXDPtk3idqxUKIMQkeBSLw/2eZe8C2iygEHAycAJwB3GmO51dmTMGGPMdGPM9LKy5O2UEHVN20HJb2DO67DiG6ejqbe7Rx9Cl5Z5/O71WWzauc/pcETiKqkS4dfHDHP0+CYFh45EJOWUAh09fi8E1vjY5hNr7S5r7UZgEtDPe0fW2uettYOttYNbtWoVs4CT0hF/cJVJvDIavrgbKpI3gWyU41qCeeuecm7SEsySZpIqEW7WKDuOR6ub9KrjgkgKSr0PuNOAbsaYYmNMDnA+8L7XNu8BRxpjsowxjYBDgQVxjjO5NSqAKyfDgItd7dT+fjSsT96+vL3aN+X2kw5mwsINvPi/lU6HIxI3SZUIO6lDfkOyMlPuhCkiKcZaWwFcA3yKK7l9w1o7zxhzpTHmSvc2C4BPgDnAVOAFa+1cp2JOWg2awOg/wwWvw84N8PwImPwUVCVnS8NfDOvMsQe34aGPFzB39TanwxGJCyXCfngPEhkDWRl6uFJd7w5NnQ5B4iwVP95aa8dba7tba7taa+93XzbWWjvWY5tHrbW9rLW9rbVPORZsKugxCq6eAt1PgC/ugpdPTsqOEsYYHj27Ly3yGnDtuJns2lfhdEgiMafMLkTGaFW2dFClVpppJ/UqI8QReS3h3H/BGc/D+vnw18NhxstJtyRq87wcnjq/Pz9t2sWv/zmdZWU7nQ5JJKaUCPvhnfMajGqE00BVkp20RCSBGAP9zoOrv4XCQfDB9fCf82DHeqcjC8vQLi148Mw+zF29jROenMQ9H8xn2+5yp8MSiQklwn54l0FoRDg9KA9OP6m2spwkgGaFcMl7MOphWPE1PDcU5v3X6ajCct6QTnx10wjOGdyRl75dwYjHvuJfU1ZSoRXoJMUoEfYj22tinEFdI9KBRoRFJCoyMmDolXDFN642a2/+Et4ZA3u2Oh1ZyFo2bsCDZ/bho2uPpEfbJtzx3jxOeuYbvlmintKSOpIuEZ56+zFxOU6GV9JrjCErM+keLgmTEuH0oxphialW3eHyz2HEbfDjW/DXw2DZV05HFZZe7Zsy7jdDGXvxIPaWV3HJP6by639OY7nqhyUFJF1mlxmns5b36K8BHj6rT1yOLc5RGiwiUZeZDSNuhV9/ATl58K/TYfzNsH+305GFzBjDqN5t+ez3R3HLqJ5MWbaJE56axH0fzmfbHtUPS/JKukQ4I26JsNdDY+Cg1vFb4tmXLq3yHD1+OtCAcPrRgLDETYeBcMUkOPQqmPo3+NtRUDrD6ajCkpudyVUjuvLVTSM4c0Ah//jfCkY+NpFXv/uJyiq9gUrySbpEOF5fY3ovnqGTZXpIttKI1k0aOB1C0lNphMRVdkM48SH4xftQvgf+cRx89QBUJteoausmuTx8dl8+uOYIDmrVmD/9dy4nP/MN3y7d6HRoImFJvkTYIyU9pmfrmB3Hu0OE0dkypgZ1bu50CEDyJcLx+oYkWX18/ZHcdmJPp8MQqavLcFebtb7nwtcPwwvHQtkip6MKW+8OzXj9iqE8d9FAdu6r4MIXvmfMK9NZuXGX06GJhCT5EmGPiEf3bx+z48z8eWvt47r/v/H47pxwSJuYHdfTuYMLa/3uVI5WkJcT82M0b5Qd82OEItkW1FAeHFhOVkbQDwv6kCuOyW0GZ4x1LcSxbRWMPRKmPJd0b0TGGE7q044v/jCcm07oweSlGznuya95cPwCtu9NrpFuST/Jlwg7dVz3ga85uhsXD+3sSAxO1V99+rujwtq+uGX4tcyJMrK5e39yLSmaKI9bospRpxdJBr1Gw1VToOvR8Olt8Mpo2LrK6ajClpudyW9HHsTEG0dwev8OPP/Nco5+bCLjpv6s+mFJWEl3lvA88cdqhNRXIudZkuFUA36n3kjCbZ983pCOYR8jURYr2b43uRLh1Vv3OB1CTL191WH1un22EmFJFk3awAXjYPSzsGamq83arP8k5Qze1k1zefScfrz/2yMoapHHbe/8yCl/nsyUZZucDk2kjqQ7S8R6AOywri346sYRAY/rK4bubRrHLig3p+pX4/HVsXffZqdo1CK2jj04vLp+74VtwpWdaYK+Z2hQXRKGMTDwErjqf9CmN/z3Knj9YtiVnBPQ+hQ2480rh/HshQPYvqecC/7+HVf+awY/b0qetnGS+pIvEfYYjbVx7PrqWY/s67wZj6+oHUuE43CMePWHTlQHtY79B6m+hc1ifoxgerZtGtb29f32JScr6d7iRKB5EVz6IRx/Hyz5zLVE88LxTkcVEWMMp/Rtz5c3DOeG47rz9eIyjn3iax76eCE7VD8sCSDpzhKxzpd85Zpf/GE4Vw3v6hFE3W3CGTUddUjbCCIDp5Z4j0eSH+6A8PvXHB6bQBzSqnHs26Bd6fkadki4H17r+9LLzszg8INaBj6GmiNKIsrIhMOuhTFfQ5O28NoF8N5vYe92pyOLSG52Jtce042JN43glH7tGPv1MkY+9jWvT1P9sDgrqRPhWAyQ+jpRN8zJrJXo+qo7DKcUsaS4IKLYKqMwk/iCkk7h3yjMPCGStOKcwaHVFffrmM83N4+kb2F+BEdJXJHUVYcr2VrDhaKwecOA12dnZnBwu6YJ05VEJGxtesGvJ8CRN7pqhsceDisnOx1VxNo0zeWJc/vz3m8Pp3OLRtzy9o+MfnYyU1dsdjo0SVNJlwhHc3Ry+p+ODWk77yP6SoTjMaoUjU/NwRKHa0YeVOeyWJfvlhQVBB2184ylY0Gj2AZUD90iKHG4ZGjnuHyFn4yDLsH+3oPl9qFMwkzzqhxJBlk5cMwd8KtPISMLXj4FPr0dyvc6HVnE+nXM560rh/H0+f3ZvGs/5/5tCr/99w+s2qz6YYmvpEuEPc9Z9R3gCrUu1fswWT5Orv7Ot4d1bVHnskhPvG2a5kZ2Qx9+OawzvzmyuM7lJ/ZpyxkDOtSqJ410sly7ZqHF672adSDBImmSmxX6zmKgZQQlDvGqdbcJMCIcbgjxSFKVB0vS6FgCV06Gwb+CKc/C88Nh7Wyno4qYMYbT+ndgwg0j+P2x3fly4XqOeeJrHv10Ibv2JVcHH0leyZcIR+nMmJvtu9F+KCdq7+WXwX9cvxhWt+dwKPfgrSuHkemVIYbbn/ekPnVrkavDzM3JJMvHyLa18OR5/Xn/miNCjjcvJ9Pn5af0bRdSnNEc5U/WpCYecSdAHhy2SF4aF5TEvsxExDE5eXDKE3Dx27B3G/z9aJj0KFQmb+LYMCeT64/txlc3juCk3m35y1fLGPHYRN6cvoqqZPwqS5JK0iXCniOvodQ8tmpSd4Tug2uO4OubRvocifS1R+9zcZaPG4ZTPhBKMp+bnckto3qEvtNQj119b2zoiVGsJ8tFs4dwoMc2ktHacEUyuhuorKZjQeBSlnDEs8uKL74+FAYTasmR54e+B8/sW3c/AV4XKo2QpHTQsXDVt9DrNJhwH7w0CjYtczqqemnXrCFPnT+Ad64+jA75DbnprTmc9pf/MW2l6ocldpIuEfY8oYVSM/v9bcfUuaxPYTPaNM31neCFMiLsszTC39k08rNsfqPwljaedvuxdUbDrj+mG0+e149Hz+7L3af2qjnpW0JPjIL3YT2wwTtXh78AQjz6FH98/ZGcH4cJaZEI9DzkZvkebY9EKHMt7z+jd9SO5+3qEQfRIMz74/2n9vtju0cxIpEk16gAzn4RzvoHbFwCY4+AqX9Pzq9/PAzs1Jx3rjqMp87rT9mOfZwzdgrX/OcHSreofliiL+kSYU/lISTCgRZq8DUS6Ssp8c7TfN3OfyIcvzekVk0a1Inj98d154wBhZwzuCOXHl5ck5Zba0N+rwwnTx3YqXnYI2yRDgh/eO0RwTeKM38jmL4mIVYLNOIezU4PoeyrvCJ2PfqMgSuGd6l12dAugTuoeD80/kaV6/MwaZlqSXp9zoarv4POh8H4G+HVM2H7GqejqpeMDMPpAzow4cbhXHdMNz6fv55jHv+axz9bpPphiaqkToQr69lY19cJcOPO/UFvV9i8IZcf4TXRLKzSiNC3Dfd2nvmAr6SsZkTY1p481aud/4UOgn09/fJlQ4IHFsDFh4b/lTlA7w51F4jw9xgZE3jkNdR65mAK8nyP4p/Srx0DO+X7vC7QoxvNj1Gh7Ku8svZWLRuH961EIMa4Sn4CqbuwSO1Hx/tvNtIJgO09JnLG4xsJkZhr2g4uegtOfgJ+/s61CMePbzkdVb01ysniD8d1Z8KNIzjhkLb8ecJSjn58Im/PKFX9sERFUifCFfX8I/A1EllSFLzHrzGGO07pFXRf7q1DuCR8//71oREl1NVJraX2KJpnglznNkGOMziExyyQo3uGt+xuIJE+to+f2y8qx3/sHP/78VcLHTARi+L7fChJ436vD5f3nd4nasf39YGqubv8p1nDbP7+i8Ece3Cb2rfxvomfhyrcfDjXY4JngqzuLVJ/xsCQy12dJVr2gLcvhzcvg93JX2PbIb8hz1wwgLevGkbbprnc8OZszvjrt8z4aYvToUmSS9pEuHWTBvXuq+srMcnPq9t4P5QJO74m0MVS/475HOpjYY5gCUGtEeEQjxXrr44TYUAuGvfx/CEdaeing4bBBEiE/e8zklf4MX4+WHj+ueRm+3697vMqjfB3fyLh636eM7iQp8/vz7Tbj+W4Xm3qbOP9vEQrafXcbwK8/ESiq0VXuOxjOOZOWPABPDcMlnzudFRRMahzAe9efTiPn9OPtVv3cNZfv+W6cTNZs3WP06FJkkrKRPily4bw/jVH1HtE2NdInK+JcKHkSJ7bdG8TZFGFKCRdlR4Z7y+Hdea6Y7qFdXuLDbn+NNJoQ69Bdh3h+F5tgmwZ+r68BZso5pkY+WsHF64nz6s9OuxrIRZwJcn+XhLR7P07uHPzWsf0pdxrRDiax/c+4hPn9mNkj9ac1r9DzYIigQaAx193pN/nN9jEz0D7DTSPQCRpZWbBkTfAbya4JtX9+2z48Pewb6fTkdVbRobhrEGFfHXjCK49+iA+nbeOox+fyJOfL2bP/kqnw5Mkk5SJ8MgerWnbLJeKyujXB/laZKO1jxZs3jxP0ON+M5SmARZ2iMZpt5FHreUJh7TlD8dVz6YPkhC447TWd2mE79tEGmV4wk3mffE/ZTHw4+KZC/149wl+t2uUk8ndp/bye73fuIz/UecM4/9DQySvcH+3CdaH+rmLBnJi77q9p6PF+/6fObAwaH2u59W92jetMyIc6TuA535VIywprV1f+M1XcNh1MP0lV2eJn793OqqoyGuQxQ3H9+DLG4Zz7MFtePrLJRz9+ET+O3N1QiwgJMkhKRPhapWh9IMKk/fo0JXDu4Z0ovTcokXjBgztUntFOV8jzYF0blF3GWHPUbyszIyIlnUOdgvfXTMSJ1EIJ3Hy5J34B9pvoBHC+feM4tLDi/1eX2ufHo+2wf9rINBdimbXCM/76OuYDXMy6VuYH7Xj1T1+8G28HyPv17i/13zYK9Z57EcDwpLysnPh+HvhsvFgK109h7/4P6gIPjk8GRQ2b8SzFw7kzSuH0bJxA373+izO/Ou3zPxZ9cMSXFInws3C7LML8NqYoQGvD3XZZW+Bbvb9H49h+p+ODWt/TXLr1ir745kDBEsIqk/6VdbW+sQcLKmecMNw3r5qWEjxRJKgp5IPrz2Ct686rM6HCn81wvWpT37lVyUhb2v8/Fwt0td+6McPvv9cr7IU75CiFaLnftQ+TdJG58Nci3AMuBgmP+FalW79PKejipohRQW899vDefTsvpRu2cMZz33L71+fxbpte50OTRJYUifCvxzWmUfP7stZAwtDvo2/9lbVMr2WT450NS7PW7VpmltrcQzv826On9rRYAJ1evB/G4/SiDCO1aVVYwZ1rl93iGiI9OuucO9vMIFKCHp3aMYgj3pccD1Xvtq9ua70n+AFW1DjqO6t6lwW6WPkc8nxiPbkmwnhZd4wO7xEOBoD5hoRlrTSoAmM/jNc8DrsXA/Pj4D/PQ1VqVFbm5FhOGdwR766cQRXj+jKRz+uZeRjE3n6iyWqHxafkjoRzsrM4JzBHcMaJSpsHnjJ2jpfX4c64Sv0EOqMjL12xVC6tvJdv3nLqJ7+9+PjoCF3jQhjslxyiWwyVbjCXxba1FlM4sA1/vfVKEZdG3yVmLRpGtslqEN5xILdX3+jt+E+u7XKRNL8GwxJUz1GwdVToPsJ8Pmd8PLJsHmF01FFTeMGWdw8qidf/mE4I3u24skvFnPM4xN5f/Ya1Q9LLUmdCIcrM8PQKMf/JDbw0bA/wLY92zbxe12gU6v3ubxV4wYc2a3uyB7Awe38HyMS1Ye2Fnq2rbuIRrjvD+cNjt6yxb5ynJE9aj8u7fJ9f5Bp5Z7QGE6NcCiTIP0JpW7a+3j+kjjjY7Jcibs1XqAFKPy1SfP3FHrGnNeg9n7HX3ck3dr4eK1F8XwRymMWbMGNaKWsSn1FgLyWcO6/4IznYf1810S6Gf9M+iWaPXUsaMRzFw3itTFDyW+Uw3XjZnL22CmqH5YaKZEIR+Nv9nH3Qgg9AiS33h492//iCfF8G/Ec7Qw68lldGgFcdGgn74vDjvuBM/0vuBCNx+BXXiv4PeTneB9ffyTvX3N4nQSneqUyX7FUJ9X3nd67vmEGZYz/5CvQ4HKgTg93jz4k4nheGzOMG4/vXvN7r/a+VxYM9npqEqA7irdQks8GWYHfkup+UHXFF/ZkOWXCIi7GQL/z4OpvocNA+OA6GHc+7FjvdGRRNbRLCz649ggePqsPP23axRnPfcvJz3zDi5NXsGnnPqfDEwelRiIcYsoV6OuQMwd24MsbhtcZmQ10m1BaMIVywvXXXzaYjs1dnSXCmViX6040crMyvb4ejkyoJQL9O+bX+n3lQyfXWubWk+dIu8Hw/jWH1/zu7762bNzAZ8eDvAauRC3Q83iInyQwkPALI/zLMD76CLvDHdApn/d+ezg9fIzW1ieZK26Zx5ijuka+A7fHA6yk5y2USWneC9N4/13VqRkO8ZmI1aQ7kZTRrBAueQ9GPQzLJ7qWaF7wodNRRVVmhuG8IZ346sYR3H1qLzKM4Z4P53PoA1/ym1em88ncdeyviH43KklsoQ/nJLIgefADZ/Thj+/+GHAbYwxdW9VdCCPQSFN9F/Q4sltLRvdrT9tmuRHVLN116iEM796qVpIZbDenD+jAqi17GHOU73rVWHltzFB63vFJrcveu+YIftq0q+Z3X0mNMYTV0qtuouTi86mqx1cJoSRS3rv3+2EpyD76dcz32dLN7+IScfw6okGQUgZPoTxm3pNV6+7D32TW8O6052st2vXjIkkrIwOGXgldj4Z3x8DrF8Hh18PRd7oW6EgRTXKzufTwYi49vJhF63bw9g+lvDtzNZ/PX0/zRtmc1r8DZw0spHeHpgnVPlRiI6ShSGPMKGPMImPMUmPMrT6uP80YM8cYM8sYM90Yc0T0Q43cGQM6RHzbQKdIzz7G4U2Wc2nZuAHnuGts/R3H1mybUyeRaJiTyYl92vnc3p/szAz+cFx3GjcI/qbWwU89bjC+3jd81X62atKAwUWBO1FU72r2nceH1IKuTt/Zml+jm+z4er4jTkAjbdkX4eHqedig/v6LwZw72NXJpVe78EbbQ+m3veyBkyKKy5PObSIBtOoOv/oUBl/u6ijxr9NhZ5nTUcVEj7ZN+ONJBzPl1qN56dIhHHZQS/4z9WdOfXYyJzw1iecnLWPDdrVfS2VBsyFjTCbwF+A4oBSYZox531o732OzL4H3rbXWGNMXeAPw3+4gRXiWNIRzYq0e3fO8TbAk6pD2flpvRZnnyPR71xxO6ZbI12+P5shks0ahl3948pwcGM2Rv3BHCSz+E1efSbVXrL6+MahvB4VQ7kEkz2FeTiYtGrsmIp7ctx3z124HQiuNCGUbz3Kc6t7H4dcIe4wIa0BYpK6sBnDKE1A4BD78HfztKDj3n9Ax9N7lySQrM4ORPVszsmdrtu0u54M5a3j7h1IeGL+Qhz5eyFHdW3H2oEKOPbhN0Em9klxCGREuAZZaa5dba/cDrwGneW5grd1pD5yp84jvXLGag3nXoVarz+hPoJNknw7NGO6jj2swJ/dpxyVDO3P7SQdHHliU9XM/ds09+h23bNzA72OaqLyf6+rEKtALMpKvvqI5oBgo+Qsct//rnrlggM/L8xtlc1cYS0QHbccXxu1iMQob6deWGhAWCVH/C+DXX0BWDrx0Ekz9e8p/emzWKJuLh3bm3asP58sbhnPl8K4sWreDa/4zk5L7v+CP7/7IDz9vURu2FBFKItwBWOXxe6n7slqMMWcYYxYCHwG/ik54oal+MUazMf4LvxgMwFmD/JdVGGNqdV4IVU5WBvee3rtm1Az8j1YOKSqga6s8bjy+R9jHCcefTu7Fh9ceQVGALgXhMgaGd2/lc6JX6DsJfLV3CzTvzWstOuL1ENfrLcxHXN51vJ77N/hPBE/p1873FeGHUGN0v/Y+L5915/Fc5l4iOhq1b8FGcGtNKI1g/8FuE2m3E5VGiIShbR8YMxEOOgbG3wjvXgH7dwW9WSro2qoxN4/qyeRbjubVyw/l6J6teeeHUs587luOefxr/vLVUtZsjfybU3FeKNXvvr+59b7A2neBd40xRwH3AnUKOo0xY4AxAJ06hZ9AxtOxvdqw8qGTg25X3XbqjAGFfLFgQ83lgT4ohlNf2rhBFl/eMCJoHKEcN5CcrAz/K59FyFr4ZzhLAPt4YIJ1BZh088ggj3X1V+d1N7rntN7c88G8mi4VV43oymCvFeHA9dh4zyT2Fdcto/x/WPEXYvVrbMn6HbW3D+V5DCOZ89WhI6TSiCDXh7OwSCwmnVQfP/jIjPH7m8Z0RELQsDmcPw6+eRy+uh/WzYXz/gUt6t99JhlkZhiO6NaSI7q1ZMfecj7+cR1vzSjl0U8X8dhnizi8a0vOGtSBUYe0o2EUF0KS2AslES4FPFdNKATW+NvYWjvJGNPVGNPSWrvR67rngecBBg8eHLXzT/WOgrUwi8UJr7B5o5pkpmnDEn7atDvobXzFqZNxbX06NOPH1dvIyQqcPHnXatV5bN2/VvlYYrl/x3zeufpAa7ZAq/iFwnMZbV+inQiGUk8bq2NXy/bR5cHfazkWSxmH8xh40kxwkQhkZMDwm1z9ht++3LU88xljoWfwQaNU0iQ3m3OHdOTcIR35edNu3v6hlLd/KOX3r8/mjgbzOKlPW84aWEhJcYHea5JAKInwNKCbMaYYWA2cD1zouYEx5iBgmXuy3EAgB9gU7WD9qR4M8j8ZKT4vxCO7teLIbu5jxqQeMvg2qdIK6p+/KuGN6asY2KnuCG0oHj27L4OLCrj17TlAPR+XCOtdA41SfnjtEazcFPyrxXC/WQh23Fq3j8Jr1N+IsK/Hu/qEcMNx3VmzrX5fJV5/TDee/nJJ5KURQLtmuazdtjfVyx1Fou+gY+CKSfDGL+C1C+GIP8DRf4KM9BsJ7dSiEb8/rjvXH9ONqSs38/aMUj6as5Y3ppfSsaAhZw0s5KyBhXQsaOR0qOJH0Bpha20FcA3wKbAAeMNaO88Yc6Ux5kr3ZmcBc40xs3B1mDjPOlBF7u/E7sQHsmiVRqQja6EgL4crh3eN+NP00C4tKG6Zd+C5t+G9DibfMjLg9fV9SfXu0IxT+vqu4/UUKIGv70hDaMtEB35hBiuN8PUh9NpjuvHgmX2DHhv8P2fVS0RH+ggYA29cMSzCW4sI+Z3gsk9g0KUw+Ql49UzYtTHozVJVRoZhaJcWPHpOP6b96VieOLcfnQoa8fSXSzjyka84729TeGP6Knbuq3A6VPESUh9ha+14a213a21Xa+397svGWmvHun9+2Fp7iLW2v7V2mLV2ciyD9taznavGs32EfW9jydeJ2vfJPUqZsPcMLQeM6OHqpBHpJLD6OLSLqy9x9Ypyni3ufjvyIC4o6ejzdt4KmzeiiXsfvxjWuc71Tn3bNf66Iw/EEIfjBa+89R1FK/dE0BZ5gctFvHVrU3dRG1+q8/OariDW9diMvXhgyMfKca+ymCrfoojEXXYunPo0nPYX+GkK/G04lM5wOirHNcrJ4syBhfz710OZfMvR3Hh8dzbs2MfNb81hyH1f8PvXZ/G/pRupqueiXBIdKbHE8pVHdeW/vz3c7+IMnr1kvb1z9WF8+rujYhecr3h81Qin0N/DQa2bsPKhkyMua6iPB8/sw2e/P4oCdwL26Nn9+NXhxRzapQVNc7NDHomEA8n05UcW17kulHKbSJ7SvJxMOhU0olML19do1a+LmvIfj8N618e28uqgEQsHey2Q4ZlEFnt0HLns8GKePr8/Zw8qDGv/rZvkcuzBrYNuV33+8ByR7tW+KaN6h/7hS5V7IlEy4GK4/DNXDfFLo2DaP1LrpFYPHfIbcs3R3Zhww3DevmoYpw/owBcL1nPRC99zxMMTePTThSwv2+l0mGktJRLhjAxD/475/muEAwzfDezUnB5t69HeywH3nd7b73XJ/NZTndRkB5kgF0iDrEy6e7Rra9sslztP7RVWd4NqgcsSXP9fPDS07ifBzwmuHR5+UEsm3TwytIbtXnfp8XP6+d80Slnfa2OG1hqV9tTOozNFZobhtP4dfC4NHQ1VYZ5kq+//TSfEtg2hSNpq3x/GfA3Fw+GjP8B/r4b9wSePpwtjDIM6F/DgmX2YdvuxPHPBALq1acJfJy7j6Me/5szn/se/v/+JbXvKnQ417aTO4uEBJMrIzxkDOvDuzNU+r6s+r+dmZ3DFUYHb0eSHuspaHLPi5y4ayM69odU+Vfdo9tatdWOuGXkQ5w0JrXwhXnyN/lYnVr3aRavlXPhPlr/kNpoDMd77atYwm2YNI1vlL5JjBht5D7dOurB53fIpDVyJREmjArjwDZj0CEx8CNb9COe9AgVdnI4soeRmZzK6X3tG92vP+u17eXfmat6eUcrt787l/z6Yz/G92nD2oEKO7NYqokEcCU9aJMJOGN69JV8sWE+XVge+Ln707L7cPfoQn9tXjz7efeohnF/ie5TxQE9c/8d1aqWbk/qE/pX0sb3a+LzcGMONCTRiF/ihrF6xLrYdGqr3Xtwyj0Xrd5CXc+BP1rs0ItAxYlXT7Osxqu9LMJSbey+iE9EhdX4Rib6MDBhxK3QYBG//2t1i7XnoMcrpyBJSm6a5XDm8K1cc1YUfV2/jrRmlvD97DR/OWUvrJg04Y0AHzhpUWOubTomutEiEq5OAQ9o3DbxhFF08tDMn921fU6sKrrXMmzX0XY3iqwbUW3WT7j3llVGLU/w70J+67nW1VqwLZV8hlkb4/g0eO7cf564orKkd9rVNzbHckR/UujE5mRnMX7s9Zi0EPT/oxVN1jXC4fYSrR7OLWhyIWwPCIjHQ7Ti44mt4/RIYdx4cdbMrQU7DFmuhMMbQtzCfvoX53H7ywXy1cANvzSjlhckr+Nuk5fQtbMZZAwsZ3a89zcOchCyBpVQi7H1O/PDaI9yXG966chgHtQ5tRnp0YjG1kuCQbxcgYWnkToR3B2i/opN6bEy7vfZCiZlxbhvRuEEWR/esPZJed+2Q2hd88YfhrNy4ixGPTazHkQO/oprkHiiTiNZDUj3aW1Lse/KraxvX/zUjwiF+IunZtikvXTqEYV1bsCPEUh4RiVDzItckuvE3usolVk+Hs/7hKqEQvxpkZTKqdztG9W7Hxp37eG/WGt6aUcpd78/jvo/mc0zPNpw1qJARPVrV6owkkUmpRNhb66YHZtH76yiRKEI5jVePCO8OcUQ4Hl0E0oX3Y3nj8T2ospazBxXyp//OdSQmfx+afOWEEZdmhPDC/Nslgyhs3pAHxi+I7CB+XDm8i9+4qyfLhTsZzxgY2dPVlUKJsEgcZDd0tVcrLHElxH8bDuf+07U6nQTVsnEDLj+imMuPKGb+mu28/UMp/525mk/mraNFXg6n9e/AWYM6cEj7aM1XST8pnQjHa0W5aKhJOAKEXOz+Ord5gKV8q/dz+EEtuPXE+i0ZHG2j+7WnZWPnk/O3rhzG/oqqkLf39ZQ0a5TN/Wf0AVydFGav2lp3o2gMzwfIROuMCMfg5R7Kh6kTDmkb1WOGVCPs/j/UuxxwO82WE4m9Qb+Etn1cq9G9eAKc9JjrMglZr/ZN6dW+F7ee2JOvF5Xx9g+lvPrdT7z4vxUc3K4pZw3swGn9O2gQLEwpnQgnk+q6zkAn7POGdKQgL4djD/Y92czTuYM7htaCqx6qSzVC9cwFA2IUSXhC/XYg1PxoaJcWDO3Soh4RwRHdWtKvY35Ne69QktrqbV75VYnf5Zrrk+K9PmZoVL5J6VvYjDml28K+XcAPsjWT5YJPIPW7/+T5nBw2Y8wo4GkgE3jBWvuQn+2GAN/hWg30rTiGKOmow0BXi7V3fg0fXAelU10JcXbiLYaVyLIzMzi2VxuO7dWGLbv288GcNbw9o5T7PlrAgx8vZET3VgwpLqBHmyb0aNuEds1y670SaSpL6UQ4GZ/3QC9WYwzHBxl9i9fY1oJ7RiXl4xue6hmMsT9S4wZZvPfbw8O6TXWieFT3VhxFK/631LW8qc/SiAhiOtSd3DfIymBfGCPo3sb9Zihbw+iNGUpSO7p/e56ZsJRT+7Vn+k9bIo4NUq+u3hiTiWup++OAUmCaMeZ9a+18H9s9DHwa/yglbeW1gIvegokPwqRHXS3Wzn3FVU8sYWuel8MvhhXxi2FFLFm/g7d+KOXD2Wv5cuGGmm2aNMiie9smdG/ThB5tGtO9bRN6tGlCiwT4hjYRpHYi7HQA4Uiys3HDMEeDk1mkJTa1W6tF9gQHulXdyXI+bh+Fr/0X3XciRbd+FPHt8xpk1azSF4oG7qWPA9X/Vq9e+M2SMiD0Nna1exSnrBJgqbV2OYAx5jXgNGC+13bXAm8DQ+IbnqS9jEw4+k/QYTC8O8ZVN3zWC65OExKxbm2acNuJB3PbiQezbXc5izfsYNG6HSxe7/r/47lrGTf1wKBEy8YN6NG2sTtBblKTLDcO4/06FaTUvU2mmmBv4dY8+t2P6h2jxsmHMpTXciitww60gEuev40HzuxD19aNOeKglmzauS/gtsn8Nx9DHYBVHr+XAod6bmCM6QCcARyNEmFxSo9RMGYivP4L+Pc5rvZqR93s6kUs9dKsUTZDigoY4lHeZq2lbMc+Fq33SJDX7+S1qatqtWXtkN+QHm1dZRU92riS466t82iQlZoDYCmVCCez6gQ2WvlKMiU+iS7Sh/Lgdp59q6P/fISzx3i+GkIdnfWnZeMG3DIqvIme3h9afrjjOCqqQivnSMHPjj6/HPD6/SngFmttZZByrDHAGIBOnUJbTlwkLAVdXC3WPrrBVS6xegac8Te1WIsBYwytm+bSumkuR3ZrVXN5VZWldMseFq0/MHq8eP0OvllSRnml660jM8NQ1KIRPdyjxj3d/3dukZf0q9+ldCKcjOe3+uavsb7Pb105jN3702NBj/o+ln0L82neKJstu8sj3lugJC3RPus4MTrr7zHw7uHte1GUBHsAo6cU8FynvBBY47XNYOA192PQEjjJGFNhrf2v50bW2ueB5wEGDx6cjG+pkgxyGsHpz0HHITD+Znh+OJz3KrTr53RkaSEjw9CpRSM6tWjEcR4rv5ZXVrFi465a5RXz12zn47nras5NDbIyOKh145rSiupR5GSaoJfSiXAyifYZJlYvv0TvxxwL9XksC/Jy3IlwmMcMqWuEnz7CSfkRsH6Cjeom4rLkMTQN6GaMKQZWA+cDF3puYK0trv7ZGPMy8KF3EiwSV8bA4F9B276uFmv/OB5OfhwGXOx0ZGkrOzOD7u7SCE979leyZEPt8opvl23inZmra7ZJpgl6KZUIN22YvHenZonl+qawKXdOd87fLhnE85OWB+zbHMyBmqoIJ9z5SNKa5Gb5XgzCxyGSPscL8rDV568lOcYqwmetrTDGXIOrG0Qm8KK1dp4x5kr39WMdDVAkkMLBcMUkeOtX8N5voXQajHoYsnOdjkzcGuZk1iwH7al6gt7CdTtYvG4Hi9bvYPyPiT9BL3kzRx9O7tOOnWdWcN9HC9i5ryKpTnQHJjVFZ3+h7uf1MUPZW4/WWKnMe6JBJP52ySDenL6Krq3ywrpd9bKZvmqvPrjmCL5fscnvbWsnv+G3gHvuooFs2b0/9Bu4dWrRCJZCU49ll+Ml2Ch4oL+HZP+s4Iu1djww3usynwmwtfbSeMQkErK8lnDJuzDhPpj8BKyd7Wqxlq869UQWygS96lHkRJqgl1KJsDGG80s68cini2Bfap7gggn3a/FD67kQhATWsaARfzi+R9i3+92x3QA4d0jHOtcVtcyjqGXdxDrQtwnhfL46qU+7MLY+4M5TenF0j9b065gf0e0jUo8PjklSviaSnjIy4di7XCPE7155oMXaQcc4HZmEIdQJetWjyJMWl1FRFd8JeimVCFdLxvNbtOoU2zR1fX3kxKicRE+T3GzuOKVXvfcTz9KI3OxMju0VfNXDaCopKuCiQztx1YiucT2uiMRJz5PdLdYugVfPgqNvhyNuUIu1JOdvgt7+iipWbqo9QW+e1wS9YV1aMG7M0KjFkpKJcDKr7yzLW0b1pF9hPkd2axmliCTZpNI3IcFq5rMyM7j/jD4h78/XNyZJX0ctkupadIVffw4f/M5VLlE6A84YCw3znY5Moiwny/cEvd37K1i6YSeL1u2Iej1xSifCyXSCi9aCGrnZmZw+oEN9w5EkpK/6w6PFOESSSE4enPk8dCyBT26D50fAef+CtqF/EJbk1Sgny+cEvWhIye8WkjIhqJ7TlIyxS2KxdX9Mln6OTkiiz8si6c0YKPkNXDYeKvbCC8fBrHFORyVJLiUT4WSUjr1fJbp8pbqt3D0bz6jntwQ92zYJvlEM1TeP9zn6q88GIsmpY4mrxVrhYPjvlfDhH6Ai8HLsIv6kdGlEMtLXtVJfnh+qmuflsPDeUTTIqt9n3g+vPYIqBz6rVSfA+qsQkVoat4ZL/gsT7oH/PQ1rZ7larDUrdDoySTIpOSL80Jl9Oah1Y5o3Sp7OCValEVJP/sofcrMz610akZWZQU49k+n6yIjhH0YKriwnkh4ys+C4e+Dcf0HZYvjbUbB8otNRSZJJyUT42F5t+OIPw8nKTJ6716lFI8C1JK9IJBpmuxqPt0zAJSzrK1p5sGfOqw+dIimi12gY8xXktYJ/nQHfPJFcs+XFUSqNSBA3HNeDoV1aMFQLXEiE+hQ245Gz+jKqT1unQ4k6lQyJSEAtu8Gvv4QProMv/w9Kp8MZf4XcZk5HJgkueYZMU1xOVgYje7R2OgxJcucO6ZhSi6mYOj/EYN8ikhoaNIaz/gGjHoYln7parK2f53RUkuCUCIvE0Z2n9OLdqw9zOoykU9+ktbm75Ciay3KKSAIyBoZeCb/8EPbvhheOhTlvOB2VJDCVRojE0a+OKHY6hDpysxP/83B9J8u9eOlgvpi/vmYJck8qJRRJQZ2HuVqsvXUZvPMbKJ0Gx98PWZqHI7UpERZJY29fNYz2+Q2dDiOo+k5sa9esIZcMK/Lap0aHRVJakzbwi/fgi7thyrOwZhac+09o2t7pyCSBJP5QkIjEzKDOBbRrlgSJcAz3rcVsRFJYZjaccD+c87KrXvhvR8GKb5yOShKIEmERSVjVo7axGL3VeLBIGjnkDFeLtYbN4ZXT4KsHXJ0l9u9yOjJxmEojRCRsJ/ZuS34cF6xR0ioi9daqB/xmArx/LXz9sOsfBgq6QJtDoE1vaNvb9XN+ZzUbTxNKhEUkbH+9eFBcjxfL85Emy4mkkQZNXGUSx97tKpVYNxfWz3X9vOADqC6VymniTo7d/9r2gdYHu24vKUWJsIgkvJiURmiwRyR9NS9y/et58oHL9u2EsoWw7kdXYrx+Hvz4Fkz/h8ftin2MHhdBhipNk5USYRFJeDEdEY7drkUkmTRoDIWDXf+qWQvbVtUdPV74EQdGjxtD615eo8e9ILepI3dDwqNEWEQSlnXXLcQiD9ayzSISlDGQ38n1r8eJBy7fvxvKFriTY/fo8bx3YMZLB7bJ7wRt+tROkJsXQUZm3O+G+KdEWEQSVvVobX0X1Ah4DA0Ji0i4chpBh0Guf9Wshe2r3aPHHuUViz8GW+XaJruRq9a4TW/3P3eS3DDfkbshSoRFJIFVJ6mxyINVIywiUWUMNCt0/et+woHLy/e4a4+rR4/nwoL34Yd/HtimWcfaiXHbPq5uFho9jjklwiKSsLTYhYgkveyG0H6A6181a2HHWq/R47mw5DOwla5tshpC654HRo/b9nbVHjcqcOZ+pCglwiKSuGpGhGNYGqFkW0TizRjXUs9N20O34w5cXr4XNi6qPXq8aDzM/NeBbZp28DF63BUyldJFQo+aiCSsqupE2NkwRETiIzsX2vVz/atmLexc70qKPRPkZV9CVYVrm8wG7tHjPtCmlyvBzmsFjVq6/m/YXC3e/FAiLCIJq3q0VgtqiEjaMgaatHX9O+jYA5dX7IONi2u3dVvyKcx61cc+MqBRC1dSnNfyQIKc1wryWhz4uVFL1/W5zdJmIoUSYRFJWDWT5WIwJpwm7/EikqqyGrjKItr2qX35ro2uEeRdZa6fd210/bzb4+e1s10/79vme98Z2a6EOK9l7ZFlz8vyWh1IrnPykvZNVYmwiCSsA+3THA1DRCR5VCeroajYB7s3uZPmMtjl8bNn4rxpmevn8l2+95PV0Efi7Jk8t6o9Ip3dMHr3t56UCItIwqqqqi6NUCYsIhJ1WQ0OTNoLxf7d7gTZa7R5V9mBhHrnelg/3/Vz5T7f+8lpHDhp9izfaNQCsnKid5+9KBEWkbSkleVERMKU0why3CvtBWMt7NtRe2TZ8//qhHrbKlgz0/V79eQ/b7nNDpRjdD4cjrkjandJibCIJKwO+Q05c2AHfnV4cdT3nWFg1CFt6dIyL+r7FhFJe8ZAblPXv4IuwbevqoK9W71KNTbWTpp3bYTy3VENU4mwiCSsjAzDE+f2j8m+szIzGHvJoOAbiohI7GVkuBYLaVQALbvF77BxO5KIiIiISAJRIiwiIiIiaUmJsIiIiIikJSXCIiIiIpKWlAiLiIiISFpSIiwiIiIiaUmJsIiIiIikJSXCIiIiIpKWlAiLiIiISFpSIiwiIiIiaUmJsIiIiIikJSXCIiIiIpKWlAiLiIiISFpSIiwiIiIiaUmJsIiIiIikJSXCIiIiIpKWlAiLiIiISFpSIiwiIiIiaclYa505sDFlwE8R3LQlsDHK4URLosaWqHGBYotUosaWqHFB9GPrbK1tFcX9JbQUfc+OhlS+f7pvySuV71+k983ne7ZjiXCkjDHTrbWDnY7Dl0SNLVHjAsUWqUSNLVHjgsSOLZWl+uOeyvdP9y15pfL9i/Z9U2mEiIiIiKQlJcIiIiIikpaSMRF+3ukAAkjU2BI1LlBskUrU2BI1Lkjs2FJZqj/uqXz/dN+SVyrfv6jet6SrERYRERERiYZkHBEWEREREam3pEmEjTGjjDGLjDFLjTG3OnD8jsaYr4wxC4wx84wx17svLzDGfG6MWeL+v7nHbW5zx7vIGHNCjOPLNMbMNMZ8mGBx5Rtj3jLGLHQ/dsMSKLbfu5/LucaYccaYXKdiM8a8aIzZYIyZ63FZ2LEYYwYZY350X/eMMcbEKLZH3c/pHGPMu8aY/HjH5isuj+tuNMZYY0zLeMclBzj9vh1LgV5/yc7f+S4VuN/npxpjZrvv2/85HVO0eecDqcQYs9L9fj3LGDM9Kju11ib8PyATWAZ0AXKA2UCvOMfQDhjo/rkJsBjoBTwC3Oq+/FbgYffPvdxxNgCK3fFnxjC+PwD/AT50/54ocf0T+LX75xwgPxFiAzoAK4CG7t/fAC51KjbgKGAgMNfjsrBjAaYCwwADfAycGKPYjgey3D8/7ERsvuJyX94R+BRXz9uWTjxm+pcY79sxvn8+X3+p8A8/5zun44rSfTNAY/fP2cD3wFCn44ryfayVD6TSP2Bl9ft6tP4ly4hwCbDUWrvcWrsfeA04LZ4BWGvXWmt/cP+8A1iAK5k6DVeyh/v/090/nwa8Zq3dZ61dASzFdT+izhhTCJwMvOBxcSLE1RTXyeIfANba/dbarYkQm1sW0NAYkwU0AtY4FZu1dhKw2evisGIxxrQDmlprp1jXO8YrHreJamzW2s+stRXuX78DCuMdm5/HDOBJ4GbAcwJEXB8zARLgfTuWArz+kl6A813Ssy473b9mu/+lzGQpP/mABJAsiXAHYJXH76U4+EdpjCkCBuD6JNnGWrsWXG8eQGv3ZvGM+SlcJ/4qj8sSIa4uQBnwkvtrmheMMXmJEJu1djXwGPAzsBbYZq39LBFi8xBuLB3cP8czRoBf4RpJdTw2Y8xoYLW1drbXVYn2mKWDhHrflsh4ne9Sgrt0YBawAfjcWpsy9w3f+UAqscBnxpgZxpgx0dhhsiTCvmr2HPkEZ4xpDLwN/M5auz3Qpj4ui3rMxphTgA3W2hmh3sTHZbF6LLNwfXX4V2vtAGAXrq/4/YlbbO5629NwfU3eHsgzxlycCLGFwF8scY/RGHM7UAH8u/oiPzHEPDZjTCPgduBOX1c7FVca02Ob5MI43yUVa22ltbY/rm+ySowxvR0OKSoiyAeS0eHW2oHAicBvjTFH1XeHyZIIl+Kq+6tWiOtr7LgyxmTjelP4t7X2HffF691fr+L+f4P78njFfDgw2hizEtdXj0cbY15NgLiqj1Xq8Wn7LVyJcSLEdiywwlpbZq0tB94BDkuQ2KqFG0spB0oUYh6jMeaXwCnARe6yAqdj64rrg81s999DIfCDMaatw3Glq4R435bI+DnfpRR3qd5EYJSzkUSNv3wgZVhr17j/3wC8SxRKFJMlEZ4GdDPGFBtjcoDzgffjGYB7Jvk/gAXW2ic8rnof+KX7518C73lcfr4xpoExphjohmtSTlRZa2+z1hZaa4twPS4TrLUXOx2XO7Z1wCpjTA/3RccA8xMhNlwlEUONMY3cz+0xuOrgEiG2amHF4i6f2GGMGeq+T7/wuE1UGWNGAbcAo621u71idiQ2a+2P1trW1toi999DKa4JP+ucjCuNOf6+LZEJcL5LesaYVsbd5cYY0xDXoMhCR4OKkgD5QEowxuQZY5pU/4xr0nb9u7ZEc+ZdLP8BJ+GauboMuN2B4x+B62u9OcAs97+TgBbAl8AS9/8FHre53R3vIuIwEx0YwYGuEQkRF9AfmO5+3P4LNE+g2P4P1xvgXOBfuDoKOBIbMA5XrXI5rgTu8khiAQa7788y4Fnci+bEILaluOo/q/8WxsY7Nl9xeV2/Eo/ZxfF8zPSv5rF19H07xvct4Osvmf/h53zndFxRum99gZnu+zYXuNPpmGJ0P0eQYl0jcM07mu3+Ny9a7ylaWU5ERERE0lKylEaIiIiIiESVEmERERERSUtKhEVEREQkLSkRFhEREZG0pERYRERERNKSEmFJeMaYSmPMLGPMbGPMD8aYw4Jsn2+MuTqE/U40xgyOXqQiIunN4/26+l+g1UTD3XeRMab+fWNFPGQ5HYBICPZY13KYGGNOAB4EhgfYPh+4Gngu5pGJiIinmvdrkWSgEWFJNk2BLQDGmMbGmC/do8Q/GmNOc2/zENDVPRrxqHvbm93bzDbGPOSxv3OMMVONMYuNMUfG966IiKQHY8xKY8zD7vfbqcaYg9yXd3a/j89x/9/JfXkbY8y77vfs2R7fBGYaY/5ujJlnjPnMvTqcSMQ0IizJoKExZhaQC7QDjnZfvhc4w1q73RjTEvjOGPM+cCvQ22MU+UTgdOBQa+1uY0yBx76zrLUlxpiTgLtwLbcpIiKRqX6/rvagtfZ198/b3e+3vwCeAk7BtZrjK9bafxpjfgU8g+v9+hnga2vtGcaYTKAxrpVJuwEXWGt/Y4x5AzgLeDUO90tSlBJhSQaepRHDgFeMMb0BAzxgjDkKqAI6AG183P5Y4CVr7W4Aa+1mj+vecf8/AyiKSfQiIukjUGnEOI//n3T/PAw40/3zv4BH3D8fDfwCwFpbCWwzxjQHVlhrZ7m30fu21JsSYUkq1top7tHfVsBJ7v8HWWvLjTErcY0aezOAv7XE97n/r0R/DyIisWT9/OxvG1/2efxcCag0QupFNcKSVIwxPYFMYBPQDNjgToJHAp3dm+0Amnjc7DPgV8aYRu59eJZGiIhIfJzn8f8U98/fAue7f74ImOz++UvgKgBjTKYxpmm8gpT0ohEwSQaeNWcG+KW1ttIY82/gA2PMdGAWsBDAWrvJGPM/d5udj621Nxlj+gPTjTH7gfHAH+N9J0RE0oB3jfAn1trqFmoNjDHf4xqEu8B92XXAi8aYm4Ay4DL35dcDzxtjLsc18nsVsDbWwUv6MdYG+xZCREREJHLu0rXB1tqNTsci4kmlESIiIiKSljQiLCIiIiJpSSPCIiIiIpKWlAiLiIiISFpSIiwiIiIiaUmJsIiIiIikJSXCIiIiIpKWlAiLiIiISFr6f7BkZ0sHcTNDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 07 | Time: 14m 49s\n",
      "\tTrain Loss: 0.358 | Train PPL:   1.431\n",
      "\t Val. Loss: 0.274 |  Val. PPL:   1.315\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import math\n",
    "import matplotlib\n",
    "matplotlib.rcParams.update({'figure.figsize': (16, 12), 'font.size': 14})\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from IPython.display import clear_output\n",
    "\n",
    "train_history = []\n",
    "valid_history = []\n",
    "\n",
    "N_EPOCHS = 7\n",
    "CLIP = 100\n",
    "\n",
    "best_valid_loss = float('inf')\n",
    "\n",
    "for epoch in range(N_EPOCHS):\n",
    "    \n",
    "    start_time = time.time()\n",
    "    \n",
    "    train_loss = train(model, train_iterator, optimizer, criterion, CLIP, train_history, valid_history)\n",
    "    valid_loss = evaluate(model, valid_iterator, criterion)\n",
    "    \n",
    "    end_time = time.time()\n",
    "    \n",
    "    epoch_mins, epoch_secs = epoch_time(start_time, end_time)\n",
    "    \n",
    "    if valid_loss < best_valid_loss:\n",
    "        best_valid_loss = valid_loss\n",
    "        torch.save(model.state_dict(), 'best-val-model.pt')\n",
    "\n",
    "    train_history.append(train_loss)\n",
    "    valid_history.append(valid_loss)\n",
    "    print(f'Epoch: {epoch+1:02} | Time: {epoch_mins}m {epoch_secs}s')\n",
    "    print(f'\\tTrain Loss: {train_loss:.3f} | Train PPL: {math.exp(train_loss):7.3f}')\n",
    "    print(f'\\t Val. Loss: {valid_loss:.3f} |  Val. PPL: {math.exp(valid_loss):7.3f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fr860UPacMeI"
   },
   "source": [
    "### Применение модели\n",
    " **(1 балл)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5sDAfAq9xol9"
   },
   "outputs": [],
   "source": [
    "def accuracy_model(model, iterator):\n",
    "    model.eval()\n",
    "    \n",
    "    true_pred = 0\n",
    "    num_pred = 0\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        for i, batch in enumerate(iterator):\n",
    "\n",
    "            words = batch.words\n",
    "            tags = batch.tags\n",
    "\n",
    "            output = model(words)\n",
    "            \n",
    "            #output = [sent len, batch size, output dim]\n",
    "            # Выбираем для каждого слова индекс тэга с максимальной вероятностью\n",
    "            output = output.argmax(axis=2)\n",
    "            \n",
    "            #output = [sent len, batch size]\n",
    "            predict_tags = output.cpu().numpy()\n",
    "            true_tags = tags.cpu().numpy()\n",
    "\n",
    "            true_pred += np.sum((true_tags == predict_tags) & (true_tags != PAD_IDX))\n",
    "            num_pred += np.prod(true_tags.shape) - (true_tags == PAD_IDX).sum()\n",
    "        \n",
    "    return round(true_pred / num_pred * 100, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "V2n0H85mxomE",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 92.042 %\n"
     ]
    }
   ],
   "source": [
    "print(\"Accuracy:\", accuracy_model(model, test_iterator), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FacTKSPJcMeP"
   },
   "source": [
    "Вы можете улучшить качество, изменяя параметры модели. Вам неоходимо добиться качества не меньше, чем `accuracy = 92 %`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TqD1lZuwxomK",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "best_model = LSTMTagger(INPUT_DIM, EMB_DIM, HID_DIM, OUTPUT_DIM, DROPOUT, NUM_LAYERS).to(device)\n",
    "best_model.load_state_dict(torch.load('best-val-model.pt'))\n",
    "assert accuracy_model(best_model, test_iterator) >= 92"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nqnaeXi6sKl5"
   },
   "source": [
    "### **Если качество сети меньше 92 процентов, то снимается половина от всех полученных баллов .<br> То есть максимум в этом случае 5 баллов за работу.**\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nVfdJM-lcMeZ"
   },
   "source": [
    "Пример решение нашей задачи:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W3GUbwldxomW"
   },
   "outputs": [],
   "source": [
    "def print_tags(model, data):\n",
    "    model.eval()\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        words, _ = data\n",
    "        example = torch.LongTensor([WORD.vocab.stoi[elem] for elem in words]).unsqueeze(1).to(device)\n",
    "        \n",
    "        output = model(example).argmax(dim=-1).cpu().numpy()\n",
    "        tags = [TAG.vocab.itos[int(elem)] for elem in output]\n",
    "\n",
    "        for token, tag in zip(words, tags):\n",
    "            print(f'{token:15s}{tag}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4mQoHc_EcMed",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From           NOUN\n",
      "what           DET\n",
      "I              NOUN\n",
      "was            VERB\n",
      "able           ADJ\n",
      "to             PRT\n",
      "gauge          NOUN\n",
      "in             ADP\n",
      "a              DET\n",
      "swift          ADJ\n",
      ",              .\n",
      "greedy         NOUN\n",
      "glance         VERB\n",
      ",              .\n",
      "the            DET\n",
      "figure         NOUN\n",
      "inside         ADP\n",
      "the            DET\n",
      "coral-colored  NOUN\n",
      "boucle         NOUN\n",
      "dress          NOUN\n",
      "was            VERB\n",
      "stupefying     VERB\n",
      ".              .\n"
     ]
    }
   ],
   "source": [
    "print_tags(model, pos_data[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "heading_collapsed": true,
    "id": "zMIJDOBmwC6v"
   },
   "source": [
    "## Вывод: \n",
    "**(0.5 балл)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Оптимальное количество эпох для обучения в этой задаче -- 7. После этого сеть начинает переобучаться."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "[homework]pos_tag.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}