notebooks/3-llm-classification-finetuned.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM Security - Prompt Injection\n",
    "## Part 3 - Classification Using a Fine-tuned LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we load the raw dataset and fine-tune a pre-trained large language model to classify malicious prompts.\n",
    "> **INPUT:** the raw dataset loaded from Hugging Face library. <br>\n",
    "> **OUTPUT:** the performance analysis of fine-tuned LLM.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. INITIALIZATION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary libraries and modules\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set display options\n",
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. LOADING DATASET"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we are going to use a pre-trained LLM and fine-tune it using the training dataset, we need to load both training and testing data sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize data set location and file name\n",
    "data_file_path = \"../data/raw/\"\n",
    "data_file_name_train = \"train-00000-of-00001-9564e8b05b4757ab\"\n",
    "data_file_name_test = \"test-00000-of-00001-701d16158af87368\"\n",
    "data_file_ext = \".parquet\"\n",
    "\n",
    "# Loading data set into a pandas DataFrame\n",
    "data_train = pd.read_parquet(data_file_path + data_file_name_train + data_file_ext)\n",
    "data_test = pd.read_parquet(data_file_path + data_file_name_test + data_file_ext)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rename \"text\" column into \"prompt\"\n",
    "data_train.rename(columns={\"text\":\"prompt\"}, inplace=True)\n",
    "data_test.rename(columns={\"text\":\"prompt\"}, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already explored the dataset in the previous notebooks, so we will directly proceed to the fine-tuning phase."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. MODEL FINE-TUNING"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this experiment, we aim at using a pre-trained LLM and fine-tuning it on the classification task.\n",
    "\n",
    "In the previous experiment, we used the pre-trained XLM-RoBERTa, the multilingual version of RoBERTa (Robustly optimized BERT approach), the enhanced version of BERT (Bidirectional Encoder Representations from Transformers).\n",
    "\n",
    "To explore the effect of model fine-tuning, we fine-tune XLM-RoBERTa on our training dataset and then re-evaluate its performance in correctly predicting prompt injections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the model, its tokenizer, and torch library\n",
    "from transformers import XLMRobertaTokenizer, XLMRobertaForSequenceClassification\n",
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the model tokenizer\n",
    "tokenizer = XLMRobertaTokenizer.from_pretrained('xlm-roberta-large')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A utility function to receive a batch of data and tokenize the prompts\n",
    "def tokenize_batch(batch):\n",
    "    return tokenizer(batch['prompt'], padding=True, truncation=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenize prompts in both training and testing datasets\n",
    "prompts_train_tokenized = tokenize_batch(data_train.to_dict(orient='list'))\n",
    "prompts_test_tokenized = tokenize_batch(data_test.to_dict(orient='list'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a dataset Class to work with PyTorch's DataLoader\n",
    "class CustomDataset(torch.utils.data.Dataset):\n",
    "    def __init__(self, encodings, labels):\n",
    "        self.encodings = encodings\n",
    "        self.labels = labels\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}\n",
    "        item['labels'] = torch.tensor(self.labels[idx])\n",
    "        return item\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare CustomDataset for training and testing to be used by the LLM\n",
    "train_dataset = CustomDataset(prompts_train_tokenized, data_train['label'])\n",
    "test_dataset = CustomDataset(prompts_test_tokenized, data_test['label'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of XLMRobertaForSequenceClassification were not initialized from the model checkpoint at xlm-roberta-large and are newly initialized: ['classifier.out_proj.bias', 'classifier.dense.bias', 'classifier.out_proj.weight', 'classifier.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "# Load pre-trained Model\n",
    "model = XLMRobertaForSequenceClassification.from_pretrained('xlm-roberta-large', num_labels=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When loading the model, we get this message indicating that some weights are not initialized. This is normal since the model is pre-trained but not fine-tuned for the downstream task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import TrainingArguments to handle the various training configurations\n",
    "from transformers import TrainingArguments\n",
    "\n",
    "# Define training arguments for fine-tuning\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=\"../output\",\n",
    "    per_device_train_batch_size=8,\n",
    "    per_device_eval_batch_size=8,\n",
    "    num_train_epochs=5,\n",
    "    evaluation_strategy=\"epoch\",\n",
    "    learning_rate=2e-5,\n",
    "    logging_dir=\"../output/logs\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize a DataFrame to track the model's performance\n",
    "results_df = pd.DataFrame(columns=[\"epoch\",\"accuracy\",\"precision\",\"recall\",\"f1\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import accuracy metrics\n",
    "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
    "\n",
    "# A utility function for model evaluation during fine-tuning\n",
    "def evaluate_model(trainer, epoch):\n",
    "    \n",
    "    # Extract predictions and labels\n",
    "    predictions, labels = trainer.predictions.argmax(axis=1), trainer.label_ids\n",
    "    \n",
    "    # Calculate accuracy\n",
    "    accuracy = accuracy_score(labels, predictions)\n",
    "    \n",
    "    # Calculate precision, recall, and f1 score\n",
    "    precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average=\"binary\")\n",
    "    \n",
    "    # Append current metrics to results\n",
    "    global results_df\n",
    "    results_df.loc[len(results_df)] = [epoch, accuracy, precision, recall, f1]\n",
    "        \n",
    "    # Return\n",
    "    return {\n",
    "        \"accuracy\": accuracy,\n",
    "        \"precision\": precision,\n",
    "        \"recall\": recall,\n",
    "        \"f1\": f1,\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the Trainer class\n",
    "from transformers import Trainer\n",
    "\n",
    "# Define trainer\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=train_dataset,\n",
    "    eval_dataset=test_dataset,\n",
    "    compute_metrics=lambda p: evaluate_model(p, trainer.state.epoch),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dcd4bc4708a5441f880016a713014206",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/345 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7378c9dd0dad424b82932ade8c5f5a07",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.6904010772705078, 'eval_accuracy': 0.6293103448275862, 'eval_precision': 1.0, 'eval_recall': 0.2833333333333333, 'eval_f1': 0.4415584415584416, 'eval_runtime': 5.1865, 'eval_samples_per_second': 22.366, 'eval_steps_per_second': 2.892, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "007cce4c4b2e417b88ba1fd4439c6564",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.6092261672019958, 'eval_accuracy': 0.9137931034482759, 'eval_precision': 1.0, 'eval_recall': 0.8333333333333334, 'eval_f1': 0.9090909090909091, 'eval_runtime': 5.1771, 'eval_samples_per_second': 22.406, 'eval_steps_per_second': 2.897, 'epoch': 2.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "53fe703b514a40069761b215dc11eab1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.553800106048584, 'eval_accuracy': 0.9310344827586207, 'eval_precision': 1.0, 'eval_recall': 0.8666666666666667, 'eval_f1': 0.9285714285714286, 'eval_runtime': 5.0967, 'eval_samples_per_second': 22.76, 'eval_steps_per_second': 2.943, 'epoch': 3.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "25dbf0c0b798460b8aa98bf6d25aa2b3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.2970550060272217, 'eval_accuracy': 0.9655172413793104, 'eval_precision': 1.0, 'eval_recall': 0.9333333333333333, 'eval_f1': 0.9655172413793104, 'eval_runtime': 5.142, 'eval_samples_per_second': 22.559, 'eval_steps_per_second': 2.917, 'epoch': 4.0}\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e4e9dfcb22ea401190510af741745bdf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.23396892845630646, 'eval_accuracy': 0.9741379310344828, 'eval_precision': 1.0, 'eval_recall': 0.95, 'eval_f1': 0.9743589743589743, 'eval_runtime': 5.1821, 'eval_samples_per_second': 22.385, 'eval_steps_per_second': 2.895, 'epoch': 5.0}\n",
      "{'train_runtime': 4743.0681, 'train_samples_per_second': 0.576, 'train_steps_per_second': 0.073, 'train_loss': 0.27316051980723505, 'epoch': 5.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=345, training_loss=0.27316051980723505, metrics={'train_runtime': 4743.0681, 'train_samples_per_second': 0.576, 'train_steps_per_second': 0.073, 'train_loss': 0.27316051980723505, 'epoch': 5.0})"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fine-tune the model\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>epoch</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>precision</th>\n",
       "      <th>recall</th>\n",
       "      <th>f1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.629310</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.283333</td>\n",
       "      <td>0.441558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>0.913793</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.833333</td>\n",
       "      <td>0.909091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>0.931034</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.866667</td>\n",
       "      <td>0.928571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.0</td>\n",
       "      <td>0.965517</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.933333</td>\n",
       "      <td>0.965517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>0.974138</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.950000</td>\n",
       "      <td>0.974359</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   epoch  accuracy  precision    recall        f1\n",
       "0    1.0  0.629310        1.0  0.283333  0.441558\n",
       "1    2.0  0.913793        1.0  0.833333  0.909091\n",
       "2    3.0  0.931034        1.0  0.866667  0.928571\n",
       "3    4.0  0.965517        1.0  0.933333  0.965517\n",
       "4    5.0  0.974138        1.0  0.950000  0.974359"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print detailed results\n",
    "results_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAHWCAYAAACbsXOkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABi0klEQVR4nO3dd3wUdeLG8Wd3k2x6KOkB6YQOChJBOBsaihH82UBPkEM5PVAUPQ+UDifq3SF6qKgH4lkRC3YUIsihFAVRQu8lkBBAUskm2Z3fH4GFJQFJ2GRSPu/Xa18ks9+ZPDNMdB9m57sWwzAMAQAAAAAuitXsAAAAAABQE1CuAAAAAMALKFcAAAAA4AWUKwAAAADwAsoVAAAAAHgB5QoAAAAAvIByBQAAAABeQLkCAAAAAC+gXAEAAACAF1CuAKCasFgsmjRpUpnX27NnjywWi+bNm+f1TDitphznefPmyWKx6KeffjI7SoW45557FBwcbHYMADUU5QoAyuDUC0+LxaIVK1aUeN4wDDVs2FAWi0U33nijCQkBAIBZfMwOAADVkb+/v9555x316NHDY/l3332nAwcOyG63m5QMZmnUqJFOnDghX19fs6MAAEzClSsAKIe+fftqwYIFKioq8lj+zjvvqHPnzoqOjjYpWe2Rm5trdgQPFotF/v7+stlsZkcBAJiEcgUA5TBo0CAdPXpUixcvdi8rKCjQBx98oDvvvLPUdXJzc/Xoo4+qYcOGstvtio+P1z//+U8ZhuExzuFw6JFHHlFERIRCQkJ000036cCBA6VuMzU1VX/6058UFRUlu92utm3bau7cueXap2PHjumxxx5T+/btFRwcrNDQUPXp00e//PJLibH5+fmaNGmSWrZsKX9/f8XExOj//u//tHPnTvcYl8ul559/Xu3bt5e/v78iIiLUu3dv970857tH6ez7yyZNmiSLxaJNmzbpzjvvVN26dd1XDX/99Vfdc889atq0qfz9/RUdHa0//elPOnr0aKnHa9iwYYqNjZXdbleTJk30wAMPqKCgQLt27ZLFYtFzzz1XYr0ffvhBFotF77777jmPX2n7c+r+ntTUVA0YMEDBwcGKiIjQY489JqfTec5tnemrr75Sz549FRQUpJCQEPXr108bN270GOOtY3Amh8Oh0aNHKyIiQkFBQbr55puVkZFxQZm3bNmiW2+9VfXq1ZO/v7+6dOmiTz/91GPMqbfYLl++XH/+859Vv359hYaGavDgwfrtt99KbPOll15S27ZtZbfbFRsbqxEjRuj48eMlxq1evVp9+/ZV3bp1FRQUpA4dOuj5558v9TiU9+8EAM6FtwUCQDk0btxY3bp107vvvqs+ffpIKn4RnJmZqYEDB+qFF17wGG8Yhm666SYtXbpUw4YNU6dOnfT111/rr3/9q1JTUz1e0N9777166623dOedd6p79+769ttv1a9fvxIZ0tPTdcUVV8hisWjkyJGKiIjQV199pWHDhikrK0sPP/xwmfZp165dWrhwoW677TY1adJE6enpeuWVV3TVVVdp06ZNio2NlSQ5nU7deOONSk5O1sCBAzVq1ChlZ2dr8eLFSklJUbNmzSRJw4YN07x589SnTx/de++9Kioq0v/+9z+tWrVKXbp0KVO2U2677Ta1aNFCTz31lLuULl68WLt27dLQoUMVHR2tjRs36tVXX9XGjRu1atUqWSwWSdLBgwfVtWtXHT9+XMOHD1erVq2UmpqqDz74QHl5eWratKmuvPJKvf3223rkkUc8fu7bb7+tkJAQ9e/fv8yZnU6nEhMTlZCQoH/+859asmSJ/vWvf6lZs2Z64IEHzrvum2++qSFDhigxMVHPPPOM8vLy9PLLL6tHjx76+eef1bhxY68eAz8/P/fPfvDBB1W3bl1NnDhRe/bs0cyZMzVy5EjNnz//vJk3btyoK6+8UnFxcRozZoyCgoL0/vvva8CAAfrwww918803e4wfOXKk6tSpo0mTJmnr1q16+eWXtXfvXi1btsyde9KkSZo8ebJ69eqlBx54wD3uxx9/1Pfff+9+K+bixYt14403KiYmRqNGjVJ0dLQ2b96szz//XKNGjfLK3wkAnJcBALhgr7/+uiHJ+PHHH41Zs2YZISEhRl5enmEYhnHbbbcZ11xzjWEYhtGoUSOjX79+7vUWLlxoSDKmTZvmsb1bb73VsFgsxo4dOwzDMIz169cbkoy//OUvHuPuvPNOQ5IxceJE97Jhw4YZMTExxpEjRzzGDhw40AgLC3Pn2r17tyHJeP3118+7b/n5+YbT6fRYtnv3bsNutxtTpkxxL5s7d64hyZgxY0aJbbhcLsMwDOPbb781JBkPPfTQOcecL9fZ+zpx4kRDkjFo0KASY0/t55neffddQ5KxfPly97LBgwcbVqvV+PHHH8+Z6ZVXXjEkGZs3b3Y/V1BQYISHhxtDhgwpsd6ZStufIUOGGJI8jp9hGMall15qdO7c+bzby87ONurUqWPcd999HsvT0tKMsLAwj+XePAanzvFevXq5lxmGYTzyyCOGzWYzjh8/ft7c1113ndG+fXsjPz/fY9vdu3c3WrRo4V526ud07tzZKCgocC9/9tlnDUnGJ598YhiGYRw+fNjw8/MzbrjhBo/zc9asWYYkY+7cuYZhGEZRUZHRpEkTo1GjRsZvv/1W6r4ZxsX9nQDA7+FtgQBQTrfffrtOnDihzz//XNnZ2fr888/P+ZbAL7/8UjabTQ899JDH8kcffVSGYeirr75yj5NUYtzZV6EMw9CHH36opKQkGYahI0eOuB+JiYnKzMzUunXryrQ/drtdVmvx/xacTqeOHj2q4OBgxcfHe2zrww8/VHh4uB588MES2zh1peHDDz+UxWLRxIkTzzmmPO6///4SywICAtxf5+fn68iRI7riiiskyZ3b5XJp4cKFSkpKKvWq2alMt99+u/z9/fX222+7n/v666915MgR/fGPf/Ra7p49e2rXrl3nXWfx4sU6fvy4Bg0a5PH3a7PZlJCQoKVLl7rHevMYnDJ8+HCPZT179pTT6dTevXvPmfnYsWP69ttvdfvttys7O9ud+ejRo0pMTNT27duVmppa4uecOQnIAw88IB8fH/fvwpIlS1RQUKCHH37YfX5K0n333afQ0FB98cUXkqSff/5Zu3fv1sMPP6w6deqcd9+k8v2dAMDv4W2BAFBOERER6tWrl9555x3l5eXJ6XTq1ltvLXXs3r17FRsbq5CQEI/lrVu3dj9/6k+r1ep+a90p8fHxHt9nZGTo+PHjevXVV/Xqq6+W+jMPHz5cpv05dY/USy+9pN27d3vcf1K/fn331zt37lR8fLx8fM79v5CdO3cqNjZW9erVK1OG39OkSZMSy44dO6bJkyfrvffeK7HPmZmZkoqPV1ZWltq1a3fe7depU0dJSUl65513NHXqVEnFbwmMi4vTtddeW67Mp+43O1PdunVLva/oTNu3b5ekc/7c0NBQ99fePAanXHLJJSUySzpv7h07dsgwDI0fP17jx48vdczhw4cVFxfn/r5FixYezwcHBysmJkZ79uyRdPp34+zfAT8/PzVt2tT9/Kn7/S5k/8r7dwIAv4dyBQAX4c4779R9992ntLQ09enTp8S/mFcUl8slSfrjH/+oIUOGlDqmQ4cOZdrmU089pfHjx+tPf/qTpk6dqnr16slqterhhx92/zxvOtcVrPNNKnDmFZpTbr/9dv3www/661//qk6dOik4OFgul0u9e/cuV+7BgwdrwYIF+uGHH9S+fXt9+umn+stf/uJx1aQsyjt74Knsb775ZqmzT55Zbr19DM6X2zhrApbSMj/22GNKTEwsdUzz5s3LlcebmNERQEWhXAHARbj55pv15z//WatWrTrvjf6NGjXSkiVLlJ2d7XH1asuWLe7nT/3pcrncV4dO2bp1q8f2Ts0k6HQ61atXL6/sywcffKBrrrlGc+bM8Vh+/PhxhYeHu79v1qyZVq9ercLCwnN+plOzZs309ddf69ixY+e8enXqSsjZM76d721nZ/vtt9+UnJysyZMna8KECe7lp676nBIREaHQ0FClpKT87jZ79+6tiIgIvf3220pISFBeXp7uvvvuC87kLaeuXkZGRp7377gijkF5NW3aVJLk6+t7wefl9u3bdc0117i/z8nJ0aFDh9S3b19Jp383tm7d6t6+VDw75+7du90/59TxSklJ8drvBACUFfdcAcBFCA4O1ssvv6xJkyYpKSnpnOP69u0rp9OpWbNmeSx/7rnnZLFY3DMOnvrz7NkGZ86c6fG9zWbTLbfcog8//LDUF8sXOmX22ds8+6rEggULStwjc8stt+jIkSMl9kU6fVXjlltukWEYmjx58jnHhIaGKjw8XMuXL/d4/qWXXipT5jO3ecrZx8tqtWrAgAH67LPP3FPBl5ZJKr4iNGjQIL3//vuaN2+e2rdvX+argN6QmJio0NBQPfXUUyosLCzx/Km/44o4BuUVGRmpq6++Wq+88ooOHTp0zsxnevXVVz327+WXX1ZRUZH7d6FXr17y8/PTCy+84JFxzpw5yszMdM+kedlll6lJkyaaOXNmicLujX0DgAvBlSsAuEjnelvemZKSknTNNdfoySef1J49e9SxY0d98803+uSTT/Twww+7/9W9U6dOGjRokF566SVlZmaqe/fuSk5O1o4dO0ps8+mnn9bSpUuVkJCg++67T23atNGxY8e0bt06LVmyRMeOHSvTftx4442aMmWKhg4dqu7du2vDhg16++23Pa4WSMVvm/vvf/+r0aNHa82aNerZs6dyc3O1ZMkS/eUvf1H//v11zTXX6O6779YLL7yg7du3u9+e9r///U/XXHONRo4cKal42vmnn35a9957r7p06aLly5dr27ZtF5w5NDRUf/jDH/Tss8+qsLBQcXFx+uabb7R79+4SY5966il98803uuqqqzR8+HC1bt1ahw4d0oIFC7RixQqPt3QOHjxYL7zwgpYuXapnnnmmTMfRW0JDQ/Xyyy/r7rvv1mWXXaaBAwcqIiJC+/bt0xdffKErr7xSs2bNqrBjUF4vvviievToofbt2+u+++5T06ZNlZ6erpUrV+rAgQMlPjetoKBA1113nW6//XZt3bpVL730knr06KGbbrpJUvEVt7Fjx2ry5Mnq3bu3brrpJve4yy+/3D3RiNVq1csvv6ykpCR16tRJQ4cOVUxMjLZs2aKNGzfq66+/vuh9A4DfZcIMhQBQbZ05Ffv5nD0Vu2EUT639yCOPGLGxsYavr6/RokUL4x//+IfHNNGGYRgnTpwwHnroIaN+/fpGUFCQkZSUZOzfv7/E9OSGYRjp6enGiBEjjIYNGxq+vr5GdHS0cd111xmvvvqqe0xZpmJ/9NFHjZiYGCMgIMC48sorjZUrVxpXXXWVcdVVV3mMzcvLM5588kmjSZMm7p976623Gjt37nSPKSoqMv7xj38YrVq1Mvz8/IyIiAijT58+xtq1az22M2zYMCMsLMwICQkxbr/9duPw4cPnnIo9IyOjRO4DBw4YN998s1GnTh0jLCzMuO2224yDBw+Werz27t1rDB482IiIiDDsdrvRtGlTY8SIEYbD4Six3bZt2xpWq9U4cODAeY/bKeeaij0oKKjE2FP7cyGWLl1qJCYmGmFhYYa/v7/RrFkz45577jF++umnCjkG5zrHly5dakgyli5d+ruZd+7caQwePNiIjo42fH19jbi4OOPGG280PvjgA/eYUz/nu+++M4YPH27UrVvXCA4ONu666y7j6NGjJbY5a9Yso1WrVoavr68RFRVlPPDAAyWmXDcMw1ixYoVx/fXXGyEhIUZQUJDRoUMH49///rf7eW/8nQDAuVgMg2vlAACc7dJLL1W9evWUnJxsdpQaad68eRo6dKh+/PHHcn+oNABUNdxzBQDAWX766SetX79egwcPNjsKAKAa4Z4rAABOSklJ0dq1a/Wvf/1LMTExuuOOO8yOBACoRrhyBQDASR988IGGDh2qwsJCvfvuu/L39zc7EgCgGuGeKwAAAADwAq5cAQAAAIAXUK4AAAAAwAuY0KIULpdLBw8eVEhIiCwWi9lxAAAAAJjEMAxlZ2crNjZWVuv5r01Rrkpx8OBBNWzY0OwYAAAAAKqI/fv3q0GDBucdQ7kqRUhIiKTiAxgaGmpyGgAAAABmycrKUsOGDd0d4XwoV6U49VbA0NBQyhUAAACAC7pdiAktAAAAAMALKFcAAAAA4AWUKwAAAADwAsoVAAAAAHgB5QoAAAAAvIByBQAAAABeQLkCAAAAAC+gXAEAAACAF1CuAAAAAMALKFcAAAAA4AWUKwAAAADwAsoVAAAAAHgB5QoAAAAAvMDH7AAAAAAAapYip0u5DqdyC4qU6yhSjqPI4/tcR5FyC5xnPHf6+9yTY2PrBOg/Q7qYvStlQrkCAAAAarlCp+ucJSjH4VRewRklyHGyBBWcfO7UegVFynM4leMokqPIddGZ8gqKvLBnlYtyBQAAAFQzjiKnR8lxl6BzXAkqtSAVnP66wHnxZag0vjaLguw+CvLzUbDdR4F2m4JPfu/+2n7yOT+bx9d1A/0qJFNFolwBAAAAFcgwDDmKXO4ik3NGISqtIOWe/fwZX+c4ipRXUKRCp1EhWf18rCcLj01BfsXFp7gcnS4+QXabAv183MXo1HNnFqNTz/n51K4pHihXAAAAwBkMw1B+oeuMK0CnS9CpclNaCTr7udNvl3PK6aqYMuTvaz1ZaE6VmzOLj8191ejM584sSe5SdPJKkq+tdpUhb6NcAQAAoFozDEN5Bc7zl6AzrgLluItPKVeJTn5fQV1IAb42d9E5XXxKlqAgu48CT33tfkvdGc/5FV8x8qEMVSmUKwAAAFQql8tQXuEZJejUW+VKuUpUYha5EgWquDAZFVSGznw7XGBpRcf91rkz7xcq/v702+ZOv5XOZrVUTFBUCZQrAAAAnJfTZZS8uvM7U2mfKj6nnjv764pgsUjBJ8vOmRMneLz97ax7gs78+uznAnxtslKGUAaUKwAAgBoo11GkzBOFpRadEiXI4VTOmZ8/dNYkCicKK6YM2awWj8kQSps44dyTKngWoSC7TQG+NlkslCGYh3IFAABQzR3PK9CG1EylpGYpJTVTG1Izte9Yntd/jo/Vco4Z4zxLUPDZs8mdcU/RmdNx232slCHUKJQrAACAauS33OIiVVymiv888NuJUsf62axnTZZQ+sQJZ0+nXXLCheJ1/WyUIeB8KFcAAABV1NEch0eJSknNUurx0otUo/qBahcXpvZxYWoXG6Z2caGqUw0/hBWozihXAAAAVUBGtkMpBzOVcuD0VamDmfmljm18RpFqHxemtnFhCgvwreTEAM5GuQIAAKhkh7Pzi69GHchyF6m0rNKLVNPwoNNXpOLC1DYuVKH+FCmgKqJcAQAAVKD0rHxtOHk1auPB4j/TsxwlxlksxUXqVIlqFxemtrGhCqFIAdUG5QoAAMALDMNQWla+UlKzPO6TysguvUg1iwh2F6n2cWFqExuqYDsvzYDqjN9gAACAMjIMQ4cy88+abCJTR3IKSoy1WqTmkcEe90i1jglVEEUKqHH4rQYAADgPwzCUevyEx4x9KamZOppbskjZrBa1OKNItYsLVeuYUAX68ZILqA34TQcAADjJMAwd+O10kSq+TypLx85TpNrHhal9g+K397WODlWAn82E5ACqAsoVAAColQzD0P5jJzw+kDflYKaO5xWWGOtjtahlVEjx1agGxVelWkWHyN+XIgXgNMoVAACo8QzD0N6jecUl6uDJIpWapcwTJYuUr82i+OiQ07P2xYYpniIF4AJQrgAAQI3ichnaeyzv9GQTB4oLVXZ+UYmxfjar4qNDPCabaBkdLLsPRQpA2Zlerl588UX94x//UFpamjp27Kh///vf6tq1a6ljCwsLNX36dL3xxhtKTU1VfHy8nnnmGfXu3ds9ZtKkSZo8ebLHevHx8dqyZUuF7gcAAKh8Lpeh3Udz3SVqQ2qmNh3MUrajlCLlY1XrM4pUu7gwtYwKkZ+P1YTkAGoiU8vV/PnzNXr0aM2ePVsJCQmaOXOmEhMTtXXrVkVGRpYYP27cOL311lt67bXX1KpVK3399de6+eab9cMPP+jSSy91j2vbtq2WLFni/t7Hx/QOCQAALpLTZWj3kVyPySY2HcxSTilFyu5jVeuYUPeMfaeKlK+NIgWg4lgMwzDM+uEJCQm6/PLLNWvWLEmSy+VSw4YN9eCDD2rMmDElxsfGxurJJ5/UiBEj3MtuueUWBQQE6K233pJUfOVq4cKFWr9+fblzZWVlKSwsTJmZmQoNDS33dgAAQPk4XYZ2ZeR4TDax6WCWcgucJcb6+55ZpIqvSjWPDKZIAfCKsnQD0y7pFBQUaO3atRo7dqx7mdVqVa9evbRy5cpS13E4HPL39/dYFhAQoBUrVngs2759u2JjY+Xv769u3bpp+vTpuuSSS86ZxeFwyOE4/enpWVlZ5dklAABQDkVOl3Zm5Hp8IO+mg1k6UViySAX42tQm1rNINYsIkg9FCkAVYFq5OnLkiJxOp6KiojyWR0VFnfP+qMTERM2YMUN/+MMf1KxZMyUnJ+ujjz6S03n6P74JCQmaN2+e4uPjdejQIU2ePFk9e/ZUSkqKQkJCSt3u9OnTS9ynBQAAvK/I6dKOjJziSSZOFalDWcovdJUYG+hnU9vYUPeMfe0bhKlZRLBsVosJyQHg91Wrm5Gef/553XfffWrVqpUsFouaNWumoUOHau7cue4xffr0cX/doUMHJSQkqFGjRnr//fc1bNiwUrc7duxYjR492v19VlaWGjZsWHE7AgBALVDodGl7eo7HPVKbD2XJUVSySAX52dQ29uTVqAbFV6aahFOkAFQvppWr8PBw2Ww2paeneyxPT09XdHR0qetERERo4cKFys/P19GjRxUbG6sxY8aoadOm5/w5derUUcuWLbVjx45zjrHb7bLb7eXbEQAAoIIil7alZ7s/iHdDapY2H8pSQSlFKtjuo7Yn39rXvkFxoWpSP0hWihSAas60cuXn56fOnTsrOTlZAwYMkFQ8oUVycrJGjhx53nX9/f0VFxenwsJCffjhh7r99tvPOTYnJ0c7d+7U3Xff7c34AADUWqeK1JmTTWw5lK0CZ8kiFeLv435L36lC1ZgiBaCGMvVtgaNHj9aQIUPUpUsXde3aVTNnzlRubq6GDh0qSRo8eLDi4uI0ffp0SdLq1auVmpqqTp06KTU1VZMmTZLL5dLjjz/u3uZjjz2mpKQkNWrUSAcPHtTEiRNls9k0aNAgU/YRAIDqzFHk1Na0bI/JJramZavQWXKy4VB/H4/PkGofF6ZL6gVSpADUGqaWqzvuuEMZGRmaMGGC0tLS1KlTJy1atMg9ycW+fftktZ6e/Sc/P1/jxo3Trl27FBwcrL59++rNN99UnTp13GMOHDigQYMG6ejRo4qIiFCPHj20atUqRUREVPbuAQBQreQXOrUl7eRb+04WqW3ppRepsABfjxLVPi5MDesFyGKhSAGovUz9nKuqis+5AgDUdPmFTm0+lHXGZBNZ2p6erSJXyZcFdQN9i2fsO6NINahLkQJQO1SLz7kCAACV40SBU5vOKFIpqZnafjhHzlKKVL0gv5Ml6vRnScXVoUgBwIWgXAEAUIPkFRRp08FTRSrrZJHKVik9SuHBfh73SLWLC1NsmD9FCgDKiXIFAEA1leso0qZDWR4fyLszI6fUIhURYi8uUSc/lLd9gzBFh1KkAMCbKFcAAFQDOY4ibTzjbX0pB7O0MyNHpd05HXmqSJ26R6pBmKJC/Ss/NADUMpQrAACqmOz8Qm08eOZkE5nafSS31CIVHep/RokKVbvYMEVSpADAFJQrAABMlJVfeMbU58WFaveR3FLHxob5e8za1y4uTBEh9kpODAA4F8oVAACVJDOvUCkHz3hrX2qm9hzNK3VsXJ0AtTtjxr52cWEKD6ZIAUBVRrkCAKACHM8rUEpqlrtIbUjN1L5jpRepBnUDPO6RahcXpnpBfpWcGABwsShXAABcpN9yC9z3Rp0qUgd+O1Hq2EvqBap9XJjanroqFRumuhQpAKgRKFcAAJTB0RzHGW/rK74ylXq89CLVqH7g6ckmThapsEDfSk4MAKgslCsAAEphGIYychzFs/YdOH1V6mBmfqnjm4QHnSxSxZ8j1TY2TGEBFCkAqE0oVwCAWsXlMvRbXoEOZzuUnpWvw9kOZZz6Osuh9OziPzOyHSpwukqsb7GcLFKxp++PahsXqlB/ihQA1HaUKwBAjeByGTqWV+AuTIdPlqUzS9ThrHxl5DhU6CzlA6NKYbFITcODPCabaBMbqhCKFACgFJQrAECV5nQZOprrOFmUTl5dOvl1epZDGdmnrz4VuS6sNElS/SA/RYTYFRXqr8hTf4baFRlS/GdUqL/Cg/1k97FV4N4BAGoSyhUAwBRFTpeO5ha4S9PZhenU90dyCuS8wNJksUj1g+wny9LpohR5ZoEKsSs82C4/H2sF7yEAoLahXAEAvKrI6dKRnNNvzzt9X5NngTqa49CFXmiyWqT6wacLU1SoXREn/4wMOV2c6gf7yddGaQIAmINyBQC4IIVOlzKyS97D5HnlyaGjuQ4ZF1iabFaLwoP9Si1MZ155qh/kJx9KEwCgiqNcAUAtV1DkUkbO6dnyTt/XdPrKU0a2Q0dzCy54mzarRZEhxW/PK1GYPEqTXTarpQL3DgCAykO5AoAaylHk9ChLZ15xOlWY0rPy9Vte4QVv09dmUURwyXuYzr6vqV6gn6yUJgBALUO5AoBqJr/QWWISiLML0+Fsh46XoTT52ayKOFmSos6YLS8ixO5RoupSmgAAOCfKFQBUEScKnCVmzTt91en0W/Wy8osueJt+PtYSkz6UmH48xK46gb6yWChNAABcDMoVAFSwXEeRe/KH9FOTQJz6/owrT9llKE3+vlaPSR9K+7ymqBB/hQb4UJoAAKgklCsAKKccR1GJSSBKXnFyKMdx4aUpwNfm+flMZ00CcWpGvVB/ShMAAFUN5QoAzmAYhrIdRcXF6KzPaTr7vqa8AucFbzfIz+ae8CEy1F9Rp+5vct/XVFycgu2UJgAAqivKFYBawTAMZeUXlShM7g+4zXIo/eTVphOFF16aQuw+ijhjEojS7muKDPVXsJ3/3AIAUNPxf3sA1ZphGMo8UejxVrz0syeBOPmno8h1wdsN8fcpMelDafc1Bfrxn1EAAFCMVwUAqiTDMPRbXuHpe5jOmATi7LfqFZShNIUF+J71+UyeZSny5Fv0AvxsFbh3AACgJqJcAahULpeh3/IKSkwzfuaseYezHMrIdqjAeeGlqW6gr8ckEMWz5Z28v+mMGfX8fSlNAACgYlCuAFSY1OMn9NryXUo9fuLkfU3F5anIZVzwNuoF+ZWYBOLMWfOK/7TL7kNpAgAA5qJcAagQhU6Xhr6+RtvSc0p9PjzYz12OPO9rOjX1uL8igu3y87FWcnIAAIDyoVwBqBBzV+zWtvQc1Qvy0yPXt1TUGfc1hQfb5WujNAEAgJqFcgXA6w4eP6GZS7ZLksb2aaXbujQ0OREAAEDF45+OAXjdlM826UShU5c3rqtbLmtgdhwAAIBKQbkC4FVLtxzWoo1pslktmjagvaxWi9mRAAAAKgXlCoDX5Bc6NfHTjZKkYT2aKD46xOREAAAAlYdyBcBrXlq6Q/uO5Sk61F+jrmthdhwAAIBKRbkC4BW7MnI0+7tdkqSJSW0UZGe+HAAAULtQrgBcNMMwNOGTjSpwunRVywj1bhdtdiQAAIBKR7kCcNG+2HBIK3YckZ+PVZNvaiuLhUksAABA7UO5AnBRsvMLNeWzTZKkEVc3V+PwIJMTAQAAmMP0cvXiiy+qcePG8vf3V0JCgtasWXPOsYWFhZoyZYqaNWsmf39/dezYUYsWLbqobQK4ODOXbNfhbIca1w/Un69qanYcAAAA05harubPn6/Ro0dr4sSJWrdunTp27KjExEQdPny41PHjxo3TK6+8on//+9/atGmT7r//ft188836+eefy71NAOW36WCW5v2wR5I0pX87+fvazA0EAABgIothGIZZPzwhIUGXX365Zs2aJUlyuVxq2LChHnzwQY0ZM6bE+NjYWD355JMaMWKEe9ktt9yigIAAvfXWW+XapiQ5HA45HA7391lZWWrYsKEyMzMVGhrqtf0FahKXy9Cts3/Qun3H1a99jF686zKzIwEAAHhdVlaWwsLCLqgbmHblqqCgQGvXrlWvXr1Oh7Fa1atXL61cubLUdRwOh/z9/T2WBQQEaMWKFeXepiRNnz5dYWFh7kfDhg0vZteAWmHB2v1at++4gvxsGn9jG7PjAAAAmM60cnXkyBE5nU5FRUV5LI+KilJaWlqp6yQmJmrGjBnavn27XC6XFi9erI8++kiHDh0q9zYlaezYscrMzHQ/9u/ff5F7B9Rsx3ILNP2rLZKkR65vqegw/99ZAwAAoOYzfUKLsnj++efVokULtWrVSn5+fho5cqSGDh0qq/XidsNutys0NNTjAeDcnl20RcfzCtUqOkT3dG9sdhwAAIAqwbRyFR4eLpvNpvT0dI/l6enpio4u/QNIIyIitHDhQuXm5mrv3r3asmWLgoOD1bRp03JvE0DZrN37m977sfjq7rQB7eRjq1b/RgMAAFBhTHtV5Ofnp86dOys5Odm9zOVyKTk5Wd26dTvvuv7+/oqLi1NRUZE+/PBD9e/f/6K3CeD3FTldGrcwRZJ0e5cG6tK4nsmJAAAAqg4fM3/46NGjNWTIEHXp0kVdu3bVzJkzlZubq6FDh0qSBg8erLi4OE2fPl2StHr1aqWmpqpTp05KTU3VpEmT5HK59Pjjj1/wNgGU3xsr92rzoSzVCfTVmD6tzY4DAABQpZharu644w5lZGRowoQJSktLU6dOnbRo0SL3hBT79u3zuJ8qPz9f48aN065duxQcHKy+ffvqzTffVJ06dS54mwDKJy0zXzO+2SpJ+lvvVqoX5GdyIgAAgKrF1M+5qqrKMpc9UFuMfGedPv/1kC69pI4+vL+7rFaL2ZEAAAAqXLX4nCsA1cf/tmfo818PyWopnsSCYgUAAFAS5QrAeTmKnJrwyUZJ0pDujdU2NszkRAAAAFUT5QrAeb363S7tPpKryBC7Rl/f0uw4AAAAVRblCsA57Tuap1lLd0iSxt3YRiH+viYnAgAAqLooVwBKZRiGJn6aIkeRS1c2r6+kDjFmRwIAAKjSKFcASvX1xnQt3ZohX5tFU/q3k8XCJBYAAADnQ7kCUEKuo0hTPiuexOLPf2imZhHBJicCAACo+ihXAEp44dvtOpiZr4b1AjTy2uZmxwEAAKgWKFcAPGxLz9ac/+2WJE1Kait/X5vJiQAAAKoHyhUAN8MwNG5hiopchm5oE6XrWkeZHQkAAKDaoFwBcPtoXarW7D6mAF+bJiS1MTsOAABAtUK5AiBJyswr1FNfbpYkPXRdCzWoG2hyIgAAgOqFcgVAkvSPb7boaG6BWkQGa1iPJmbHAQAAqHYoVwD0y/7jenv1PknS1AHt5OfDfxoAAADKildQQC3ndBVPYmEY0v9dGqcrmtY3OxIAAEC1RLkCarm3V+/VhtRMhfj7aGzf1mbHAQAAqLYoV0Atdjg7X//4eqsk6fHEeEWE2E1OBAAAUH1RroBabPqXW5SdX6T2cWG6M6GR2XEAAACqNcoVUEut3HlUH/+cKotF+vvN7WSzWsyOBAAAUK1RroBaqKDIpfGfpEiS/pjQSB0a1DE3EAAAQA1AuQJqoTkrdmvH4RyFB/vpsRvizY4DAABQI1CugFrmwG95eiF5uyTpib6tFRboa3IiAACAmoFyBdQykz/bpBOFTnVtUk83XxpndhwAAIAag3IF1CJLNqVr8aZ0+VgtmjagnSwWJrEAAADwFsoVUEucKHBq0mcbJUnDejZRy6gQkxMBAADULJQroJZ4cekOHfjthGLD/DXquhZmxwEAAKhxKFdALbAzI0evLN8pSZp4U1sF+vmYnAgAAKDmoVwBNZxhGJrwSYoKnYaubRWpG9pEmR0JAACgRqJcATXcp78c1Pc7jsruY9WkpLZMYgEAAFBBKFdADZaVX6hpX2yWJI28prkuqR9ociIAAICai3IF1GAzvtmmjGyHmoQHafhVTc2OAwAAUKNRroAaKiU1U/9duUeSNLV/O9l9bOYGAgAAqOEoV0AN5HIZGrcwRS5DSuoYqx4tws2OBAAAUONRroAaaP5P+7V+/3EF2300rl9rs+MAAADUCpQroIY5muPQ019tkSSNvr6lokL9TU4EAABQO1CugBrm6a+2KPNEoVrHhGpwt0ZmxwEAAKg1KFdADfLjnmNasPaAJGnagHbysfErDgAAUFl45QXUEIVOl8Z9nCJJGnh5Q3VuVNfkRAAAALUL5QqoId74YY+2pmerbqCv/ta7ldlxAAAAah3Ty9WLL76oxo0by9/fXwkJCVqzZs15x8+cOVPx8fEKCAhQw4YN9cgjjyg/P9/9/KRJk2SxWDwerVrxQhM126HME3pu8TZJ0tg+rVU3yM/kRAAAALWPj5k/fP78+Ro9erRmz56thIQEzZw5U4mJidq6dasiIyNLjH/nnXc0ZswYzZ07V927d9e2bdt0zz33yGKxaMaMGe5xbdu21ZIlS9zf+/iYuptAhZv2+WblFjjVuVFd3dq5gdlxAAAAaiVTr1zNmDFD9913n4YOHao2bdpo9uzZCgwM1Ny5c0sd/8MPP+jKK6/UnXfeqcaNG+uGG27QoEGDSlzt8vHxUXR0tPsRHs4HqKLm+m5bhr7YcEg2q0XTBrST1WoxOxIAAECtZFq5Kigo0Nq1a9WrV6/TYaxW9erVSytXrix1ne7du2vt2rXuMrVr1y59+eWX6tu3r8e47du3KzY2Vk2bNtVdd92lffv2nTeLw+FQVlaWxwOoDvILnZrwSfEkFvd0b6zWMaEmJwIAAKi9THu/3JEjR+R0OhUVFeWxPCoqSlu2bCl1nTvvvFNHjhxRjx49ZBiGioqKdP/99+uJJ55wj0lISNC8efMUHx+vQ4cOafLkyerZs6dSUlIUEhJS6nanT5+uyZMne2/ngEoy+7ud2ns0T1Ghdj1yfUuz4wAAANRqpk9oURbLli3TU089pZdeeknr1q3TRx99pC+++EJTp051j+nTp49uu+02dejQQYmJifryyy91/Phxvf/+++fc7tixY5WZmel+7N+/vzJ2B7goe47k6qVlOyVJE25sq2A79xYCAACYybRXY+Hh4bLZbEpPT/dYnp6erujo6FLXGT9+vO6++27de++9kqT27dsrNzdXw4cP15NPPimrtWRXrFOnjlq2bKkdO3acM4vdbpfdbr+IvQEql2EYmvDpRhUUudSzRbj6ti/9dwYAAACVx7QrV35+furcubOSk5Pdy1wul5KTk9WtW7dS18nLyytRoGw2m6TiF5ulycnJ0c6dOxUTE+Ol5ID5FqWkafm2DPnZrJrSv50sFiaxAAAAMJup7yMaPXq0hgwZoi5duqhr166aOXOmcnNzNXToUEnS4MGDFRcXp+nTp0uSkpKSNGPGDF166aVKSEjQjh07NH78eCUlJblL1mOPPaakpCQ1atRIBw8e1MSJE2Wz2TRo0CDT9hPwphxHkSZ/tkmSdP/VzdQkPMjkRAAAAJBMLld33HGHMjIyNGHCBKWlpalTp05atGiRe5KLffv2eVypGjdunCwWi8aNG6fU1FRFREQoKSlJf//7391jDhw4oEGDBuno0aOKiIhQjx49tGrVKkVERFT6/gEV4fkl25SWla9L6gXqL1c3MzsOAAAATrIY53o/XS2WlZWlsLAwZWZmKjSUqa1RdWxJy1K/F1bI6TL0+tDLdU18yQ/bBgAAgPeUpRtUq9kCgdrM5TI07uMUOV2G+rSLplgBAABUMZQroJr4cN0B/bT3NwX62TT+xjZmxwEAAMBZKFdANXA8r0DTvyr+cO2He7VQbJ0AkxMBAADgbJQroBp4ZtFWHcstUMuoYA29sonZcQAAAFAKyhVQxa3b95ve+3GfJGnagPbytfFrCwAAUBXxKg2owoqcLo1fmCLDkG65rIG6NqlndiQAAACcA+UKqMLeWrVXGw9mKSzAV0/0bWV2HAAAAJwH5Qqoog5n5etf32yTJD3eO171g+0mJwIAAMD5UK6AKurvX25WtqNIHRvW0cDLLzE7DgAAAH4H5Qqogr7fcUSfrD8oq0Wa1r+dbFaL2ZEAAADwOyhXQBXjKHJq/CcpkqS7r2ik9g3CTE4EAACAC0G5AqqY//xvt3Zl5Co82K7RN8SbHQcAAAAXiHIFVCH7j+XpheTtkqRx/VorLMDX5EQAAAC4UJQroAqZ/NlGOYpc6ta0vvp3ijU7DgAAAMqgzOWqcePGmjJlivbt21cReYBaa/GmdC3ZfFi+NoumDmgri4VJLAAAAKqTMperhx9+WB999JGaNm2q66+/Xu+9954cDkdFZANqjbyCIk36dKMk6b6eTdU8MsTkRAAAACircpWr9evXa82aNWrdurUefPBBxcTEaOTIkVq3bl1FZARqvH9/u0Opx08ork6AHry2hdlxAAAAUA7lvufqsssu0wsvvKCDBw9q4sSJ+s9//qPLL79cnTp10ty5c2UYhjdzAjXW9vRsvbZ8lyRp0k1tFeBnMzkRAAAAysOnvCsWFhbq448/1uuvv67Fixfriiuu0LBhw3TgwAE98cQTWrJkid555x1vZgVqHMMwNP6TFBW5DPVqHanr20SZHQkAAADlVOZytW7dOr3++ut69913ZbVaNXjwYD333HNq1aqVe8zNN9+syy+/3KtBgZrok/UHtWrXMfn7WjUxqa3ZcQAAAHARylyuLr/8cl1//fV6+eWXNWDAAPn6lvwcniZNmmjgwIFeCQjUVJknCjXti82SpAevbaGG9QJNTgQAAICLUeZytWvXLjVq1Oi8Y4KCgvT666+XOxRQG8z4ZquO5DjULCJI9/VsanYcAAAAXKQyT2hx+PBhrV69usTy1atX66effvJKKKCm23AgU2+u2itJmtq/nfx8+DxvAACA6q7Mr+hGjBih/fv3l1iempqqESNGeCUUUJM5XYbGLdwglyH17xSr7s3DzY4EAAAALyhzudq0aZMuu+yyEssvvfRSbdq0ySuhgJrs3TX79MuBTIXYffRkv9ZmxwEAAICXlLlc2e12paenl1h+6NAh+fiUe2Z3oFY4kuPQs4u2SJIeS4xXZIi/yYkAAADgLWUuVzfccIPGjh2rzMxM97Ljx4/riSee0PXXX+/VcEBNM/3LLcrKL1Lb2FD98YrzTwwDAACA6qXMl5r++c9/6g9/+IMaNWqkSy+9VJK0fv16RUVF6c033/R6QKCmWL3rqD5cd0AWizRtQDvZrBazIwEAAMCLylyu4uLi9Ouvv+rtt9/WL7/8ooCAAA0dOlSDBg0q9TOvAEiFTpfGf5IiSRrU9RJdekldkxMBAADA28p1k1RQUJCGDx/u7SxAjTV3xW5tS89RvSA/PZ4Yb3YcAAAAVIByz0CxadMm7du3TwUFBR7Lb7rpposOBdQkB4+f0Mwl2yVJY/u0Up1AP5MTAQAAoCKUuVzt2rVLN998szZs2CCLxSLDMCRJFkvx/SNOp9O7CYFqbspnm3Si0Kmujevp1s4NzI4DAACAClLm2QJHjRqlJk2a6PDhwwoMDNTGjRu1fPlydenSRcuWLauAiED1tXTLYS3amCab1aKpA9q5/xECAAAANU+Zr1ytXLlS3377rcLDw2W1WmW1WtWjRw9Nnz5dDz30kH7++eeKyAlUO/mFTk38dKMkaViPJoqPDjE5EQAAACpSma9cOZ1OhYQUv0gMDw/XwYMHJUmNGjXS1q1bvZsOqMZeWrZT+47lKTrUX6Oua2F2HAAAAFSwMl+5ateunX755Rc1adJECQkJevbZZ+Xn56dXX31VTZs2rYiMQLWzKyNHs5ftlCRNTGqjIHu5544BAABANVHmV3zjxo1Tbm6uJGnKlCm68cYb1bNnT9WvX1/z58/3ekCgujEMQxM+2agCp0tXtYxQ73bRZkcCAABAJShzuUpMTHR/3bx5c23ZskXHjh1T3bp1uVkfkPTFhkNaseOI/HysmtK/Lb8XAAAAtUSZ7rkqLCyUj4+PUlJSPJbXq1ePF5CApOz8Qk35bJMkacTVzdWofpDJiQAAAFBZylSufH19dckll/BZVsA5zFyyXYezHWpcP1B/vop7EAEAAGqTMs8W+OSTT+qJJ57QsWPHvBLgxRdfVOPGjeXv76+EhAStWbPmvONnzpyp+Ph4BQQEqGHDhnrkkUeUn59/UdsEvGHTwSzN+2GPJGlK/3by97WZGwgAAACVqsz3XM2aNUs7duxQbGysGjVqpKAgz7c9rVu37oK3NX/+fI0ePVqzZ89WQkKCZs6cqcTERG3dulWRkZElxr/zzjsaM2aM5s6dq+7du2vbtm265557ZLFYNGPGjHJtE/AGl8vQuIUb5HQZ6tc+Rn9oGWF2JAAAAFQyi2EYRllWmDx58nmfnzhx4gVvKyEhQZdffrlmzZolSXK5XGrYsKEefPBBjRkzpsT4kSNHavPmzUpOTnYve/TRR7V69WqtWLGiXNssTVZWlsLCwpSZmanQ0NAL3h/UXvN/3Ke/fbhBQX42JT96taLD/M2OBAAAAC8oSzco85WrspSn8ykoKNDatWs1duxY9zKr1apevXpp5cqVpa7TvXt3vfXWW1qzZo26du2qXbt26csvv9Tdd99d7m1KksPhkMPhcH+flZV1sbuHWuRYboGmf7VFkvTI9S0pVgAAALWUaZ9seuTIETmdTkVFRXksj4qK0pYtW0pd584779SRI0fUo0cPGYahoqIi3X///XriiSfKvU1Jmj59+u9ekQPO5dlFW3Q8r1CtokN0T/fGZscBAACASco8oYXVapXNZjvnoyItW7ZMTz31lF566SWtW7dOH330kb744gtNnTr1orY7duxYZWZmuh/79+/3UmLUdGv3/qb3fiw+X6YNaCcfW5l/pQAAAFBDlPnK1ccff+zxfWFhoX7++We98cYbZbr6Ex4eLpvNpvT0dI/l6enpio6OLnWd8ePH6+6779a9994rSWrfvr1yc3M1fPhwPfnkk+XapiTZ7XbZ7fYLzg5IUpHTpXELiz/z7fYuDdSlcT2TEwEAAMBMZS5X/fv3L7Hs1ltvVdu2bTV//nwNGzbsgrbj5+enzp07Kzk5WQMGDJBUPPlEcnKyRo4cWeo6eXl5slo9rwyculpmGEa5tgmU1xsr92rzoSzVCfTVmD6tzY4DAAAAk3ntnqsrrrhCw4cPL9M6o0eP1pAhQ9SlSxd17dpVM2fOVG5uroYOHSpJGjx4sOLi4jR9+nRJUlJSkmbMmKFLL71UCQkJ2rFjh8aPH6+kpCR3yfq9bQLekJaZrxnfbJUk/a13K9UL8jM5EQAAAMzmlXJ14sQJvfDCC4qLiyvTenfccYcyMjI0YcIEpaWlqVOnTlq0aJF7Qop9+/Z5XKkaN26cLBaLxo0bp9TUVEVERCgpKUl///vfL3ibgDdM+2KTcgucuvSSOrqjS0Oz4wAAAKAKKPPnXNWtW1cWi8X9vWEYys7OVmBgoN566y3ddNNNXg9Z2ficK5zP/7Zn6O45a2S1SJ892ENtY8PMjgQAAIAKUqGfc/Xcc895lCur1aqIiAglJCSobt26ZU8LVCOOIqcmfLJRkjSke2OKFQAAANzKXK7uueeeCogBVA+vfrdLu4/kKjLErtHXtzQ7DgAAAKqQMn8oz+uvv64FCxaUWL5gwQK98cYbXgkFVEX7juZp1tIdkqRxN7ZRiL+vyYkAAABQlZS5XE2fPl3h4eEllkdGRuqpp57ySiigqjEMQxM/TZGjyKUrm9dXUocYsyMBAACgiilzudq3b5+aNGlSYnmjRo20b98+r4QCqpqvN6Zr6dYM+dmsmtK/ncd9hwAAAIBUjnIVGRmpX3/9tcTyX375RfXr1/dKKKAqyXUUacpnxZNY/PmqpmoWEWxyIgAAAFRFZS5XgwYN0kMPPaSlS5fK6XTK6XTq22+/1ahRozRw4MCKyAiY6oVvt+tgZr4a1gvQiGuamx0HAAAAVVSZZwucOnWq9uzZo+uuu04+PsWru1wuDR48mHuuUONsS8/WnP/tliRNSmorf1+byYkAAABQVZW5XPn5+Wn+/PmaNm2a1q9fr4CAALVv316NGjWqiHyAaQzD0LiFKSpyGbqhTZSuax1ldiQAAABUYWUuV6e0aNFCLVq08GYWoEr5aF2q1uw+pgBfmyYktTE7DgAAAKq4Mt9zdcstt+iZZ54psfzZZ5/Vbbfd5pVQgNky8wr11JebJUkPXddCDeoGmpwIAAAAVV2Zy9Xy5cvVt2/fEsv79Omj5cuXeyUUYLZ/fLNFR3ML1CIyWMN6lPzoAQAAAOBsZS5XOTk58vPzK7Hc19dXWVlZXgkFmOmX/cf19uriz2ybOqCd/HzK/GsCAACAWqjMrxrbt2+v+fPnl1j+3nvvqU0b7ktB9eZ0FU9iYRjS/10apyua8tltAAAAuDBlntBi/Pjx+r//+z/t3LlT1157rSQpOTlZ77zzjj744AOvBwQq0zur92pDaqZC/H00tm9rs+MAAACgGilzuUpKStLChQv11FNP6YMPPlBAQIA6duyob7/9VvXq1auIjEClOJydr2e/3ipJejwxXhEhdpMTAQAAoDop11Ts/fr1U79+/SRJWVlZevfdd/XYY49p7dq1cjqdXg0IVJbpX25Rdn6R2seF6c4EPrcNAAAAZVPuO/WXL1+uIUOGKDY2Vv/617907bXXatWqVd7MBlSalTuP6uOfU2WxSH+/uZ1sVovZkQAAAFDNlOnKVVpamubNm6c5c+YoKytLt99+uxwOhxYuXMhkFqi2CopcGv9JiiTpjwmN1KFBHXMDAQAAoFq64CtXSUlJio+P16+//qqZM2fq4MGD+ve//12R2YBKMWfFbu04nKPwYD89dkO82XEAAABQTV3wlauvvvpKDz30kB544AG1aNGiIjMBlebAb3l6IXm7JOmJvq0VFuhrciIAAABUVxd85WrFihXKzs5W586dlZCQoFmzZunIkSMVmQ2ocJM/26QThU51bVJPN18aZ3YcAAAAVGMXXK6uuOIKvfbaazp06JD+/Oc/67333lNsbKxcLpcWL16s7OzsiswJeN2STelavCldPlaLpg1oJ4uFSSwAAABQfmWeLTAoKEh/+tOftGLFCm3YsEGPPvqonn76aUVGRuqmm26qiIyA150ocGrSZxslScN6NlHLqBCTEwEAAKC6K/dU7JIUHx+vZ599VgcOHNC7777rrUxAhXtx6Q4d+O2EYsP8Neo67iEEAADAxbuocnWKzWbTgAED9Omnn3pjc0CF2pmRo1eW75QkTbyprQL9yvVZ2gAAAIAHr5QroLowDEMTPklRodPQta0idUObKLMjAQAAoIagXKFW+ezXQ/p+x1HZfayalNSWSSwAAADgNZQr1BpZ+YWa+vkmSdLIa5rrkvqBJicCAABATUK5Qq0x45ttysh2qGl4kIZf1dTsOAAAAKhhKFeoFVJSM/XflXskSVP6t5Pdx2ZuIAAAANQ4lCvUeC6XoXELU+QypKSOserRItzsSAAAAKiBKFeo8eb/tF/r9x9XsN1H4/q1NjsOAAAAaijKFWq0ozkOPf3VFknS6OtbKirU3+REAAAAqKkoV6jRnv5qizJPFKp1TKgGd2tkdhwAAADUYJQr1Fg/7jmmBWsPSJKmDWgnHxunOwAAACoOrzZRIxU6XRr3cYokaVDXhurcqK7JiQAAAFDTUa5QI73xwx5tTc9W3UBfPZ7Yyuw4AAAAqAUoV6hxDmWe0HOLt0mSxvZprbpBfiYnAgAAQG1AuUKNM+3zzcotcKpzo7q6tXMDs+MAAACglqgS5erFF19U48aN5e/vr4SEBK1Zs+acY6+++mpZLJYSj379+rnH3HPPPSWe7927d2XsCkz23bYMfbHhkGxWi6YNaCer1WJ2JAAAANQSPmYHmD9/vkaPHq3Zs2crISFBM2fOVGJiorZu3arIyMgS4z/66CMVFBS4vz969Kg6duyo2267zWNc79699frrr7u/t9vtFbcTqBLyC52a8EnxJBb3dG+s1jGhJicCAABAbWL6lasZM2bovvvu09ChQ9WmTRvNnj1bgYGBmjt3bqnj69Wrp+joaPdj8eLFCgwMLFGu7Ha7x7i6dZktrqab/d1O7T2ap6hQux65vqXZcQAAAFDLmFquCgoKtHbtWvXq1cu9zGq1qlevXlq5cuUFbWPOnDkaOHCggoKCPJYvW7ZMkZGRio+P1wMPPKCjR4+ecxsOh0NZWVkeD1Qve47k6qVlOyVJE25sq2C76RdlAQAAUMuYWq6OHDkip9OpqKgoj+VRUVFKS0v73fXXrFmjlJQU3XvvvR7Le/furf/+979KTk7WM888o++++059+vSR0+ksdTvTp09XWFiY+9GwYcPy7xQqnWEYmvDpRhUUudSzRbj6to82OxIAAABqoWr9z/tz5sxR+/bt1bVrV4/lAwcOdH/dvn17dejQQc2aNdOyZct03XXXldjO2LFjNXr0aPf3WVlZFKxqZFFKmpZvy5Cfzaop/dvJYmESCwAAAFQ+U69chYeHy2azKT093WN5enq6oqPPf/UhNzdX7733noYNG/a7P6dp06YKDw/Xjh07Sn3ebrcrNDTU44HqIcdRpMmfbZIk3X91MzUJD/qdNQAAAICKYWq58vPzU+fOnZWcnOxe5nK5lJycrG7dup133QULFsjhcOiPf/zj7/6cAwcO6OjRo4qJibnozKhanl+yTWlZ+bqkXqD+cnUzs+MAAACgFjN9tsDRo0frtdde0xtvvKHNmzfrgQceUG5uroYOHSpJGjx4sMaOHVtivTlz5mjAgAGqX7++x/KcnBz99a9/1apVq7Rnzx4lJyerf//+at68uRITEytln1A5tqRlae73eyRJk/u3lb+vzdxAAAAAqNVMv+fqjjvuUEZGhiZMmKC0tDR16tRJixYtck9ysW/fPlmtnh1w69atWrFihb755psS27PZbPr111/1xhtv6Pjx44qNjdUNN9ygqVOn8llXNYjLZWjcxylyugz1aReta+JLfiYaAAAAUJkshmEYZoeoarKyshQWFqbMzEzuv6qiFvy0X3/94FcF+tm0ZPRViq0TYHYkAAAA1EBl6Qamvy0QKKvjeQWa/tUWSdLDvVpQrAAAAFAlUK5Q7Tz79VYdyy1Qy6hgDb2yidlxAAAAAEmUK1QzP+/7Te+u2SdJmjagvXxtnMIAAACoGnhlimqjyOnSuIUpMgzplssaqGuTemZHAgAAANwoV6g23lq1VxsPZikswFdP9G1ldhwAAADAA+UK1cLhrHz965ttkqTHe8erfjDT6gMAAKBqoVyhWvj7l5uV7ShSx4Z1NPDyS8yOAwAAAJRAuUKV98OOI/pk/UFZLdK0/u1ks1rMjgQAAACUQLlCleYocmrcJymSpLuvaKT2DcJMTgQAAACUjnKFKu0//9utXRm5Cg+2a/QN8WbHAQAAAM6JcoUqa/+xPL2QvF2SNP7G1goL8DU5EQAAAHBulCtUWZM/2yhHkUvdmtbXTR1jzY4DAAAAnBflClXS4k3pWrL5sHxtFk0d0FYWC5NYAAAAoGqjXKHKySso0qRPN0qS7uvZVM0jQ0xOBAAAAPw+yhWqnH9/u0Opx08ork6AHry2hdlxAAAAgAtCuUKVsj09W68t3yVJmnRTWwX42UxOBAAAAFwYyhWqDMMwNP6TFBW5DPVqHaXr20SZHQkAAAC4YJQrVBmfrD+oVbuOyd/XqolJbcyOAwAAAJQJ5QpVQuaJQk37YrMk6cFrW6hhvUCTEwEAAABlQ7lClTDjm606kuNQs4gg3dezqdlxAAAAgDKjXMF0Gw5k6s1VeyVJU/u3k58PpyUAAACqH17FwlROl6FxCzfIZUj9O8Wqe/NwsyMBAAAA5UK5gqneXbNPvxzIVIjdR0/2a212HAAAAKDcKFcwzZEch55dtEWS9FhivCJD/E1OBAAAAJQf5Qqmmf7lFmXlF6ltbKj+eEUjs+MAAAAAF4VyBVOs3nVUH647IItFmjagnWxWi9mRAAAAgItCuUKlK3S6NP6TFEnSoK6X6NJL6pqcCAAAALh4lCtUurkrdmtbeo7qBfnp8cR4s+MAAAAAXkG5QqU6ePyEZi7ZLkka26eV6gT6mZwIAAAA8A7KFSrVlM826UShU10b19OtnRuYHQcAAADwGsoVKs3SLYe1aGOabFaLpg5oJ4uFSSwAAABQc1CuUCnyC52a+OlGSdKwHk0UHx1iciIAAADAuyhXqBQvLdupfcfyFB3qr1HXtTA7DgAAAOB1lCtUuF0ZOZq9bKckaWJSGwXZfUxOBAAAAHgf5QoVyjAMTfhkowqcLl3VMkK920WbHQkAAACoEJQrVKgvNhzSih1H5Odj1ZT+bZnEAgAAADUW5QoVJju/UFM+2yRJGnF1czWqH2RyIgAAAKDiUK5QYWYu2a7D2Q41rh+oP1/V1Ow4AAAAQIWiXKFCbDqYpXk/7JEkTenfTv6+NnMDAQAAABWMcgWvc7kMjVu4QU6XoX7tY/SHlhFmRwIAAAAqXJUoVy+++KIaN24sf39/JSQkaM2aNecce/XVV8tisZR49OvXzz3GMAxNmDBBMTExCggIUK9evbR9+/bK2BVIWrB2v9btO64gP5vG39jG7DgAAABApTC9XM2fP1+jR4/WxIkTtW7dOnXs2FGJiYk6fPhwqeM/+ugjHTp0yP1ISUmRzWbTbbfd5h7z7LPP6oUXXtDs2bO1evVqBQUFKTExUfn5+ZW1W7XWsdwCTf9qiyTpketbKjrM3+REAAAAQOUwvVzNmDFD9913n4YOHao2bdpo9uzZCgwM1Ny5c0sdX69ePUVHR7sfixcvVmBgoLtcGYahmTNnaty4cerfv786dOig//73vzp48KAWLlxY6jYdDoeysrI8HiifZxdt0fG8QrWKDtE93RubHQcAAACoNKaWq4KCAq1du1a9evVyL7NarerVq5dWrlx5QduYM2eOBg4cqKCg4mm+d+/erbS0NI9thoWFKSEh4ZzbnD59usLCwtyPhg0bXsRe1V5r9/6m937cL0maNqCdfGymd3cAAACg0pj66vfIkSNyOp2KioryWB4VFaW0tLTfXX/NmjVKSUnRvffe6152ar2ybHPs2LHKzMx0P/bv31/WXan1ipwujVuYIkm6vUsDdWlcz+REAAAAQOXyMTvAxZgzZ47at2+vrl27XtR27Ha77Ha7l1LVTv9duVebD2WpTqCvxvRpbXYcAAAAoNKZeuUqPDxcNptN6enpHsvT09MVHR193nVzc3P13nvvadiwYR7LT61Xnm2ifNKz8jVj8TZJ0t96t1K9ID+TEwEAAACVz9Ry5efnp86dOys5Odm9zOVyKTk5Wd26dTvvugsWLJDD4dAf//hHj+VNmjRRdHS0xzazsrK0evXq390mymfq55uU4yjSpZfU0R1duF8NAAAAtZPpbwscPXq0hgwZoi5duqhr166aOXOmcnNzNXToUEnS4MGDFRcXp+nTp3usN2fOHA0YMED169f3WG6xWPTwww9r2rRpatGihZo0aaLx48crNjZWAwYMqKzdqjX+tz1Dn/96SFZL8SQWVqvF7EgAAACAKUwvV3fccYcyMjI0YcIEpaWlqVOnTlq0aJF7Qop9+/bJavW8wLZ161atWLFC33zzTanbfPzxx5Wbm6vhw4fr+PHj6tGjhxYtWiR/fz5zyZscRU5N+GSjJGlI98ZqGxtmciIAAADAPBbDMAyzQ1Q1WVlZCgsLU2ZmpkJDQ82OU2X9O3m7/rV4myJD7Ep+9CqF+PuaHQkAAADwqrJ0Az6ICOWy72ieZi3dIUkad2MbihUAAABqPcoVyswwDE38NEWOIpeubF5fSR1izI4EAAAAmI5yhTL7emO6lm7NkJ/Nqin928liYRILAAAAgHKFMsl1FGnKZ8WTWPz5qqZqFhFsciIAAACgaqBcoUxe+Ha7Dmbmq2G9AI24prnZcQAAAIAqg3KFC7YtPVtz/rdbkjQpqa38fW0mJwIAAACqDsoVLohhGBq3MEVFLkM3tInSda2jzI4EAAAAVCmUK1yQj9alas3uYwrwtWlCUhuz4wAAAABVDuUKvyszr1BPfblZkvTQdS3UoG6gyYkAAACAqodyhd/1j2+26GhugVpEBmtYjyZmxwEAAACqJMoVzuuX/cf19up9kqSpA9rJz4dTBgAAACgNr5RxTk5X8SQWhiH936VxuqJpfbMjAQAAAFUW5Qrn9M7qvdqQmqkQfx+N7dva7DgAAABAlUa5Qqkysh169uutkqTHE+MVEWI3OREAAABQtVGuUKqnvtys7PwidWgQpjsTGpkdBwAAAKjyKFcoYeXOo/r451RZLNK0Ae1ks1rMjgQAAABUeZQreCgocmn8JymSpD8mNFKHBnXMDQQAAABUE5QreJizYrd2HM5ReLCfHrsh3uw4AAAAQLVBuYLbgd/y9ELydknSE31bKyzQ1+REAAAAQPVBuYLblM826UShU12b1NPNl8aZHQcAAACoVihXkCQlb07XN5vS5WO1aNqAdrJYmMQCAAAAKAvKFXSiwKmJn26UJN3bs6laRoWYnAgAAACofihX0ItLd+jAbycUG+avh65rbnYcAAAAoFqiXNVyOzNy9MrynZKkiTe1VaCfj8mJAAAAgOqJclWLGYahCZ+kqNBp6NpWkbqhTZTZkQAAAIBqi3JVi3326yF9v+Oo7D5WTUpqyyQWAAAAwEWgXNVSWfmFmvr5JknSyGua65L6gSYnAgAAAKo3ylUtNeObbcrIdqhpeJCGX9XU7DgAAABAtUe5qoVSUjP135V7JElT+reT3cdmbiAAAACgBqBc1TIul6FxC1PkMqSkjrHq0SLc7EgAAABAjUC5qmXm/7Rf6/cfV7DdR+P6tTY7DgAAAFBjUK5qkaM5Dj391RZJ0ujrWyoq1N/kRAAAAEDNQbmqRZ7+aosyTxSqdUyoBndrZHYcAAAAoEahXNUSP+45pgVrD0iSpg1oJx8bf/UAAACAN/EKuxYodLo07uMUSdKgrg3VuVFdkxMBAAAANQ/lqhZ444c92pqerbqBvno8sZXZcQAAAIAaiXJVwx3KPKHnFm+TJI3t01p1g/xMTgQAAADUTJSrGm7a55uVW+BU50Z1dWvnBmbHAQAAAGosylUN9t22DH2x4ZBsVoumDWgnq9VidiQAAACgxjK9XL344otq3Lix/P39lZCQoDVr1px3/PHjxzVixAjFxMTIbrerZcuW+vLLL93PT5o0SRaLxePRqlXtu88ov9CpiZ8UT2JxT/fGah0TanIiAAAAoGbzMfOHz58/X6NHj9bs2bOVkJCgmTNnKjExUVu3blVkZGSJ8QUFBbr++usVGRmpDz74QHFxcdq7d6/q1KnjMa5t27ZasmSJ+3sfH1N30xSzv9upPUfzFBVq1yPXtzQ7DgAAAFDjmdo6ZsyYofvuu09Dhw6VJM2ePVtffPGF5s6dqzFjxpQYP3fuXB07dkw//PCDfH19JUmNGzcuMc7Hx0fR0dEVmr0q23MkVy8t2ylJmnBjWwXba1+5BAAAACqbaW8LLCgo0Nq1a9WrV6/TYaxW9erVSytXrix1nU8//VTdunXTiBEjFBUVpXbt2umpp56S0+n0GLd9+3bFxsaqadOmuuuuu7Rv377zZnE4HMrKyvJ4VFeGYWjCpxtVUORSzxbh6tu+9pZMAAAAoDKZVq6OHDkip9OpqKgoj+VRUVFKS0srdZ1du3bpgw8+kNPp1Jdffqnx48frX//6l6ZNm+Yek5CQoHnz5mnRokV6+eWXtXv3bvXs2VPZ2dnnzDJ9+nSFhYW5Hw0bNvTOTppgUUqalm/LkJ/Nqin928liYRILAAAAoDJUq/eLuVwuRUZG6tVXX5XNZlPnzp2Vmpqqf/zjH5o4caIkqU+fPu7xHTp0UEJCgho1aqT3339fw4YNK3W7Y8eO1ejRo93fZ2VlVcuCleMo0uTPNkmS7r+6mZqEB5mcCAAAAKg9TCtX4eHhstlsSk9P91ienp5+zvulYmJi5OvrK5vN5l7WunVrpaWlqaCgQH5+JT8gt06dOmrZsqV27Nhxzix2u112u72ce1J1PL9km9Ky8nVJvUD95epmZscBAAAAahXT3hbo5+enzp07Kzk52b3M5XIpOTlZ3bp1K3WdK6+8Ujt27JDL5XIv27Ztm2JiYkotVpKUk5OjnTt3KiYmxrs7UMVsScvS3O/3SJIm928rf1/b+VcAAAAA4FWmfs7V6NGj9dprr+mNN97Q5s2b9cADDyg3N9c9e+DgwYM1duxY9/gHHnhAx44d06hRo7Rt2zZ98cUXeuqppzRixAj3mMcee0zfffed9uzZox9++EE333yzbDabBg0aVOn7V1lcLkPjPk6R02WoT7toXRNfchp7AAAAABXL1Huu7rjjDmVkZGjChAlKS0tTp06dtGjRIvckF/v27ZPVerr/NWzYUF9//bUeeeQRdejQQXFxcRo1apT+9re/ucccOHBAgwYN0tGjRxUREaEePXpo1apVioiIqPT9qywfrjugn/b+pkA/m8bf2MbsOAAAAECtZDEMwzA7RFWTlZWlsLAwZWZmKjQ01Ow453U8r0DX/us7Hcst0BN9W2n4H7jXCgAAAPCWsnQDU98WiIv37NdbdSy3QC2jgjX0yiZmxwEAAABqLcpVNfbzvt/07priD0ieNqC9fG38dQIAAABm4dV4NVXkdGncwhQZhnRr5wbq2qSe2ZEAAACAWo1yVU29tWqvNh7MUliAr8b2aWV2HAAAAKDWo1xVQ4ez8vWvb7ZJkh7vHa/6wdX/A5ABAACA6o5yVQ39/cvNynYUqWPDOhp4+SVmxwEAAAAgylW188OOI/pk/UFZLdK0/u1ks1rMjgQAAABAlKtqpaDIpXGfpEiS7r6ikdo3CDM5EQAAAIBTKFfVyGv/26VdGbkKD7br0cR4s+MAAAAAOAPlqprYfyxPLyRvlySNv7G1Qv19TU4EAAAA4EyUq2pi8mcb5ShyqVvT+rqpY6zZcQAAAACchXJVDSzelK4lmw/L12bR1AFtZbEwiQUAAABQ1VCuqri8giJN+nSjJOm+nk3VPDLE5EQAAAAASkO5quJmfbtDqcdPKK5OgB68toXZcQAAAACcA+WqCssvdOqjdamSpEk3tVWAn83kRAAAAADOxcfsADg3f1+bFj3cU5/9clDXt4kyOw4AAACA8+DKVRVXJ9BPd3drbHYMAAAAAL+DcgUAAAAAXkC5AgAAAAAvoFwBAAAAgBdQrgAAAADACyhXAAAAAOAFlCsAAAAA8ALKFQAAAAB4AeUKAAAAALyAcgUAAAAAXkC5AgAAAAAvoFwBAAAAgBdQrgAAAADACyhXAAAAAOAFlCsAAAAA8AIfswNURYZhSJKysrJMTgIAAADATKc6wamOcD6Uq1JkZ2dLkho2bGhyEgAAAABVQXZ2tsLCws47xmJcSAWrZVwulw4ePKiQkBBZLBZTs2RlZalhw4bav3+/QkNDTc1SE3F8KxbHt2JxfCsex7hicXwrFse3YnF8K1ZVOr6GYSg7O1uxsbGyWs9/VxVXrkphtVrVoEEDs2N4CA0NNf3Eqsk4vhWL41uxOL4Vj2NcsTi+FYvjW7E4vhWrqhzf37tidQoTWgAAAACAF1CuAAAAAMALKFdVnN1u18SJE2W3282OUiNxfCsWx7dicXwrHse4YnF8KxbHt2JxfCtWdT2+TGgBAAAAAF7AlSsAAAAA8ALKFQAAAAB4AeUKAAAAALyAcgUAAAAAXkC5Mtny5cuVlJSk2NhYWSwWLVy48HfXWbZsmS677DLZ7XY1b95c8+bNq/Cc1VVZj++yZctksVhKPNLS0ioncDUyffp0XX755QoJCVFkZKQGDBigrVu3/u56CxYsUKtWreTv76/27dvryy+/rIS01VN5jvG8efNKnL/+/v6VlLh6efnll9WhQwf3B1R269ZNX3311XnX4fy9cGU9vpy75ff000/LYrHo4YcfPu84zt/yuZDjy/lbNpMmTSpxvFq1anXedarL+Uu5Mllubq46duyoF1988YLG7969W/369dM111yj9evX6+GHH9a9996rr7/+uoKTVk9lPb6nbN26VYcOHXI/IiMjKyhh9fXdd99pxIgRWrVqlRYvXqzCwkLdcMMNys3NPec6P/zwgwYNGqRhw4bp559/1oABAzRgwAClpKRUYvLqozzHWCr+NPszz9+9e/dWUuLqpUGDBnr66ae1du1a/fTTT7r22mvVv39/bdy4sdTxnL9lU9bjK3HulsePP/6oV155RR06dDjvOM7f8rnQ4ytx/pZV27ZtPY7XihUrzjm2Wp2/BqoMScbHH3983jGPP/640bZtW49ld9xxh5GYmFiByWqGCzm+S5cuNSQZv/32W6VkqkkOHz5sSDK+++67c465/fbbjX79+nksS0hIMP785z9XdLwa4UKO8euvv26EhYVVXqgapm7dusZ//vOfUp/j/L145zu+nLtll52dbbRo0cJYvHixcdVVVxmjRo0651jO37Iry/Hl/C2biRMnGh07drzg8dXp/OXKVTWzcuVK9erVy2NZYmKiVq5caVKimqlTp06KiYnR9ddfr++//97sONVCZmamJKlevXrnHMP5e3Eu5BhLUk5Ojho1aqSGDRv+7pUCFHM6nXrvvfeUm5urbt26lTqG87f8LuT4Spy7ZTVixAj169evxHlZGs7fsivL8ZU4f8tq+/btio2NVdOmTXXXXXdp37595xxbnc5fH7MDoGzS0tIUFRXlsSwqKkpZWVk6ceKEAgICTEpWM8TExGj27Nnq0qWLHA6H/vOf/+jqq6/W6tWrddlll5kdr8pyuVx6+OGHdeWVV6pdu3bnHHeu85d72n7fhR7j+Ph4zZ07Vx06dFBmZqb++c9/qnv37tq4caMaNGhQiYmrhw0bNqhbt27Kz89XcHCwPv74Y7Vp06bUsZy/ZVeW48u5Wzbvvfee1q1bpx9//PGCxnP+lk1Zjy/nb9kkJCRo3rx5io+P16FDhzR58mT17NlTKSkpCgkJKTG+Op2/lCvgDPHx8YqPj3d/3717d+3cuVPPPfec3nzzTROTVW0jRoxQSkrKed8vjYtzoce4W7duHlcGunfvrtatW+uVV17R1KlTKzpmtRMfH6/169crMzNTH3zwgYYMGaLvvvvunAUAZVOW48u5e+H279+vUaNGafHixUyaUAHKc3w5f8umT58+7q87dOighIQENWrUSO+//76GDRtmYrKLR7mqZqKjo5Wenu6xLD09XaGhoVy1qiBdu3alNJzHyJEj9fnnn2v58uW/+69z5zp/o6OjKzJitVeWY3w2X19fXXrppdqxY0cFpave/Pz81Lx5c0lS586d9eOPP+r555/XK6+8UmIs52/ZleX4no1z99zWrl2rw4cPe7yjwul0avny5Zo1a5YcDodsNpvHOpy/F648x/dsnL9lU6dOHbVs2fKcx6s6nb/cc1XNdOvWTcnJyR7LFi9efN73sOPirF+/XjExMWbHqHIMw9DIkSP18ccf69tvv1WTJk1+dx3O37IpzzE+m9Pp1IYNGziHL5DL5ZLD4Sj1Oc7fi3e+43s2zt1zu+6667RhwwatX7/e/ejSpYvuuusurV+/vtQX/py/F648x/dsnL9lk5OTo507d57zeFWr89fsGTVqu+zsbOPnn382fv75Z0OSMWPGDOPnn3829u7daxiGYYwZM8a4++673eN37dplBAYGGn/961+NzZs3Gy+++KJhs9mMRYsWmbULVVpZj+9zzz1nLFy40Ni+fbuxYcMGY9SoUYbVajWWLFli1i5UWQ888IARFhZmLFu2zDh06JD7kZeX5x5z9913G2PGjHF///333xs+Pj7GP//5T2Pz5s3GxIkTDV9fX2PDhg1m7EKVV55jPHnyZOPrr782du7caaxdu9YYOHCg4e/vb2zcuNGMXajSxowZY3z33XfG7t27jV9//dUYM2aMYbFYjG+++cYwDM7fi1XW48u5e3HOns2O89e7fu/4cv6WzaOPPmosW7bM2L17t/H9998bvXr1MsLDw43Dhw8bhlG9z1/KlclOTf199mPIkCGGYRjGkCFDjKuuuqrEOp06dTL8/PyMpk2bGq+//nql564uynp8n3nmGaNZs2aGv7+/Ua9ePePqq682vv32W3PCV3GlHVdJHufjVVdd5T7Wp7z//vtGy5YtDT8/P6Nt27bGF198UbnBq5HyHOOHH37YuOSSSww/Pz8jKirK6Nu3r7Fu3brKD18N/OlPfzIaNWpk+Pn5GREREcZ1113nfuFvGJy/F6usx5dz9+Kc/eKf89e7fu/4cv6WzR133GHExMQYfn5+RlxcnHHHHXcYO3bscD9fnc9fi2EYRuVdJwMAAACAmol7rgAAAADACyhXAAAAAOAFlCsAAAAA8ALKFQAAAAB4AeUKAAAAALyAcgUAAAAAXkC5AgAAAAAvoFwBAAAAgBdQrgAA8DKLxaKFCxeaHQMAUMkoVwCAGuWee+6RxWIp8ejdu7fZ0QAANZyP2QEAAPC23r176/XXX/dYZrfbTUoDAKgtuHIFAKhx7Ha7oqOjPR5169aVVPyWvZdffll9+vRRQECAmjZtqg8++MBj/Q0bNujaa69VQECA6tevr+HDhysnJ8djzNy5c9W2bVvZ7XbFxMRo5MiRHs8fOXJEN998swIDA9WiRQt9+umnFbvTAADTUa4AALXO+PHjdcstt+iXX37RXXfdpYEDB2rz5s2SpNzcXCUmJqpu3br68ccftWDBAi1ZssSjPL388ssaMWKEhg8frg0bNujTTz9V8+bNPX7G5MmTdfvtt+vXX39V3759ddddd+nYsWOVup8AgMplMQzDMDsEAADecs899+itt96Sv7+/x/InnnhCTzzxhCwWi+6//369/PLL7ueuuOIKXXbZZXrppZf02muv6W9/+5v279+voKAgSdKXX36ppKQkHTx4UFFRUYqLi9PQoUM1bdq0UjNYLBaNGzdOU6dOlVRc2IKDg/XVV19x7xcA1GDccwUAqHGuueYaj/IkSfXq1XN/3a1bN4/nunXrpvXr10uSNm/erI4dO7qLlSRdeeWVcrlc2rp1qywWiw4ePKjrrrvuvBk6dOjg/jooKEihoaE6fPhweXcJAFANUK4AADVOUFBQibfpeUtAQMAFjfP19fX43mKxyOVyVUQkAEAVwT1XAIBaZ9WqVSW+b926tSSpdevW+uWXX5Sbm+t+/vvvv5fValV8fLxCQkLUuHFjJScnV2pmAEDVx5UrAECN43A4lJaW5rHMx8dH4eHhkqQFCxaoS5cu6tGjh95++22tWbNGc+bMkSTdddddmjhxooYMGaJJkyYpIyNDDz74oO6++25FRUVJkiZNmqT7779fkZGR6tOnj7Kzs/X999/rwQcfrNwdBQBUKZQrAECNs2jRIsXExHgsi4+P15YtWyQVz+T33nvv6S9/+YtiYmL07rvvqk2bNpKkwMBAff311xo1apQuv/xyBQYG6pZbbtGMGTPc2xoyZIjy8/P13HPP6bHHHlN4eLhuvfXWyttBAECVxGyBAIBaxWKx6OOPP9aAAQPMjgIAqGG45woAAAAAvIByBQAAAABewD1XAIBahXfDAwAqCleuAAAAAMALKFcAAAAA4AWUKwAAAADwAsoVAAAAAHgB5QoAAAAAvIByBQAAAABeQLkCAAAAAC+gXAEAAACAF/w/nmK94fIc3kIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the accuracy in each epoch\n",
    "\n",
    "# Create a figure\n",
    "plt.figure(figsize=(10,5))\n",
    "\n",
    "# Plot samples\n",
    "sns.lineplot(data=results_df, x=\"epoch\", y=\"accuracy\")\n",
    "\n",
    "# Set figure title and axes labels\n",
    "plt.title(\"Model accuracy in each epoch\")\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Accuracy\")\n",
    "\n",
    "# Show the plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is our reading of this chart and epoch performance:\n",
    "\n",
    "- Accuracy: The accuracy has significantly improved over the epochs, starting from 62.93% in the first epoch and reaching 97.41% in the fifth epoch. This indicates that the model is performing well in making correct predictions overall.\n",
    "\n",
    "- Precision: The precision is consistently 1.0 (100%) across all epochs. This suggests that when the model predicts a prompt as an injection, it is always correct. This is an excellent result and indicates a low rate of false positives.\n",
    "\n",
    "- Recall: The recall improves from 28.33% in the first epoch to 95.00% in the fifth epoch. This indicates that the model has become much better at identifying actual positive instances as the training progresses.\n",
    "\n",
    "- F1 Score: The F1 score starts at 44.16% in the first epoch and increases to 97.44% in the fifth epoch. This suggests that the model is achieving a good balance between precision and recall, resulting in a robust performance.\n",
    "\n",
    "In summary, the model is performing well on the given task, with high accuracy, precision, recall, and F1 score. The improvement in metrics across epochs suggests that the model is learning and refining its predictions over time.\n",
    "\n",
    "Finally, these results indicate a successful fine-tuning process, and the model seems to be effective in classifying prompts as injection or non-injection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5d54f52f512e486abbbf4946c5e79789",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.23396892845630646, 'eval_accuracy': 0.9741379310344828, 'eval_precision': 1.0, 'eval_recall': 0.95, 'eval_f1': 0.9743589743589743, 'eval_runtime': 1.8938, 'eval_samples_per_second': 61.254, 'eval_steps_per_second': 7.921, 'epoch': 5.0}\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the final model\n",
    "final_results = trainer.evaluate()\n",
    "print(final_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fine-tuning is finished now, we will save the model to a local path.\n",
    "\n",
    "This step is preferred so we can reload it later for evaluation and inference without the need to repeat the fine-tuning process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set local model path\n",
    "models_path = \"../models/xlm_roberta\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('../models/xlm_roberta\\\\tokenizer_config.json',\n",
       " '../models/xlm_roberta\\\\special_tokens_map.json',\n",
       " '../models/xlm_roberta\\\\sentencepiece.bpe.model',\n",
       " '../models/xlm_roberta\\\\added_tokens.json')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save the model and tokenizer for a later use \n",
    "model.save_pretrained(models_path)\n",
    "tokenizer.save_pretrained(models_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. RESULT ANALYSIS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6cb13359b715406db923de930aa5192b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/15 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.23396892845630646, 'eval_accuracy': 0.9741379310344828, 'eval_precision': 1.0, 'eval_recall': 0.95, 'eval_f1': 0.9743589743589743, 'eval_runtime': 2.0254, 'eval_samples_per_second': 57.273, 'eval_steps_per_second': 7.406, 'epoch': 5.0}\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the final model\n",
    "final_results = trainer.evaluate()\n",
    "print(final_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final results demonstrate the following:\n",
    "- The model achieved impressive accuracy, which suggests that the model is effective in making correct predictions for the prompt injection classification problem.\n",
    "- The precision indicates that when the model predicts a positive class, it is correct most of the time.\n",
    "- The high recall also indicates that the model is able to capture the majority of the actual positive instances.\n",
    "- The F1 score is high and suggests that the model is performing well in both precision and recall.\n",
    "\n",
    "Overall, the results indicate that the fine-tuned RoBERTa model is highly effective for the classification task, achieving a high level of accuracy, precision, recall, and F1 score. \n",
    "\n",
    "A next step might be to further analyze the wrong predictions to make sense of any potential patterns the model might have missed or was unable to capture in the dataset.\n",
    "\n",
    "**Considering these metrics, and comparing the achieved performance with all other experiments conducted in the other notebooks, we conclude that the fine-tuned LLM has outperformed all other methods across all classification metrics in predicting prompt injections.**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "nlp_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}