models/model_train/image-detection_training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "00024d19",
   "metadata": {},
   "source": [
    "<td>\n",
    "   <a target=\"_blank\" href=\"https://www.clarifai.com/\" ><img src=\"https://upload.wikimedia.org/wikipedia/commons/b/bc/Clarifai_Logo_FC_Web.png\" width=256/></a>\n",
    "</td>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4da69603",
   "metadata": {},
   "source": [
    "<td>\n",
    "<a href=\"https://colab.research.google.com/github/Clarifai/examples/blob/main/models/model_train/image-detection_training.ipynb\" target=\"_blank\"><img\n",
    "src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"></a>\n",
    "</td>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2482f50",
   "metadata": {},
   "source": [
    "# Models\n",
    "\n",
    "\n",
    "\n",
    "Clarifai offers a range of powerful model types, each designed to generate meaningful outputs based on user specific inputs and AI tasks.\n",
    "\n",
    "There are wide variety of models that can be used as standalone solutions, or as building blocks for your own custom business solutions.\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db356341",
   "metadata": {},
   "source": [
    "Clarifai Models are the recommended starting points for many users because they offer incredibly fast training times when you customize them using the \"embedding-classifier\" (Transfer Learning Classifier) model type.\n",
    "\n",
    "But there are many cases where accuracy and the ability to carefully target solutions take priority over speed and ease of use. Additionally, you may need a model to learn new features, not recognized by existing Clarifai Models. For these cases, it is possible to \"deep fine-tune\" your custom models and integrate them directly within your workflows.\n",
    "\n",
    "You might consider deep training if you have:\n",
    "\n",
    "- A custom tailored dataset\n",
    "- Accurate labels\n",
    "- Expertise and time to fine-tune models\n",
    "\n",
    "\n",
    "_______\n",
    "On the [Clarifai Community](https://clarifai.com/explore) explore page, you can click the [Models](https://clarifai.com/explore/models) tab to search and access the models available for everyone to use."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cde3b72",
   "metadata": {},
   "source": [
    "This notebook contains Model Train demo for **visual-detector** Model Type with **MMDetection** Template."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "800de27a",
   "metadata": {},
   "source": [
    "# Getting Started"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7fd7f15",
   "metadata": {},
   "source": [
    "### Installation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfb70f5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install clarifai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ed766dc8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"CLARIFAI_PAT\"] = \"PAT\" # replace with your own PAT key here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49de2e2b",
   "metadata": {},
   "source": [
    "*Note: Guide to get your [PAT](https://docs.clarifai.com/clarifai-basics/authentication/personal-access-tokens)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46eaf1e6",
   "metadata": {},
   "source": [
    "### For Colab\n",
    "To access data files from Clarifai examples repo, you can clone the repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f3c537b",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/Clarifai/examples.git\n",
    "%cd /content/examples/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3193b97c",
   "metadata": {},
   "source": [
    "### For Local"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "116bcb48",
   "metadata": {},
   "outputs": [],
   "source": [
    "git clone https://github.com/Clarifai/examples.git\n",
    "cd examples/models/model_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "407c786d",
   "metadata": {},
   "source": [
    "## VISUAL-DETECTION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8315500",
   "metadata": {},
   "source": [
    "**Input: Images and videos**\n",
    "\n",
    "**Output: Regions**\n",
    "\n",
    "\n",
    "Visual Detector, also known as Object Detection, is a type of deep fine-tuned model designed to identify and locate objects within images or video frames. It goes beyond simple image classification, where the goal is to assign a single label to an entire image.\n",
    "\n",
    "Instead, an object detection model can identify multiple objects of different classes within an image and provide their corresponding bounding box coordinates."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "259ecd1a",
   "metadata": {},
   "source": [
    "### Creating an App"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5eaf6277",
   "metadata": {},
   "outputs": [],
   "source": [
    "from clarifai.client.user import User\n",
    "#replace your \"user_id\"\n",
    "client = User(user_id=\"user_id\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28056328",
   "metadata": {},
   "outputs": [],
   "source": [
    "app = client.create_app(app_id=\"app_id\", base_workflow=\"Universal\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd607e6",
   "metadata": {},
   "source": [
    "### Uploading Image Detection Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3bd7e01",
   "metadata": {},
   "source": [
    "#### Training Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7bee071a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/adithyansukumar/work/ml_training_error/examples/datasets/upload/image_detection/voc'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "module_path = os.path.join(os.getcwd().split('/models/model_train')[0],'datasets/upload/image_detection/voc')\n",
    "module_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c54a91c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "#importing load_module_dataloader for calling the dataloader object in dataset.py in the local data folder\n",
    "from clarifai.datasets.upload.utils import load_module_dataloader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e7bfcd22",
   "metadata": {},
   "outputs": [],
   "source": [
    "voc_dataloader = load_module_dataloader(module_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "24264654",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">2024-03-20 14:19:20 </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> clarifai.client.app:                                                        <a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">app.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py#310\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">310</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         Dataset created                                                             <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         code: SUCCESS                                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         description: <span style=\"color: #008000; text-decoration-color: #008000\">\"Ok\"</span>                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         req_id: <span style=\"color: #008000; text-decoration-color: #008000\">\"b60943e53bd93d20cf97392a8757fd75\"</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>                                                                                     <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m2024-03-20 14:19:20\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m clarifai.client.app:                                                        \u001b]8;id=315178;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py\u001b\\\u001b[2mapp.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=705511;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py#310\u001b\\\u001b[2m310\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         Dataset created                                                             \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         code: SUCCESS                                                               \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         description: \u001b[32m\"Ok\"\u001b[0m                                                           \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         req_id: \u001b[32m\"b60943e53bd93d20cf97392a8757fd75\"\u001b[0m                                  \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m                                                                                     \u001b[2m          \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Uploading Dataset: 100%|██████████| 1/1 [00:14<00:00, 14.16s/it]\n"
     ]
    }
   ],
   "source": [
    "dataset = app.create_dataset(dataset_id=\"train_dataset\")\n",
    "dataset.upload_dataset(dataloader=voc_dataloader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a40825c",
   "metadata": {},
   "source": [
    "### List Trainable Model Types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "71254639",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['visual-classifier',\n",
       " 'visual-detector',\n",
       " 'visual-segmenter',\n",
       " 'visual-embedder',\n",
       " 'clusterer',\n",
       " 'text-classifier',\n",
       " 'embedding-classifier',\n",
       " 'text-to-text']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "app.list_trainable_model_types()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e02f87b2",
   "metadata": {},
   "source": [
    "### Create a Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "467498cb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">2024-03-20 14:20:04 </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> clarifai.client.app:                                                        <a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">app.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py#334\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">334</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         Model created                                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         code: SUCCESS                                                               <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         description: <span style=\"color: #008000; text-decoration-color: #008000\">\"Ok\"</span>                                                           <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         req_id: <span style=\"color: #008000; text-decoration-color: #008000\">\"12b78798d4d1c1ed35c7f7adb0dabee2\"</span>                                  <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>                                                                                     <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">          </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m2024-03-20 14:20:04\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m clarifai.client.app:                                                        \u001b]8;id=861922;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py\u001b\\\u001b[2mapp.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=965;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/app.py#334\u001b\\\u001b[2m334\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         Model created                                                               \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         code: SUCCESS                                                               \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         description: \u001b[32m\"Ok\"\u001b[0m                                                           \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         req_id: \u001b[32m\"12b78798d4d1c1ed35c7f7adb0dabee2\"\u001b[0m                                  \u001b[2m          \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m                                                                                     \u001b[2m          \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "MODEL_ID = \"model_detector\"\n",
    "MODEL_TYPE_ID = \"visual-detector\"\n",
    "model = app.create_model(model_id=MODEL_ID, model_type_id=MODEL_TYPE_ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4fa1b8c",
   "metadata": {},
   "source": [
    "### List Templates for the Model Type\n",
    "\n",
    "Templates give you the control to choose the specific architecture used by your neural network, as well as define a set of hyperparameters you can use to fine-tune the way your model learns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9f2d3b9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['MMDetection_AdvancedConfig',\n",
       " 'MMDetection_FasterRCNN',\n",
       " 'MMDetection_SSD',\n",
       " 'MMDetection_YoloF',\n",
       " 'Clarifai_InceptionV2',\n",
       " 'Clarifai_InceptionV4',\n",
       " 'detection_msc10']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.list_training_templates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f3ca868",
   "metadata": {},
   "source": [
    "### Save params\n",
    "Save the parameters for the specific model template in a YAML file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "aa53dc65",
   "metadata": {},
   "outputs": [],
   "source": [
    "YAML_FILE = 'model_params.yaml'\n",
    "model_params = model.get_params(template='MMDetection_AdvancedConfig', save_to=YAML_FILE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8c87de00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'dataset_id': '',\n",
       " 'dataset_version_id': '',\n",
       " 'concepts': [],\n",
       " 'train_params': {'invalid_data_tolerance_percent': 5.0,\n",
       "  'template': 'MMDetection_AdvancedConfig',\n",
       "  'custom_config': 'MMDetection_AdvancedConfig.py',\n",
       "  'image_size': [320.0]},\n",
       " 'inference_params': {'detection_threshold': 0.0}}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import yaml\n",
    "# Preview YAML content\n",
    "file = open(YAML_FILE)\n",
    "data = yaml.safe_load(file)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19407a99",
   "metadata": {},
   "source": [
    "#### Get param info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "72bf3322",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'fieldType': 'ARRAY_OF_NUMBERS', 'defaultValue': [320.0], 'description': 'the image size for inference. can be 1 or 2 elements. when a single value, specifies min side', 'param': 'image_size'}\n"
     ]
    }
   ],
   "source": [
    "print(model.get_param_info(param = 'image_size'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d9bfade",
   "metadata": {},
   "source": [
    "### Edit the parameters in the YAML file to pass it to model.train()\n",
    "**Here using the already edited and saved YAML file**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a1d77a82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'dataset_id': 'train_dataset',\n",
       " 'dataset_version_id': '',\n",
       " 'concepts': ['id-cow',\n",
       "  'id-horse',\n",
       "  'id-bottle',\n",
       "  'id-sofa',\n",
       "  'id-bird',\n",
       "  'id-cat',\n",
       "  'id-dog',\n",
       "  'id-person'],\n",
       " 'train_params': {'invalid_data_tolerance_percent': 5.0,\n",
       "  'template': 'MMDetection_AdvancedConfig',\n",
       "  'seed': -1.0,\n",
       "  'custom_config': 'saved_MMDetection.py',\n",
       "  'num_gpus': 1.0,\n",
       "  'image_size': [320.0]},\n",
       " 'inference_params': {'detection_threshold': 0.0}}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Preview YAML content\n",
    "file = open('saved_mmdetection.yaml')\n",
    "data = yaml.safe_load(file)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8925534c",
   "metadata": {},
   "source": [
    "## Custom Config"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0955b4e9",
   "metadata": {},
   "source": [
    "Clarifai training templates employ two training toolboxes: **HuggingFace and MMDetection.** \n",
    "\n",
    "These toolboxes offer various training templates that can be configured at great length by the ML Builder. Here an example for Visual Detection(via MMDetection)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60463326",
   "metadata": {},
   "source": [
    "The custom config param in the yaml will have a python script **MMDetection.py**. This script can be configured to use the MMDetection configs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56b5c641",
   "metadata": {},
   "source": [
    "#### Preview of saved MMDetection.py custom config file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "9aced6cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_base_ = '/mmdetection/configs/yolof/yolof_r50_c5_8x8_1x_coco.py'\n",
      "model=dict(\n",
      "  bbox_head=dict(num_classes=0))\n",
      "data=dict(\n",
      "  train=dict(\n",
      "    ann_file='',\n",
      "    img_prefix='',\n",
      "    classes=''\n",
      "    ),\n",
      "  val=dict(\n",
      "    ann_file='',\n",
      "    img_prefix='',\n",
      "    classes=''))\n",
      "optimizer=dict(\n",
      "  _delete_=True,\n",
      "  type='Adam',\n",
      "  lr=0.0001,\n",
      "  weight_decay=0.0001)\n",
      "lr_config = dict(\n",
      "  _delete_=True,\n",
      "  policy='CosineAnnealing',\n",
      "  warmup='linear',\n",
      "  warmup_iters=1000,\n",
      "  warmup_ratio=0.1,\n",
      "  min_lr_ratio=1e-5)\n",
      "runner = dict(\n",
      "  _delete_=True,\n",
      "  type='EpochBasedRunner',\n",
      "  max_epochs=10)\n"
     ]
    }
   ],
   "source": [
    "file = open('saved_MMDetection.py')\n",
    "script = file.read()\n",
    "print(script)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41f151ad",
   "metadata": {},
   "source": [
    "### Introduction to the provided custom config example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8827998",
   "metadata": {},
   "source": [
    "- **_base_** is the base model details to get from. Refer here for given [example](https://github.com/open-mmlab/mmdetection/blob/main/configs/yolof/yolof_r50-c5_8xb8-1x_coco.py)\n",
    "\n",
    "- Remaining all the variables, for example **model**, **runner** are to over write from the base config specific to our need.\n",
    "\n",
    "- The reason **data** variable is initalized is to over write the coco annotation label file and replace it with our Clarifai Dataset's Label file."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eefa05d2",
   "metadata": {},
   "source": [
    "## Important Note: \n",
    "- See [MMDetection](https://mmdetection.readthedocs.io/en/dev/tutorials/config.html) link for general configuration guidelines.\n",
    "- Here we are starting with the base config for yolof (see [MMDetection code](https://github.com/open-mmlab/mmdetection/blob/main/configs/yolof/yolof_r50-c5_8xb8-1x_coco.py)) \n",
    "- Changing the optimizer to Adam, customizing optimizer config. On top of that we are modifying the learning rate config.\n",
    "- Adopting a cosine annealing policy.\n",
    "- Setting the number of epochs to 10. \n",
    "  - Note the usage of _delete_ above. If not used, the dict that is being defined is merged to the _base_ config; which might define an invalid configuration. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92dcf007",
   "metadata": {},
   "source": [
    "### Start Model Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0b5383b0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">2024-03-20 14:23:26 </span><span style=\"color: #000080; text-decoration-color: #000080\">INFO    </span> clarifai.client.model:                                                    <a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/model.py\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">model.py</span></a><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">:</span><a href=\"file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/model.py#241\" target=\"_blank\"><span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">241</span></a>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         Model Training Started                                                    <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         code: SUCCESS                                                             <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         description: <span style=\"color: #008000; text-decoration-color: #008000\">\"Ok\"</span>                                                         <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>         req_id: <span style=\"color: #008000; text-decoration-color: #008000\">\"774410fed9d30dedd0fafd90f651e31a\"</span>                                <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "<span style=\"color: #7fbfbf; text-decoration-color: #7fbfbf\">                    </span>                                                                                   <span style=\"color: #7f7f7f; text-decoration-color: #7f7f7f\">            </span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[2;36m2024-03-20 14:23:26\u001b[0m\u001b[2;36m \u001b[0m\u001b[34mINFO    \u001b[0m clarifai.client.model:                                                    \u001b]8;id=555966;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/model.py\u001b\\\u001b[2mmodel.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=189596;file:///Users/adithyansukumar/.pyenv/versions/3.9.10/lib/python3.9/site-packages/clarifai/client/model.py#241\u001b\\\u001b[2m241\u001b[0m\u001b]8;;\u001b\\\n",
       "\u001b[2;36m                    \u001b[0m         Model Training Started                                                    \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         code: SUCCESS                                                             \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         description: \u001b[32m\"Ok\"\u001b[0m                                                         \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m         req_id: \u001b[32m\"774410fed9d30dedd0fafd90f651e31a\"\u001b[0m                                \u001b[2m            \u001b[0m\n",
       "\u001b[2;36m                    \u001b[0m                                                                                   \u001b[2m            \u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_version_id = model.train(yaml_file='saved_mmdetection.yaml')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e74d8bb",
   "metadata": {},
   "source": [
    "### Check Model Training Status\n",
    "Note: If the status code is 'MODEL_TRAINED', then the user can know the Model is Trained and ready to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "6830219e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "code: MODEL_TRAINED\n",
      "description: \"Model is trained and ready\"\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#To store training logs in a file, fix training_logs param as True\n",
    "status = model.training_status(version_id=model_version_id,training_logs=False)\n",
    "print(status)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdd71761",
   "metadata": {},
   "source": [
    "### Model Prediction\n",
    "Predicting with the Trained Model.\n",
    "Note: Refer this [notebook](https://github.com/Clarifai/examples/blob/main/models/model_predict.ipynb) for more info on Model Predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "c5fdb80c",
   "metadata": {},
   "outputs": [],
   "source": [
    "IMAGE_PATH = os.path.join(os.getcwd().split('/models')[0],'datasets/upload/image_detection/voc/images/2007_000464.jpg')\n",
    "\n",
    "prediction_response = model.predict_by_filepath(IMAGE_PATH, input_type=\"image\")\n",
    "\n",
    "# Get the output\n",
    "regions = prediction_response.outputs[0].data.regions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "008be74e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the prediction bboxes\n",
    "import cv2\n",
    "from urllib.request import urlopen\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "img = plt.imread(IMAGE_PATH)\n",
    "\n",
    "for region in regions:\n",
    "    # Accessing and rounding the bounding box values\n",
    "    top_row = round(region.region_info.bounding_box.top_row, 3) * img.shape[0]\n",
    "    left_col = round(region.region_info.bounding_box.left_col, 3)* img.shape[1]\n",
    "    bottom_row = round(region.region_info.bounding_box.bottom_row, 3)* img.shape[0]\n",
    "    right_col = round(region.region_info.bounding_box.right_col, 3)* img.shape[1]\n",
    "\n",
    "    cv2.rectangle(img, (int(left_col),int(top_row)), (int(right_col),int(bottom_row)), (36,255,12), 2)\n",
    "\n",
    "    # Get concept name\n",
    "    concept_name = region.data.concepts[0].name\n",
    "\n",
    "    # Display text\n",
    "    cv2.putText(img, concept_name, (int(left_col),int(top_row-10)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (36,255,12), 2)\n",
    "\n",
    "plt.axis('off')\n",
    "plt.imshow(img[...,::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b72097e8",
   "metadata": {},
   "source": [
    "## Note\n",
    "\n",
    "- This Notebook is a demo to get started with Model Training in Clarifai Platform with Python SDK.\n",
    "- For better accuracy of the Model, Choose your own data and different Templates and Hyperparams."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66f7c3f9",
   "metadata": {},
   "source": [
    "## Clarifai Resources"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e83d01c6",
   "metadata": {},
   "source": [
    "**Website**: [https://www.clarifai.com](https://www.clarifai.com/)\n",
    "\n",
    "**Demo**: [https://clarifai.com/demo](https://clarifai.com/demo)\n",
    "\n",
    "**Sign up for a free Account**: [https://clarifai.com/signup](https://clarifai.com/signup)\n",
    "\n",
    "**Developer Guide**: [https://docs.clarifai.com](https://docs.clarifai.com/)\n",
    "\n",
    "**Clarifai Community**: [https://clarifai.com/explore](https://clarifai.com/explore)\n",
    "\n",
    "**Python SDK Docs**: [https://docs.clarifai.com/python-sdk/api-reference](https://docs.clarifai.com/python-sdk/api-reference)\n",
    "\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}