workflows/create_workflow.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<td>\n",
    "   <a target=\"_blank\" href=\"https://www.clarifai.com/\" ><img src=\"https://upload.wikimedia.org/wikipedia/commons/b/bc/Clarifai_Logo_FC_Web.png\" width=256/></a>\n",
    "</td>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<td>\n",
    "<a href=\"https://colab.research.google.com/github/Clarifai/examples/blob/main/workflows/create_workflow.ipynb\" target=\"_blank\"><img\n",
    "src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Colab\"></a>\n",
    "</td>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Workflows offer a versatile framework for constructing your inference pipeline. It equips you with the fundamental components for sophisticated machine learning ensemble modeling and incorporating business logic.\n",
    "\n",
    "Leverage Workflows to handle intricate processes, extract valuable insights, and construct tailored solutions. With Workflows, your machine learning models can seamlessly function as interconnected nodes within a graph.\n",
    "\n",
    "Clarifai simplifies the process of integrating diverse models, enabling you to execute intricate data operations and design solutions tailored to your precise business requirements.\n",
    "\n",
    "Utilize workflows to link multiple models in a sequence, steering and influencing their behavior effectively."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting Started"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The Clarifai Python SDK empowers you to define and construct intricate workflows through a YAML configuration. \n",
    "In this notebook, we've included several workflows to assist you in getting started:\n",
    "\n",
    "**Prompter + LLM**: Create a suitable prompt using Prompter and then utilize LLM to generate text.\n",
    "\n",
    "**Audio Speech Recognition + Sentiment Analysis**: Convert audio to text and employ sentiment analysis to determine the sentiment of the text.\n",
    "\n",
    "**Language-Aware OCR**: Extract text from images using OCR and then employ language detection to identify the language of the text.\n",
    "\n",
    "**Face Sentiment**: Detect faces within images and subsequently apply sentiment analysis to gauge the sentiment expressed by those faces.\n",
    "\n",
    "**Face Search**: This workflow integrates detection, recognition, and embedding to produce face landmarks and facilitates visual search using the embeddings of detected faces.\n",
    "\n",
    "**Demographics**: A multi-model workflow that identifies, crops, and recognizes demographic characteristics of faces, including age, gender, and multicultural attributes.\n",
    "\n",
    "**General**: A versatile image workflow that combines detection, classification, and embedding to recognize general concepts, encompassing objects, themes, moods, and more."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation and Setup"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is required to set **CLARIFAI_PAT** as Environment Variable.\n",
    "\n",
    "More info on [Personal Access Tokens](https://docs.clarifai.com/clarifai-basics/authentication/personal-access-tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install clarifai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"CLARIFAI_PAT\"] = \"CLARIFAI_PAT\" # replace with your own PAT key here\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For Colab\n",
    "\n",
    "To access config yamls from examples repo, you can clone the repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/Clarifai/examples.git\n",
    "%cd /content/examples/workflows/"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating Workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from clarifai.client.user import User\n",
    "# Create a new app (replace USER_ID and APP_ID with your own)\n",
    "app = User(user_id=\"USER_ID\").create_app(app_id=\"APP_ID\", base_workflow=\"Empty\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompter + LLM\n",
    "\n",
    "This workflow utilizes Prompter to create a suitable prompt and then utilizes LLM to generate text.\n",
    "A prompt template serves as a pre-configured piece of text used to instruct a text-to-text model. It acts as a structured query or input that guides the model in generating the desired response.\n",
    "\n",
    "Your prompt template should include at least one instance of the placeholder **{data.text.raw}**. When you input your text data at inference time, all occurrences of {data.text.raw} within the template will be replaced with the provided text.\n",
    "\n",
    "Here is an example of a prompt template for classifying text as positive or negative:\n",
    "\n",
    "```Classify sentiment between postive and negative for the text {data.text.raw}```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Prompter+LLM workflow\n",
    "prompter_llm_workflow = app.create_workflow(config_filepath=\"configs/prompter_llm.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run workflow on a text input\n",
    "prediction = prompter_llm_workflow.predict_by_bytes(b\"Good Morning! I think it's going to be a great day!\", input_type=\"text\")\n",
    "\n",
    "# Get workflow results\n",
    "print(prediction.results[0].outputs[-1].data.text.raw)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Audio Speech Recognition - Sentiment\n",
    "\n",
    "A workflow for obtaining sentiment of audio\n",
    " - Audio to Text Node\n",
    " - Text Sentiment Analysis Node"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create ASR-Sentiment workflow\n",
    "asr_sentiment_workflow = app.create_workflow(config_filepath=\"configs/asr_sentiment.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Infer workflow on an audio input\n",
    "prediction = asr_sentiment_workflow.predict_by_url(\"https://s3.amazonaws.com/samples.clarifai.com/GreatPresentation2.wav\", input_type=\"audio\")\n",
    "\n",
    "# Get workflow results\n",
    "print(prediction.results[0].outputs[-1].data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Language Aware OCR\n",
    "\n",
    "Optical character recognition workflow\n",
    " - Text Detection Node\n",
    " - Text Aggregation Node - Operator that combines text detections into text body for the whole Image\n",
    " - Language ID Operator Node - Operator for language identification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create OCR workflow\n",
    "ocr_workflow = app.create_workflow(config_filepath=\"configs/language_aware_ocr.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predict workflow on an image input\n",
    "prediction = ocr_workflow.predict_by_url(\"https://s3.amazonaws.com/samples.clarifai.com/ocr_img_00417302.jpg\", input_type=\"image\")\n",
    "\n",
    "# Get workflow results\n",
    "print(prediction.results[0].outputs[-1].data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Face Sentiment\n",
    "\n",
    "Multi-model workflow that combines face detection and sentiment classification of 7 concepts: anger, disgust, fear, neutral, happiness, sadness, contempt, and\n",
    "\n",
    "Contains:\n",
    " - Visual Detector - To detect faces\n",
    " - Image Cropper - Crop faces from the image\n",
    " - Visual Classifier - To classify sentiment of the face"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Face-Sentiment workflow\n",
    "face_sentiment_workflow = app.create_workflow(config_filepath=\"configs/face_sentiment.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Predict workflow on an image input\n",
    "prediction = face_sentiment_workflow.predict_by_url(\"https://samples.clarifai.com/face-det.jpg\", input_type=\"image\")\n",
    "\n",
    "# Get workflow results\n",
    "print(prediction.results[0].outputs[-1].data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Demographics\n",
    "\n",
    "Multi-model workflow that detects, crops, and recognizes demographic characteristics of faces. Visually classifies age, gender, and multi-culture\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create demographic workflow\n",
    "demographics_workflow = app.create_workflow(config_filepath=\"configs/demographics.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run workflow on an image input\n",
    "demographics_workflow.predict_by_url(\"https://s3.amazonaws.com/samples.clarifai.com/face-det.jpg\", input_type=\"image\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## General\n",
    " - Contains visual image classifier, image embedder and clustering model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "general_workflow = app.create_workflow(config_filepath=\"configs/general.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "general_workflow.predict_by_url(\"https://samples.clarifai.com/metro-north.jpg\", input_type=\"image\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Face Search\n",
    "A workflow that combines detection, recognition, and embedding to generate face landmarks and enable visual search using detected faces's embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Workflow for Visual Face Search\n",
    "face_workflow = app.create_workflow(config_filepath=\"configs/face_search.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Running Workflow\n",
    "face_workflow.predict_by_url(url='https://samples.clarifai.com/face-det.jpg', input_type=\"image\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clarifai Resources\n",
    "\n",
    "**Website**: [https://www.clarifai.com](https://www.clarifai.com/)\n",
    "\n",
    "**Demo**: [https://clarifai.com/demo](https://clarifai.com/demo)\n",
    "\n",
    "**Sign up for a free Account**: [https://clarifai.com/signup](https://clarifai.com/signup)\n",
    "\n",
    "**Developer Guide**: [https://docs.clarifai.com](https://docs.clarifai.com/)\n",
    "\n",
    "**Clarifai Community**: [https://clarifai.com/explore](https://clarifai.com/explore)\n",
    "\n",
    "**Python SDK Docs**: [https://docs.clarifai.com/python-sdk/api-reference](https://docs.clarifai.com/python-sdk/api-reference)\n",
    "\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}