examples/sense_of_humor/sense_of_humor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example notebook, we mine-tune an AI program's sense of humor, by giving it feedback on its jokes. Run the cells in order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#you need to install these\n",
    "%pip install dspy-ai\n",
    "%pip install ipywidgets\n",
    "%pip install IPython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import\n",
    "import dspy\n",
    "import os\n",
    "import dotenv\n",
    "\n",
    "#setup dspy\n",
    "dotenv.load_dotenv(os.path.expanduser(\"~/.env\"))  # load OpenAI API key from .env file, or you can set directly with \n",
    "llm = dspy.OpenAI(model='gpt-4o', max_tokens=4096, temperature=0.8)\n",
    "#llm = dspy.OpenAI(model='gpt-4o', max_tokens=4096, api_key=\"sk-\") #or you can set the API key directly here\n",
    "dspy.settings.configure(lm=llm)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next cell, you will be prompted to request a type of joke that you want. 5 jokes will be generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Why did the baby elephant bring a suitcase to the zoo? Because it wanted to pack its trunk!',\n",
       " \"Why did the baby kangaroo get in trouble at school? Because he couldn't stop jumping to conclusions!\",\n",
       " \"Why don't baby ducks tell jokes? Because they always crack up!\",\n",
       " 'Why did the baby kitten bring a pencil to bed? Because it wanted to draw some purr-fect dreams!',\n",
       " 'Why did the baby lion always get lost? Because he couldn\\'t stop \"lion\" around!']"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = input(\"What kind of jokes do you want to generate?\") #from interactive input\n",
    "#query = \"Jokes about Elvis.\" #alternatively write the string directly into the code\n",
    "\n",
    "# 1) Declare with a signature, and request 5 jokes.\n",
    "comedian = dspy.ChainOfThought('query -> joke', n=5)\n",
    "\n",
    "# 2) Call with input argument.\n",
    "response = comedian(query=query)\n",
    "\n",
    "# 3) Access the outputs.\n",
    "response.completions.joke\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the human feedback part. For each joke, you must provide a rating (1-5) and give feedback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Why did the baby elephant bring a suitcase to the zoo? Because it wanted to pack its trunk!\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Why did the baby kangaroo get in trouble at school? Because he couldn't stop jumping to conclusions!\n",
      "Why don't baby ducks tell jokes? Because they always crack up!\n",
      "Why did the baby kitten bring a pencil to bed? Because it wanted to draw some purr-fect dreams!\n",
      "Why did the baby lion always get lost? Because he couldn't stop \"lion\" around!\n"
     ]
    }
   ],
   "source": [
    "def get_user_rating(joke):\n",
    "    while True:\n",
    "        try:\n",
    "            rating = int(input(f\"{joke}\\n---\\nPlease rate on a 5-point scale (1-5): \"))\n",
    "            if 1 <= rating <= 5:\n",
    "                return rating  # Cast the integer to a string before returning\n",
    "            else:\n",
    "                print(\"Invalid input. Please enter a number between 1 and 5.\")\n",
    "        except ValueError:\n",
    "            print(\"Invalid input. Please enter a number.\")\n",
    "def get_user_feedback(joke):\n",
    "    feedback = input(f\"{joke}\\n---\\nPlease provide your feedback: \")\n",
    "    return feedback\n",
    "\n",
    "history = []\n",
    "\n",
    "# Get rating and feedback for each joke\n",
    "for joke in response.completions.joke:\n",
    "    print(joke)\n",
    "    rating = get_user_rating(joke)\n",
    "    feedback = get_user_feedback(joke)\n",
    "    this_joke = {\n",
    "        'query': query,\n",
    "        'rationale': response.completions.rationale[response.completions.joke.index(joke)],\n",
    "        'joke': joke,\n",
    "        'rating': rating,\n",
    "        'feedback': feedback\n",
    "    }\n",
    "    history.append(this_joke)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the `history` variable saves everything"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'query': 'jokes about baby animals',\n",
       "  'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We should consider a common characteristic or behavior of baby animals that can be humorously exaggerated or word-play that is easy to understand.\",\n",
       "  'joke': 'Why did the baby elephant bring a suitcase to the zoo? Because it wanted to pack its trunk!',\n",
       "  'rating': 4,\n",
       "  'feedback': 'moderately funny'},\n",
       " {'query': 'jokes about baby animals',\n",
       "  'rationale': \"Query: jokes about baby animals  \\nReasoning: Let's think step by step in order to produce the joke. We can start by considering characteristics or behaviors of baby animals that are endearing or amusing. For example, baby animals are often cute, clumsy, curious, and have unique names that can be used cleverly in wordplay. By combining these traits, we can create a lighthearted and funny joke.\",\n",
       "  'joke': \"Why did the baby kangaroo get in trouble at school? Because he couldn't stop jumping to conclusions!\",\n",
       "  'rating': 5,\n",
       "  'feedback': 'this is genius. i love office space. '},\n",
       " {'query': 'jokes about baby animals',\n",
       "  'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We need to consider the characteristics and behaviors of baby animals that might be endearing or funny. Baby animals are often associated with cuteness, innocence, clumsiness, and playful antics. We can play on these traits to create a humorous scenario or a witty remark.\",\n",
       "  'joke': \"Why don't baby ducks tell jokes? Because they always crack up!\",\n",
       "  'rating': 3,\n",
       "  'feedback': 'should be \"quack up\"'},\n",
       " {'query': 'jokes about baby animals',\n",
       "  'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We need to consider baby animals that people commonly find cute or endearing, such as kittens, puppies, or ducklings. Next, we need to incorporate a playful or humorous twist that reflects the innocence and charm of baby animals.\",\n",
       "  'joke': 'Why did the baby kitten bring a pencil to bed? Because it wanted to draw some purr-fect dreams!',\n",
       "  'rating': 2,\n",
       "  'feedback': 'trying too hard'},\n",
       " {'query': 'jokes about baby animals',\n",
       "  'rationale': \"Reasoning: Let's think step by step in order to produce the joke. We know that baby animals are often seen as cute and endearing, so a joke involving them should play on their innocence and charm. Additionally, using a common scenario or play on words can make the joke more relatable and funny.\",\n",
       "  'joke': 'Why did the baby lion always get lost? Because he couldn\\'t stop \"lion\" around!',\n",
       "  'rating': 3,\n",
       "  'feedback': 'meh'}]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now we convert `history` into DSPy.Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Example({'query': 'jokes about baby animals', 'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We should consider a common characteristic or behavior of baby animals that can be humorously exaggerated or word-play that is easy to understand.\", 'joke': 'Why did the baby elephant bring a suitcase to the zoo? Because it wanted to pack its trunk!', 'rating': 4, 'feedback': 'moderately funny'}) (input_keys={'rating', 'feedback', 'query'}),\n",
       " Example({'query': 'jokes about baby animals', 'rationale': \"Query: jokes about baby animals  \\nReasoning: Let's think step by step in order to produce the joke. We can start by considering characteristics or behaviors of baby animals that are endearing or amusing. For example, baby animals are often cute, clumsy, curious, and have unique names that can be used cleverly in wordplay. By combining these traits, we can create a lighthearted and funny joke.\", 'joke': \"Why did the baby kangaroo get in trouble at school? Because he couldn't stop jumping to conclusions!\", 'rating': 5, 'feedback': 'this is genius. i love office space. '}) (input_keys={'rating', 'feedback', 'query'}),\n",
       " Example({'query': 'jokes about baby animals', 'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We need to consider the characteristics and behaviors of baby animals that might be endearing or funny. Baby animals are often associated with cuteness, innocence, clumsiness, and playful antics. We can play on these traits to create a humorous scenario or a witty remark.\", 'joke': \"Why don't baby ducks tell jokes? Because they always crack up!\", 'rating': 3, 'feedback': 'should be \"quack up\"'}) (input_keys={'rating', 'feedback', 'query'}),\n",
       " Example({'query': 'jokes about baby animals', 'rationale': \"Query: jokes about baby animals\\nReasoning: Let's think step by step in order to produce the joke. We need to consider baby animals that people commonly find cute or endearing, such as kittens, puppies, or ducklings. Next, we need to incorporate a playful or humorous twist that reflects the innocence and charm of baby animals.\", 'joke': 'Why did the baby kitten bring a pencil to bed? Because it wanted to draw some purr-fect dreams!', 'rating': 2, 'feedback': 'trying too hard'}) (input_keys={'rating', 'feedback', 'query'}),\n",
       " Example({'query': 'jokes about baby animals', 'rationale': \"Reasoning: Let's think step by step in order to produce the joke. We know that baby animals are often seen as cute and endearing, so a joke involving them should play on their innocence and charm. Additionally, using a common scenario or play on words can make the joke more relatable and funny.\", 'joke': 'Why did the baby lion always get lost? Because he couldn\\'t stop \"lion\" around!', 'rating': 3, 'feedback': 'meh'}) (input_keys={'rating', 'feedback', 'query'})]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from dspy import Example\n",
    "history_examples = [\n",
    "    Example(base=item).with_inputs('query','rating','feedback') for item in history #what should the example inputs be?\n",
    "]\n",
    "display(history_examples)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we are going to optimize the comedian to our joke preferences, using BootstrapFewShot. The quantitative rating is normalized as float 0-1, and serves as the optimization metric. the optimized program is compiled as `minetuned`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 80%|████████  | 4/5 [00:00<00:00, 381.14it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bootstrapped 4 full traces after 5 examples in round 0.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "class ComedianModule(dspy.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.signature = comedian.signature\n",
    "        self.predictor = dspy.ChainOfThought(self.signature)\n",
    "\n",
    "    def forward(self, query, rating=None, feedback=None):\n",
    "        result = self.predictor(query=query, rating=rating, feedback=feedback)\n",
    "        return dspy.Prediction(rationale=result.rationale, joke=result.joke)\n",
    "\n",
    "from dspy.teleprompt.bootstrap import BootstrapFewShot\n",
    "minetuned = BootstrapFewShot(\n",
    "    metric=lambda example, prediction, *args: float(example['rating']/5) #normalize rating to 0-1\n",
    ").compile(\n",
    "    student=ComedianModule(),\n",
    "    trainset=history_examples,\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "optionally, we can inspect the history to see what happened. change n to go back further in the llm history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm.inspect_history(n=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now we can see if `minetuned` is any funnier than the base model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Why did the baby bunny carry a pencil? Because it wanted to draw some \"hare\"-raising pictures!'"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "am_i_funny_now = minetuned(query=query)\n",
    "\n",
    "display(am_i_funny_now.joke)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "my-env310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}