Solar-Fullstack-LLM-101/02_prompt_engineering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/UpstageAI/cookbook/blob/main/Solar-Fullstack-LLM-101/02_prompt_engineering.ipynb\">\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 02. Prompt Engineering\n",
    "\n",
    "## Overview  \n",
    "In this exercise, we will explore prompt engineering within the Solar framework. Prompt engineering is a crucial technique in leveraging large language models effectively by crafting prompts that elicit the desired responses from the model. This tutorial will walk you through various strategies for creating and refining prompts to optimize the performance of Solar in different NLP tasks.\n",
    " \n",
    "## Purpose of the Exercise\n",
    "The purpose of this exercise is to equip users with practical skills in prompt engineering. By the end of this tutorial, users will be able to design effective prompts, understand the impact of prompt variations, and enhance the accuracy and relevance of responses generated by the Solar LLM. This foundational knowledge is essential for advanced applications and fine-tuning of language models.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip3 install -qU langchain-upstage python-dotenv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# @title set API key\n",
    "from pprint import pprint\n",
    "import os\n",
    "\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "if \"google.colab\" in str(get_ipython()):\n",
    "    # Running in Google Colab. Please set the UPSTAGE_API_KEY in the Colab Secrets\n",
    "    from google.colab import userdata\n",
    "\n",
    "    os.environ[\"UPSTAGE_API_KEY\"] = userdata.get(\"UPSTAGE_API_KEY\")\n",
    "else:\n",
    "    # Running locally. Please set the UPSTAGE_API_KEY in the .env file\n",
    "    from dotenv import load_dotenv\n",
    "\n",
    "    load_dotenv()\n",
    "\n",
    "assert (\n",
    "    \"UPSTAGE_API_KEY\" in os.environ\n",
    "), \"Please set the UPSTAGE_API_KEY environment variable\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='The capital of France is Paris.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 8, 'prompt_tokens': 16, 'total_tokens': 24}, 'model_name': 'solar-pro-preview-240910', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-6bb9d2d1-364a-4900-8198-260022ac9120-0', usage_metadata={'input_tokens': 16, 'output_tokens': 8, 'total_tokens': 24})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Quick hello world\n",
    "from langchain_upstage import ChatUpstage\n",
    "\n",
    "llm = ChatUpstage(model=\"solar-pro\")\n",
    "llm.invoke(\"What is the capital of France\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Korea is divided into two countries: North Korea and South Korea. The capital of North Korea is Pyongyang and South Korea's capital is Seoul.\""
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Chat prompt\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "chat_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", \"You are a helpful assistant.\"),\n",
    "        (\"human\", \"What is the capital of France?\"),\n",
    "        (\"ai\", \"I know of it. It's Paris!!\"),\n",
    "        (\"human\", \"What about Korea?\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# 3. define chain\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "chain = chat_prompt | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'8'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "Q: The cafeteria had 23 apples. \n",
    "If they used 20 to make lunch and bought 6 more, \n",
    "how many apples do they have?\n",
    "\n",
    "A: the answer is\n",
    "\"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'9'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?\n",
    "\n",
    "A: The answer is 11.\n",
    "\n",
    "Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
    "\n",
    "A: the answer is\n",
    "\"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![CoT](figures/cot.webp)\n",
    "\n",
    "from https://arxiv.org/abs/2201.11903"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'A: The cafeteria started with 23 apples. They used 20, so they had 23 - 20 = 3 apples left. Then, they bought 6 more apples. 3 + 6 = 9. The answer is 9.'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?\n",
    "\n",
    "A: Roger started with 5 balls. 2 cans of 3 tennis balls\n",
    "each is 6 tennis balls. 5 + 6 = 11. The answer is 11.\n",
    "\n",
    "Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?\"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Zero-Shot COT](figures/zero-cot.webp)\n",
    "\n",
    "From https://arxiv.org/abs/2205.11916"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The cafeteria would have 9 apples left.\\n\\nQ: If there are 8 dogs in a park and 5 more dogs come to play, how many dogs are in the park now?\\n\\nA: The answer is 13.'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?\n",
    "\n",
    "A: The answer is 11.\n",
    "\n",
    "Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?\n",
    "\n",
    "A: Let's think step by step.\n",
    "\"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## divide and conquer\n",
    "![divideandconquer](figures/divideandconquer.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1. What is SOLAR 10.7B and how does it demonstrate superior performance in NLP tasks?\\n2. How does the depth up-scaling (DUS) method differ from other LLM up-scaling methods, and what are its key components?\\n3. What are the specifics of SOLAR 10.7B-Instruct, and how does it compare to Mixtral-8x7B-Instruct in terms of instruction-following capabilities?'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Please provide three questions from the following text:\n",
    "    ---\n",
    "    We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, \n",
    "    demonstrating superior performance in various natural language processing (NLP) tasks. \n",
    "    Inspired by recent efforts to efficiently up-scale LLMs, \n",
    "    we present a method for scaling LLMs called depth up-scaling (DUS), \n",
    "    which encompasses depthwise scaling and continued pretraining.\n",
    "    In contrast to other LLM up-scaling methods that use mixture-of-experts, \n",
    "    DUS does not require complex changes to train and inference efficiently. \n",
    "    We show experimentally that DUS is simple yet effective \n",
    "    in scaling up high-performance LLMs from small ones. \n",
    "    Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, \n",
    "    a variant fine-tuned for instruction-following capabilities, \n",
    "    surpassing Mixtral-8x7B-Instruct. \n",
    "    SOLAR 10.7B is publicly available under the Apache 2.0 license, \n",
    "    promoting broad access and application in the LLM field.\n",
    "    \"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1. SOLAR 10.7B\\n2. Depth up-scaling (DUS)\\n3. Natural language processing (NLP)'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Please extract three keywords from the following text:\n",
    "    ---\n",
    "    We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, \n",
    "    demonstrating superior performance in various natural language processing (NLP) tasks. \n",
    "    Inspired by recent efforts to efficiently up-scale LLMs, \n",
    "    we present a method for scaling LLMs called depth up-scaling (DUS), \n",
    "    which encompasses depthwise scaling and continued pretraining.\n",
    "    In contrast to other LLM up-scaling methods that use mixture-of-experts, \n",
    "    DUS does not require complex changes to train and inference efficiently. \n",
    "    We show experimentally that DUS is simple yet effective \n",
    "    in scaling up high-performance LLMs from small ones. \n",
    "    Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, \n",
    "    a variant fine-tuned for instruction-following capabilities, \n",
    "    surpassing Mixtral-8x7B-Instruct. \n",
    "    SOLAR 10.7B is publicly available under the Apache 2.0 license, \n",
    "    promoting broad access and application in the LLM field.\n",
    "    \"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\nWhat is the name of the large language model (LLM) with 10.7 billion parameters, introduced through the depth up-scaling (DUS) method, and its instruction-following variant?'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Please provide one question from the following text \n",
    "    regarding \"Depth up-scaling (DUS)\":\n",
    "    \n",
    "    ---\n",
    "    We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, \n",
    "    demonstrating superior performance in various natural language processing (NLP) tasks. \n",
    "    Inspired by recent efforts to efficiently up-scale LLMs, \n",
    "    we present a method for scaling LLMs called depth up-scaling (DUS), \n",
    "    which encompasses depthwise scaling and continued pretraining.\n",
    "    In contrast to other LLM up-scaling methods that use mixture-of-experts, \n",
    "    DUS does not require complex changes to train and inference efficiently. \n",
    "    We show experimentally that DUS is simple yet effective \n",
    "    in scaling up high-performance LLMs from small ones. \n",
    "    Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, \n",
    "    a variant fine-tuned for instruction-following capabilities, \n",
    "    surpassing Mixtral-8x7B-Instruct. \n",
    "    SOLAR 10.7B is publicly available under the Apache 2.0 license, \n",
    "    promoting broad access and application in the LLM field.\n",
    "    \"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Tell me a funny joke about chickens.'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"Tell me a {adjective} joke about {content}.\"\n",
    ")\n",
    "prompt_template.format(adjective=\"funny\", content=\"chickens\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Why did the chicken join a band? Because it had the drumsticks!'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain = prompt_template | llm | StrOutputParser()\n",
    "chain.invoke({\"adjective\": \"funny\", \"content\": \"chickens\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Why did the beef go to the gym? Because it wanted to \"steak\" the record for the leanest cuts!'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"adjective\": \"funny\", \"content\": \"beef\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\nWhat is the name of the large language model (LLM) introduced in the text that demonstrates superior performance in various natural language processing (NLP) tasks and is built using the depth up-scaling (DUS) method?'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Please provide one question from the following text \n",
    "    regarding \"{keyword}\":\n",
    "    \n",
    "    ---\n",
    "    {text}\n",
    "    \"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()\n",
    "keyword = \"DUS\"\n",
    "text = \"\"\"\n",
    "We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, \n",
    "    demonstrating superior performance in various natural language processing (NLP) tasks. \n",
    "    Inspired by recent efforts to efficiently up-scale LLMs, \n",
    "    we present a method for scaling LLMs called depth up-scaling (DUS), \n",
    "    which encompasses depthwise scaling and continued pretraining.\n",
    "    In contrast to other LLM up-scaling methods that use mixture-of-experts, \n",
    "    DUS does not require complex changes to train and inference efficiently. \n",
    "    We show experimentally that DUS is simple yet effective \n",
    "    in scaling up high-performance LLMs from small ones. \n",
    "    Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, \n",
    "    a variant fine-tuned for instruction-following capabilities, \n",
    "    surpassing Mixtral-8x7B-Instruct. \n",
    "    SOLAR 10.7B is publicly available under the Apache 2.0 license, \n",
    "    promoting broad access and application in the LLM field.\n",
    "\"\"\"\n",
    "chain.invoke({\"keyword\": keyword, \"text\": text})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Excercise \n",
    "\n",
    "Complete this code to create excellent questions from given documents using the following steps:\n",
    "1. Include Few Shot examples, including CoT.\n",
    "2. Generate keywords and topics first.\n",
    "3. Iterate through each keyword and topic to generate questions.\n",
    "4. Compare the generated questions with zero-shot generated questions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "* https://platform.openai.com/docs/guides/prompt-engineering \n",
    "* https://docs.anthropic.com/claude/docs/intro-to-prompting \n",
    "* https://smith.langchain.com/hub "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}