02b - Speech.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Speech\n",
    "\n",
    "Increasingly, we expect to be able to communicate with artificial intelligence (AI) systems by talking to them, often with the expectation of a spoken response.\n",
    "\n",
    "<p style='text-align:center'><img src='./images/speech.jpg' alt='A robot speaking'/></p>\n",
    "\n",
    "*Speech recognition* (an AI system interpreting spoken language) and *speech synthesis* (an AI system generating a spoken response) are the key components of a speech-enabled AI solution.\n",
    "\n",
    "## Create a Cognitive Services resource\n",
    "\n",
    "To build software that can interpret audible speech and respond verbally, you can use the **Speech** cognitive service, which provides a simple way to transcribe spoken language into text and vice-versa.\n",
    "\n",
    "If you don't already have one, use the following steps to create a **Cognitive Services** resource in your Azure subscription:\n",
    "\n",
    "1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n",
    "2. Click the **&#65291;Create a resource** button, search for *Cognitive Services*, and create a **Cognitive Services** resource with the following settings:\n",
    "    - **Name**: *Enter a unique name*.\n",
    "    - **Subscription**: *Your Azure subscription*.\n",
    "    - **Location**: *Any available location*.\n",
    "    - **Pricing tier**: S0\n",
    "    - **Resource group**: *Create a resource group with a unique name*.\n",
    "3. Wait for deployment to complete. Then go to your cognitive services resource, and on the **Overview** page, click the link to manage the keys for the service. You will need the endpoint and keys to connect to your cognitive services resource from client applications.\n",
    "\n",
    "### Get the Key and Endpoint for your Cognitive Services resource\n",
    "\n",
    "To use your cognitive services resource, client applications need its  endpoint and authentication key:\n",
    "\n",
    "1. In the Azure portal, on the **Keys and Endpoint** page for your cognitive service resource, copy the **Key1** for your resource and paste it in the code below, replacing **YOUR_COG_KEY**.\n",
    "2. Copy the **endpoint** for your resource and and paste it in the code below, replacing **YOUR_COG_ENDPOINT**.\n",
    "3. Run the code in the cell below by clicking its green <span style=\"color:green\">&#9655</span> button."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cog_key = 'YOUR_COG_KEY'\n",
    "cog_endpoint = 'YOUR_COG_ENDPOINT'\n",
    "\n",
    "# Get region from endpoint\n",
    "cog_region = cog_endpoint[8:cog_endpoint.find('.')]\n",
    "\n",
    "print('Ready to use cognitive services in {} using key {}'.format(cog_region, cog_key))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Speech recognition\n",
    "\n",
    "Suppose you want to build a home automation system that accepts spoken instructions, such as \"turn the light on\" or \"turn the light off\". Your application needs to be able to take the audio-based input (your spoken instruction), and interpret it by transcribing it to text that it can then parse and analyze.\n",
    "\n",
    "Now you're ready to transcribe some speech. The input can be a microphone or an audio file. In this case, you'll use an audio file. Run the cell below to hear it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import IPython\n",
    "import os\n",
    "\n",
    "# Get spoken command from audio file\n",
    "file_name = 'light-on.wav'\n",
    "audio_file = os.path.join('data', 'speech', file_name)\n",
    "IPython.display.Audio(data=audio_file, autoplay=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's use the speech-to-text capabilities of the Speech service to transcribe the audio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import IPython\n",
    "from azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, AudioConfig\n",
    "\n",
    "# Get spoken command from audio file\n",
    "file_name = 'light-on.wav'\n",
    "audio_file = os.path.join('data', 'speech', file_name)\n",
    "\n",
    "# Configure speech recognizer\n",
    "speech_config = SpeechConfig(cog_key, cog_region)\n",
    "audio_config = AudioConfig(filename=audio_file) # Use file instead of default (microphone)\n",
    "speech_recognizer = SpeechRecognizer(speech_config, audio_config)\n",
    "\n",
    "# Use a one-time, synchronous call to transcribe the speech\n",
    "speech = speech_recognizer.recognize_once()\n",
    "\n",
    "# Play audio and show transcribed text\n",
    "IPython.display.display(IPython.display.Audio(audio_file, autoplay=True),\n",
    "                        IPython.display.HTML(speech.text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try changing the **file_name** variable to *light-off.wav*, and run the cell again. The service should be able to transcribe both files correctly to text.\n",
    "\n",
    "## Speech synthesis\n",
    "\n",
    "So now you've seen how the Speech service can be used to transcribe speech into text; but what about the opposite? How can you convert text into speech?\n",
    "\n",
    "Well, let's assume your home automation system has interpreted a command to turn the light on. An appropriate response might be to acknowledge the command verbally (as well as actually performing the commanded task!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import IPython\n",
    "from azure.cognitiveservices.speech import SpeechConfig, SpeechSynthesizer, AudioConfig\n",
    "\n",
    "# Get text to be spoken\n",
    "response_text = 'Turning the light on.'\n",
    "\n",
    "# Configure speech synthesis\n",
    "speech_config = SpeechConfig(cog_key, cog_region)\n",
    "output_file = os.path.join('data', 'speech', 'response.wav')\n",
    "audio_output = AudioConfig(filename=output_file) # Use a file instead of default (speakers)\n",
    "speech_synthesizer = SpeechSynthesizer(speech_config, audio_output)\n",
    "\n",
    "# Transcribe text into speech\n",
    "result = speech_synthesizer.speak_text(response_text)\n",
    "\n",
    "# Play the output audio file\n",
    "IPython.display.display(IPython.display.Audio(output_file, autoplay=True),\n",
    "                        IPython.display.Image(data=os.path.join(\"data\", \"speech\" , response_text.lower() + 'jpg')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try changing the **response_text** variable to *Turning the light off.* (including the period at the end) and run the cell again to hear the result.\n",
    "\n",
    "## Learn more\n",
    "\n",
    "You've seen a very simple example of using the Speech cognitive service in this notebook. You can learn more about [speech-to-text](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-speech-to-text) and [text-to-speech](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-text-to-speech) in the Speech service documentation."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "version": "3.5.3-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}