Ch10_Recurrent_Neural_Networks/Beam_Search.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Beam Search\n",
    "\n",
    "In `chapter_seq2seq`, we discussed how to train an encoder-decoder with input and output sequences that are both of variable length. In this section, we are going to introduce how to use the encoder-decoder to predict sequences of variable length.\n",
    "\n",
    "As in the previous section, when preparing to train the data set, we normally attach a special symbol \"&lt;eos&gt;\" after each sentence to indicate the termination of the sequence. We will continue to use this mathematical symbol in the discussion below. For ease of discussion, we assume that the output of the decoder is a sequence of text. Let the size of output text dictionary $\\mathcal{Y}$ (contains special symbol \"&lt;eos&gt;\") be $\\left|\\mathcal{Y}\\right|$, and the maximum length of the output sequence be $T'$. There are a total $\\mathcal{O}(\\left|\\mathcal{Y}\\right|^{T'})$ types of possible output sequences. All the subsequences after the special symbol \"&lt;eos&gt;\" in these output sequences will be discarded.\n",
    "\n",
    "\n",
    "## Greedy Search\n",
    "\n",
    "First, we will take a look at a simple solution: greedy search. For any time step $t'$ of the output sequence, we are going to search for the word with the highest conditional probability from $|\\mathcal{Y}|$ numbers of words, with\n",
    "\n",
    "$$y_{t'} = \\operatorname*{argmax}_{y \\in \\mathcal{Y}} \\mathbb{P}(y \\mid y_1, \\ldots, y_{t'-1}, \\boldsymbol{c})$$\n",
    "\n",
    "as the output.  Once the \"&lt;eos&gt;\" symbol is detected, or the output sequence has reached its maximum length $T'$, the output is completed.\n",
    "\n",
    "As we mentioned in out discussion of the decoder, the conditional probability of generating an output sequence based on the input sequence is $\\prod_{t'=1}^{T'} \\mathbb{P}(y_{t'} \\mid y_1, \\ldots, y_{t'-1}, \\boldsymbol{c})$. We will take the output sequence with the highest conditional probability as the optimal sequence. The main problem with greedy search is that there is no guarantee that the optimal sequence will be obtained.\n",
    "\n",
    "Take a look at the example below. We assume that there are four words \"A\", \"B\", \"C\", and \"&lt;eos&gt;\" in the output dictionary.  The four numbers under each time step in Figure 11.9 represent the conditional probabilities of generating \"A\", \"B\", \"C\", and \"&lt;eos&gt;\" at that time step.  At each time step, greedy search selects the word with the highest conditional probability. Therefore, the output sequence \"A\", \"B\", \"C\", \"&lt;eos&gt;\" will be generated in Figure 11.9. The conditional probability of this output sequence is $0.5\\times0.4\\times0.4\\times0.6 = 0.048$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"87pt\" version=\"1.1\" viewBox=\"0 0 148 87\" width=\"148pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol id=\"glyph0-0\" overflow=\"visible\">\n",
       "<path d=\"M 1.125 0 L 1.125 -5.625 L 5.625 -5.625 L 5.625 0 Z M 1.265625 -0.140625 L 5.484375 -0.140625 L 5.484375 -5.484375 L 1.265625 -5.484375 Z M 1.265625 -0.140625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-1\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -3.171875 C 0.375 -3.929688 0.453125 -4.546875 0.609375 -5.015625 C 0.765625 -5.484375 0.992188 -5.84375 1.296875 -6.09375 C 1.609375 -6.34375 2 -6.46875 2.46875 -6.46875 C 2.820312 -6.46875 3.128906 -6.394531 3.390625 -6.25 C 3.648438 -6.113281 3.863281 -5.914062 4.03125 -5.65625 C 4.195312 -5.394531 4.328125 -5.078125 4.421875 -4.703125 C 4.523438 -4.328125 4.578125 -3.816406 4.578125 -3.171875 C 4.578125 -2.421875 4.5 -1.8125 4.34375 -1.34375 C 4.1875 -0.882812 3.953125 -0.523438 3.640625 -0.265625 C 3.335938 -0.015625 2.945312 0.109375 2.46875 0.109375 C 1.851562 0.109375 1.367188 -0.113281 1.015625 -0.5625 C 0.585938 -1.09375 0.375 -1.960938 0.375 -3.171875 Z M 1.1875 -3.171875 C 1.1875 -2.117188 1.304688 -1.414062 1.546875 -1.0625 C 1.796875 -0.71875 2.101562 -0.546875 2.46875 -0.546875 C 2.832031 -0.546875 3.140625 -0.71875 3.390625 -1.0625 C 3.640625 -1.414062 3.765625 -2.117188 3.765625 -3.171875 C 3.765625 -4.234375 3.640625 -4.9375 3.390625 -5.28125 C 3.140625 -5.632812 2.832031 -5.8125 2.46875 -5.8125 C 2.101562 -5.8125 1.8125 -5.660156 1.59375 -5.359375 C 1.320312 -4.960938 1.1875 -4.234375 1.1875 -3.171875 Z M 1.1875 -3.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-2\" overflow=\"visible\">\n",
       "<path d=\"M 0.8125 0 L 0.8125 -0.90625 L 1.71875 -0.90625 L 1.71875 0 Z M 0.8125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-3\" overflow=\"visible\">\n",
       "<path d=\"M 3.359375 0 L 2.5625 0 L 2.5625 -5.046875 C 2.375 -4.859375 2.125 -4.671875 1.8125 -4.484375 C 1.5 -4.304688 1.222656 -4.175781 0.984375 -4.09375 L 0.984375 -4.859375 C 1.421875 -5.054688 1.804688 -5.300781 2.140625 -5.59375 C 2.472656 -5.894531 2.707031 -6.1875 2.84375 -6.46875 L 3.359375 -6.46875 Z M 3.359375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-4\" overflow=\"visible\">\n",
       "<path d=\"M 4.53125 -0.765625 L 4.53125 0 L 0.265625 0 C 0.265625 -0.1875 0.296875 -0.367188 0.359375 -0.546875 C 0.472656 -0.835938 0.648438 -1.125 0.890625 -1.40625 C 1.128906 -1.6875 1.472656 -2.007812 1.921875 -2.375 C 2.617188 -2.957031 3.085938 -3.414062 3.328125 -3.75 C 3.578125 -4.082031 3.703125 -4.398438 3.703125 -4.703125 C 3.703125 -5.015625 3.585938 -5.273438 3.359375 -5.484375 C 3.140625 -5.703125 2.851562 -5.8125 2.5 -5.8125 C 2.113281 -5.8125 1.804688 -5.695312 1.578125 -5.46875 C 1.347656 -5.238281 1.234375 -4.921875 1.234375 -4.515625 L 0.421875 -4.609375 C 0.472656 -5.210938 0.679688 -5.671875 1.046875 -5.984375 C 1.410156 -6.304688 1.898438 -6.46875 2.515625 -6.46875 C 3.128906 -6.46875 3.613281 -6.296875 3.96875 -5.953125 C 4.332031 -5.609375 4.515625 -5.1875 4.515625 -4.6875 C 4.515625 -4.425781 4.460938 -4.171875 4.359375 -3.921875 C 4.253906 -3.671875 4.078125 -3.40625 3.828125 -3.125 C 3.585938 -2.851562 3.1875 -2.476562 2.625 -2 C 2.144531 -1.601562 1.835938 -1.332031 1.703125 -1.1875 C 1.566406 -1.039062 1.457031 -0.898438 1.375 -0.765625 Z M 4.53125 -0.765625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-5\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.6875 L 1.203125 -1.765625 C 1.265625 -1.359375 1.40625 -1.050781 1.625 -0.84375 C 1.851562 -0.644531 2.125 -0.546875 2.4375 -0.546875 C 2.820312 -0.546875 3.144531 -0.6875 3.40625 -0.96875 C 3.675781 -1.257812 3.8125 -1.640625 3.8125 -2.109375 C 3.8125 -2.566406 3.679688 -2.925781 3.421875 -3.1875 C 3.171875 -3.445312 2.84375 -3.578125 2.4375 -3.578125 C 2.175781 -3.578125 1.941406 -3.515625 1.734375 -3.390625 C 1.535156 -3.273438 1.375 -3.128906 1.25 -2.953125 L 0.515625 -3.046875 L 1.140625 -6.359375 L 4.34375 -6.359375 L 4.34375 -5.59375 L 1.765625 -5.59375 L 1.421875 -3.875 C 1.804688 -4.132812 2.210938 -4.265625 2.640625 -4.265625 C 3.203125 -4.265625 3.675781 -4.070312 4.0625 -3.6875 C 4.445312 -3.300781 4.640625 -2.800781 4.640625 -2.1875 C 4.640625 -1.601562 4.472656 -1.097656 4.140625 -0.671875 C 3.722656 -0.148438 3.15625 0.109375 2.4375 0.109375 C 1.851562 0.109375 1.375 -0.0507812 1 -0.375 C 0.632812 -0.707031 0.425781 -1.144531 0.375 -1.6875 Z M 0.375 -1.6875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-6\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 0 L 2.328125 -5.6875 L 0.21875 -5.6875 L 0.21875 -6.4375 L 5.3125 -6.4375 L 5.3125 -5.6875 L 3.1875 -5.6875 L 3.1875 0 Z M 2.328125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-7\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 -5.53125 L 0.59375 -6.4375 L 1.390625 -6.4375 L 1.390625 -5.53125 Z M 0.59375 0 L 0.59375 -4.671875 L 1.390625 -4.671875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-8\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -4.671875 L 1.296875 -4.671875 L 1.296875 -4.015625 C 1.441406 -4.242188 1.632812 -4.425781 1.875 -4.5625 C 2.125 -4.695312 2.40625 -4.765625 2.71875 -4.765625 C 3.0625 -4.765625 3.34375 -4.691406 3.5625 -4.546875 C 3.78125 -4.410156 3.9375 -4.210938 4.03125 -3.953125 C 4.40625 -4.492188 4.882812 -4.765625 5.46875 -4.765625 C 5.9375 -4.765625 6.296875 -4.632812 6.546875 -4.375 C 6.796875 -4.125 6.921875 -3.734375 6.921875 -3.203125 L 6.921875 0 L 6.125 0 L 6.125 -2.9375 C 6.125 -3.257812 6.097656 -3.488281 6.046875 -3.625 C 6.003906 -3.757812 5.914062 -3.867188 5.78125 -3.953125 C 5.644531 -4.046875 5.484375 -4.09375 5.296875 -4.09375 C 4.972656 -4.09375 4.703125 -3.984375 4.484375 -3.765625 C 4.265625 -3.546875 4.15625 -3.195312 4.15625 -2.71875 L 4.15625 0 L 3.359375 0 L 3.359375 -3.03125 C 3.359375 -3.382812 3.296875 -3.648438 3.171875 -3.828125 C 3.046875 -4.003906 2.835938 -4.09375 2.546875 -4.09375 C 2.316406 -4.09375 2.109375 -4.03125 1.921875 -3.90625 C 1.734375 -3.789062 1.597656 -3.617188 1.515625 -3.390625 C 1.429688 -3.171875 1.390625 -2.847656 1.390625 -2.421875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-9\" overflow=\"visible\">\n",
       "<path d=\"M 3.78125 -1.5 L 4.609375 -1.40625 C 4.472656 -0.925781 4.226562 -0.550781 3.875 -0.28125 C 3.53125 -0.0195312 3.085938 0.109375 2.546875 0.109375 C 1.867188 0.109375 1.328125 -0.0976562 0.921875 -0.515625 C 0.523438 -0.941406 0.328125 -1.535156 0.328125 -2.296875 C 0.328125 -3.078125 0.53125 -3.679688 0.9375 -4.109375 C 1.34375 -4.546875 1.867188 -4.765625 2.515625 -4.765625 C 3.140625 -4.765625 3.644531 -4.550781 4.03125 -4.125 C 4.425781 -3.707031 4.625 -3.113281 4.625 -2.34375 C 4.625 -2.289062 4.625 -2.21875 4.625 -2.125 L 1.140625 -2.125 C 1.171875 -1.613281 1.316406 -1.222656 1.578125 -0.953125 C 1.835938 -0.679688 2.164062 -0.546875 2.5625 -0.546875 C 2.851562 -0.546875 3.097656 -0.617188 3.296875 -0.765625 C 3.503906 -0.921875 3.664062 -1.164062 3.78125 -1.5 Z M 1.1875 -2.78125 L 3.796875 -2.78125 C 3.765625 -3.175781 3.664062 -3.472656 3.5 -3.671875 C 3.25 -3.972656 2.921875 -4.125 2.515625 -4.125 C 2.148438 -4.125 1.84375 -4 1.59375 -3.75 C 1.351562 -3.507812 1.21875 -3.1875 1.1875 -2.78125 Z M 1.1875 -2.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-10\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-11\" overflow=\"visible\">\n",
       "<path d=\"M 0.28125 -1.390625 L 1.0625 -1.515625 C 1.101562 -1.203125 1.222656 -0.960938 1.421875 -0.796875 C 1.628906 -0.628906 1.910156 -0.546875 2.265625 -0.546875 C 2.628906 -0.546875 2.898438 -0.617188 3.078125 -0.765625 C 3.253906 -0.910156 3.34375 -1.082031 3.34375 -1.28125 C 3.34375 -1.46875 3.265625 -1.609375 3.109375 -1.703125 C 2.992188 -1.773438 2.722656 -1.867188 2.296875 -1.984375 C 1.710938 -2.128906 1.304688 -2.253906 1.078125 -2.359375 C 0.859375 -2.460938 0.691406 -2.609375 0.578125 -2.796875 C 0.460938 -2.984375 0.40625 -3.191406 0.40625 -3.421875 C 0.40625 -3.628906 0.453125 -3.820312 0.546875 -4 C 0.640625 -4.175781 0.769531 -4.328125 0.9375 -4.453125 C 1.0625 -4.535156 1.226562 -4.609375 1.4375 -4.671875 C 1.65625 -4.734375 1.882812 -4.765625 2.125 -4.765625 C 2.488281 -4.765625 2.804688 -4.710938 3.078125 -4.609375 C 3.359375 -4.503906 3.566406 -4.363281 3.703125 -4.1875 C 3.835938 -4.007812 3.929688 -3.769531 3.984375 -3.46875 L 3.203125 -3.359375 C 3.171875 -3.597656 3.066406 -3.785156 2.890625 -3.921875 C 2.722656 -4.054688 2.488281 -4.125 2.1875 -4.125 C 1.820312 -4.125 1.5625 -4.0625 1.40625 -3.9375 C 1.25 -3.820312 1.171875 -3.679688 1.171875 -3.515625 C 1.171875 -3.410156 1.203125 -3.320312 1.265625 -3.25 C 1.328125 -3.15625 1.429688 -3.082031 1.578125 -3.03125 C 1.648438 -3 1.878906 -2.929688 2.265625 -2.828125 C 2.828125 -2.679688 3.21875 -2.5625 3.4375 -2.46875 C 3.664062 -2.375 3.84375 -2.234375 3.96875 -2.046875 C 4.09375 -1.867188 4.15625 -1.644531 4.15625 -1.375 C 4.15625 -1.101562 4.078125 -0.851562 3.921875 -0.625 C 3.765625 -0.394531 3.539062 -0.210938 3.25 -0.078125 C 2.96875 0.046875 2.640625 0.109375 2.265625 0.109375 C 1.660156 0.109375 1.195312 -0.015625 0.875 -0.265625 C 0.5625 -0.523438 0.363281 -0.898438 0.28125 -1.390625 Z M 0.28125 -1.390625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-12\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 -0.703125 L 2.4375 -0.015625 C 2.207031 0.0351562 2.007812 0.0625 1.84375 0.0625 C 1.550781 0.0625 1.328125 0.015625 1.171875 -0.078125 C 1.015625 -0.171875 0.898438 -0.289062 0.828125 -0.4375 C 0.765625 -0.582031 0.734375 -0.890625 0.734375 -1.359375 L 0.734375 -4.046875 L 0.15625 -4.046875 L 0.15625 -4.671875 L 0.734375 -4.671875 L 0.734375 -5.828125 L 1.53125 -6.296875 L 1.53125 -4.671875 L 2.328125 -4.671875 L 2.328125 -4.046875 L 1.53125 -4.046875 L 1.53125 -1.328125 C 1.53125 -1.097656 1.539062 -0.953125 1.5625 -0.890625 C 1.59375 -0.828125 1.640625 -0.773438 1.703125 -0.734375 C 1.765625 -0.691406 1.851562 -0.671875 1.96875 -0.671875 C 2.0625 -0.671875 2.179688 -0.679688 2.328125 -0.703125 Z M 2.328125 -0.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-13\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 1.78125 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4.0625 C 1.476562 -4.300781 1.664062 -4.476562 1.875 -4.59375 C 2.09375 -4.707031 2.359375 -4.765625 2.671875 -4.765625 C 3.066406 -4.765625 3.414062 -4.660156 3.71875 -4.453125 C 4.019531 -4.253906 4.25 -3.96875 4.40625 -3.59375 C 4.5625 -3.21875 4.640625 -2.8125 4.640625 -2.375 C 4.640625 -1.894531 4.550781 -1.460938 4.375 -1.078125 C 4.207031 -0.691406 3.960938 -0.394531 3.640625 -0.1875 C 3.316406 0.0078125 2.972656 0.109375 2.609375 0.109375 C 2.347656 0.109375 2.113281 0.0507812 1.90625 -0.0625 C 1.695312 -0.175781 1.523438 -0.316406 1.390625 -0.484375 L 1.390625 1.78125 Z M 1.3125 -2.3125 C 1.3125 -1.707031 1.429688 -1.257812 1.671875 -0.96875 C 1.921875 -0.6875 2.21875 -0.546875 2.5625 -0.546875 C 2.90625 -0.546875 3.203125 -0.691406 3.453125 -0.984375 C 3.710938 -1.285156 3.84375 -1.75 3.84375 -2.375 C 3.84375 -2.96875 3.71875 -3.410156 3.46875 -3.703125 C 3.226562 -4.003906 2.9375 -4.15625 2.59375 -4.15625 C 2.257812 -4.15625 1.960938 -3.992188 1.703125 -3.671875 C 1.441406 -3.359375 1.3125 -2.90625 1.3125 -2.3125 Z M 1.3125 -2.3125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-14\" overflow=\"visible\">\n",
       "<path d=\"M 2.90625 0 L 2.90625 -1.546875 L 0.109375 -1.546875 L 0.109375 -2.265625 L 3.046875 -6.4375 L 3.703125 -6.4375 L 3.703125 -2.265625 L 4.578125 -2.265625 L 4.578125 -1.546875 L 3.703125 -1.546875 L 3.703125 0 Z M 2.90625 -2.265625 L 2.90625 -5.171875 L 0.890625 -2.265625 Z M 2.90625 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-15\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.703125 L 1.171875 -1.8125 C 1.265625 -1.363281 1.414062 -1.039062 1.625 -0.84375 C 1.84375 -0.644531 2.113281 -0.546875 2.4375 -0.546875 C 2.800781 -0.546875 3.109375 -0.671875 3.359375 -0.921875 C 3.617188 -1.179688 3.75 -1.503906 3.75 -1.890625 C 3.75 -2.253906 3.628906 -2.550781 3.390625 -2.78125 C 3.160156 -3.019531 2.863281 -3.140625 2.5 -3.140625 C 2.34375 -3.140625 2.15625 -3.109375 1.9375 -3.046875 L 2.03125 -3.75 C 2.082031 -3.738281 2.125 -3.734375 2.15625 -3.734375 C 2.488281 -3.734375 2.789062 -3.820312 3.0625 -4 C 3.332031 -4.175781 3.46875 -4.445312 3.46875 -4.8125 C 3.46875 -5.101562 3.367188 -5.34375 3.171875 -5.53125 C 2.972656 -5.71875 2.71875 -5.8125 2.40625 -5.8125 C 2.101562 -5.8125 1.847656 -5.710938 1.640625 -5.515625 C 1.441406 -5.328125 1.3125 -5.039062 1.25 -4.65625 L 0.453125 -4.796875 C 0.554688 -5.328125 0.773438 -5.738281 1.109375 -6.03125 C 1.453125 -6.320312 1.878906 -6.46875 2.390625 -6.46875 C 2.742188 -6.46875 3.066406 -6.390625 3.359375 -6.234375 C 3.660156 -6.085938 3.890625 -5.882812 4.046875 -5.625 C 4.203125 -5.363281 4.28125 -5.085938 4.28125 -4.796875 C 4.28125 -4.515625 4.203125 -4.257812 4.046875 -4.03125 C 3.898438 -3.800781 3.679688 -3.617188 3.390625 -3.484375 C 3.773438 -3.398438 4.070312 -3.21875 4.28125 -2.9375 C 4.488281 -2.664062 4.59375 -2.320312 4.59375 -1.90625 C 4.59375 -1.34375 4.382812 -0.863281 3.96875 -0.46875 C 3.5625 -0.0820312 3.046875 0.109375 2.421875 0.109375 C 1.859375 0.109375 1.390625 -0.0546875 1.015625 -0.390625 C 0.640625 -0.722656 0.425781 -1.160156 0.375 -1.703125 Z M 0.375 -1.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-16\" overflow=\"visible\">\n",
       "<path d=\"M 4.484375 -4.859375 L 3.6875 -4.796875 C 3.625 -5.109375 3.523438 -5.335938 3.390625 -5.484375 C 3.179688 -5.703125 2.921875 -5.8125 2.609375 -5.8125 C 2.347656 -5.8125 2.125 -5.742188 1.9375 -5.609375 C 1.6875 -5.421875 1.488281 -5.15625 1.34375 -4.8125 C 1.207031 -4.46875 1.132812 -3.972656 1.125 -3.328125 C 1.3125 -3.609375 1.539062 -3.816406 1.8125 -3.953125 C 2.09375 -4.097656 2.382812 -4.171875 2.6875 -4.171875 C 3.21875 -4.171875 3.664062 -3.976562 4.03125 -3.59375 C 4.40625 -3.207031 4.59375 -2.707031 4.59375 -2.09375 C 4.59375 -1.6875 4.503906 -1.304688 4.328125 -0.953125 C 4.148438 -0.609375 3.910156 -0.34375 3.609375 -0.15625 C 3.304688 0.0195312 2.960938 0.109375 2.578125 0.109375 C 1.921875 0.109375 1.382812 -0.128906 0.96875 -0.609375 C 0.550781 -1.097656 0.34375 -1.898438 0.34375 -3.015625 C 0.34375 -4.253906 0.570312 -5.160156 1.03125 -5.734375 C 1.425781 -6.222656 1.96875 -6.46875 2.65625 -6.46875 C 3.15625 -6.46875 3.566406 -6.320312 3.890625 -6.03125 C 4.210938 -5.75 4.410156 -5.359375 4.484375 -4.859375 Z M 1.25 -2.09375 C 1.25 -1.8125 1.304688 -1.546875 1.421875 -1.296875 C 1.535156 -1.054688 1.695312 -0.867188 1.90625 -0.734375 C 2.113281 -0.609375 2.332031 -0.546875 2.5625 -0.546875 C 2.894531 -0.546875 3.179688 -0.675781 3.421875 -0.9375 C 3.660156 -1.207031 3.78125 -1.578125 3.78125 -2.046875 C 3.78125 -2.492188 3.660156 -2.84375 3.421875 -3.09375 C 3.191406 -3.351562 2.894531 -3.484375 2.53125 -3.484375 C 2.175781 -3.484375 1.875 -3.351562 1.625 -3.09375 C 1.375 -2.84375 1.25 -2.507812 1.25 -2.09375 Z M 1.25 -2.09375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-17\" overflow=\"visible\">\n",
       "<path d=\"M 5.296875 -2.265625 L 6.140625 -2.046875 C 5.960938 -1.347656 5.640625 -0.8125 5.171875 -0.4375 C 4.710938 -0.0703125 4.144531 0.109375 3.46875 0.109375 C 2.78125 0.109375 2.21875 -0.03125 1.78125 -0.3125 C 1.34375 -0.59375 1.007812 -1 0.78125 -1.53125 C 0.5625 -2.070312 0.453125 -2.648438 0.453125 -3.265625 C 0.453125 -3.941406 0.578125 -4.53125 0.828125 -5.03125 C 1.085938 -5.53125 1.453125 -5.90625 1.921875 -6.15625 C 2.398438 -6.414062 2.921875 -6.546875 3.484375 -6.546875 C 4.128906 -6.546875 4.671875 -6.378906 5.109375 -6.046875 C 5.554688 -5.722656 5.863281 -5.265625 6.03125 -4.671875 L 5.1875 -4.484375 C 5.039062 -4.953125 4.828125 -5.289062 4.546875 -5.5 C 4.265625 -5.71875 3.90625 -5.828125 3.46875 -5.828125 C 2.976562 -5.828125 2.566406 -5.707031 2.234375 -5.46875 C 1.898438 -5.226562 1.664062 -4.90625 1.53125 -4.5 C 1.394531 -4.101562 1.328125 -3.695312 1.328125 -3.28125 C 1.328125 -2.726562 1.40625 -2.242188 1.5625 -1.828125 C 1.726562 -1.421875 1.976562 -1.117188 2.3125 -0.921875 C 2.644531 -0.722656 3.007812 -0.625 3.40625 -0.625 C 3.882812 -0.625 4.285156 -0.757812 4.609375 -1.03125 C 4.941406 -1.3125 5.171875 -1.722656 5.296875 -2.265625 Z M 5.296875 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-18\" overflow=\"visible\">\n",
       "<path d=\"M 0.65625 0 L 0.65625 -6.4375 L 3.078125 -6.4375 C 3.566406 -6.4375 3.957031 -6.367188 4.25 -6.234375 C 4.550781 -6.109375 4.785156 -5.910156 4.953125 -5.640625 C 5.128906 -5.367188 5.21875 -5.085938 5.21875 -4.796875 C 5.21875 -4.515625 5.140625 -4.253906 4.984375 -4.015625 C 4.835938 -3.773438 4.613281 -3.578125 4.3125 -3.421875 C 4.695312 -3.304688 4.992188 -3.113281 5.203125 -2.84375 C 5.421875 -2.570312 5.53125 -2.25 5.53125 -1.875 C 5.53125 -1.570312 5.460938 -1.289062 5.328125 -1.03125 C 5.203125 -0.769531 5.046875 -0.566406 4.859375 -0.421875 C 4.671875 -0.285156 4.4375 -0.179688 4.15625 -0.109375 C 3.875 -0.0351562 3.523438 0 3.109375 0 Z M 1.515625 -3.734375 L 2.90625 -3.734375 C 3.28125 -3.734375 3.550781 -3.757812 3.71875 -3.8125 C 3.9375 -3.875 4.097656 -3.976562 4.203125 -4.125 C 4.316406 -4.28125 4.375 -4.46875 4.375 -4.6875 C 4.375 -4.90625 4.320312 -5.09375 4.21875 -5.25 C 4.113281 -5.414062 3.96875 -5.53125 3.78125 -5.59375 C 3.59375 -5.65625 3.265625 -5.6875 2.796875 -5.6875 L 1.515625 -5.6875 Z M 1.515625 -0.765625 L 3.109375 -0.765625 C 3.390625 -0.765625 3.585938 -0.773438 3.703125 -0.796875 C 3.890625 -0.828125 4.050781 -0.882812 4.1875 -0.96875 C 4.320312 -1.050781 4.429688 -1.171875 4.515625 -1.328125 C 4.597656 -1.484375 4.640625 -1.664062 4.640625 -1.875 C 4.640625 -2.113281 4.578125 -2.320312 4.453125 -2.5 C 4.328125 -2.675781 4.15625 -2.796875 3.9375 -2.859375 C 3.71875 -2.929688 3.40625 -2.96875 3 -2.96875 L 1.515625 -2.96875 Z M 1.515625 -0.765625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-19\" overflow=\"visible\">\n",
       "<path d=\"M -0.015625 0 L 2.46875 -6.4375 L 3.375 -6.4375 L 6.015625 0 L 5.046875 0 L 4.296875 -1.953125 L 1.59375 -1.953125 L 0.890625 0 Z M 1.84375 -2.640625 L 4.03125 -2.640625 L 3.359375 -4.4375 C 3.148438 -4.976562 3 -5.421875 2.90625 -5.765625 C 2.820312 -5.347656 2.703125 -4.9375 2.546875 -4.53125 Z M 1.84375 -2.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-0\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-1\" overflow=\"visible\">\n",
       "<path d=\"M 5 -4.640625 L 0.453125 -2.609375 L 0.453125 -2.046875 L 5 -0.015625 L 5 -0.6875 L 1.296875 -2.328125 L 5 -3.96875 Z M 5 -4.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-2\" overflow=\"visible\">\n",
       "<path d=\"M 2.46875 -4.78125 C 1.796875 -4.78125 1.265625 -4.53125 0.875 -4.046875 C 0.484375 -3.59375 0.28125 -3.015625 0.28125 -2.328125 C 0.28125 -1.578125 0.484375 -0.984375 0.90625 -0.53125 C 1.296875 -0.09375 1.84375 0.125 2.5 0.125 C 3.109375 0.125 3.59375 -0.046875 3.984375 -0.390625 C 4.296875 -0.6875 4.515625 -1.046875 4.609375 -1.5 L 3.890625 -1.5 C 3.78125 -1.171875 3.625 -0.921875 3.421875 -0.765625 C 3.1875 -0.578125 2.875 -0.484375 2.5 -0.484375 C 2.0625 -0.484375 1.703125 -0.625 1.453125 -0.90625 C 1.203125 -1.203125 1.078125 -1.609375 1.03125 -2.125 L 4.703125 -2.125 C 4.6875 -2.9375 4.515625 -3.578125 4.171875 -4.015625 C 3.78125 -4.53125 3.21875 -4.78125 2.46875 -4.78125 Z M 2.5 -4.171875 C 3.375 -4.171875 3.875 -3.6875 3.953125 -2.6875 L 1.046875 -2.6875 C 1.09375 -3.140625 1.234375 -3.5 1.5 -3.765625 C 1.75 -4.046875 2.078125 -4.171875 2.5 -4.171875 Z M 2.5 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-3\" overflow=\"visible\">\n",
       "<path d=\"M 2.640625 -4.78125 C 1.9375 -4.78125 1.390625 -4.546875 0.96875 -4.0625 C 0.578125 -3.59375 0.375 -3.03125 0.375 -2.328125 C 0.375 -1.625 0.578125 -1.046875 0.96875 -0.609375 C 1.390625 -0.109375 1.9375 0.125 2.640625 0.125 C 3.328125 0.125 3.890625 -0.109375 4.3125 -0.609375 C 4.703125 -1.046875 4.890625 -1.625 4.890625 -2.328125 C 4.890625 -3.03125 4.703125 -3.59375 4.296875 -4.0625 C 3.890625 -4.546875 3.328125 -4.78125 2.640625 -4.78125 Z M 2.640625 -4.171875 C 3.125 -4.171875 3.515625 -3.984375 3.796875 -3.609375 C 4.046875 -3.265625 4.171875 -2.84375 4.171875 -2.328125 C 4.171875 -1.8125 4.046875 -1.390625 3.796875 -1.046875 C 3.515625 -0.671875 3.125 -0.484375 2.640625 -0.484375 C 2.140625 -0.484375 1.75 -0.671875 1.484375 -1.046875 C 1.234375 -1.390625 1.109375 -1.8125 1.109375 -2.328125 C 1.109375 -2.84375 1.234375 -3.265625 1.484375 -3.609375 C 1.75 -3.984375 2.140625 -4.171875 2.640625 -4.171875 Z M 2.640625 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-4\" overflow=\"visible\">\n",
       "<path d=\"M 2.203125 -4.78125 C 1.6875 -4.78125 1.25 -4.65625 0.921875 -4.421875 C 0.578125 -4.1875 0.40625 -3.890625 0.40625 -3.484375 C 0.40625 -3.09375 0.578125 -2.796875 0.9375 -2.578125 C 1.15625 -2.4375 1.53125 -2.296875 2.09375 -2.171875 C 2.59375 -2.046875 2.921875 -1.9375 3.078125 -1.875 C 3.359375 -1.734375 3.515625 -1.53125 3.515625 -1.265625 C 3.515625 -0.75 3.09375 -0.484375 2.265625 -0.484375 C 1.875 -0.484375 1.578125 -0.546875 1.375 -0.6875 C 1.171875 -0.859375 1.046875 -1.109375 0.984375 -1.484375 L 0.28125 -1.484375 C 0.390625 -0.40625 1.0625 0.125 2.28125 0.125 C 3.59375 0.125 4.25 -0.359375 4.25 -1.3125 C 4.25 -1.71875 4.0625 -2.03125 3.703125 -2.265625 C 3.453125 -2.421875 3.0625 -2.5625 2.5 -2.703125 C 2.03125 -2.828125 1.703125 -2.921875 1.53125 -3 C 1.265625 -3.140625 1.140625 -3.3125 1.140625 -3.515625 C 1.140625 -3.71875 1.234375 -3.875 1.4375 -4 C 1.625 -4.125 1.875 -4.171875 2.203125 -4.171875 C 2.5625 -4.171875 2.859375 -4.109375 3.046875 -3.953125 C 3.234375 -3.828125 3.34375 -3.625 3.40625 -3.328125 L 4.109375 -3.328125 C 3.984375 -4.296875 3.359375 -4.78125 2.203125 -4.78125 Z M 2.203125 -4.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-5\" overflow=\"visible\">\n",
       "<path d=\"M 0.453125 -4.640625 L 0.453125 -3.96875 L 4.15625 -2.328125 L 0.453125 -0.6875 L 0.453125 -0.015625 L 5 -2.046875 L 5 -2.609375 Z M 0.453125 -4.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<rect height=\"87\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"148\" x=\"0\" y=\"0\"/>\n",
       "<rect height=\"87\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"148\" x=\"0\" y=\"0\"/>\n",
       "<path d=\"M 131.445312 149.800781 L 152 149.800781 L 152 165.898438 L 131.445312 165.898438 Z M 131.445312 149.800781 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.466605\" xlink:href=\"#glyph0-1\" y=\"80.702789\"/>\n",
       "  <use x=\"55.472405\" xlink:href=\"#glyph0-2\" y=\"80.702789\"/>\n",
       "  <use x=\"57.976205\" xlink:href=\"#glyph0-3\" y=\"80.702789\"/>\n",
       "</g>\n",
       "<path d=\"M 131.445312 133.699219 L 152 133.699219 L 152 149.800781 L 131.445312 149.800781 Z M 131.445312 133.699219 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.466605\" xlink:href=\"#glyph0-1\" y=\"64.602783\"/>\n",
       "  <use x=\"55.472405\" xlink:href=\"#glyph0-2\" y=\"64.602783\"/>\n",
       "  <use x=\"57.976205\" xlink:href=\"#glyph0-4\" y=\"64.602783\"/>\n",
       "</g>\n",
       "<path d=\"M 131.445312 117.601562 L 152 117.601562 L 152 133.699219 L 131.445312 133.699219 Z M 131.445312 117.601562 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.466605\" xlink:href=\"#glyph0-1\" y=\"48.502783\"/>\n",
       "  <use x=\"55.472405\" xlink:href=\"#glyph0-2\" y=\"48.502783\"/>\n",
       "  <use x=\"57.976205\" xlink:href=\"#glyph0-4\" y=\"48.502783\"/>\n",
       "</g>\n",
       "<path d=\"M 131.445312 101.5 L 152 101.5 L 152 117.601562 L 131.445312 117.601562 Z M 131.445312 101.5 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.466605\" xlink:href=\"#glyph0-1\" y=\"32.402783\"/>\n",
       "  <use x=\"55.472405\" xlink:href=\"#glyph0-2\" y=\"32.402783\"/>\n",
       "  <use x=\"57.976205\" xlink:href=\"#glyph0-5\" y=\"32.402783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.44553\" xlink:href=\"#glyph0-6\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"10.60883\" xlink:href=\"#glyph0-7\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"12.60863\" xlink:href=\"#glyph0-8\" y=\"13.852783\"/>\n",
       "  <use x=\"20.10563\" xlink:href=\"#glyph0-9\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"25.11143\" xlink:href=\"#glyph0-10\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"27.61163\" xlink:href=\"#glyph0-11\" y=\"13.852783\"/>\n",
       "  <use x=\"32.11163\" xlink:href=\"#glyph0-12\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"34.61183\" xlink:href=\"#glyph0-9\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"39.61763\" xlink:href=\"#glyph0-13\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"54.219534\" xlink:href=\"#glyph0-3\" y=\"13.952783\"/>\n",
       "</g>\n",
       "<path d=\"M 158.042969 149.800781 L 178.597656 149.800781 L 178.597656 165.898438 L 158.042969 165.898438 Z M 158.042969 149.800781 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.064985\" xlink:href=\"#glyph0-1\" y=\"80.702789\"/>\n",
       "  <use x=\"82.070785\" xlink:href=\"#glyph0-2\" y=\"80.702789\"/>\n",
       "  <use x=\"84.574585\" xlink:href=\"#glyph0-4\" y=\"80.702789\"/>\n",
       "</g>\n",
       "<path d=\"M 158.042969 101.5 L 178.597656 101.5 L 178.597656 117.601562 L 158.042969 117.601562 Z M 158.042969 101.5 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.064985\" xlink:href=\"#glyph0-1\" y=\"32.402783\"/>\n",
       "  <use x=\"82.070785\" xlink:href=\"#glyph0-2\" y=\"32.402783\"/>\n",
       "  <use x=\"84.574585\" xlink:href=\"#glyph0-3\" y=\"32.402783\"/>\n",
       "</g>\n",
       "<path d=\"M 158.042969 117.601562 L 178.597656 117.601562 L 178.597656 133.699219 L 158.042969 133.699219 Z M 158.042969 117.601562 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.064985\" xlink:href=\"#glyph0-1\" y=\"48.502783\"/>\n",
       "  <use x=\"82.070785\" xlink:href=\"#glyph0-2\" y=\"48.502783\"/>\n",
       "  <use x=\"84.574585\" xlink:href=\"#glyph0-14\" y=\"48.502783\"/>\n",
       "</g>\n",
       "<path d=\"M 158.042969 133.699219 L 178.597656 133.699219 L 178.597656 149.800781 L 158.042969 149.800781 Z M 158.042969 133.699219 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.064985\" xlink:href=\"#glyph0-1\" y=\"64.602783\"/>\n",
       "  <use x=\"82.070785\" xlink:href=\"#glyph0-2\" y=\"64.602783\"/>\n",
       "  <use x=\"84.574585\" xlink:href=\"#glyph0-15\" y=\"64.602783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"80.817914\" xlink:href=\"#glyph0-4\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 184.730469 149.890625 L 205.289062 149.890625 L 205.289062 165.992188 L 184.730469 165.992188 Z M 184.730469 149.890625 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"103.754285\" xlink:href=\"#glyph0-1\" y=\"80.793698\"/>\n",
       "  <use x=\"108.760085\" xlink:href=\"#glyph0-2\" y=\"80.793698\"/>\n",
       "  <use x=\"111.263885\" xlink:href=\"#glyph0-4\" y=\"80.793698\"/>\n",
       "</g>\n",
       "<path d=\"M 184.730469 133.789062 L 205.289062 133.789062 L 205.289062 149.890625 L 184.730469 149.890625 Z M 184.730469 133.789062 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"103.754285\" xlink:href=\"#glyph0-1\" y=\"64.693693\"/>\n",
       "  <use x=\"108.760085\" xlink:href=\"#glyph0-2\" y=\"64.693693\"/>\n",
       "  <use x=\"111.263885\" xlink:href=\"#glyph0-14\" y=\"64.693693\"/>\n",
       "</g>\n",
       "<path d=\"M 184.730469 117.691406 L 205.289062 117.691406 L 205.289062 133.789062 L 184.730469 133.789062 Z M 184.730469 117.691406 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"103.754285\" xlink:href=\"#glyph0-1\" y=\"48.593693\"/>\n",
       "  <use x=\"108.760085\" xlink:href=\"#glyph0-2\" y=\"48.593693\"/>\n",
       "  <use x=\"111.263885\" xlink:href=\"#glyph0-4\" y=\"48.593693\"/>\n",
       "</g>\n",
       "<path d=\"M 184.730469 101.589844 L 205.289062 101.589844 L 205.289062 117.691406 L 184.730469 117.691406 Z M 184.730469 101.589844 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"103.754285\" xlink:href=\"#glyph0-1\" y=\"32.493693\"/>\n",
       "  <use x=\"108.760085\" xlink:href=\"#glyph0-2\" y=\"32.493693\"/>\n",
       "  <use x=\"111.263885\" xlink:href=\"#glyph0-4\" y=\"32.493693\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"107.416314\" xlink:href=\"#glyph0-15\" y=\"13.943693\"/>\n",
       "</g>\n",
       "<path d=\"M 211.421875 149.980469 L 231.976562 149.980469 L 231.976562 166.082031 L 211.421875 166.082031 Z M 211.421875 149.980469 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.443585\" xlink:href=\"#glyph0-1\" y=\"80.884607\"/>\n",
       "  <use x=\"135.449385\" xlink:href=\"#glyph0-2\" y=\"80.884607\"/>\n",
       "  <use x=\"137.953185\" xlink:href=\"#glyph0-16\" y=\"80.884607\"/>\n",
       "</g>\n",
       "<path d=\"M 211.421875 133.882812 L 231.976562 133.882812 L 231.976562 149.980469 L 211.421875 149.980469 Z M 211.421875 133.882812 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.443585\" xlink:href=\"#glyph0-1\" y=\"64.784603\"/>\n",
       "  <use x=\"135.449385\" xlink:href=\"#glyph0-2\" y=\"64.784603\"/>\n",
       "  <use x=\"137.953185\" xlink:href=\"#glyph0-4\" y=\"64.784603\"/>\n",
       "</g>\n",
       "<path d=\"M 211.421875 117.78125 L 231.976562 117.78125 L 231.976562 133.882812 L 211.421875 133.882812 Z M 211.421875 117.78125 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.443585\" xlink:href=\"#glyph0-1\" y=\"48.684603\"/>\n",
       "  <use x=\"135.449385\" xlink:href=\"#glyph0-2\" y=\"48.684603\"/>\n",
       "  <use x=\"137.953185\" xlink:href=\"#glyph0-4\" y=\"48.684603\"/>\n",
       "</g>\n",
       "<path d=\"M 211.421875 101.683594 L 231.976562 101.683594 L 231.976562 117.78125 L 211.421875 117.78125 Z M 211.421875 101.683594 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-85,-80)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.443585\" xlink:href=\"#glyph0-1\" y=\"32.584603\"/>\n",
       "  <use x=\"135.449385\" xlink:href=\"#glyph0-2\" y=\"32.584603\"/>\n",
       "  <use x=\"137.953185\" xlink:href=\"#glyph0-1\" y=\"32.584603\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"134.196514\" xlink:href=\"#glyph0-14\" y=\"14.034603\"/>\n",
       "</g>\n",
       "<path d=\"M 9 69.800781 L 41.066406 69.800781 L 41.066406 85.898438 L 9 85.898438 Z M 9 69.800781 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"12.18215\" xlink:href=\"#glyph1-1\" y=\"81.350006\"/>\n",
       "  <use x=\"17.62715\" xlink:href=\"#glyph1-2\" y=\"81.350006\"/>\n",
       "  <use x=\"22.62215\" xlink:href=\"#glyph1-3\" y=\"81.350006\"/>\n",
       "  <use x=\"27.89615\" xlink:href=\"#glyph1-4\" y=\"81.350006\"/>\n",
       "  <use x=\"32.44115\" xlink:href=\"#glyph1-5\" y=\"81.350006\"/>\n",
       "</g>\n",
       "<path d=\"M 9 53.699219 L 41.066406 53.699219 L 41.066406 69.800781 L 9 69.800781 Z M 9 53.699219 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"21.784394\" xlink:href=\"#glyph0-17\" y=\"64.602783\"/>\n",
       "</g>\n",
       "<path d=\"M 9 37.601562 L 41.066406 37.601562 L 41.066406 53.699219 L 9 53.699219 Z M 9 37.601562 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"22.032685\" xlink:href=\"#glyph0-18\" y=\"48.502783\"/>\n",
       "</g>\n",
       "<path d=\"M 9 21.5 L 41.066406 21.5 L 41.066406 37.601562 L 9 37.601562 Z M 9 21.5 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"22.032685\" xlink:href=\"#glyph0-19\" y=\"32.402783\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import SVG\n",
    "SVG('../img/s2s_prob1.svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Fig.10.15.1: The four numbers under each time step represent the conditional probabilities of generating \"A\", \"B\", \"C\", and \"&lt;eos&gt;\" at that time step.  At each time step, greedy search selects the word with the highest conditional probability. \n",
    "\n",
    "\n",
    "Now, we will look at another example shown in Figure 11.10. Unlike in Figure 11.9, Figure 11.10 selects the word \"C\" for it has the second highest conditional probability at time step 2. Since the output subsequences of time steps 1 and 2, on which time step 3 is based, are changed from \"A\" and \"B\" in Fig. 10.9 to \"A\" and \"C\" in Fig. 10.10, the conditional probability of each word generated at time step 3 has also changed in Fig. 10.10. We choose the word \"B\", which has the highest conditional probability. Now, the output subsequences of time step 4 based on the first three time steps are \"A\", \"C\", and \"B\", which are different from \"A\", \"B\", and \"C\" in Fig. 10.9. Therefore, the conditional probability of generating each word in time step 4 in Figure 11.10 is also different from that in Figure 11.9. We find that the conditional probability of the output sequence \"A\", \"C\", \"B\", \"&lt;eos&gt;\" at the current time step is $0.5\\times0.3 \\times0.6\\times0.6=0.054$, which is higher than the conditional probability of the output sequence obtained by greedy search. Therefore, the output sequence \"A\", \"B\", \"C\", \"&lt;eos&gt;\" obtained by the greedy search is not an optimal sequence.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"87pt\" version=\"1.1\" viewBox=\"0 0 148 87\" width=\"148pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol id=\"glyph0-0\" overflow=\"visible\">\n",
       "<path d=\"M 1.125 0 L 1.125 -5.625 L 5.625 -5.625 L 5.625 0 Z M 1.265625 -0.140625 L 5.484375 -0.140625 L 5.484375 -5.484375 L 1.265625 -5.484375 Z M 1.265625 -0.140625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-1\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -3.171875 C 0.375 -3.929688 0.453125 -4.546875 0.609375 -5.015625 C 0.765625 -5.484375 0.992188 -5.84375 1.296875 -6.09375 C 1.609375 -6.34375 2 -6.46875 2.46875 -6.46875 C 2.820312 -6.46875 3.128906 -6.394531 3.390625 -6.25 C 3.648438 -6.113281 3.863281 -5.914062 4.03125 -5.65625 C 4.195312 -5.394531 4.328125 -5.078125 4.421875 -4.703125 C 4.523438 -4.328125 4.578125 -3.816406 4.578125 -3.171875 C 4.578125 -2.421875 4.5 -1.8125 4.34375 -1.34375 C 4.1875 -0.882812 3.953125 -0.523438 3.640625 -0.265625 C 3.335938 -0.015625 2.945312 0.109375 2.46875 0.109375 C 1.851562 0.109375 1.367188 -0.113281 1.015625 -0.5625 C 0.585938 -1.09375 0.375 -1.960938 0.375 -3.171875 Z M 1.1875 -3.171875 C 1.1875 -2.117188 1.304688 -1.414062 1.546875 -1.0625 C 1.796875 -0.71875 2.101562 -0.546875 2.46875 -0.546875 C 2.832031 -0.546875 3.140625 -0.71875 3.390625 -1.0625 C 3.640625 -1.414062 3.765625 -2.117188 3.765625 -3.171875 C 3.765625 -4.234375 3.640625 -4.9375 3.390625 -5.28125 C 3.140625 -5.632812 2.832031 -5.8125 2.46875 -5.8125 C 2.101562 -5.8125 1.8125 -5.660156 1.59375 -5.359375 C 1.320312 -4.960938 1.1875 -4.234375 1.1875 -3.171875 Z M 1.1875 -3.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-2\" overflow=\"visible\">\n",
       "<path d=\"M 0.8125 0 L 0.8125 -0.90625 L 1.71875 -0.90625 L 1.71875 0 Z M 0.8125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-3\" overflow=\"visible\">\n",
       "<path d=\"M 3.359375 0 L 2.5625 0 L 2.5625 -5.046875 C 2.375 -4.859375 2.125 -4.671875 1.8125 -4.484375 C 1.5 -4.304688 1.222656 -4.175781 0.984375 -4.09375 L 0.984375 -4.859375 C 1.421875 -5.054688 1.804688 -5.300781 2.140625 -5.59375 C 2.472656 -5.894531 2.707031 -6.1875 2.84375 -6.46875 L 3.359375 -6.46875 Z M 3.359375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-4\" overflow=\"visible\">\n",
       "<path d=\"M 4.53125 -0.765625 L 4.53125 0 L 0.265625 0 C 0.265625 -0.1875 0.296875 -0.367188 0.359375 -0.546875 C 0.472656 -0.835938 0.648438 -1.125 0.890625 -1.40625 C 1.128906 -1.6875 1.472656 -2.007812 1.921875 -2.375 C 2.617188 -2.957031 3.085938 -3.414062 3.328125 -3.75 C 3.578125 -4.082031 3.703125 -4.398438 3.703125 -4.703125 C 3.703125 -5.015625 3.585938 -5.273438 3.359375 -5.484375 C 3.140625 -5.703125 2.851562 -5.8125 2.5 -5.8125 C 2.113281 -5.8125 1.804688 -5.695312 1.578125 -5.46875 C 1.347656 -5.238281 1.234375 -4.921875 1.234375 -4.515625 L 0.421875 -4.609375 C 0.472656 -5.210938 0.679688 -5.671875 1.046875 -5.984375 C 1.410156 -6.304688 1.898438 -6.46875 2.515625 -6.46875 C 3.128906 -6.46875 3.613281 -6.296875 3.96875 -5.953125 C 4.332031 -5.609375 4.515625 -5.1875 4.515625 -4.6875 C 4.515625 -4.425781 4.460938 -4.171875 4.359375 -3.921875 C 4.253906 -3.671875 4.078125 -3.40625 3.828125 -3.125 C 3.585938 -2.851562 3.1875 -2.476562 2.625 -2 C 2.144531 -1.601562 1.835938 -1.332031 1.703125 -1.1875 C 1.566406 -1.039062 1.457031 -0.898438 1.375 -0.765625 Z M 4.53125 -0.765625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-5\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.6875 L 1.203125 -1.765625 C 1.265625 -1.359375 1.40625 -1.050781 1.625 -0.84375 C 1.851562 -0.644531 2.125 -0.546875 2.4375 -0.546875 C 2.820312 -0.546875 3.144531 -0.6875 3.40625 -0.96875 C 3.675781 -1.257812 3.8125 -1.640625 3.8125 -2.109375 C 3.8125 -2.566406 3.679688 -2.925781 3.421875 -3.1875 C 3.171875 -3.445312 2.84375 -3.578125 2.4375 -3.578125 C 2.175781 -3.578125 1.941406 -3.515625 1.734375 -3.390625 C 1.535156 -3.273438 1.375 -3.128906 1.25 -2.953125 L 0.515625 -3.046875 L 1.140625 -6.359375 L 4.34375 -6.359375 L 4.34375 -5.59375 L 1.765625 -5.59375 L 1.421875 -3.875 C 1.804688 -4.132812 2.210938 -4.265625 2.640625 -4.265625 C 3.203125 -4.265625 3.675781 -4.070312 4.0625 -3.6875 C 4.445312 -3.300781 4.640625 -2.800781 4.640625 -2.1875 C 4.640625 -1.601562 4.472656 -1.097656 4.140625 -0.671875 C 3.722656 -0.148438 3.15625 0.109375 2.4375 0.109375 C 1.851562 0.109375 1.375 -0.0507812 1 -0.375 C 0.632812 -0.707031 0.425781 -1.144531 0.375 -1.6875 Z M 0.375 -1.6875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-6\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 0 L 2.328125 -5.6875 L 0.21875 -5.6875 L 0.21875 -6.4375 L 5.3125 -6.4375 L 5.3125 -5.6875 L 3.1875 -5.6875 L 3.1875 0 Z M 2.328125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-7\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 -5.53125 L 0.59375 -6.4375 L 1.390625 -6.4375 L 1.390625 -5.53125 Z M 0.59375 0 L 0.59375 -4.671875 L 1.390625 -4.671875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-8\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -4.671875 L 1.296875 -4.671875 L 1.296875 -4.015625 C 1.441406 -4.242188 1.632812 -4.425781 1.875 -4.5625 C 2.125 -4.695312 2.40625 -4.765625 2.71875 -4.765625 C 3.0625 -4.765625 3.34375 -4.691406 3.5625 -4.546875 C 3.78125 -4.410156 3.9375 -4.210938 4.03125 -3.953125 C 4.40625 -4.492188 4.882812 -4.765625 5.46875 -4.765625 C 5.9375 -4.765625 6.296875 -4.632812 6.546875 -4.375 C 6.796875 -4.125 6.921875 -3.734375 6.921875 -3.203125 L 6.921875 0 L 6.125 0 L 6.125 -2.9375 C 6.125 -3.257812 6.097656 -3.488281 6.046875 -3.625 C 6.003906 -3.757812 5.914062 -3.867188 5.78125 -3.953125 C 5.644531 -4.046875 5.484375 -4.09375 5.296875 -4.09375 C 4.972656 -4.09375 4.703125 -3.984375 4.484375 -3.765625 C 4.265625 -3.546875 4.15625 -3.195312 4.15625 -2.71875 L 4.15625 0 L 3.359375 0 L 3.359375 -3.03125 C 3.359375 -3.382812 3.296875 -3.648438 3.171875 -3.828125 C 3.046875 -4.003906 2.835938 -4.09375 2.546875 -4.09375 C 2.316406 -4.09375 2.109375 -4.03125 1.921875 -3.90625 C 1.734375 -3.789062 1.597656 -3.617188 1.515625 -3.390625 C 1.429688 -3.171875 1.390625 -2.847656 1.390625 -2.421875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-9\" overflow=\"visible\">\n",
       "<path d=\"M 3.78125 -1.5 L 4.609375 -1.40625 C 4.472656 -0.925781 4.226562 -0.550781 3.875 -0.28125 C 3.53125 -0.0195312 3.085938 0.109375 2.546875 0.109375 C 1.867188 0.109375 1.328125 -0.0976562 0.921875 -0.515625 C 0.523438 -0.941406 0.328125 -1.535156 0.328125 -2.296875 C 0.328125 -3.078125 0.53125 -3.679688 0.9375 -4.109375 C 1.34375 -4.546875 1.867188 -4.765625 2.515625 -4.765625 C 3.140625 -4.765625 3.644531 -4.550781 4.03125 -4.125 C 4.425781 -3.707031 4.625 -3.113281 4.625 -2.34375 C 4.625 -2.289062 4.625 -2.21875 4.625 -2.125 L 1.140625 -2.125 C 1.171875 -1.613281 1.316406 -1.222656 1.578125 -0.953125 C 1.835938 -0.679688 2.164062 -0.546875 2.5625 -0.546875 C 2.851562 -0.546875 3.097656 -0.617188 3.296875 -0.765625 C 3.503906 -0.921875 3.664062 -1.164062 3.78125 -1.5 Z M 1.1875 -2.78125 L 3.796875 -2.78125 C 3.765625 -3.175781 3.664062 -3.472656 3.5 -3.671875 C 3.25 -3.972656 2.921875 -4.125 2.515625 -4.125 C 2.148438 -4.125 1.84375 -4 1.59375 -3.75 C 1.351562 -3.507812 1.21875 -3.1875 1.1875 -2.78125 Z M 1.1875 -2.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-10\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-11\" overflow=\"visible\">\n",
       "<path d=\"M 0.28125 -1.390625 L 1.0625 -1.515625 C 1.101562 -1.203125 1.222656 -0.960938 1.421875 -0.796875 C 1.628906 -0.628906 1.910156 -0.546875 2.265625 -0.546875 C 2.628906 -0.546875 2.898438 -0.617188 3.078125 -0.765625 C 3.253906 -0.910156 3.34375 -1.082031 3.34375 -1.28125 C 3.34375 -1.46875 3.265625 -1.609375 3.109375 -1.703125 C 2.992188 -1.773438 2.722656 -1.867188 2.296875 -1.984375 C 1.710938 -2.128906 1.304688 -2.253906 1.078125 -2.359375 C 0.859375 -2.460938 0.691406 -2.609375 0.578125 -2.796875 C 0.460938 -2.984375 0.40625 -3.191406 0.40625 -3.421875 C 0.40625 -3.628906 0.453125 -3.820312 0.546875 -4 C 0.640625 -4.175781 0.769531 -4.328125 0.9375 -4.453125 C 1.0625 -4.535156 1.226562 -4.609375 1.4375 -4.671875 C 1.65625 -4.734375 1.882812 -4.765625 2.125 -4.765625 C 2.488281 -4.765625 2.804688 -4.710938 3.078125 -4.609375 C 3.359375 -4.503906 3.566406 -4.363281 3.703125 -4.1875 C 3.835938 -4.007812 3.929688 -3.769531 3.984375 -3.46875 L 3.203125 -3.359375 C 3.171875 -3.597656 3.066406 -3.785156 2.890625 -3.921875 C 2.722656 -4.054688 2.488281 -4.125 2.1875 -4.125 C 1.820312 -4.125 1.5625 -4.0625 1.40625 -3.9375 C 1.25 -3.820312 1.171875 -3.679688 1.171875 -3.515625 C 1.171875 -3.410156 1.203125 -3.320312 1.265625 -3.25 C 1.328125 -3.15625 1.429688 -3.082031 1.578125 -3.03125 C 1.648438 -3 1.878906 -2.929688 2.265625 -2.828125 C 2.828125 -2.679688 3.21875 -2.5625 3.4375 -2.46875 C 3.664062 -2.375 3.84375 -2.234375 3.96875 -2.046875 C 4.09375 -1.867188 4.15625 -1.644531 4.15625 -1.375 C 4.15625 -1.101562 4.078125 -0.851562 3.921875 -0.625 C 3.765625 -0.394531 3.539062 -0.210938 3.25 -0.078125 C 2.96875 0.046875 2.640625 0.109375 2.265625 0.109375 C 1.660156 0.109375 1.195312 -0.015625 0.875 -0.265625 C 0.5625 -0.523438 0.363281 -0.898438 0.28125 -1.390625 Z M 0.28125 -1.390625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-12\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 -0.703125 L 2.4375 -0.015625 C 2.207031 0.0351562 2.007812 0.0625 1.84375 0.0625 C 1.550781 0.0625 1.328125 0.015625 1.171875 -0.078125 C 1.015625 -0.171875 0.898438 -0.289062 0.828125 -0.4375 C 0.765625 -0.582031 0.734375 -0.890625 0.734375 -1.359375 L 0.734375 -4.046875 L 0.15625 -4.046875 L 0.15625 -4.671875 L 0.734375 -4.671875 L 0.734375 -5.828125 L 1.53125 -6.296875 L 1.53125 -4.671875 L 2.328125 -4.671875 L 2.328125 -4.046875 L 1.53125 -4.046875 L 1.53125 -1.328125 C 1.53125 -1.097656 1.539062 -0.953125 1.5625 -0.890625 C 1.59375 -0.828125 1.640625 -0.773438 1.703125 -0.734375 C 1.765625 -0.691406 1.851562 -0.671875 1.96875 -0.671875 C 2.0625 -0.671875 2.179688 -0.679688 2.328125 -0.703125 Z M 2.328125 -0.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-13\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 1.78125 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4.0625 C 1.476562 -4.300781 1.664062 -4.476562 1.875 -4.59375 C 2.09375 -4.707031 2.359375 -4.765625 2.671875 -4.765625 C 3.066406 -4.765625 3.414062 -4.660156 3.71875 -4.453125 C 4.019531 -4.253906 4.25 -3.96875 4.40625 -3.59375 C 4.5625 -3.21875 4.640625 -2.8125 4.640625 -2.375 C 4.640625 -1.894531 4.550781 -1.460938 4.375 -1.078125 C 4.207031 -0.691406 3.960938 -0.394531 3.640625 -0.1875 C 3.316406 0.0078125 2.972656 0.109375 2.609375 0.109375 C 2.347656 0.109375 2.113281 0.0507812 1.90625 -0.0625 C 1.695312 -0.175781 1.523438 -0.316406 1.390625 -0.484375 L 1.390625 1.78125 Z M 1.3125 -2.3125 C 1.3125 -1.707031 1.429688 -1.257812 1.671875 -0.96875 C 1.921875 -0.6875 2.21875 -0.546875 2.5625 -0.546875 C 2.90625 -0.546875 3.203125 -0.691406 3.453125 -0.984375 C 3.710938 -1.285156 3.84375 -1.75 3.84375 -2.375 C 3.84375 -2.96875 3.71875 -3.410156 3.46875 -3.703125 C 3.226562 -4.003906 2.9375 -4.15625 2.59375 -4.15625 C 2.257812 -4.15625 1.960938 -3.992188 1.703125 -3.671875 C 1.441406 -3.359375 1.3125 -2.90625 1.3125 -2.3125 Z M 1.3125 -2.3125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-14\" overflow=\"visible\">\n",
       "<path d=\"M 2.90625 0 L 2.90625 -1.546875 L 0.109375 -1.546875 L 0.109375 -2.265625 L 3.046875 -6.4375 L 3.703125 -6.4375 L 3.703125 -2.265625 L 4.578125 -2.265625 L 4.578125 -1.546875 L 3.703125 -1.546875 L 3.703125 0 Z M 2.90625 -2.265625 L 2.90625 -5.171875 L 0.890625 -2.265625 Z M 2.90625 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-15\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.703125 L 1.171875 -1.8125 C 1.265625 -1.363281 1.414062 -1.039062 1.625 -0.84375 C 1.84375 -0.644531 2.113281 -0.546875 2.4375 -0.546875 C 2.800781 -0.546875 3.109375 -0.671875 3.359375 -0.921875 C 3.617188 -1.179688 3.75 -1.503906 3.75 -1.890625 C 3.75 -2.253906 3.628906 -2.550781 3.390625 -2.78125 C 3.160156 -3.019531 2.863281 -3.140625 2.5 -3.140625 C 2.34375 -3.140625 2.15625 -3.109375 1.9375 -3.046875 L 2.03125 -3.75 C 2.082031 -3.738281 2.125 -3.734375 2.15625 -3.734375 C 2.488281 -3.734375 2.789062 -3.820312 3.0625 -4 C 3.332031 -4.175781 3.46875 -4.445312 3.46875 -4.8125 C 3.46875 -5.101562 3.367188 -5.34375 3.171875 -5.53125 C 2.972656 -5.71875 2.71875 -5.8125 2.40625 -5.8125 C 2.101562 -5.8125 1.847656 -5.710938 1.640625 -5.515625 C 1.441406 -5.328125 1.3125 -5.039062 1.25 -4.65625 L 0.453125 -4.796875 C 0.554688 -5.328125 0.773438 -5.738281 1.109375 -6.03125 C 1.453125 -6.320312 1.878906 -6.46875 2.390625 -6.46875 C 2.742188 -6.46875 3.066406 -6.390625 3.359375 -6.234375 C 3.660156 -6.085938 3.890625 -5.882812 4.046875 -5.625 C 4.203125 -5.363281 4.28125 -5.085938 4.28125 -4.796875 C 4.28125 -4.515625 4.203125 -4.257812 4.046875 -4.03125 C 3.898438 -3.800781 3.679688 -3.617188 3.390625 -3.484375 C 3.773438 -3.398438 4.070312 -3.21875 4.28125 -2.9375 C 4.488281 -2.664062 4.59375 -2.320312 4.59375 -1.90625 C 4.59375 -1.34375 4.382812 -0.863281 3.96875 -0.46875 C 3.5625 -0.0820312 3.046875 0.109375 2.421875 0.109375 C 1.859375 0.109375 1.390625 -0.0546875 1.015625 -0.390625 C 0.640625 -0.722656 0.425781 -1.160156 0.375 -1.703125 Z M 0.375 -1.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-16\" overflow=\"visible\">\n",
       "<path d=\"M 4.484375 -4.859375 L 3.6875 -4.796875 C 3.625 -5.109375 3.523438 -5.335938 3.390625 -5.484375 C 3.179688 -5.703125 2.921875 -5.8125 2.609375 -5.8125 C 2.347656 -5.8125 2.125 -5.742188 1.9375 -5.609375 C 1.6875 -5.421875 1.488281 -5.15625 1.34375 -4.8125 C 1.207031 -4.46875 1.132812 -3.972656 1.125 -3.328125 C 1.3125 -3.609375 1.539062 -3.816406 1.8125 -3.953125 C 2.09375 -4.097656 2.382812 -4.171875 2.6875 -4.171875 C 3.21875 -4.171875 3.664062 -3.976562 4.03125 -3.59375 C 4.40625 -3.207031 4.59375 -2.707031 4.59375 -2.09375 C 4.59375 -1.6875 4.503906 -1.304688 4.328125 -0.953125 C 4.148438 -0.609375 3.910156 -0.34375 3.609375 -0.15625 C 3.304688 0.0195312 2.960938 0.109375 2.578125 0.109375 C 1.921875 0.109375 1.382812 -0.128906 0.96875 -0.609375 C 0.550781 -1.097656 0.34375 -1.898438 0.34375 -3.015625 C 0.34375 -4.253906 0.570312 -5.160156 1.03125 -5.734375 C 1.425781 -6.222656 1.96875 -6.46875 2.65625 -6.46875 C 3.15625 -6.46875 3.566406 -6.320312 3.890625 -6.03125 C 4.210938 -5.75 4.410156 -5.359375 4.484375 -4.859375 Z M 1.25 -2.09375 C 1.25 -1.8125 1.304688 -1.546875 1.421875 -1.296875 C 1.535156 -1.054688 1.695312 -0.867188 1.90625 -0.734375 C 2.113281 -0.609375 2.332031 -0.546875 2.5625 -0.546875 C 2.894531 -0.546875 3.179688 -0.675781 3.421875 -0.9375 C 3.660156 -1.207031 3.78125 -1.578125 3.78125 -2.046875 C 3.78125 -2.492188 3.660156 -2.84375 3.421875 -3.09375 C 3.191406 -3.351562 2.894531 -3.484375 2.53125 -3.484375 C 2.175781 -3.484375 1.875 -3.351562 1.625 -3.09375 C 1.375 -2.84375 1.25 -2.507812 1.25 -2.09375 Z M 1.25 -2.09375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-17\" overflow=\"visible\">\n",
       "<path d=\"M 5.296875 -2.265625 L 6.140625 -2.046875 C 5.960938 -1.347656 5.640625 -0.8125 5.171875 -0.4375 C 4.710938 -0.0703125 4.144531 0.109375 3.46875 0.109375 C 2.78125 0.109375 2.21875 -0.03125 1.78125 -0.3125 C 1.34375 -0.59375 1.007812 -1 0.78125 -1.53125 C 0.5625 -2.070312 0.453125 -2.648438 0.453125 -3.265625 C 0.453125 -3.941406 0.578125 -4.53125 0.828125 -5.03125 C 1.085938 -5.53125 1.453125 -5.90625 1.921875 -6.15625 C 2.398438 -6.414062 2.921875 -6.546875 3.484375 -6.546875 C 4.128906 -6.546875 4.671875 -6.378906 5.109375 -6.046875 C 5.554688 -5.722656 5.863281 -5.265625 6.03125 -4.671875 L 5.1875 -4.484375 C 5.039062 -4.953125 4.828125 -5.289062 4.546875 -5.5 C 4.265625 -5.71875 3.90625 -5.828125 3.46875 -5.828125 C 2.976562 -5.828125 2.566406 -5.707031 2.234375 -5.46875 C 1.898438 -5.226562 1.664062 -4.90625 1.53125 -4.5 C 1.394531 -4.101562 1.328125 -3.695312 1.328125 -3.28125 C 1.328125 -2.726562 1.40625 -2.242188 1.5625 -1.828125 C 1.726562 -1.421875 1.976562 -1.117188 2.3125 -0.921875 C 2.644531 -0.722656 3.007812 -0.625 3.40625 -0.625 C 3.882812 -0.625 4.285156 -0.757812 4.609375 -1.03125 C 4.941406 -1.3125 5.171875 -1.722656 5.296875 -2.265625 Z M 5.296875 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-18\" overflow=\"visible\">\n",
       "<path d=\"M 0.65625 0 L 0.65625 -6.4375 L 3.078125 -6.4375 C 3.566406 -6.4375 3.957031 -6.367188 4.25 -6.234375 C 4.550781 -6.109375 4.785156 -5.910156 4.953125 -5.640625 C 5.128906 -5.367188 5.21875 -5.085938 5.21875 -4.796875 C 5.21875 -4.515625 5.140625 -4.253906 4.984375 -4.015625 C 4.835938 -3.773438 4.613281 -3.578125 4.3125 -3.421875 C 4.695312 -3.304688 4.992188 -3.113281 5.203125 -2.84375 C 5.421875 -2.570312 5.53125 -2.25 5.53125 -1.875 C 5.53125 -1.570312 5.460938 -1.289062 5.328125 -1.03125 C 5.203125 -0.769531 5.046875 -0.566406 4.859375 -0.421875 C 4.671875 -0.285156 4.4375 -0.179688 4.15625 -0.109375 C 3.875 -0.0351562 3.523438 0 3.109375 0 Z M 1.515625 -3.734375 L 2.90625 -3.734375 C 3.28125 -3.734375 3.550781 -3.757812 3.71875 -3.8125 C 3.9375 -3.875 4.097656 -3.976562 4.203125 -4.125 C 4.316406 -4.28125 4.375 -4.46875 4.375 -4.6875 C 4.375 -4.90625 4.320312 -5.09375 4.21875 -5.25 C 4.113281 -5.414062 3.96875 -5.53125 3.78125 -5.59375 C 3.59375 -5.65625 3.265625 -5.6875 2.796875 -5.6875 L 1.515625 -5.6875 Z M 1.515625 -0.765625 L 3.109375 -0.765625 C 3.390625 -0.765625 3.585938 -0.773438 3.703125 -0.796875 C 3.890625 -0.828125 4.050781 -0.882812 4.1875 -0.96875 C 4.320312 -1.050781 4.429688 -1.171875 4.515625 -1.328125 C 4.597656 -1.484375 4.640625 -1.664062 4.640625 -1.875 C 4.640625 -2.113281 4.578125 -2.320312 4.453125 -2.5 C 4.328125 -2.675781 4.15625 -2.796875 3.9375 -2.859375 C 3.71875 -2.929688 3.40625 -2.96875 3 -2.96875 L 1.515625 -2.96875 Z M 1.515625 -0.765625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-19\" overflow=\"visible\">\n",
       "<path d=\"M -0.015625 0 L 2.46875 -6.4375 L 3.375 -6.4375 L 6.015625 0 L 5.046875 0 L 4.296875 -1.953125 L 1.59375 -1.953125 L 0.890625 0 Z M 1.84375 -2.640625 L 4.03125 -2.640625 L 3.359375 -4.4375 C 3.148438 -4.976562 3 -5.421875 2.90625 -5.765625 C 2.820312 -5.347656 2.703125 -4.9375 2.546875 -4.53125 Z M 1.84375 -2.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-0\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-1\" overflow=\"visible\">\n",
       "<path d=\"M 5 -4.640625 L 0.453125 -2.609375 L 0.453125 -2.046875 L 5 -0.015625 L 5 -0.6875 L 1.296875 -2.328125 L 5 -3.96875 Z M 5 -4.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-2\" overflow=\"visible\">\n",
       "<path d=\"M 2.46875 -4.78125 C 1.796875 -4.78125 1.265625 -4.53125 0.875 -4.046875 C 0.484375 -3.59375 0.28125 -3.015625 0.28125 -2.328125 C 0.28125 -1.578125 0.484375 -0.984375 0.90625 -0.53125 C 1.296875 -0.09375 1.84375 0.125 2.5 0.125 C 3.109375 0.125 3.59375 -0.046875 3.984375 -0.390625 C 4.296875 -0.6875 4.515625 -1.046875 4.609375 -1.5 L 3.890625 -1.5 C 3.78125 -1.171875 3.625 -0.921875 3.421875 -0.765625 C 3.1875 -0.578125 2.875 -0.484375 2.5 -0.484375 C 2.0625 -0.484375 1.703125 -0.625 1.453125 -0.90625 C 1.203125 -1.203125 1.078125 -1.609375 1.03125 -2.125 L 4.703125 -2.125 C 4.6875 -2.9375 4.515625 -3.578125 4.171875 -4.015625 C 3.78125 -4.53125 3.21875 -4.78125 2.46875 -4.78125 Z M 2.5 -4.171875 C 3.375 -4.171875 3.875 -3.6875 3.953125 -2.6875 L 1.046875 -2.6875 C 1.09375 -3.140625 1.234375 -3.5 1.5 -3.765625 C 1.75 -4.046875 2.078125 -4.171875 2.5 -4.171875 Z M 2.5 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-3\" overflow=\"visible\">\n",
       "<path d=\"M 2.640625 -4.78125 C 1.9375 -4.78125 1.390625 -4.546875 0.96875 -4.0625 C 0.578125 -3.59375 0.375 -3.03125 0.375 -2.328125 C 0.375 -1.625 0.578125 -1.046875 0.96875 -0.609375 C 1.390625 -0.109375 1.9375 0.125 2.640625 0.125 C 3.328125 0.125 3.890625 -0.109375 4.3125 -0.609375 C 4.703125 -1.046875 4.890625 -1.625 4.890625 -2.328125 C 4.890625 -3.03125 4.703125 -3.59375 4.296875 -4.0625 C 3.890625 -4.546875 3.328125 -4.78125 2.640625 -4.78125 Z M 2.640625 -4.171875 C 3.125 -4.171875 3.515625 -3.984375 3.796875 -3.609375 C 4.046875 -3.265625 4.171875 -2.84375 4.171875 -2.328125 C 4.171875 -1.8125 4.046875 -1.390625 3.796875 -1.046875 C 3.515625 -0.671875 3.125 -0.484375 2.640625 -0.484375 C 2.140625 -0.484375 1.75 -0.671875 1.484375 -1.046875 C 1.234375 -1.390625 1.109375 -1.8125 1.109375 -2.328125 C 1.109375 -2.84375 1.234375 -3.265625 1.484375 -3.609375 C 1.75 -3.984375 2.140625 -4.171875 2.640625 -4.171875 Z M 2.640625 -4.171875 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-4\" overflow=\"visible\">\n",
       "<path d=\"M 2.203125 -4.78125 C 1.6875 -4.78125 1.25 -4.65625 0.921875 -4.421875 C 0.578125 -4.1875 0.40625 -3.890625 0.40625 -3.484375 C 0.40625 -3.09375 0.578125 -2.796875 0.9375 -2.578125 C 1.15625 -2.4375 1.53125 -2.296875 2.09375 -2.171875 C 2.59375 -2.046875 2.921875 -1.9375 3.078125 -1.875 C 3.359375 -1.734375 3.515625 -1.53125 3.515625 -1.265625 C 3.515625 -0.75 3.09375 -0.484375 2.265625 -0.484375 C 1.875 -0.484375 1.578125 -0.546875 1.375 -0.6875 C 1.171875 -0.859375 1.046875 -1.109375 0.984375 -1.484375 L 0.28125 -1.484375 C 0.390625 -0.40625 1.0625 0.125 2.28125 0.125 C 3.59375 0.125 4.25 -0.359375 4.25 -1.3125 C 4.25 -1.71875 4.0625 -2.03125 3.703125 -2.265625 C 3.453125 -2.421875 3.0625 -2.5625 2.5 -2.703125 C 2.03125 -2.828125 1.703125 -2.921875 1.53125 -3 C 1.265625 -3.140625 1.140625 -3.3125 1.140625 -3.515625 C 1.140625 -3.71875 1.234375 -3.875 1.4375 -4 C 1.625 -4.125 1.875 -4.171875 2.203125 -4.171875 C 2.5625 -4.171875 2.859375 -4.109375 3.046875 -3.953125 C 3.234375 -3.828125 3.34375 -3.625 3.40625 -3.328125 L 4.109375 -3.328125 C 3.984375 -4.296875 3.359375 -4.78125 2.203125 -4.78125 Z M 2.203125 -4.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-5\" overflow=\"visible\">\n",
       "<path d=\"M 0.453125 -4.640625 L 0.453125 -3.96875 L 4.15625 -2.328125 L 0.453125 -0.6875 L 0.453125 -0.015625 L 5 -2.046875 L 5 -2.609375 Z M 0.453125 -4.640625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<rect height=\"87\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"148\" x=\"0\" y=\"0\"/>\n",
       "<rect height=\"87\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"148\" x=\"0\" y=\"0\"/>\n",
       "<path d=\"M 129.910156 254.699219 L 150.464844 254.699219 L 150.464844 270.800781 L 129.910156 270.800781 Z M 129.910156 254.699219 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.932465\" xlink:href=\"#glyph0-1\" y=\"80.602788\"/>\n",
       "  <use x=\"55.938265\" xlink:href=\"#glyph0-2\" y=\"80.602788\"/>\n",
       "  <use x=\"58.442065\" xlink:href=\"#glyph0-3\" y=\"80.602788\"/>\n",
       "</g>\n",
       "<path d=\"M 129.910156 238.601562 L 150.464844 238.601562 L 150.464844 254.699219 L 129.910156 254.699219 Z M 129.910156 238.601562 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.932465\" xlink:href=\"#glyph0-1\" y=\"64.502783\"/>\n",
       "  <use x=\"55.938265\" xlink:href=\"#glyph0-2\" y=\"64.502783\"/>\n",
       "  <use x=\"58.442065\" xlink:href=\"#glyph0-4\" y=\"64.502783\"/>\n",
       "</g>\n",
       "<path d=\"M 129.910156 222.5 L 150.464844 222.5 L 150.464844 238.601562 L 129.910156 238.601562 Z M 129.910156 222.5 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.932465\" xlink:href=\"#glyph0-1\" y=\"48.402783\"/>\n",
       "  <use x=\"55.938265\" xlink:href=\"#glyph0-2\" y=\"48.402783\"/>\n",
       "  <use x=\"58.442065\" xlink:href=\"#glyph0-4\" y=\"48.402783\"/>\n",
       "</g>\n",
       "<path d=\"M 129.910156 206.398438 L 150.464844 206.398438 L 150.464844 222.5 L 129.910156 222.5 Z M 129.910156 206.398438 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"50.932465\" xlink:href=\"#glyph0-1\" y=\"32.302783\"/>\n",
       "  <use x=\"55.938265\" xlink:href=\"#glyph0-2\" y=\"32.302783\"/>\n",
       "  <use x=\"58.442065\" xlink:href=\"#glyph0-5\" y=\"32.302783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.91138\" xlink:href=\"#glyph0-6\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"11.07468\" xlink:href=\"#glyph0-7\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"13.07448\" xlink:href=\"#glyph0-8\" y=\"13.752783\"/>\n",
       "  <use x=\"20.57148\" xlink:href=\"#glyph0-9\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"25.57728\" xlink:href=\"#glyph0-10\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"28.07748\" xlink:href=\"#glyph0-11\" y=\"13.752783\"/>\n",
       "  <use x=\"32.57748\" xlink:href=\"#glyph0-12\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"35.07768\" xlink:href=\"#glyph0-9\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"40.08348\" xlink:href=\"#glyph0-13\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"54.685394\" xlink:href=\"#glyph0-3\" y=\"13.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 156.507812 254.699219 L 177.0625 254.699219 L 177.0625 270.800781 L 156.507812 270.800781 Z M 156.507812 254.699219 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.530835\" xlink:href=\"#glyph0-1\" y=\"80.602788\"/>\n",
       "  <use x=\"82.536635\" xlink:href=\"#glyph0-2\" y=\"80.602788\"/>\n",
       "  <use x=\"85.040435\" xlink:href=\"#glyph0-4\" y=\"80.602788\"/>\n",
       "</g>\n",
       "<path d=\"M 156.507812 206.398438 L 177.0625 206.398438 L 177.0625 222.5 L 156.507812 222.5 Z M 156.507812 206.398438 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.530835\" xlink:href=\"#glyph0-1\" y=\"32.302783\"/>\n",
       "  <use x=\"82.536635\" xlink:href=\"#glyph0-2\" y=\"32.302783\"/>\n",
       "  <use x=\"85.040435\" xlink:href=\"#glyph0-3\" y=\"32.302783\"/>\n",
       "</g>\n",
       "<path d=\"M 156.507812 222.5 L 177.0625 222.5 L 177.0625 238.601562 L 156.507812 238.601562 Z M 156.507812 222.5 \" style=\"fill-rule:nonzero;fill:rgb(99.998474%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.530835\" xlink:href=\"#glyph0-1\" y=\"48.402783\"/>\n",
       "  <use x=\"82.536635\" xlink:href=\"#glyph0-2\" y=\"48.402783\"/>\n",
       "  <use x=\"85.040435\" xlink:href=\"#glyph0-14\" y=\"48.402783\"/>\n",
       "</g>\n",
       "<path d=\"M 156.507812 238.601562 L 177.0625 238.601562 L 177.0625 254.699219 L 156.507812 254.699219 Z M 156.507812 238.601562 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"77.530835\" xlink:href=\"#glyph0-1\" y=\"64.502783\"/>\n",
       "  <use x=\"82.536635\" xlink:href=\"#glyph0-2\" y=\"64.502783\"/>\n",
       "  <use x=\"85.040435\" xlink:href=\"#glyph0-15\" y=\"64.502783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"81.283764\" xlink:href=\"#glyph0-4\" y=\"13.752783\"/>\n",
       "</g>\n",
       "<path d=\"M 183.199219 254.789062 L 203.753906 254.789062 L 203.753906 270.890625 L 183.199219 270.890625 Z M 183.199219 254.789062 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"104.220085\" xlink:href=\"#glyph0-1\" y=\"80.693697\"/>\n",
       "  <use x=\"109.225885\" xlink:href=\"#glyph0-2\" y=\"80.693697\"/>\n",
       "  <use x=\"111.729685\" xlink:href=\"#glyph0-3\" y=\"80.693697\"/>\n",
       "</g>\n",
       "<path d=\"M 183.199219 238.691406 L 203.753906 238.691406 L 203.753906 254.789062 L 183.199219 254.789062 Z M 183.199219 238.691406 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"104.220085\" xlink:href=\"#glyph0-1\" y=\"64.593693\"/>\n",
       "  <use x=\"109.225885\" xlink:href=\"#glyph0-2\" y=\"64.593693\"/>\n",
       "  <use x=\"111.729685\" xlink:href=\"#glyph0-4\" y=\"64.593693\"/>\n",
       "</g>\n",
       "<path d=\"M 183.199219 222.589844 L 203.753906 222.589844 L 203.753906 238.691406 L 183.199219 238.691406 Z M 183.199219 222.589844 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"104.220085\" xlink:href=\"#glyph0-1\" y=\"48.493693\"/>\n",
       "  <use x=\"109.225885\" xlink:href=\"#glyph0-2\" y=\"48.493693\"/>\n",
       "  <use x=\"111.729685\" xlink:href=\"#glyph0-16\" y=\"48.493693\"/>\n",
       "</g>\n",
       "<path d=\"M 183.199219 206.492188 L 203.753906 206.492188 L 203.753906 222.589844 L 183.199219 222.589844 Z M 183.199219 206.492188 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"104.220085\" xlink:href=\"#glyph0-1\" y=\"32.393693\"/>\n",
       "  <use x=\"109.225885\" xlink:href=\"#glyph0-2\" y=\"32.393693\"/>\n",
       "  <use x=\"111.729685\" xlink:href=\"#glyph0-3\" y=\"32.393693\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"107.882114\" xlink:href=\"#glyph0-15\" y=\"13.843693\"/>\n",
       "</g>\n",
       "<path d=\"M 209.886719 254.882812 L 230.441406 254.882812 L 230.441406 270.980469 L 209.886719 270.980469 Z M 209.886719 254.882812 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.909385\" xlink:href=\"#glyph0-1\" y=\"80.784606\"/>\n",
       "  <use x=\"135.915185\" xlink:href=\"#glyph0-2\" y=\"80.784606\"/>\n",
       "  <use x=\"138.418985\" xlink:href=\"#glyph0-16\" y=\"80.784606\"/>\n",
       "</g>\n",
       "<path d=\"M 209.886719 238.78125 L 230.441406 238.78125 L 230.441406 254.882812 L 209.886719 254.882812 Z M 209.886719 238.78125 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.909385\" xlink:href=\"#glyph0-1\" y=\"64.684603\"/>\n",
       "  <use x=\"135.915185\" xlink:href=\"#glyph0-2\" y=\"64.684603\"/>\n",
       "  <use x=\"138.418985\" xlink:href=\"#glyph0-3\" y=\"64.684603\"/>\n",
       "</g>\n",
       "<path d=\"M 209.886719 222.683594 L 230.441406 222.683594 L 230.441406 238.78125 L 209.886719 238.78125 Z M 209.886719 222.683594 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.909385\" xlink:href=\"#glyph0-1\" y=\"48.584603\"/>\n",
       "  <use x=\"135.915185\" xlink:href=\"#glyph0-2\" y=\"48.584603\"/>\n",
       "  <use x=\"138.418985\" xlink:href=\"#glyph0-4\" y=\"48.584603\"/>\n",
       "</g>\n",
       "<path d=\"M 209.886719 206.582031 L 230.441406 206.582031 L 230.441406 222.683594 L 209.886719 222.683594 Z M 209.886719 206.582031 \" style=\"fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-83,-185)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"130.909385\" xlink:href=\"#glyph0-1\" y=\"32.484603\"/>\n",
       "  <use x=\"135.915185\" xlink:href=\"#glyph0-2\" y=\"32.484603\"/>\n",
       "  <use x=\"138.418985\" xlink:href=\"#glyph0-3\" y=\"32.484603\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"134.662314\" xlink:href=\"#glyph0-14\" y=\"13.934603\"/>\n",
       "</g>\n",
       "<path d=\"M 9.464844 69.699219 L 41.535156 69.699219 L 41.535156 85.800781 L 9.464844 85.800781 Z M 9.464844 69.699219 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"12.648\" xlink:href=\"#glyph1-1\" y=\"81.250005\"/>\n",
       "  <use x=\"18.093\" xlink:href=\"#glyph1-2\" y=\"81.250005\"/>\n",
       "  <use x=\"23.088\" xlink:href=\"#glyph1-3\" y=\"81.250005\"/>\n",
       "  <use x=\"28.362\" xlink:href=\"#glyph1-4\" y=\"81.250005\"/>\n",
       "  <use x=\"32.907\" xlink:href=\"#glyph1-5\" y=\"81.250005\"/>\n",
       "</g>\n",
       "<path d=\"M 9.464844 53.601562 L 41.535156 53.601562 L 41.535156 69.699219 L 9.464844 69.699219 Z M 9.464844 53.601562 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"22.250244\" xlink:href=\"#glyph0-17\" y=\"64.502783\"/>\n",
       "</g>\n",
       "<path d=\"M 9.464844 37.5 L 41.535156 37.5 L 41.535156 53.601562 L 9.464844 53.601562 Z M 9.464844 37.5 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"22.498535\" xlink:href=\"#glyph0-18\" y=\"48.402783\"/>\n",
       "</g>\n",
       "<path d=\"M 9.464844 21.398438 L 41.535156 21.398438 L 41.535156 37.5 L 9.464844 37.5 Z M 9.464844 21.398438 \" style=\" stroke:none;fill-rule:nonzero;fill:rgb(100%,100%,100%);fill-opacity:1;\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"22.498535\" xlink:href=\"#glyph0-19\" y=\"32.302783\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SVG('../img/s2s_prob2.svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Fig.10.15.2: The four numbers under each time step represent the conditional probabilities of generating \"A\", \"B\", \"C\", and \"&lt;eos&gt;\" at that time step.  At time step 2, the word \"C\", which has the second highest conditional probability, is selected. \n",
    "\n",
    "## Exhaustive Search\n",
    "\n",
    "If the goal is to obtain the optimal sequence, we may consider using exhaustive search: an exhaustive examination of all possible output sequences, which outputs the sequence with the highest conditional probability.\n",
    "\n",
    "Although we can use an exhaustive search to obtain the optimal sequence, its computational overhead $\\mathcal{O}(\\left|\\mathcal{Y}\\right|^{T'})$ is likely to be excessively high. For example, when $|\\mathcal{Y}|=10000$ and $T'=10$, we will need to evaluate $10000^{10} = 10^{40}$ sequences. This is next to impossible to complete. The computational overhead of greedy search is $\\mathcal{O}(\\left|\\mathcal{Y}\\right|T')$, which is usually significantly less than the computational overhead of an exhaustive search. For example, when $|\\mathcal{Y}|=10000$ and $T'=10$, we only need to evaluate $10000\\times10=1\\times10^5$ sequences.\n",
    "\n",
    "\n",
    "## Beam Search\n",
    "\n",
    "Beam search is an improved algorithm based on greedy search. It has a hyper-parameter named beam size. We set it to $k$. At time step 1, we select $k$ words with the highest conditional probability to be the first words of the $k$ candidate output sequences. For each subsequent time step, we are going to select the $k$ output sequences with the highest conditional probability from the total of $k\\left|\\mathcal{Y}\\right|$ possible output sequences based on the $k$ candidate output sequences from the previous time step. These will be the candidate output sequence for that time step. Finally, we will filter out the sequences containing the special symbol \"&lt;eos&gt;\" from the candidate output sequences of each time step and discard all the subsequences after it to obtain a set of final candidate output sequences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg height=\"234pt\" version=\"1.1\" viewBox=\"0 0 364 234\" width=\"364pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<defs>\n",
       "<g>\n",
       "<symbol id=\"glyph0-0\" overflow=\"visible\">\n",
       "<path d=\"M 1.125 0 L 1.125 -5.625 L 5.625 -5.625 L 5.625 0 Z M 1.265625 -0.140625 L 5.484375 -0.140625 L 5.484375 -5.484375 L 1.265625 -5.484375 Z M 1.265625 -0.140625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-1\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 0 L 2.328125 -5.6875 L 0.21875 -5.6875 L 0.21875 -6.4375 L 5.3125 -6.4375 L 5.3125 -5.6875 L 3.1875 -5.6875 L 3.1875 0 Z M 2.328125 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-2\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 -5.53125 L 0.59375 -6.4375 L 1.390625 -6.4375 L 1.390625 -5.53125 Z M 0.59375 0 L 0.59375 -4.671875 L 1.390625 -4.671875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-3\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -4.671875 L 1.296875 -4.671875 L 1.296875 -4.015625 C 1.441406 -4.242188 1.632812 -4.425781 1.875 -4.5625 C 2.125 -4.695312 2.40625 -4.765625 2.71875 -4.765625 C 3.0625 -4.765625 3.34375 -4.691406 3.5625 -4.546875 C 3.78125 -4.410156 3.9375 -4.210938 4.03125 -3.953125 C 4.40625 -4.492188 4.882812 -4.765625 5.46875 -4.765625 C 5.9375 -4.765625 6.296875 -4.632812 6.546875 -4.375 C 6.796875 -4.125 6.921875 -3.734375 6.921875 -3.203125 L 6.921875 0 L 6.125 0 L 6.125 -2.9375 C 6.125 -3.257812 6.097656 -3.488281 6.046875 -3.625 C 6.003906 -3.757812 5.914062 -3.867188 5.78125 -3.953125 C 5.644531 -4.046875 5.484375 -4.09375 5.296875 -4.09375 C 4.972656 -4.09375 4.703125 -3.984375 4.484375 -3.765625 C 4.265625 -3.546875 4.15625 -3.195312 4.15625 -2.71875 L 4.15625 0 L 3.359375 0 L 3.359375 -3.03125 C 3.359375 -3.382812 3.296875 -3.648438 3.171875 -3.828125 C 3.046875 -4.003906 2.835938 -4.09375 2.546875 -4.09375 C 2.316406 -4.09375 2.109375 -4.03125 1.921875 -3.90625 C 1.734375 -3.789062 1.597656 -3.617188 1.515625 -3.390625 C 1.429688 -3.171875 1.390625 -2.847656 1.390625 -2.421875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-4\" overflow=\"visible\">\n",
       "<path d=\"M 3.78125 -1.5 L 4.609375 -1.40625 C 4.472656 -0.925781 4.226562 -0.550781 3.875 -0.28125 C 3.53125 -0.0195312 3.085938 0.109375 2.546875 0.109375 C 1.867188 0.109375 1.328125 -0.0976562 0.921875 -0.515625 C 0.523438 -0.941406 0.328125 -1.535156 0.328125 -2.296875 C 0.328125 -3.078125 0.53125 -3.679688 0.9375 -4.109375 C 1.34375 -4.546875 1.867188 -4.765625 2.515625 -4.765625 C 3.140625 -4.765625 3.644531 -4.550781 4.03125 -4.125 C 4.425781 -3.707031 4.625 -3.113281 4.625 -2.34375 C 4.625 -2.289062 4.625 -2.21875 4.625 -2.125 L 1.140625 -2.125 C 1.171875 -1.613281 1.316406 -1.222656 1.578125 -0.953125 C 1.835938 -0.679688 2.164062 -0.546875 2.5625 -0.546875 C 2.851562 -0.546875 3.097656 -0.617188 3.296875 -0.765625 C 3.503906 -0.921875 3.664062 -1.164062 3.78125 -1.5 Z M 1.1875 -2.78125 L 3.796875 -2.78125 C 3.765625 -3.175781 3.664062 -3.472656 3.5 -3.671875 C 3.25 -3.972656 2.921875 -4.125 2.515625 -4.125 C 2.148438 -4.125 1.84375 -4 1.59375 -3.75 C 1.351562 -3.507812 1.21875 -3.1875 1.1875 -2.78125 Z M 1.1875 -2.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-5\" overflow=\"visible\">\n",
       "<path d=\"\" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-6\" overflow=\"visible\">\n",
       "<path d=\"M 0.28125 -1.390625 L 1.0625 -1.515625 C 1.101562 -1.203125 1.222656 -0.960938 1.421875 -0.796875 C 1.628906 -0.628906 1.910156 -0.546875 2.265625 -0.546875 C 2.628906 -0.546875 2.898438 -0.617188 3.078125 -0.765625 C 3.253906 -0.910156 3.34375 -1.082031 3.34375 -1.28125 C 3.34375 -1.46875 3.265625 -1.609375 3.109375 -1.703125 C 2.992188 -1.773438 2.722656 -1.867188 2.296875 -1.984375 C 1.710938 -2.128906 1.304688 -2.253906 1.078125 -2.359375 C 0.859375 -2.460938 0.691406 -2.609375 0.578125 -2.796875 C 0.460938 -2.984375 0.40625 -3.191406 0.40625 -3.421875 C 0.40625 -3.628906 0.453125 -3.820312 0.546875 -4 C 0.640625 -4.175781 0.769531 -4.328125 0.9375 -4.453125 C 1.0625 -4.535156 1.226562 -4.609375 1.4375 -4.671875 C 1.65625 -4.734375 1.882812 -4.765625 2.125 -4.765625 C 2.488281 -4.765625 2.804688 -4.710938 3.078125 -4.609375 C 3.359375 -4.503906 3.566406 -4.363281 3.703125 -4.1875 C 3.835938 -4.007812 3.929688 -3.769531 3.984375 -3.46875 L 3.203125 -3.359375 C 3.171875 -3.597656 3.066406 -3.785156 2.890625 -3.921875 C 2.722656 -4.054688 2.488281 -4.125 2.1875 -4.125 C 1.820312 -4.125 1.5625 -4.0625 1.40625 -3.9375 C 1.25 -3.820312 1.171875 -3.679688 1.171875 -3.515625 C 1.171875 -3.410156 1.203125 -3.320312 1.265625 -3.25 C 1.328125 -3.15625 1.429688 -3.082031 1.578125 -3.03125 C 1.648438 -3 1.878906 -2.929688 2.265625 -2.828125 C 2.828125 -2.679688 3.21875 -2.5625 3.4375 -2.46875 C 3.664062 -2.375 3.84375 -2.234375 3.96875 -2.046875 C 4.09375 -1.867188 4.15625 -1.644531 4.15625 -1.375 C 4.15625 -1.101562 4.078125 -0.851562 3.921875 -0.625 C 3.765625 -0.394531 3.539062 -0.210938 3.25 -0.078125 C 2.96875 0.046875 2.640625 0.109375 2.265625 0.109375 C 1.660156 0.109375 1.195312 -0.015625 0.875 -0.265625 C 0.5625 -0.523438 0.363281 -0.898438 0.28125 -1.390625 Z M 0.28125 -1.390625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-7\" overflow=\"visible\">\n",
       "<path d=\"M 2.328125 -0.703125 L 2.4375 -0.015625 C 2.207031 0.0351562 2.007812 0.0625 1.84375 0.0625 C 1.550781 0.0625 1.328125 0.015625 1.171875 -0.078125 C 1.015625 -0.171875 0.898438 -0.289062 0.828125 -0.4375 C 0.765625 -0.582031 0.734375 -0.890625 0.734375 -1.359375 L 0.734375 -4.046875 L 0.15625 -4.046875 L 0.15625 -4.671875 L 0.734375 -4.671875 L 0.734375 -5.828125 L 1.53125 -6.296875 L 1.53125 -4.671875 L 2.328125 -4.671875 L 2.328125 -4.046875 L 1.53125 -4.046875 L 1.53125 -1.328125 C 1.53125 -1.097656 1.539062 -0.953125 1.5625 -0.890625 C 1.59375 -0.828125 1.640625 -0.773438 1.703125 -0.734375 C 1.765625 -0.691406 1.851562 -0.671875 1.96875 -0.671875 C 2.0625 -0.671875 2.179688 -0.679688 2.328125 -0.703125 Z M 2.328125 -0.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-8\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 1.78125 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4.0625 C 1.476562 -4.300781 1.664062 -4.476562 1.875 -4.59375 C 2.09375 -4.707031 2.359375 -4.765625 2.671875 -4.765625 C 3.066406 -4.765625 3.414062 -4.660156 3.71875 -4.453125 C 4.019531 -4.253906 4.25 -3.96875 4.40625 -3.59375 C 4.5625 -3.21875 4.640625 -2.8125 4.640625 -2.375 C 4.640625 -1.894531 4.550781 -1.460938 4.375 -1.078125 C 4.207031 -0.691406 3.960938 -0.394531 3.640625 -0.1875 C 3.316406 0.0078125 2.972656 0.109375 2.609375 0.109375 C 2.347656 0.109375 2.113281 0.0507812 1.90625 -0.0625 C 1.695312 -0.175781 1.523438 -0.316406 1.390625 -0.484375 L 1.390625 1.78125 Z M 1.3125 -2.3125 C 1.3125 -1.707031 1.429688 -1.257812 1.671875 -0.96875 C 1.921875 -0.6875 2.21875 -0.546875 2.5625 -0.546875 C 2.90625 -0.546875 3.203125 -0.691406 3.453125 -0.984375 C 3.710938 -1.285156 3.84375 -1.75 3.84375 -2.375 C 3.84375 -2.96875 3.71875 -3.410156 3.46875 -3.703125 C 3.226562 -4.003906 2.9375 -4.15625 2.59375 -4.15625 C 2.257812 -4.15625 1.960938 -3.992188 1.703125 -3.671875 C 1.441406 -3.359375 1.3125 -2.90625 1.3125 -2.3125 Z M 1.3125 -2.3125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-9\" overflow=\"visible\">\n",
       "<path d=\"M 3.359375 0 L 2.5625 0 L 2.5625 -5.046875 C 2.375 -4.859375 2.125 -4.671875 1.8125 -4.484375 C 1.5 -4.304688 1.222656 -4.175781 0.984375 -4.09375 L 0.984375 -4.859375 C 1.421875 -5.054688 1.804688 -5.300781 2.140625 -5.59375 C 2.472656 -5.894531 2.707031 -6.1875 2.84375 -6.46875 L 3.359375 -6.46875 Z M 3.359375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-10\" overflow=\"visible\">\n",
       "<path d=\"M 5.296875 -2.265625 L 6.140625 -2.046875 C 5.960938 -1.347656 5.640625 -0.8125 5.171875 -0.4375 C 4.710938 -0.0703125 4.144531 0.109375 3.46875 0.109375 C 2.78125 0.109375 2.21875 -0.03125 1.78125 -0.3125 C 1.34375 -0.59375 1.007812 -1 0.78125 -1.53125 C 0.5625 -2.070312 0.453125 -2.648438 0.453125 -3.265625 C 0.453125 -3.941406 0.578125 -4.53125 0.828125 -5.03125 C 1.085938 -5.53125 1.453125 -5.90625 1.921875 -6.15625 C 2.398438 -6.414062 2.921875 -6.546875 3.484375 -6.546875 C 4.128906 -6.546875 4.671875 -6.378906 5.109375 -6.046875 C 5.554688 -5.722656 5.863281 -5.265625 6.03125 -4.671875 L 5.1875 -4.484375 C 5.039062 -4.953125 4.828125 -5.289062 4.546875 -5.5 C 4.265625 -5.71875 3.90625 -5.828125 3.46875 -5.828125 C 2.976562 -5.828125 2.566406 -5.707031 2.234375 -5.46875 C 1.898438 -5.226562 1.664062 -4.90625 1.53125 -4.5 C 1.394531 -4.101562 1.328125 -3.695312 1.328125 -3.28125 C 1.328125 -2.726562 1.40625 -2.242188 1.5625 -1.828125 C 1.726562 -1.421875 1.976562 -1.117188 2.3125 -0.921875 C 2.644531 -0.722656 3.007812 -0.625 3.40625 -0.625 C 3.882812 -0.625 4.285156 -0.757812 4.609375 -1.03125 C 4.941406 -1.3125 5.171875 -1.722656 5.296875 -2.265625 Z M 5.296875 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-11\" overflow=\"visible\">\n",
       "<path d=\"M 3.640625 -0.578125 C 3.347656 -0.328125 3.066406 -0.148438 2.796875 -0.046875 C 2.523438 0.0546875 2.234375 0.109375 1.921875 0.109375 C 1.410156 0.109375 1.015625 -0.015625 0.734375 -0.265625 C 0.460938 -0.515625 0.328125 -0.835938 0.328125 -1.234375 C 0.328125 -1.460938 0.378906 -1.671875 0.484375 -1.859375 C 0.585938 -2.046875 0.722656 -2.195312 0.890625 -2.3125 C 1.054688 -2.425781 1.242188 -2.515625 1.453125 -2.578125 C 1.609375 -2.609375 1.84375 -2.644531 2.15625 -2.6875 C 2.800781 -2.757812 3.273438 -2.851562 3.578125 -2.96875 C 3.578125 -3.070312 3.578125 -3.140625 3.578125 -3.171875 C 3.578125 -3.492188 3.503906 -3.71875 3.359375 -3.84375 C 3.148438 -4.03125 2.847656 -4.125 2.453125 -4.125 C 2.078125 -4.125 1.800781 -4.054688 1.625 -3.921875 C 1.445312 -3.796875 1.316406 -3.566406 1.234375 -3.234375 L 0.46875 -3.328125 C 0.53125 -3.660156 0.640625 -3.925781 0.796875 -4.125 C 0.960938 -4.332031 1.195312 -4.488281 1.5 -4.59375 C 1.8125 -4.707031 2.164062 -4.765625 2.5625 -4.765625 C 2.96875 -4.765625 3.289062 -4.71875 3.53125 -4.625 C 3.78125 -4.53125 3.960938 -4.410156 4.078125 -4.265625 C 4.203125 -4.128906 4.285156 -3.953125 4.328125 -3.734375 C 4.359375 -3.597656 4.375 -3.359375 4.375 -3.015625 L 4.375 -1.953125 C 4.375 -1.222656 4.390625 -0.757812 4.421875 -0.5625 C 4.453125 -0.363281 4.519531 -0.175781 4.625 0 L 3.796875 0 C 3.710938 -0.164062 3.660156 -0.359375 3.640625 -0.578125 Z M 3.578125 -2.34375 C 3.285156 -2.226562 2.851562 -2.128906 2.28125 -2.046875 C 1.957031 -1.992188 1.726562 -1.9375 1.59375 -1.875 C 1.457031 -1.820312 1.351562 -1.738281 1.28125 -1.625 C 1.207031 -1.507812 1.171875 -1.382812 1.171875 -1.25 C 1.171875 -1.039062 1.25 -0.863281 1.40625 -0.71875 C 1.5625 -0.582031 1.796875 -0.515625 2.109375 -0.515625 C 2.410156 -0.515625 2.679688 -0.582031 2.921875 -0.71875 C 3.160156 -0.851562 3.335938 -1.035156 3.453125 -1.265625 C 3.535156 -1.441406 3.578125 -1.703125 3.578125 -2.046875 Z M 3.578125 -2.34375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-12\" overflow=\"visible\">\n",
       "<path d=\"M 0.59375 0 L 0.59375 -4.671875 L 1.3125 -4.671875 L 1.3125 -4 C 1.644531 -4.507812 2.140625 -4.765625 2.796875 -4.765625 C 3.078125 -4.765625 3.332031 -4.710938 3.5625 -4.609375 C 3.800781 -4.515625 3.976562 -4.382812 4.09375 -4.21875 C 4.207031 -4.0625 4.289062 -3.867188 4.34375 -3.640625 C 4.375 -3.492188 4.390625 -3.238281 4.390625 -2.875 L 4.390625 0 L 3.59375 0 L 3.59375 -2.84375 C 3.59375 -3.164062 3.5625 -3.40625 3.5 -3.5625 C 3.4375 -3.71875 3.328125 -3.84375 3.171875 -3.9375 C 3.015625 -4.039062 2.832031 -4.09375 2.625 -4.09375 C 2.289062 -4.09375 2 -3.984375 1.75 -3.765625 C 1.507812 -3.554688 1.390625 -3.148438 1.390625 -2.546875 L 1.390625 0 Z M 0.59375 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-13\" overflow=\"visible\">\n",
       "<path d=\"M 3.625 0 L 3.625 -0.59375 C 3.320312 -0.125 2.882812 0.109375 2.3125 0.109375 C 1.945312 0.109375 1.609375 0.00390625 1.296875 -0.203125 C 0.984375 -0.410156 0.738281 -0.695312 0.5625 -1.0625 C 0.394531 -1.425781 0.3125 -1.847656 0.3125 -2.328125 C 0.3125 -2.796875 0.390625 -3.21875 0.546875 -3.59375 C 0.703125 -3.976562 0.929688 -4.269531 1.234375 -4.46875 C 1.546875 -4.664062 1.894531 -4.765625 2.28125 -4.765625 C 2.5625 -4.765625 2.8125 -4.707031 3.03125 -4.59375 C 3.25 -4.476562 3.425781 -4.320312 3.5625 -4.125 L 3.5625 -6.4375 L 4.359375 -6.4375 L 4.359375 0 Z M 1.125 -2.328125 C 1.125 -1.734375 1.25 -1.285156 1.5 -0.984375 C 1.75 -0.691406 2.046875 -0.546875 2.390625 -0.546875 C 2.734375 -0.546875 3.023438 -0.6875 3.265625 -0.96875 C 3.515625 -1.25 3.640625 -1.679688 3.640625 -2.265625 C 3.640625 -2.898438 3.515625 -3.367188 3.265625 -3.671875 C 3.015625 -3.972656 2.710938 -4.125 2.359375 -4.125 C 2.003906 -4.125 1.707031 -3.976562 1.46875 -3.6875 C 1.238281 -3.394531 1.125 -2.941406 1.125 -2.328125 Z M 1.125 -2.328125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-14\" overflow=\"visible\">\n",
       "<path d=\"M 0.609375 1.78125 L 0.609375 -6.4375 L 2.359375 -6.4375 L 2.359375 -5.78125 L 1.40625 -5.78125 L 1.40625 1.140625 L 2.359375 1.140625 L 2.359375 1.78125 Z M 0.609375 1.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-15\" overflow=\"visible\">\n",
       "<path d=\"M 1.921875 1.78125 L 0.171875 1.78125 L 0.171875 1.140625 L 1.125 1.140625 L 1.125 -5.78125 L 0.171875 -5.78125 L 0.171875 -6.4375 L 1.921875 -6.4375 Z M 1.921875 1.78125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-16\" overflow=\"visible\">\n",
       "<path d=\"M 4.53125 -0.765625 L 4.53125 0 L 0.265625 0 C 0.265625 -0.1875 0.296875 -0.367188 0.359375 -0.546875 C 0.472656 -0.835938 0.648438 -1.125 0.890625 -1.40625 C 1.128906 -1.6875 1.472656 -2.007812 1.921875 -2.375 C 2.617188 -2.957031 3.085938 -3.414062 3.328125 -3.75 C 3.578125 -4.082031 3.703125 -4.398438 3.703125 -4.703125 C 3.703125 -5.015625 3.585938 -5.273438 3.359375 -5.484375 C 3.140625 -5.703125 2.851562 -5.8125 2.5 -5.8125 C 2.113281 -5.8125 1.804688 -5.695312 1.578125 -5.46875 C 1.347656 -5.238281 1.234375 -4.921875 1.234375 -4.515625 L 0.421875 -4.609375 C 0.472656 -5.210938 0.679688 -5.671875 1.046875 -5.984375 C 1.410156 -6.304688 1.898438 -6.46875 2.515625 -6.46875 C 3.128906 -6.46875 3.613281 -6.296875 3.96875 -5.953125 C 4.332031 -5.609375 4.515625 -5.1875 4.515625 -4.6875 C 4.515625 -4.425781 4.460938 -4.171875 4.359375 -3.921875 C 4.253906 -3.671875 4.078125 -3.40625 3.828125 -3.125 C 3.585938 -2.851562 3.1875 -2.476562 2.625 -2 C 2.144531 -1.601562 1.835938 -1.332031 1.703125 -1.1875 C 1.566406 -1.039062 1.457031 -0.898438 1.375 -0.765625 Z M 4.53125 -0.765625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph0-17\" overflow=\"visible\">\n",
       "<path d=\"M 0.375 -1.703125 L 1.171875 -1.8125 C 1.265625 -1.363281 1.414062 -1.039062 1.625 -0.84375 C 1.84375 -0.644531 2.113281 -0.546875 2.4375 -0.546875 C 2.800781 -0.546875 3.109375 -0.671875 3.359375 -0.921875 C 3.617188 -1.179688 3.75 -1.503906 3.75 -1.890625 C 3.75 -2.253906 3.628906 -2.550781 3.390625 -2.78125 C 3.160156 -3.019531 2.863281 -3.140625 2.5 -3.140625 C 2.34375 -3.140625 2.15625 -3.109375 1.9375 -3.046875 L 2.03125 -3.75 C 2.082031 -3.738281 2.125 -3.734375 2.15625 -3.734375 C 2.488281 -3.734375 2.789062 -3.820312 3.0625 -4 C 3.332031 -4.175781 3.46875 -4.445312 3.46875 -4.8125 C 3.46875 -5.101562 3.367188 -5.34375 3.171875 -5.53125 C 2.972656 -5.71875 2.71875 -5.8125 2.40625 -5.8125 C 2.101562 -5.8125 1.847656 -5.710938 1.640625 -5.515625 C 1.441406 -5.328125 1.3125 -5.039062 1.25 -4.65625 L 0.453125 -4.796875 C 0.554688 -5.328125 0.773438 -5.738281 1.109375 -6.03125 C 1.453125 -6.320312 1.878906 -6.46875 2.390625 -6.46875 C 2.742188 -6.46875 3.066406 -6.390625 3.359375 -6.234375 C 3.660156 -6.085938 3.890625 -5.882812 4.046875 -5.625 C 4.203125 -5.363281 4.28125 -5.085938 4.28125 -4.796875 C 4.28125 -4.515625 4.203125 -4.257812 4.046875 -4.03125 C 3.898438 -3.800781 3.679688 -3.617188 3.390625 -3.484375 C 3.773438 -3.398438 4.070312 -3.21875 4.28125 -2.9375 C 4.488281 -2.664062 4.59375 -2.320312 4.59375 -1.90625 C 4.59375 -1.34375 4.382812 -0.863281 3.96875 -0.46875 C 3.5625 -0.0820312 3.046875 0.109375 2.421875 0.109375 C 1.859375 0.109375 1.390625 -0.0546875 1.015625 -0.390625 C 0.640625 -0.722656 0.425781 -1.160156 0.375 -1.703125 Z M 0.375 -1.703125 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-0\" overflow=\"visible\">\n",
       "<path d=\"M 1.125 0 L 1.125 -5.625 L 5.625 -5.625 L 5.625 0 Z M 1.265625 -0.140625 L 5.484375 -0.140625 L 5.484375 -5.484375 L 1.265625 -5.484375 Z M 1.265625 -0.140625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-1\" overflow=\"visible\">\n",
       "<path d=\"M -0.1875 0 L 3.46875 -6.4375 L 4.484375 -6.4375 L 5.546875 0 L 4.71875 0 L 4.40625 -1.859375 L 1.796875 -1.859375 L 0.765625 0 Z M 2.171875 -2.515625 L 4.296875 -2.515625 L 4.046875 -4.140625 C 3.953125 -4.773438 3.894531 -5.304688 3.875 -5.734375 C 3.71875 -5.367188 3.5 -4.929688 3.21875 -4.421875 Z M 2.171875 -2.515625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-2\" overflow=\"visible\">\n",
       "<path d=\"M 0.390625 0 L 1.734375 -6.4375 L 3.734375 -6.4375 C 4.097656 -6.4375 4.367188 -6.425781 4.546875 -6.40625 C 4.828125 -6.351562 5.066406 -6.265625 5.265625 -6.140625 C 5.460938 -6.015625 5.613281 -5.84375 5.71875 -5.625 C 5.832031 -5.40625 5.890625 -5.164062 5.890625 -4.90625 C 5.890625 -4.550781 5.789062 -4.238281 5.59375 -3.96875 C 5.40625 -3.707031 5.109375 -3.503906 4.703125 -3.359375 C 5.054688 -3.242188 5.316406 -3.070312 5.484375 -2.84375 C 5.660156 -2.613281 5.75 -2.347656 5.75 -2.046875 C 5.75 -1.679688 5.644531 -1.335938 5.4375 -1.015625 C 5.238281 -0.691406 4.96875 -0.441406 4.625 -0.265625 C 4.289062 -0.0859375 3.921875 0 3.515625 0 Z M 2.03125 -3.6875 L 3.328125 -3.6875 C 3.953125 -3.6875 4.398438 -3.785156 4.671875 -3.984375 C 4.953125 -4.191406 5.09375 -4.484375 5.09375 -4.859375 C 5.09375 -5.046875 5.046875 -5.203125 4.953125 -5.328125 C 4.867188 -5.460938 4.753906 -5.554688 4.609375 -5.609375 C 4.472656 -5.671875 4.207031 -5.703125 3.8125 -5.703125 L 2.453125 -5.703125 Z M 1.40625 -0.734375 L 2.875 -0.734375 C 3.257812 -0.734375 3.519531 -0.742188 3.65625 -0.765625 C 3.925781 -0.816406 4.144531 -0.894531 4.3125 -1 C 4.488281 -1.113281 4.625 -1.265625 4.71875 -1.453125 C 4.820312 -1.640625 4.875 -1.832031 4.875 -2.03125 C 4.875 -2.332031 4.773438 -2.5625 4.578125 -2.71875 C 4.390625 -2.875 4.03125 -2.953125 3.5 -2.953125 L 1.875 -2.953125 Z M 1.40625 -0.734375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-3\" overflow=\"visible\">\n",
       "<path d=\"M 0.40625 0 L 1.75 -6.4375 L 3.6875 -6.4375 C 4.15625 -6.4375 4.507812 -6.40625 4.75 -6.34375 C 5.101562 -6.25 5.40625 -6.082031 5.65625 -5.84375 C 5.90625 -5.613281 6.09375 -5.320312 6.21875 -4.96875 C 6.34375 -4.625 6.40625 -4.238281 6.40625 -3.8125 C 6.40625 -3.289062 6.328125 -2.816406 6.171875 -2.390625 C 6.015625 -1.960938 5.804688 -1.585938 5.546875 -1.265625 C 5.296875 -0.941406 5.03125 -0.6875 4.75 -0.5 C 4.476562 -0.320312 4.15625 -0.1875 3.78125 -0.09375 C 3.488281 -0.03125 3.132812 0 2.71875 0 Z M 1.421875 -0.734375 L 2.4375 -0.734375 C 2.894531 -0.734375 3.300781 -0.773438 3.65625 -0.859375 C 3.882812 -0.910156 4.078125 -0.988281 4.234375 -1.09375 C 4.441406 -1.21875 4.628906 -1.390625 4.796875 -1.609375 C 5.023438 -1.898438 5.203125 -2.226562 5.328125 -2.59375 C 5.460938 -2.957031 5.53125 -3.375 5.53125 -3.84375 C 5.53125 -4.363281 5.4375 -4.765625 5.25 -5.046875 C 5.070312 -5.328125 4.84375 -5.515625 4.5625 -5.609375 C 4.351562 -5.671875 4.03125 -5.703125 3.59375 -5.703125 L 2.453125 -5.703125 Z M 1.421875 -0.734375 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-4\" overflow=\"visible\">\n",
       "<path d=\"M 0.40625 0 L 1.75 -6.4375 L 6.40625 -6.4375 L 6.25 -5.703125 L 2.46875 -5.703125 L 2.046875 -3.703125 L 5.734375 -3.703125 L 5.578125 -2.96875 L 1.890625 -2.96875 L 1.421875 -0.734375 L 5.46875 -0.734375 L 5.328125 0 Z M 0.40625 0 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "<symbol id=\"glyph1-5\" overflow=\"visible\">\n",
       "<path d=\"M 5.421875 -2.265625 L 6.28125 -2.15625 C 6.007812 -1.40625 5.613281 -0.835938 5.09375 -0.453125 C 4.582031 -0.078125 4.003906 0.109375 3.359375 0.109375 C 2.578125 0.109375 1.957031 -0.128906 1.5 -0.609375 C 1.039062 -1.085938 0.8125 -1.769531 0.8125 -2.65625 C 0.8125 -3.820312 1.160156 -4.78125 1.859375 -5.53125 C 2.484375 -6.207031 3.253906 -6.546875 4.171875 -6.546875 C 4.859375 -6.546875 5.410156 -6.363281 5.828125 -6 C 6.253906 -5.632812 6.503906 -5.144531 6.578125 -4.53125 L 5.765625 -4.453125 C 5.679688 -4.921875 5.503906 -5.269531 5.234375 -5.5 C 4.972656 -5.726562 4.632812 -5.84375 4.21875 -5.84375 C 3.425781 -5.84375 2.789062 -5.492188 2.3125 -4.796875 C 1.882812 -4.191406 1.671875 -3.472656 1.671875 -2.640625 C 1.671875 -1.972656 1.832031 -1.46875 2.15625 -1.125 C 2.488281 -0.78125 2.914062 -0.609375 3.4375 -0.609375 C 3.875 -0.609375 4.273438 -0.75 4.640625 -1.03125 C 5.003906 -1.320312 5.265625 -1.734375 5.421875 -2.265625 Z M 5.421875 -2.265625 \" style=\"stroke:none;\"/>\n",
       "</symbol>\n",
       "</g>\n",
       "</defs>\n",
       "<g id=\"surface1\">\n",
       "<rect height=\"234\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"364\" x=\"0\" y=\"0\"/>\n",
       "<rect height=\"234\" style=\"fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;\" width=\"364\" x=\"0\" y=\"0\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"86.65845\" xlink:href=\"#glyph0-1\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"91.82175\" xlink:href=\"#glyph0-2\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"93.82155\" xlink:href=\"#glyph0-3\" y=\"13.205566\"/>\n",
       "  <use x=\"101.31855\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"106.32435\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"108.82455\" xlink:href=\"#glyph0-6\" y=\"13.205566\"/>\n",
       "  <use x=\"113.32455\" xlink:href=\"#glyph0-7\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"115.82475\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"120.83055\" xlink:href=\"#glyph0-8\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"125.83635\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"128.33655\" xlink:href=\"#glyph0-9\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"87.23413\" xlink:href=\"#glyph0-10\" y=\"23.5\"/>\n",
       "  <use x=\"93.73393\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"98.73973\" xlink:href=\"#glyph0-12\" y=\"23.5\"/>\n",
       "  <use x=\"103.74553\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"108.75133\" xlink:href=\"#glyph0-2\" y=\"23.5\"/>\n",
       "  <use x=\"110.75113\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"115.75693\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"120.76273\" xlink:href=\"#glyph0-7\" y=\"23.5\"/>\n",
       "  <use x=\"123.26653\" xlink:href=\"#glyph0-4\" y=\"23.5\"/>\n",
       "  <use x=\"128.27233\" xlink:href=\"#glyph0-6\" y=\"23.5\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"5.999512\" xlink:href=\"#glyph0-14\" y=\"140.352823\"/>\n",
       "  <use x=\"8.499712\" xlink:href=\"#glyph0-15\" y=\"140.352823\"/>\n",
       "</g>\n",
       "<path d=\"M 215.085938 237.550781 C 217.820312 240.285156 217.820312 244.714844 215.085938 247.449219 C 212.355469 250.183594 207.921875 250.183594 205.1875 247.449219 C 202.453125 244.714844 202.453125 240.285156 205.1875 237.550781 C 207.921875 234.816406 212.355469 234.816406 215.085938 237.550781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"63.136165\" xlink:href=\"#glyph1-1\" y=\"105.352783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"63.136165\" xlink:href=\"#glyph1-2\" y=\"122.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"62.887874\" xlink:href=\"#glyph1-3\" y=\"157.852863\"/>\n",
       "</g>\n",
       "<path d=\"M 160.5 272.640625 L 199.109375 249.195312 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 202.527344 247.121094 L 199.109375 249.195312 M 198.332031 247.914062 L 202.527344 247.121094 L 199.886719 250.480469 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 160.5 275.070312 L 195.992188 264.292969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 199.820312 263.132812 L 195.992188 264.292969 M 195.554688 262.859375 L 199.820312 263.132812 L 196.429688 265.730469 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 160.5 279.929688 L 195.992188 290.707031 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 199.820312 291.867188 L 195.992188 290.707031 M 196.429688 289.269531 L 199.820312 291.867188 L 195.554688 292.140625 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"63.136165\" xlink:href=\"#glyph1-4\" y=\"173.898933\"/>\n",
       "</g>\n",
       "<path d=\"M 215.085938 272.550781 C 217.820312 275.285156 217.820312 279.714844 215.085938 282.449219 C 212.355469 285.183594 207.921875 285.183594 205.1875 282.449219 C 202.453125 279.714844 202.453125 275.285156 205.1875 272.550781 C 207.921875 269.816406 212.355469 269.816406 215.085938 272.550781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"62.887874\" xlink:href=\"#glyph1-5\" y=\"140.352823\"/>\n",
       "</g>\n",
       "<path d=\"M 160.5 277.5 L 197.238281 277.5 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 201.238281 277.5 L 197.238281 277.5 M 197.238281 276 L 201.238281 277.5 L 197.238281 279 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 160.5 282.15625 L 196.539062 303.132812 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 199.996094 305.144531 L 196.539062 303.132812 M 197.292969 301.835938 L 199.996094 305.144531 L 195.785156 304.425781 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"106.998535\" xlink:href=\"#glyph1-1\" y=\"92.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 316.585938 206.550781 C 319.320312 209.285156 319.320312 213.714844 316.585938 216.449219 C 313.855469 219.183594 309.421875 219.183594 306.6875 216.449219 C 303.953125 213.714844 303.953125 209.285156 306.6875 206.550781 C 309.421875 203.816406 313.855469 203.816406 316.585938 206.550781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-2\" y=\"74.352783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-1\" y=\"56.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.387844\" xlink:href=\"#glyph1-3\" y=\"110.352883\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 227.273438 L 299.351562 215.441406 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 303.160156 214.21875 L 299.351562 215.441406 M 298.894531 214.015625 L 303.160156 214.21875 L 299.8125 216.871094 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 262.5 224.691406 L 298.132812 202.433594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.527344 200.316406 L 298.132812 202.433594 M 297.339844 201.164062 L 301.527344 200.316406 L 298.929688 203.707031 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 262.5 232.582031 L 297.492188 243.207031 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.320312 244.367188 L 297.492188 243.207031 M 297.929688 241.769531 L 301.320312 244.367188 L 297.054688 244.640625 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-4\" y=\"126.398883\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 234.945312 L 298.039062 255.632812 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.496094 257.644531 L 298.039062 255.632812 M 298.792969 254.335938 L 301.496094 257.644531 L 297.285156 256.925781 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"106.750244\" xlink:href=\"#glyph1-5\" y=\"189.352823\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-2\" y=\"171.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.387844\" xlink:href=\"#glyph1-3\" y=\"204.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 323.917969 L 297.492188 313.292969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.320312 312.132812 L 297.492188 313.292969 M 297.054688 311.859375 L 301.320312 312.132812 L 297.929688 314.730469 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 262.5 328.785156 L 297.441406 338.183594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.304688 339.222656 L 297.441406 338.183594 M 297.828125 336.734375 L 301.304688 339.222656 L 297.050781 339.628906 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.387844\" xlink:href=\"#glyph1-5\" y=\"188.352783\"/>\n",
       "</g>\n",
       "<path d=\"M 316.585938 354.550781 C 319.320312 357.285156 319.320312 361.714844 316.585938 364.449219 C 313.855469 367.183594 309.421875 367.183594 306.6875 364.449219 C 303.953125 361.714844 303.953125 357.285156 306.6875 354.550781 C 309.421875 351.816406 313.855469 351.816406 316.585938 354.550781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-4\" y=\"222.352783\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 331.367188 L 300.441406 353.089844 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 303.914062 355.078125 L 300.441406 353.089844 M 301.1875 351.789062 L 303.914062 355.078125 L 299.695312 354.390625 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 262.5 326.351562 L 297.238281 325.75 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.238281 325.679688 L 297.238281 325.75 M 297.210938 324.25 L 301.238281 325.679688 L 297.265625 327.25 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.387844\" xlink:href=\"#glyph1-5\" y=\"92.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 230 L 297.238281 230 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.238281 230 L 297.238281 230 M 297.238281 228.5 L 301.238281 230 L 297.238281 231.5 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"164.636135\" xlink:href=\"#glyph1-1\" y=\"157.375853\"/>\n",
       "</g>\n",
       "<path d=\"M 262.5 321.785156 L 297.976562 302.101562 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 301.476562 300.160156 L 297.976562 302.101562 M 297.25 300.789062 L 301.476562 300.160156 L 298.707031 303.414062 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"210.99707\" xlink:href=\"#glyph1-1\" y=\"92.852783\"/>\n",
       "  <use x=\"217.00007\" xlink:href=\"#glyph1-2\" y=\"92.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 423.585938 242.550781 C 426.320312 245.285156 426.320312 249.714844 423.585938 252.449219 C 420.855469 255.183594 416.421875 255.183594 413.6875 252.449219 C 410.953125 249.714844 410.953125 245.285156 413.6875 242.550781 C 416.421875 239.816406 420.855469 239.816406 423.585938 242.550781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.387844\" xlink:href=\"#glyph1-3\" y=\"110.352883\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-1\" y=\"56.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-2\" y=\"74.352883\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 233.492188 L 406.292969 243.75 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 410.121094 244.914062 L 406.292969 243.75 M 406.726562 242.316406 L 410.121094 244.914062 L 405.855469 245.1875 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 372.5 222.816406 L 405.132812 202.433594 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.527344 200.316406 L 405.132812 202.433594 M 404.339844 201.164062 L 408.527344 200.316406 L 405.929688 203.707031 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 372.5 226.308594 L 404.519531 216.03125 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.328125 214.808594 L 404.519531 216.03125 M 404.0625 214.601562 L 408.328125 214.808594 L 404.976562 217.460938 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-4\" y=\"127.852883\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 236.984375 L 405.09375 256.777344 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.511719 258.851562 L 405.09375 256.777344 M 405.875 255.492188 L 408.511719 258.851562 L 404.316406 258.058594 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"210.748779\" xlink:href=\"#glyph1-5\" y=\"189.352823\"/>\n",
       "  <use x=\"217.248579\" xlink:href=\"#glyph1-4\" y=\"189.352823\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-2\" y=\"171.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-4\" y=\"225.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 323.007812 L 404.492188 313.292969 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.320312 312.132812 L 404.492188 313.292969 M 404.054688 311.859375 L 408.320312 312.132812 L 404.929688 314.730469 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 372.5 333.78125 L 405.152344 354.460938 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.53125 356.601562 L 405.152344 354.460938 M 405.957031 353.195312 L 408.53125 356.601562 L 404.351562 355.726562 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.387844\" xlink:href=\"#glyph1-5\" y=\"188.352783\"/>\n",
       "</g>\n",
       "<path d=\"M 423.585938 337.050781 C 426.320312 339.785156 426.320312 344.214844 423.585938 346.949219 C 420.855469 349.683594 416.421875 349.683594 413.6875 346.949219 C 410.953125 344.214844 410.953125 339.785156 413.6875 337.050781 C 416.421875 334.316406 420.855469 334.316406 423.585938 337.050781 \" style=\"fill-rule:nonzero;fill:rgb(69.802856%,85.096741%,100%);fill-opacity:1;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.387844\" xlink:href=\"#glyph1-3\" y=\"204.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 329.59375 L 406.179688 338.648438 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 410.042969 339.6875 L 406.179688 338.648438 M 406.570312 337.199219 L 410.042969 339.6875 L 405.789062 340.097656 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 372.5 326.300781 L 404.238281 325.75 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.238281 325.679688 L 404.238281 325.75 M 404.210938 324.25 L 408.238281 325.679688 L 404.265625 327.25 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.387844\" xlink:href=\"#glyph1-5\" y=\"92.852783\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 230 L 404.238281 230 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.238281 230 L 404.238281 230 M 404.238281 228.5 L 408.238281 230 L 404.238281 231.5 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"271.636135\" xlink:href=\"#glyph1-1\" y=\"157.375853\"/>\n",
       "</g>\n",
       "<path d=\"M 372.5 320.121094 L 404.976562 302.101562 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 408.476562 300.160156 L 404.976562 302.101562 M 404.25 300.789062 L 408.476562 300.160156 L 405.707031 303.414062 \" style=\"fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"325.747314\" xlink:href=\"#glyph1-1\" y=\"92.852783\"/>\n",
       "  <use x=\"331.750314\" xlink:href=\"#glyph1-2\" y=\"92.852783\"/>\n",
       "  <use x=\"337.753314\" xlink:href=\"#glyph1-3\" y=\"92.852783\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"325.499023\" xlink:href=\"#glyph1-5\" y=\"189.352823\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"331.998823\" xlink:href=\"#glyph1-4\" y=\"189.352823\"/>\n",
       "  <use x=\"338.001823\" xlink:href=\"#glyph1-3\" y=\"189.352823\"/>\n",
       "</g>\n",
       "<path d=\"M 216.871094 240.582031 L 245.5 232.421875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 214.804688 282.714844 L 245.5 317.003906 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 318.195312 213.957031 L 349.5 225.691406 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 317.457031 355.609375 L 349.5 334.1875 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 425.363281 245.550781 L 464.5 234.203125 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<path d=\"M 425.417969 340.257812 L 464 330.351562 \" style=\"fill:none;stroke-width:1;stroke-linecap:round;stroke-linejoin:round;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:4,4;stroke-miterlimit:10;\" transform=\"matrix(1,0,0,1,-144,-140)\"/>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"193.65845\" xlink:href=\"#glyph0-1\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"198.82175\" xlink:href=\"#glyph0-2\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"200.82155\" xlink:href=\"#glyph0-3\" y=\"13.205566\"/>\n",
       "  <use x=\"208.31855\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"213.32435\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"215.82455\" xlink:href=\"#glyph0-6\" y=\"13.205566\"/>\n",
       "  <use x=\"220.32455\" xlink:href=\"#glyph0-7\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"222.82475\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"227.83055\" xlink:href=\"#glyph0-8\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"232.83635\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"235.33655\" xlink:href=\"#glyph0-16\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"194.23413\" xlink:href=\"#glyph0-10\" y=\"23.5\"/>\n",
       "  <use x=\"200.73393\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"205.73973\" xlink:href=\"#glyph0-12\" y=\"23.5\"/>\n",
       "  <use x=\"210.74553\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"215.75133\" xlink:href=\"#glyph0-2\" y=\"23.5\"/>\n",
       "  <use x=\"217.75113\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"222.75693\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"227.76273\" xlink:href=\"#glyph0-7\" y=\"23.5\"/>\n",
       "  <use x=\"230.26653\" xlink:href=\"#glyph0-4\" y=\"23.5\"/>\n",
       "  <use x=\"235.27233\" xlink:href=\"#glyph0-6\" y=\"23.5\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"311.65845\" xlink:href=\"#glyph0-1\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"316.82175\" xlink:href=\"#glyph0-2\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"318.82155\" xlink:href=\"#glyph0-3\" y=\"13.205566\"/>\n",
       "  <use x=\"326.31855\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"331.32435\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"333.82455\" xlink:href=\"#glyph0-6\" y=\"13.205566\"/>\n",
       "  <use x=\"338.32455\" xlink:href=\"#glyph0-7\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"340.82475\" xlink:href=\"#glyph0-4\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"345.83055\" xlink:href=\"#glyph0-8\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"350.83635\" xlink:href=\"#glyph0-5\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"353.33655\" xlink:href=\"#glyph0-17\" y=\"13.205566\"/>\n",
       "</g>\n",
       "<g style=\"fill:rgb(0%,0%,0%);fill-opacity:1;\">\n",
       "  <use x=\"312.23413\" xlink:href=\"#glyph0-10\" y=\"23.5\"/>\n",
       "  <use x=\"318.73393\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"323.73973\" xlink:href=\"#glyph0-12\" y=\"23.5\"/>\n",
       "  <use x=\"328.74553\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"333.75133\" xlink:href=\"#glyph0-2\" y=\"23.5\"/>\n",
       "  <use x=\"335.75113\" xlink:href=\"#glyph0-13\" y=\"23.5\"/>\n",
       "  <use x=\"340.75693\" xlink:href=\"#glyph0-11\" y=\"23.5\"/>\n",
       "  <use x=\"345.76273\" xlink:href=\"#glyph0-7\" y=\"23.5\"/>\n",
       "  <use x=\"348.26653\" xlink:href=\"#glyph0-4\" y=\"23.5\"/>\n",
       "  <use x=\"353.27233\" xlink:href=\"#glyph0-6\" y=\"23.5\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SVG('../img/beam_search.svg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fig.10.15.3: The beam search process. The beam size is 2 and the maximum length of the output sequence is 3. The candidate output sequences are $A$, $C$, $AB$, $CE$, $ABD$, and $CED$. \n",
    "\n",
    "Figure 11.11 demonstrates the process of beam search with an example. Suppose that the vocabulary of the output sequence only contains five elements: $\\mathcal{Y} = \\{A, B, C, D, E\\}$ where one of them is a special symbol “&lt;eos&gt;”. Set beam size to 2, the maximum length of the output sequence to 3. At time step 1 of the output sequence, suppose the words with the highest conditional probability $\\mathbb{P}(y_1 \\mid \\boldsymbol{c})$ are $A$ and $C$. At time step 2, we compute $\\mathbb{P}(A, y_2 \\mid \\boldsymbol{c}) = \\mathbb{P}(A \\mid \\boldsymbol{c})\\mathbb{P}(y_2 \\mid A, \\boldsymbol{c})$ and $\\mathbb{P}(C, y_2 \\mid \\boldsymbol{c}) = \\mathbb{P}(C \\mid \\boldsymbol{c})\\mathbb{P}(y_2 \\mid C, \\boldsymbol{c})$ for all $y_2 \\in \\mathcal{Y}$, and pick the largest two among these 10 values: say $\\mathbb{P}(A, B \\mid \\boldsymbol{c})$ and $\\mathbb{P}(C, E \\mid \\boldsymbol{c})$. Then at time step 3, we compute $\\mathbb{P}(A, B, y_3 \\mid \\boldsymbol{c}) = \\mathbb{P}(A, B \\mid \\boldsymbol{c})\\mathbb{P}(y_3 \\mid A, B, \\boldsymbol{c})$ and $\\mathbb{P}(C, E, y_3 \\mid \\boldsymbol{c}) = \\mathbb{P}(C, E \\mid \\boldsymbol{c})\\mathbb{P}(y_3 \\mid C, E, \\boldsymbol{c})$ for all $y_3 \\in \\mathcal{Y}$, and pick the largest two among these 10 values: say $\\mathbb{P}(A, B, D \\mid \\boldsymbol{c})$ and $\\mathbb{P}(C, E, D \\mid  \\boldsymbol{c})$. As a result, we obtain 6 candidates output sequences: (1) $A$; (2)$C$; (3) $A$, $B$; (4)$C$, $E$; (5)$A$, $B$, $D$; and (6)$C$, $E$, $D$. In the end, we will get the set of final candidate output sequences based on these 6 sequences.\n",
    "\n",
    "In the set of final candidate output sequences, we will take the sequence with the highest score as the output sequence from those below:\n",
    "\n",
    "$$ \\frac{1}{L^\\alpha} \\log \\mathbb{P}(y_1, \\ldots, y_{L}) = \\frac{1}{L^\\alpha} \\sum_{t'=1}^L \\log \\mathbb{P}(y_{t'} \\mid y_1, \\ldots, y_{t'-1}, \\boldsymbol{c}),$$\n",
    "\n",
    "Here, $L$ is the length of the final candidate sequence and the selection for $\\alpha$ is generally 0.75. The $L^\\alpha$ on the denominator is a penalty on the logarithmic addition scores for the longer sequences above. The computational overhead $\\mathcal{O}(k\\left|\\mathcal{Y}\\right|T')$ of the beam search can be obtained through analysis. The result is a computational overhead between those of greedy search and exhaustive search. In addition, greedy search can be treated as a beam search with a beam size of 1. Beam search strikes a balance between computational overhead and search quality using a flexible beam size of $k$.\n",
    "\n",
    "\n",
    "## Summary\n",
    "\n",
    "* Methods for predicting variable-length sequences include greedy search, exhaustive search, and beam search.\n",
    "* Beam search strikes a balance between computational overhead and search quality using a flexible beam size.\n",
    "\n",
    "\n",
    "## Exercises\n",
    "\n",
    "* Can we treat an exhaustive search as a beam search with a special beam size? Why?\n",
    "* We used language models to generate sentences in `chapter_rnn_scratch`. Which kind of search does this output use? Can you improve it?\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}