Example_8.3_Shortcut Maze.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#######################################################################\n",
    "# Copyright (C)                                                       #\n",
    "# 2016-2018 Shangtong Zhang(zhangshangtong.cpp@gmail.com)             #\n",
    "# 2016 Kenta Shimada(hyperkentakun@gmail.com)                         #\n",
    "# Permission given to modify the code as long as you keep this        #\n",
    "# declaration at the top                                              #\n",
    "#######################################################################\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "這次的範例主要參考[ShangtongZhang](https://github.com/ShangtongZhang/reinforcement-learning-an-introduction/blob/master/chapter07/random_walk.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Chapter 8主要結合planning與learning，讓過往的經驗可以再次得到一次利用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example 8.3: Shortcut Maze\n",
    "\n",
    "![](https://i.imgur.com/U56FymZ.png)\n",
    "Figure 8.5: Average performance of Dyna agents on a shortcut task. The left environment was used for the first 3000 steps, the right environment for the rest.\n",
    "\n",
    "Figure 8.5說明上述環境變化所造成的問題。最一開始，最佳路徑就在障礙物的左邊(左上圖)。在3000個steps之後，右邊出現一個捷徑，這個捷徑並不干擾到原本較長的路徑(右上圖)。這圖說明著，regular Dyna-Q agent並沒有切換過去那條捷徑，never。事實上，它也從來沒有發現過有這條捷徑。它的模型(model)並不覺得有這麼一條捷徑，所以plan的愈多，就愈不可能往右，也就不可能去發現捷徑。即使你用$\\epsilon-$greedy也不大可能讓agent做那麼多的探索行為(exploratory actions)來發現這條捷徑。\n",
    "\n",
    "事實上這邊我們遇到的普遍問題是探索(exploration)與開發(exploitation)之間矛盾的另一種版本問題。在planning context中，探索意味著那些能夠改善model的actions，而開發則意味著當前model給出的最佳方式下的行為。我們希望agent能夠利用探索來發現環境的變化，但又不希望效能大幅的降低。這跟先前談論過的探索/開發(exploration/exploitation)的矛盾是一樣的，沒有特效藥，不過簡單的啟發式方法(heuristics)通常是有效的。\n",
    "\n",
    "其實解決shortcut maze(捷徑迷宮)問題的Dyna-Q+ agent就是使用這種啟發式方法。這個agent持續的追蹤每一個state-action pair，它記錄從上次與環境的實際交互過程中嚐試這個pair已經經過多少個time steps。經過的時間愈長，我們就愈能推敲這個pair的動態性已經改變，而且模型也可能已經不正確。為了鼓勵測試長時間沒有試過的actions，只要在模擬經驗中有這些action，那就有一個特別的\"bouns reward\"。特別是，如果model對於轉移的reward是$r$，然後這個轉移在$\\tau$個time steps中沒有被嚐試過，那planning updates的完成就好像是這個轉移產生了$r + k\\sqrt{\\tau}$的reward，其中$k$是比較小的數值。這鼓勵了agent持續的測試所有可能的state transitions(狀態轉移)，即使為了實現這樣的測試需要執行很多很多的actions。當然，這樣的測試是需要成本的，不過更多情況下，這種computational curiosity(好奇心)是值得的。\n",
    "\n",
    "註：Dyna-Q agent以另外兩種方式改變。首先，允許在上面框框中的Tabular Dyna-Q algorithm的planning step ($f$)中考慮一個state中從來沒有出現過的actions。第二，這些actions的初始模型(initial model)會以0的reward回到相同的state。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "基本上這個範例跟Example 8.2大致相同，只是一些環境的參數做些許調整就可以直接執行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先載入需求套件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tqdm import tqdm\n",
    "import heapq\n",
    "from copy import deepcopy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先設置Environment，要注意一下這邊的一些設置是跟Example 8.1不一樣的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Maze:\n",
    "    def __init__(self):\n",
    "        \"\"\"初始化環境\n",
    "        \n",
    "        根據範例說明：\n",
    "        1. 這是一個寬9高6的gridworld\n",
    "        2. 每個episode的start state都是[5, 3]\n",
    "        3. goal state為[0, 8]\n",
    "        4. 我們能做的action就是上下左右\n",
    "        5. 路面上是有障礙\n",
    "        6. 在1000個step之後障礙就會變動\n",
    "        \"\"\"\n",
    "        self.WORLD_WIDTH = 9\n",
    "        self.WORLD_HEIGHT = 6\n",
    "        \n",
    "        self.START_STATE = [5, 3]\n",
    "        self.GOAL_STATE = [0, 8]\n",
    "        \n",
    "        self.ACTION_UP = 0\n",
    "        self.ACTION_DOWN = 1\n",
    "        self.ACTION_LEFT = 2\n",
    "        self.ACTION_RIGHT = 3\n",
    "        self.action_space = [self.ACTION_UP, self.ACTION_DOWN, self.ACTION_LEFT, self.ACTION_RIGHT]\n",
    "        \n",
    "        # 障礙設置\n",
    "        # 這次的範例中，障礙是會變動的\n",
    "        self.obstacles = None\n",
    "        self.old_obstacles = [[3, i] for i in range(1, 9)]\n",
    "        self.new_obstacles = [[3, i] for i in range(1, 8)]\n",
    "        \n",
    "        self.value_size = (self.WORLD_HEIGHT, self.WORLD_WIDTH, len(self.action_space))\n",
    "        \n",
    "        # 根據範例的圖片來設置\n",
    "        self.max_steps = 6000\n",
    "        \n",
    "        # 1000個step之後環境變換\n",
    "        self.time_to_switch = 3000\n",
    "        \n",
    "    def step(self, state, action):\n",
    "        \"\"\"run step according action\n",
    "        \n",
    "        state: 當前的狀態\n",
    "        action: 要執行的action\n",
    "        \n",
    "        幾點要注意：\n",
    "            1. 不能超出邊界，這可以用max、min來處理\n",
    "            2. 撞到障礙就是停在原地\n",
    "            3. 每次的reward都是0，除非你到達goal state\n",
    "            \n",
    "        return next_state, reward\n",
    "        \"\"\"\n",
    "        x, y = state\n",
    "        \n",
    "        if action == self.ACTION_UP:\n",
    "            x = max(x - 1, 0)\n",
    "        elif  action == self.ACTION_DOWN:\n",
    "            x = min(x + 1, self.WORLD_HEIGHT - 1)\n",
    "        elif  action == self.ACTION_LEFT:\n",
    "            y = max(y - 1, 0)\n",
    "        elif  action == self.ACTION_RIGHT:\n",
    "            y = min(y + 1, self.WORLD_WIDTH - 1)\n",
    "        \n",
    "        if [x, y] in self.obstacles:\n",
    "            x, y = state\n",
    "        \n",
    "        if [x, y] == self.GOAL_STATE:\n",
    "            reward = 1.0\n",
    "        else:\n",
    "            reward = 0.0\n",
    "        \n",
    "        return [x, y], reward\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "maze = Maze()\n",
    "maze.obstacles = maze.old_obstacles\n",
    "assert maze.START_STATE == [5, 3]\n",
    "assert maze.GOAL_STATE == [0, 8]\n",
    "assert (maze.step([5, 3], 0)) == ([4, 3], 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我們會需要一個model來記錄曾經有過的經驗，這個model是for Dyna-Q的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TrivialModel:\n",
    "    def __init__(self, rand=np.random):\n",
    "        # 用dict來做模型，反正我們只會記錄該state-action pair最後一次的經驗\n",
    "        self.model = dict()\n",
    "        self.rand = rand\n",
    "#         self.rand.seed(10)\n",
    "    \n",
    "    def feed(self, state, action, next_state, reward):\n",
    "        \"\"\"記錄經驗\"\"\"\n",
    "        state = deepcopy(state)\n",
    "        next_state = deepcopy(next_state)\n",
    "        \n",
    "        # 判斷該組state是否存在目前的model\n",
    "        # 如不存在就賦予一個預設值\n",
    "        if tuple(state) not in self.model.keys():\n",
    "            self.model[tuple(state)] = dict()\n",
    "        self.model[tuple(state)][action] = [list(next_state), reward]\n",
    "        \n",
    "    def sample(self):\n",
    "        \"\"\"隨機取一筆經驗出來\"\"\"\n",
    "        state_index = self.rand.choice(range(len(self.model.keys())))\n",
    "        state = list(self.model)[state_index]\n",
    "        action_index = self.rand.choice(range(len(self.model[state].keys())))\n",
    "        action = list(self.model[state])[action_index]\n",
    "        next_state, reward = self.model[state][action]\n",
    "        state = deepcopy(state)\n",
    "        next_state = deepcopy(next_state)\n",
    "        return list(state), action, list(next_state), reward"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我們需要另一個model來滿足Dyna-Q+的需求"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TimeModel:\n",
    "    def __init__(self, maze, time_weight=1e-4, rand=np.random):\n",
    "        \"\"\"\n",
    "        maze: Maze object\n",
    "        time_weight: 時間所賦予的權重\n",
    "        rand: numpy random function\n",
    "        \"\"\"\n",
    "        self.rand = rand\n",
    "        self.model = dict()\n",
    "        # 追蹤記錄total time\n",
    "        self.time = 0\n",
    "        self.time_weight = time_weight\n",
    "        self.maze = maze\n",
    "        \n",
    "    def feed(self, state, action, next_state, reward):\n",
    "        \"\"\"記錄經驗\n",
    "        \n",
    "        我們需要先判斷一下相關經驗是否已經存在，不存在就先設置初始格式再寫入\n",
    "        剩下這個state沒有被執行過的action我們就另外做個記錄\n",
    "        只是記錄的next state會是current state        \n",
    "        \"\"\"\n",
    "        state = deepcopy(state)\n",
    "        next_state = deepcopy(next_state)\n",
    "        \n",
    "        self.time +=1\n",
    "        \n",
    "        if tuple(state) not in self.model.keys():\n",
    "            self.model[tuple(state)] = dict()\n",
    "            \n",
    "            for _action in self.maze.action_space:\n",
    "                if _action != action:\n",
    "                    # 這邊要注意，next_state記錄的是current state\n",
    "                    # 意思就是，當我們後面去遇到這個state-action pair的時候\n",
    "                    # 我們就會從current state再往下走，而不會真正的轉至下一個state\n",
    "                    # 當這個pair第一次被sample到的時候，雖然reward是0，但是還是會有一個bouns reward\n",
    "                    # 這也合理，沒走過的你也不知道走了以後會怎麼樣，但是它逐漸的累積之後\n",
    "                    # 總是有機會被選到，那就能真正的知道它的下一個狀態是怎麼了\n",
    "                    self.model[tuple(state)][_action] =[list(state), 0, 1]\n",
    "        \n",
    "        self.model[tuple(state)][action] = [list(next_state), reward, self.time]\n",
    "    \n",
    "    def sample(self):\n",
    "        \"\"\"隨機取一筆經驗出來\"\"\"\n",
    "        state_index = self.rand.choice(range(len(self.model.keys())))\n",
    "        state  = list(self.model)[state_index]\n",
    "        \n",
    "        action_index = self.rand.choice(range(len(self.model[state].keys())))\n",
    "        action = list(self.model[state])[action_index]\n",
    "        \n",
    "        # 這邊我們假設我們真的去sample到那個沒被選過的pair\n",
    "        # next_state還會是state\n",
    "        # 然後這個pair已經好久沒有出現，所以有bouns reward之後，其reward還是會增長\n",
    "        next_state, reward, time = self.model[state][action]\n",
    "        reward += self.time_weight * np.sqrt(self.time - time)\n",
    "        \n",
    "        state = deepcopy(state)\n",
    "        next_state = deepcopy(next_state)\n",
    "        \n",
    "        return list(state), action, list(next_state), reward"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後我們來設置一下Dyna-Q所需要的參數：\n",
    "* gamma: discount factor\n",
    "* epsilon: 探索的機率\n",
    "* planning_steps: planning steps\n",
    "* average_runs: 幾次的run計算平均"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "dyna_q_params = {\n",
    "    'gamma': 0.95,\n",
    "    'epsilon': 0.1,\n",
    "    'planning_steps': 50,\n",
    "    'average_runs': 5,     \n",
    "    'alpha': 1.0,\n",
    "    'time_weight': 1e-3,\n",
    "    'methods': ['Dyna-Q', 'Dyna-Q+']\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gamma': 0.95,\n",
       " 'epsilon': 0.1,\n",
       " 'planning_steps': 50,\n",
       " 'average_runs': 5,\n",
       " 'alpha': 1.0,\n",
       " 'time_weight': 0.001,\n",
       " 'methods': ['Dyna-Q', 'Dyna-Q+']}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dyna_q_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "設置一個已經很熟悉的，用來選擇action的function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def action_selection(state, value_function, maze, dyna_params):\n",
    "    \"\"\"選擇action的function\n",
    "    \n",
    "    後來自己被很多q_value參數搞混，所以就試著分開來命名\n",
    "    \n",
    "    state: current state\n",
    "    value_function: action value function, that is q_value\n",
    "    maze: object for Maze\n",
    "    dyna_params: algorithm's parameters\n",
    "    \"\"\"\n",
    "    if np.random.binomial(1, dyna_q_params['epsilon']) == 1:\n",
    "        return np.random.choice(maze.action_space)\n",
    "    else:\n",
    "        # 取得該state的所有可能的action\n",
    "        values = value_function[state[0], state[1], :]\n",
    "        return np.random.choice([action for action, value in enumerate(values) if value == np.max(values)])\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接著來設置Dyna-Q algorithm，下面給出完整演算法的pseudocode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/jo168Rh.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dyna_q_algorithm(q_value, model, maze, params):\n",
    "    \"\"\"dyna_q algorithm\n",
    "    \n",
    "    q_value: action-value function\n",
    "    model: for planning\n",
    "    maze: environment\n",
    "    params: for dyna_q's parameters\n",
    "    \n",
    "    見上演算法說明\n",
    "    \"\"\"\n",
    "    # (a) current state\n",
    "    state = maze.START_STATE\n",
    "    steps = 0\n",
    "    \n",
    "    # 一直到agent走到目標之前就是一路迴圈執行\n",
    "    while state != maze.GOAL_STATE:        \n",
    "        # 記錄當下的steps\n",
    "        steps += 1\n",
    "        \n",
    "        # (b) 取得action\n",
    "        action = action_selection(state, q_value, maze, params)\n",
    "        \n",
    "        # (c) 執行action，得到next state與reward\n",
    "        next_state, reward = maze.step(state, action)\n",
    "        \n",
    "        # (d) 做一次Q-learning的更新\n",
    "        q_value[state[0], state[1], action] += \\\n",
    "            params['alpha'] * (reward + params['gamma'] * np.max(q_value[next_state[0], next_state[1], :]) \\\n",
    "                - q_value[state[0], state[1], action])\n",
    "        \n",
    "        # (e) 資料寫入模型\n",
    "        model.feed(state, action, next_state, reward)\n",
    "        \n",
    "        # (f) 迴圈執行planning_steps次\n",
    "        for t in range(0, params['planning_steps']):            \n",
    "            # 隨機的從模型中取一筆資料\n",
    "            # 從經驗中學習，其實就是用sample到的資料再做一次更新\n",
    "            _state, _action, _next_state, _reward = model.sample()\n",
    "            q_value[_state[0], _state[1], _action] += \\\n",
    "            params['alpha'] * (_reward + params['gamma'] * np.max(q_value[_next_state[0], _next_state[1], :]) \\\n",
    "                - q_value[_state[0], _state[1], _action])\n",
    "            \n",
    "        state = next_state\n",
    "        \n",
    "        if steps > maze.max_steps:\n",
    "            break\n",
    "    \n",
    "    return steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "演算法的部份也已經好了，現在我們可以來處理實際要執行的過程了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先初始化需求的物件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "shortcut_maze  = Maze()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後我們也已經有env的參數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gamma': 0.95,\n",
       " 'epsilon': 0.1,\n",
       " 'planning_steps': 50,\n",
       " 'average_runs': 5,\n",
       " 'alpha': 1.0,\n",
       " 'time_weight': 0.001,\n",
       " 'methods': ['Dyna-Q', 'Dyna-Q+']}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dyna_q_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下來設置一些過程中需要的參數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 2, 6000)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_steps = shortcut_maze.max_steps\n",
    "rewards = np.zeros((dyna_q_params['average_runs'], 2, max_steps))\n",
    "# 20run、2個model、3000個steps\n",
    "rewards.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 5/5 [02:52<00:00, 34.60s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 53s, sys: 883 ms, total: 2min 53s\n",
      "Wall time: 2min 52s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# 執行20次\n",
    "for run in tqdm(range(dyna_q_params['average_runs'])):\n",
    "    # 初始化兩個model\n",
    "    models = [TrivialModel(), TimeModel(shortcut_maze,  dyna_q_params['time_weight'])]\n",
    "    \n",
    "    # 初始化兩種演算法的action value\n",
    "    q_values = [np.zeros(shortcut_maze.value_size), np.zeros(shortcut_maze.value_size)]\n",
    "    \n",
    "    for i in range(len(dyna_q_params['methods'])):\n",
    "        # 一開始是左邊開個洞的迷宮\n",
    "        shortcut_maze.obstacles = shortcut_maze.old_obstacles\n",
    "        \n",
    "        # 初始化step\n",
    "        steps = 0\n",
    "        last_steps = steps\n",
    "        # 3000個step為極限做迴圈\n",
    "        while steps < max_steps:\n",
    "            steps += dyna_q_algorithm(q_values[i], models[i], shortcut_maze, dyna_q_params)\n",
    "            # 更新reward\n",
    "            # 這個執行區間內的reward都會變成起始那一個reward\n",
    "            rewards[run, i, last_steps: steps] = rewards[run, i, last_steps]\n",
    "            # 累加reward\n",
    "            rewards[run, i, min(steps, max_steps - 1)] = rewards[run, i, last_steps] + 1\n",
    "            \n",
    "            last_steps = steps\n",
    "            \n",
    "            # 1000個step之後環境變換\n",
    "            if steps > shortcut_maze.time_to_switch:\n",
    "                shortcut_maze.obstacles = shortcut_maze.new_obstacles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "計算平均"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 6000)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 延著0軸計算平均，所以0軸消失\n",
    "rewards = rewards.mean(axis=0)\n",
    "rewards.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "繪製圖面"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7MAAAJNCAYAAADj8SKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdebyXY+L/8dfVadeiXatKpEKrMogsjX0pe7asYzAYjDFj5oufYYy9RISEbFkz1pE9JEULiSK00Kq9U2e5fn/cZxJDnZbPuc/nnNfz8ehxruv+bO9P0/Tw7rrv6w4xRiRJkiRJyiYV0g4gSZIkSdLGssxKkiRJkrKOZVaSJEmSlHUss5IkSZKkrGOZlSRJkiRlHcusJEmSJCnrVEw7wOaoX79+bNmyZdoxJEmSJEkZMH78+AUxxga/9FhWl9mWLVsybty4tGNIkiRJkjIghPDNrz3macaSJEmSpKxjmZUkSZIkZR3LrCRJkiQp62T1NbO/JC8vj1mzZpGbm5t2lKxVtWpVmjVrRqVKldKOIkmSJEm/qMyV2VmzZlGzZk1atmxJCCHtOFknxsjChQuZNWsWrVq1SjuOJEmSJP2iMneacW5uLvXq1bPIbqIQAvXq1XNlW5IkSVKpVubKLGCR3Uz+/kmSJEkq7cpkmU1bTk4OnTp1okOHDnTs2JGbb76ZwsLCjH/u6NGj6d69OzvuuCNt27blzjvvzPhnSpIkSVIaytw1s6VBtWrVmDBhAgDz5s2jX79+LF26lKuvvjpjn/n999/Tr18/nn32Wbp06cKCBQs44IADaNy4MX369MnY50qSJElSGlyZzbCGDRsyZMgQBg0aRIyRvfbaa23RBdhzzz2ZOHEiV111Faeffjq9evWidevWDBw4cO1zjjzySLp27UqHDh0YMmTIL37OHXfcQf/+/enSpQsA9evX54YbbuDGG2/M7BeUJEmSpBRYZktA69atKSgoYN68eZxxxhkMGzYMgC+++ILc3Fw6duwIwNSpU3nllVcYO3YsV199NXl5eQAMHTqU8ePHM27cOAYOHMjChQv/5zM+/fRTunbt+pNj3bp1Y8qUKZn9cpIkSZKUgjJ9mvHV//6UKXOWbtH3bN+kFlce1mGTX3/MMcdwzTXXcOONNzJ06FD69++/9rFDDjmEKlWqUKVKFRo2bMjcuXNp1qwZAwcO5JlnngFg5syZTJs2jXr16m3uV5EkSZKkrOXKbAn46quvyMnJoWHDhlSvXp3evXszcuRIRowYwYknnrj2eVWqVFk7zsnJIT8/nzfffJNRo0bx/vvvM3HiRDp37kxubi7PPPMMnTp1olOnTowbN4727dszfvz4n3zu+PHj6datW4l9T0mSJEkqKWV6ZXZzVlC3lPnz53POOedw/vnnr73lzZlnnslhhx1Gz549qVOnznpfv2TJEurUqUP16tWZOnUqY8aMAaBPnz4/2dipadOm9OjRg759+9KpUycWLlzIFVdcwfXXX5+5LydJkiRJKSnTZTYtq1atolOnTuTl5VGxYkVOPvlkLr744rWPd+3alVq1anHaaadt8L0OPPBA7rrrLtq1a0fbtm3ZbbfdfvF5jRs3Zvjw4Zx99tksWbKEr7/+mmHDhrH33ntvse8lSZIkSaVFiDGmnWGTdevWLY4bN+4nxz777DPatWuXUqLimTNnDr169WLq1KlUqJCZM73vvPNOBg8ezNtvv73B1d9fkg2/j5IkSZLKthDC+BjjL1476TWzJezBBx+kR48eXHvttRkrsgDnnnsukydP3qQiK0mSJEmlnacZl7BTTjmFU045Je0YkiRJkpTVXJmVJEmSJGUdy6wkSZIkKetYZiVJkiRJWccyK0mSJEnKOpbZDMjJyaFTp0506NCBjh07cvPNN1NYWJjxzx09ejTdu3dnxx13pG3bttx5550Z/0xJkiRJWWj6azBoV5j/RdpJNpm7GWdAtWrVmDBhAgDz5s2jX79+LF26lKuvvjpjn/n999/Tr18/nn32Wbp06cKCBQs44IADaNy4MX369PnJc3v16sWwYcNo2bJlxvJIkiRJKqV++BpeugwWTofKW6WdZpO5MpthDRs2ZMiQIQwaNIgYI3vttdfaoguw5557MnHiRK666ipOP/10evXqRevWrRk4cODa5xx55JF07dqVDh06MGTIkF/8nDvuuIP+/fvTpUsXAOrXr88NN9zAjTfemNkvKEmSJCl7zHgHBnVPimzt5lCjUdqJNplltgS0bt2agoIC5s2bxxlnnMGwYcMA+OKLL8jNzaVjx44ATJ06lVdeeYWxY8dy9dVXk5eXB8DQoUMZP34848aNY+DAgSxcuPB/PuPTTz+la9euPznWrVs3pkyZktkvJ0mSJCk7jB8GDxwKBavhgH/ChZMgJ3tP1s3e5MXx0uXw/eQt+57b7AwHXb/JLz/mmGO45ppruPHGGxk6dCj9+/df+9ghhxxClSpVqFKlCg0bNmTu3Lk0a9aMgQMH8swzzwAwc+ZMpk2bRr169Tbqc++//34GDBgAwPTp0zn44IOpXLkyrVq1WvvekiRJksqggjx47gKY+EgyP+ExaHtQupm2gLJdZkuJr776ipycHBo2bEgIgd69ezNy5EhGjBjB+PHj1z6vSpUqa8c5OTnk5+fz5ptvMmrUKN5//32qV69Or169yM3N5Zlnnll7De69995L+/btGT9+PEccccTa9xg/fjzdunUD4LTTTuO0004DvGZWkiRJKjfy18ADh8HMMVC9PvzuLajdLO1UW0TZLrObsYK6pcyfP59zzjmH888/nxACAGeeeSaHHXYYPXv2pE6dOut9/ZIlS6hTpw7Vq1dn6tSpjBkzBoA+ffr8ZGOnpk2b0qNHD/r27UunTp1YuHAhV1xxBddfn/7vgSRJkqQULP0OBu8OqxZBo53gnNFQ1EnKgrJdZlOyatUqOnXqRF5eHhUrVuTkk0/m4osvXvt4165dqVWr1tqV0vU58MADueuuu2jXrh1t27Zlt912+8XnNW7cmOHDh3P22WezZMkSvv76a4YNG8bee++9xb6XJEmSpCzx9WgYdkgybnc49L2nTBVZgBBjTDvDJuvWrVscN27cT4599tlntGvXLqVExTNnzhx69erF1KlTqVAhM3tw3XnnnQwePJi33357g6u/vyQbfh8lSZIk/YIpz8GIk5Pxfv8HPS9JN89mCCGMjzF2+6XH3M24hD344IP06NGDa6+9NmNFFuDcc89l8uTJm1RkJUmSJGWpV//vxyJ7+O1ZXWQ3xNOMS9gpp5zCKaecknYMSZIkSWXNC5fAh/cm49+/D43ap5snwyyzkiRJkpTNCgvh6bPgkydhq4bwh3FQtXbaqTKuTJbZGOPanYO18bL5OmpJkiSpXCkshLv3grmToXq9clNkoQxeM1u1alUWLlxoIdtEMUYWLlxI1apV044iSZIkaX0Wz4TbuyRFtnFHuOTzclNkoQyuzDZr1oxZs2Yxf/78tKNkrapVq9KsWdm4kbIkSZJUJn3xCjzWDwrzoflu0P95yKmUdqoSlbEyG0KoCrwNVCn6nCdjjFeGEIYBewNLip7aP8Y4ISTnBQ8ADgZWFh3/aGM/t1KlSrRq1WpLfAVJkiRJKn0WfwuPHJuM9/oT7Pu3dPOkJJMrs6uBfWOMy0MIlYDRIYSXih77U4zxyZ89/yBg+6JfPYDBRT8lSZIkSQAz3oEHDk3GxzwAHY5MN0+KMnbNbEwsL5pWKvq1vgtZjwAeLHrdGGDrEELjTOWTJEmSpKzy8cM/Ftk+Q8p1kYUMbwAVQsgJIUwA5gGvxhg/KHro2hDCpBDCrSGEKkXHmgIz13n5rKJjkiRJklR+xQhPng4jz03m+10JHY9LN1MpkNEyG2MsiDF2ApoB3UMIOwF/AXYEdgXqAn/emPcMIZwdQhgXQhjnJk+SJEmSyrSCPHjqDPjkKaixDfzhI+h5cdqpSoUSuTVPjHEx8AZwYIzxu6JTiVcD9wPdi542G2i+zsuaFR37+XsNiTF2izF2a9CgQaajS5IkSVI61qyEu3omRbZOS/jjJ1Bvu7RTlRoZK7MhhAYhhK2LxtWA3sDU/14HW7R78ZHAJ0UveQ44JSR2A5bEGL/LVD5JkiRJKrXyV8MdPWD+Z7D9AXDBhHJ3650NyeRuxo2BB0IIOSSleUSM8fkQwushhAZAACYA5xQ9/0WS2/JMJ7k1z2kZzCZJkiRJpdPKRXDPPrDkW+h8Ehw6AEJIO1Wpk7EyG2OcBHT+heP7/srzI3BepvJIkiRJUqn3n7/Be7cn44794PBBFtlfkcmVWUmSJElScaxeDvfun5xWXKk6HHAddPNk1fWxzEqSJElSmqaPguFHJeMmXeDEJ2Cr+ulmygKWWUmSJElKQ2EhfPkaPHx0Mt/zj7D/VWkmyiqWWUmSJElKw1v/greuT8ZH3Qc7H51unixjmZUkSZKkklRYCE/2hykjoWYTOPkZaLhj2qmyjmVWkiRJkkpK3iq4pT2sWgQ1G1tkN4NlVpIkSZJKwqKvYOhBSZFtvU+y0VNOpbRTZS3LrCRJkiRl2vL5MLBzMm53GBw3PN08ZUCFtANIkiRJUplVWABj74Gb2iTzg2+yyG4hrsxKkiRJUibM/RSeOQe+n5TMD70Nup2WbqYyxDIrSZIkSVtSjPDUmfDJk8m80c5w1D3QsF26ucoYy6wkSZIkbSmFhXDnbrDg82R+0tPJZk8VvMJzS7PMSpIkSdKWkLsE7uoJi79JVmNPfxmq1Eg7VZllmZUkSZKkzZW7FAbvCUu+hW33TO4fW7Fy2qnKNMusJEmSJG2OqS/CYyck4w594JhhqcYpLzxxW5IkSZI21ezxPxbZnpfCUUPTzVOOuDIrSZIkSZvi2XNhwsPJ+Ow3oUnnNNOUO5ZZSZIkSdoYq5fD3T1h0VfJvP8LFtkUWGYlSZIkqbgWfQUjTkl+Nu8Bfe6Cuq3TTlUuWWYlSZIkqTjWrIQHDoclM6F1Lzj5WQgh7VTllmVWkiRJkjZk5li4r3cy3uU46Dsk3TyyzEqSJEnSeo0fBv++MBnv+3f4zXmpxlHCMitJkiRJv+bN6+HNfybjk56CNvunm0drWWYlSZIk6eeWfQ/Dj4K5nyTzcz+Ahjumm0k/YZmVJEmSpHUt/Q5uKSqu2+wCZ7wKlaqmm0n/o0LaASRJkiSp1Jg59sci2/U0OOcdi2wpZZmVJEmSJICZH/64Y3HX/nDYbanG0fp5mrEkSZIkTRkJI05JxkfcAZ1PSjePNsgyK0mSJKl8e+sGeOPaZHzUfbDz0enmUbFYZiVJkiSVTzHCc+fDx8OhWl04+Rlo0intVComy6wkSZKk8qcgH1685Mcie/or0GCHtFNpI1hmJUmSJJUvy+fBoyfA7HFQuQZc/Jk7Fmchy6wkSZKk8mPVYrhp+2Tcai84/hGLbJby1jySJEmSyodpr8K/tk3GOx0FJz0NVWqmm0mbzDIrSZIkqWwrLIQxg+Hhol2Ke/0Vjh4KOZXSzaXN4mnGkiRJksquH76Bu/eC3MVQoRKc9To03iXtVNoCXJmVJEmSVDaNux8G7JIU2fZHwgUfWWTLEFdmJUmSJJUtMcI7N8Pr1yTzo4cm18iqTLHMSpIkSSo7YoTn/wjj70/mJzwGbQ9KN5MywjIrSZIkqWz44Wu48zeQtxKq14OLJkPlrdJOpQzxmllJkiRJ2a2wECY9AYP3SIrszsfCBR9bZMs4V2YlSZIkZa9FM5LdilcvTeZ7XAS9r043k0qEZVaSJElSdvrwXnjhkmS88zGw/1VQu1maiVSCLLOSJEmSss97g+A/VyTjo+6DnY9ON49KnGVWkiRJUvYoyIM3/5ncegfgd+9479hyyjIrSZIkKTvECEN6wdxPoGptOH8c1GiYdiqlxN2MJUmSJJV++auTjZ7mfgJ1W1tk5cqsJEmSpFJuzODk1OLcJdC4E5z6XLIyq3LNMitJkiSp9Jr4OLx8eTLe7Vw48J/p5lGpYZmVJEmSVDp99CA894dkfObr0KxrunlUqlhmJUmSJJU+DxwGM96GqlvDeWOhZqO0E6mUcQMoSZIkSaXHklnw+MlJka3RCM541SKrX+TKrCRJkqTSYfpr8NwFsHwuNOsOR98HW7dIO5VKKcusJEmSpPQtnw/D+ybjff4Ge/8p3Twq9TzNWJIkSVK6Jj0BN7VJxgf80yKrYrHMSpIkSUrP6Nvg6TOT8YH/gt1+n24eZQ1PM5YkSZJU8mKEp86AT55K5qe9DNv+Jt1MyiqWWUmSJEkla9lcGH4UzJ0MFSrB6a94D1ltNMusJEmSpJKzchE80T8pso12gpOe9tY72iQZu2Y2hFA1hDA2hDAxhPBpCOHqouOtQggfhBCmhxAeDyFULjpepWg+vejxlpnKJkmSJCkFi7+FG1rBt+9Bu8Ph9+9aZLXJMrkB1Gpg3xhjR6ATcGAIYTfgX8CtMcY2wA/AGUXPPwP4oej4rUXPkyRJklQWTHoCbts5GR/wTzjqvnTzKOtlrMzGxPKiaaWiXxHYF3iy6PgDwJFF4yOK5hQ9vl8IIWQqnyRJkqQSsu6OxYffDr85FypWTjeTsl5Gr5kNIeQA44E2wB3Al8DiGGN+0VNmAU2Lxk2BmQAxxvwQwhKgHrAgkxklSZIkZUj+Ghh1FYy5I5mf+m9otVeqkVR2ZLTMxhgLgE4hhK2BZ4AdN/c9QwhnA2cDtGjRYnPfTpIkSVIm5K2CIb1g/tRkfvFnUKtJqpFUtmTymtm1YoyLgTeA3wBbhxD+W6KbAbOLxrOB5gBFj9cGFv7Cew2JMXaLMXZr0KBBxrNLkiRJ2khLZsGN2ydFtmk3+Mssi6y2uEzuZtygaEWWEEI1oDfwGUmpPbroaacCI4vGzxXNKXr89RhjzFQ+SZIkSRnwxnVwawdYswx2PgbOeg2q1Ew7lcqgTJ5m3Bh4oOi62QrAiBjj8yGEKcBjIYR/AB8D/93G7D7goRDCdGARcHwGs0mSJEnakmKEB4+AGW8l80NugV3PWP9rpM2QsTIbY5wEdP6F418B3X/heC5wTKbySJIkScqQFQvgpu0hFkJOZbjsK1djlXElcs2sJEmSpDJq3FC4o0dSZNvsD5fPtMiqRGR0N2NJkiRJZdSKBTD6Vnh/UDLf5Xjoe3e6mVSuWGYlSZIkbZzvJsHdPZNxlVpw3lio1TjdTCp3LLOSJEmSim/upz8W2e0PgOOGQ8XK6WZSuWSZlSRJklQ8r1zx42nF/UbADgekm0flmmVWkiRJ0vrFmNw/9v1BUKESnPAobN877VQq5yyzkiRJkn7dqh/g3t6wcBrkVIELPoLazdJOJVlmJUmSJP2KeZ/Bg0fC8u+hWXc46UmoWjvtVBJgmZUkSZL0czHC02fD5BHJfNez4JCb0s0k/YxlVpIkSdKPZo6FV/4Ksz6ErRrAEXe40ZNKJcusJEmSpMTHw2Hkecm4aVc4/RXIqZRuJulXWGYlSZIkwXuD4D9XJOMjB0OnfunmkTbAMitJkiSVdy//FcbckYwvnQ41GqSbRyoGy6wkSZJUnj3UB758PRmf96FFVlmjQtoBJEmSJKVg2ffwyHFJka1SG879ABrskHYqqdhcmZUkSZLKk9XL4LVrYOzdybxuazjzNaheN91c0kayzEqSJEnlxeJv4badf5z3GQIdj0svj7QZLLOSJElSefDlG/DQkcm404lw4PVQtVa6maTNYJmVJEmSyrrJT8JTZyTjg26EHmenm0faAiyzkiRJUlm1YBqMugqmPp/MjxkGHfqkmUjaYiyzkiRJUlkTIyydA0N6wZrlUKspnPYS1Nk27WTSFmOZlSRJksqajx6Ef1+QjPe6DPb5K4SQbiZpC7PMSpIkSWVFjPDcH+Djh6DSVnDkHdD+SIusyiTLrCRJklQWLJoBT54Ocz6C6vXhqHthu33STiVljGVWkiRJynbTRsHDRyXjVnvDSU9BTqV0M0kZZpmVJEmSstnLf4Exdybjg2+C7melm0cqIZZZSZIkKRvFCK9f82ORPfV5aNUz3UxSCbLMSpIkSdmmsADuPwhmfpDM//QVbFUv3UxSCbPMSpIkSdlkwXQY3hcWfwN1WsJZb0D1ummnkkqcZVaSJEnKFvOmwp09knHz3eC0l6BChXQzSSnxT74kSZKUDT4e/mOR/e21cMYrFlmVa67MSpIkSaXdpCdg5HnJ+Kj7YOej080jlQKWWUmSJKm0ihEmPAIjz03mxz4I7Y9IN5NUSlhmJUmSpNLqxUvhw3uT8cVToVbjdPNIpYhlVpIkSSpt8lfDg0fCt+9Bxarwu7ctstLPWGYlSZKk0mTFArijB6xcAHW3g5OfgTrbpp1KKnUss5IkSVJpsXoZ3LYL5K2AtgfDCY+mnUgqtdzLW5IkSSoNvpsE/2yWFNm9L7fIShtgmZUkSZLSNvYeuLtnMt79AtjnL+nmkbKApxlLkiRJaYkR/vM3eH9QMj/mAehwZLqZpCxhmZUkSZLSkJcLr/wVxt0HBPj9e9CofdqppKxhmZUkSZLS8PxFMPFRqNEILpoMFauknUjKKpZZSZIkqSQV5MGgbvDD11CzCZz7nkVW2gRuACVJkiSVlPmfwx3dkyLbsD387i2oViftVFJWcmVWkiRJyrS8XHjhYpjwcDLfbj846SkIId1cUhazzEqSJEmZtGQW3Nrhx/kJj0Hbg9LLI5URlllJkiQpU74dAw8W3Wpn17Pgt9dApWrpZpLKCMusJEmStKXFCNNfg4ePSub7/A32/lO6maQyxjIrSZIkbUkxwst/gQ8GJ/Oj7oOdj043k1QGWWYlSZKkLSUvF+7skexWDHDay7Dtb1KNJJVVlllJkiRpS/j6XRh2cDKutz2cOALqtk43k1SGWWYlSZKkzVGQB69fA+8OSOZdToXDB6abSSoHLLOSJEnSpvp+Mtx/MKxemsz73gu7HJNuJqmcsMxKkiRJm2LqC/DYiUCEzifDobdCTqW0U0nlhmVWkiRJ2lhfvAKP9UvGhw+CLienm0cqhyyzkiRJ0saYMwEeOTYZHzcc2h2Wbh6pnLLMSpIkScVRWAgjz4OJjyTzc96FbXZKN5NUjlVIO4AkSZJU6uWvgTt2/bHInvC4RVZZ7fWpc+n2j1F8MXdZ2lE2mSuzkiRJ0vrMGg/37puMm3aFk56CanXSzSRtphcmfc+C5aupVTV7Ny3L2MpsCKF5COGNEMKUEMKnIYQLi45fFUKYHUKYUPTr4HVe85cQwvQQwuchhAMylU2SJEnaoBhh2qgfi2yPc+CMURZZZa0YIxNmLuaMYR/y1EezqFmlIvVrVE471ibL5MpsPnBJjPGjEEJNYHwI4dWix26NMd607pNDCO2B44EOQBNgVAhhhxhjQQYzSpIkSf/ry9fhidMgd3EyP+hG6HF2upmkzRBj5OyHxvPqlLlrj714YU8q5mTvlacZK7Mxxu+A74rGy0IInwFN1/OSI4DHYoyrgRkhhOlAd+D9TGWUJEmSfiJGePly+OCuZL7DgbD/VdCwXZqppE1WWBi54tlPeHL8TPIKIrWqVmRo/13p3KIOORVC2vE2S4lcMxtCaAl0Bj4A9gDODyGcAowjWb39gaTojlnnZbNYf/mVJEmStpy8XLhzN/hhRjI/41Vo3j3dTNImWpNfyF1vfcnwMd8wb9lqalWtSN/OjbnmyJ2oXDF7V2PXlfEyG0KoATwFXBRjXBpCGAxcA8SinzcDp2/E+50NnA3QokWLLR9YkiRJ5c+S2TD0AFgyE5rtCic/A1Vqpp1K2iSfzlnChY9NYPq85QB03bYOD53RneqVy9b+vxn9NiGESiRF9uEY49MAMca56zx+D/B80XQ20HydlzcrOvYTMcYhwBCAbt26xcwklyRJUrmxfB4MOzgpstvtmxRZKQvFGPn7yE8YPuZbABrUrMJ7l+9LpSy+LnZ9MlZmQwgBuA/4LMZ4yzrHGxddTwvQB/ikaPwc8EgI4RaSDaC2B8ZmKp8kSZLKucICmDISnjwtmff6C+z953QzSZsgN6+A/xv5CSPGzVp77OEze9C9Vd0yW2QhsyuzewAnA5NDCBOKjv0VOCGE0InkNOOvgd8BxBg/DSGMAKaQ7IR8njsZS5IkKSPWrIRHj4cZbyXzg2+CbmdAyO4NcVS+xBh5cvws/vTkpLXHfrd3a/rv3pLGtaulmKxkZHI349HAL/1t8OJ6XnMtcG2mMkmSJEkU5MGAjrBiHlSvB6e9BA3app1K2ihjZyzitPvHsmJNsv530m4tuPrwnbJ+h+KNUbauAJYkSZLWZ/ooeLQfFKyGzifBIbdAxSppp5KKbeyMRVz34mdMmJncA3n/dg25ru/ONKxZNeVkJc8yK0mSpLJv5ofw9o0w7ZVk3vkkOHyQpxUra0yft4x735nBYx/OBGDHbWry/47Yie6t6qacLD2WWUmSJJVdKxbCxEfhP1ck89rN4bAB0Ga/dHNJxVRYGHnsw5n89ZnJAFSvnMMjZ+1Gp+Zbp5wsfZZZSZIklU1zJsCQvX+cn/AYtD0ovTzSRigojDw3cTaXPzWZ1fmFANx+Qmd6t29E1Uo5KacrHSyzkiRJKntmjYN7i1Zf97gQdr8AtqqfbiapmKZ+v5R+93zAohVrAOi/e0tO3b0lrepvlXKy0sUyK0mSpLJlykgYcUoy7vcE7PDbdPNIxbQ6v4DLnpzEyAlzANizTX2uP2pnmtWpnnKy0skyK0mSpLLj/Tvglb8m45Oegjb7p5tHKoaCwsifn5rEk+NnAVA5pwK39+vMAR22STlZ6WaZlSRJUtnwwqXw4T0QKsBZb0CTTmknktYrxsjwMd9wzzsz+HbRSipXrMDpe7Ti4t47ULlihbTjlXqWWUmSJGW3gnwYeR5MegxyKsNZr8M2O6edSlqvGQtWcN2Ln/HqlLkAHLTTNtx+Qmcq5lhii8syK0mSpOxVkAcPHgnfjIZqdeD370GtJmmnkn7VmvxCThs2lnenLwSgfo3KvHzRXtSvUSXlZNnHMitJkqTs9MPXcPfekLsY2vSGE5+AENJOJf2iGCPXvzyVu9/6CoA61Svxz747s9cODahe2Vq2KfxdkyRJUvaZ8Ag8+/tk3P4I6DPEImmitIYAACAASURBVKtSadGKNbz1xTzueXsGU75bCiS32rnysPYE/8xuFsusJEmSssvo22DUlcn4qPtg56PTzSP9ivyCQs544EM+/nYxAF23rcMjZ/WgSsWclJOVDZZZSZIkZY93B/5YZM8fB/W3TzeP9AvyCwo54Z4xfPj1DwDs364RVx7WnuZ1vV/slmSZlSRJUnZ4+yZ4/Zpkx+LzP4Q6LdNOJP3E6vwC/vXS57w2dS7fLFxJn85NaV63Ouf22o6qlVyN3dIss5IkSSr93vgnvHU9VN0azvsAam6TdiJprWW5eXz87WIuenwCi1asoWKFwP7tGnHzMR2pUMHrYjPFMitJkqTSqyAPXvozjLsPqtaGs9+0yKpU+WT2Eg69ffTa+WEdm3D7CZ1TTFR+WGYlSZJUOsUIT58Nnz4NIQfOG2uRVakx7utF/PmpSXw5fwUA5+/ThoN23oYOTWqnnKz8sMxKkiSp9FmxEO7oDisXQKu9k3vIVqySdiqJd6cvYOyMRQx4bRp1qlfiyE5N6NR8a/rv0SrtaOWOZVaSJEmly+KZMGhXyF8FOx4KR9xhkVXqFi5fzejpC7jwsQlrj53ZszXn7dMmxVTlm2VWkiRJpcfnL8Gzv0+K7K5nwiE3p51I5VyMkRtf+Zw73/xy7bG7TurCgTs1TjGVwDIrSZKk0iBGePFS+PDeZH7gv2C3c9LNpHLvyfGzGPDaF8xctIrKORX4+6Ht6NW2ofeLLSUss5IkSUpXQR48diJMewWq1IJTnoWmXdNOpXLsuYlz+NdLU5m9eBUAB++8Df/suwu1q1VKOZnWZZmVJElSemKEZ36XFNlqdeGPn0DlrdJOpXLq/S8Xcsurn/Ph1z8A0HP7+vzrqF1osnW1lJPpl1hmJUmSlI7cJfDgkTDnI2jcEc56AyrkpJ1K5dCSlXmccM8Ypny3FIA2DWvw4OndLbGlnGVWkiRJJasgH8YOgVf+kswbtoeTn7XIqsStzi+g3z0fMP6bZCW2Vf2tuPHoXejcog45FULK6bQhlllJkiSVnPw18Ohx8OXryXzPP8I+f4Mc/7NUJWfF6nyuf2kqT4yfSW5eIY1qVeHC/XagX48WaUfTRvBvDUmSJJWM2R/B0AOhYDU07QYnPQXVtk47lcqRlWvy+WT2Us584EOW5uZTKSdwyM6NGdSvMyG4EpttLLOSJEnKvCkjYcQpybj1PnDiE5DjzrAqOTFGDrt9NF/OXwHA/u0acs8p3SyxWcwyK0mSpMzJX53sVvzpM8n8+Edgx0PSzaRyZfHKNVz6xEQ+mLGIZbn5bNdgK67rszNdtq1jkc1ylllJkiRlxqIZMPwoWPQl1GwMxz4EzXdNO5XKiRgj97/7Nf/v+SkAtK6/FX06N+VPB7SlZlXPCigLLLOSJEna8pbPg7t6wpplsOOhcNxwcBVMJaCgMPL1whWcct9YZi9eBUCfzk256ZiO7lBcxlhmJUmStOUsmwuTHodX/57Mu/aHwwakGknlR15BIb1veYuvF64EoE71Soz5635Uqehtn8oiy6wkSZK2jAmPwMjzIBYm84Nvgl3PTDeTyoXZi1fxj+enMOqzueQVRFo32IrLDmjLb9tvQwVXY8ssy6wkSZI2T4zw7m0w6qpkvtdl0OMc2KpeqrFU9i1YvprnJszh3ne+Ys6SXBrXrsoeberzr6N28ZTicsAyK0mSpM3z8uXwwV3J+I+fQu1m6eZRmZdfUMiy3HzOeWg84775AYBTf7MtVx+xU8rJVJIss5IkSdo0uUvgob4we1wyv2wGVK+bbiaVeYWFkUNvH83U75cBsGeb+tzRrwu1qlltyhv/F5ckSdLGW7MCrm+RjOu2hlOft8gqo3LzCvjTk5P4+NsfmPXDKk7o3oL2jWty8M6NqV3dW+2UR5ZZSZIkbZxZ42HoAcm4Yz/oMzjdPCrTCgsjr342lytHfsr3S3NpVqcah+zSmOv67ETwdk/lmmVWkiRJxTduKDz/x2Tc6SQ4YlC6eVSmLc3N47DbR/NN0a12Wtffitcv7ZVuKJUalllJkiRt2NLv4IWL4fMXIacK9L4advt92qlURi1cvprfP/wRY2csAmD/dg352yHtaVG3esrJVJpYZiVJkrR+cz6Ge/ZN7h+79bZw7hiobKnQlldQGLnuxc+4b/QMABrUrMLFvXfghO4tUk6m0sgyK0mSpF/3+Uvw6PHJeK8/wT5XgNcpKgPGzljEifeOIa8gAnDR/ttz0f47pJxKpdmvltkQwmQg/trjMcZdMpJIkiRJpcPYe+DFS5Nxvydgh9+mm0dl0sxFKzn09tEsWZUHwEE7bcOgfl3IqeA/mmj91rcye2jRz/OKfj5U9PPEzMWRJElS6mKEcff9WGRPeNwiqy3uhxVruOjxCbz1xXwA9t2xIRfstz2dmm+dcjJli18tszHGbwBCCL1jjJ3XeejyEMJHwOWZDidJkqQUvHUDvHldMj79P9CiR7p5VKasyS/k8Q+/5e8jPwWg7laV+dsh7ejbpVnKyZRtinPNbAgh7BFjfLdosjtQIbOxJEmSVOLyV8Odv4FFXybzSz6Hmtukm0llRoyRAa9N47ZR09Ye6797S648rL33i9UmKU6ZPR24P4RQu2i+uOiYJEmSyorl8+GmNsm4aTc4+j6LrLaIGCPPTZzDZU9OYnV+IQB/OqAtJ3RvQd2tKqecTtlsvWU2hFABaBNj7PjfMhtjXFIiySRJklQypr4Aj/VLxrscB32HpJtHZcIPK9Zw3+gZ/HvSHL5ZuJImtauyf/tGXLDf9tSvUSXteCoD1ltmY4yFIYTLgBGWWEmSpDJo3SLb+//BHhemm0dlwvtfLuS8Rz5i0Yo1AHRsVpubj+1Em4Y1Uk6msqQ4pxmPCiFcCjwOrPjvwRjjooylkiRJUuatW2RPfwVa7JZuHmW9vIJC+t75HpNnJ+tgh3dswq3HdfI2O8qI4pTZ44p+nrfOsQi03vJxJEmSVCLG3AUv/zkZn/U6NO2abh5ltQXLV/OP56fw7IQ5ADSvW43BJ3Zlp6a1N/BKadNtsMzGGFuVRBBJkiSVkFeugPcHJeMTn7LIapOtzi/g5U++58LHJgBQs0pFTvrNtvxx/x2oXNEboCizirMySwhhJ6A9UPW/x2KMD2YqlCRJkjKgIC8psmPvTuZnvwlNOqeZSFkqxsj0ecs5avB7LM3NB6Bv56bcclynlJOpPNlgmQ0hXAn0IimzLwIHAaMBy6wkSVK2WPUD3NIB8oq2QLn4M6jVJN1MykozF62k7+D3mL9sNQBtGtZg6Km70qJe9ZSTqbwpzsrs0UBH4OMY42khhEbA8MzGkiRJ0hbz3US4e69k3GgnOHMUVKqWbiZlnfemL+CON6fz7vSFAPRu34gTujen1w4NqeAGT0pBccrsqqJb9OSHEGoB84DmGc4lSZKkLWHSCHj6rGTc/XdwwHWQU6wrzSQAJs5czNtfzOfmV78AoEntqlzy27Yc1bVZyslU3hXnb7JxIYStgXuA8cBy4P2MppIkSdLmG3UVjL41Gfe9F3Y5JtU4yi4FhZGnxs/isqcmrT320oU9ade4VoqppB8VZzfjc4uGd4UQXgZqxRgnre81kiRJSlFBPjxwKHxbtP5w7gfQcMd0MymrXP3vT7n/3a/Xzh89azd2alqLmlUrpRdK+pnibAD1EPA28E6McWpx3ziE0Jxkk6hGJPelHRJjHBBCqAs8DrQEvgaOjTH+EEIIwADgYGAl0D/G+NHGfR1JkqRy7pv3k9OKl8yErRrCqc9ZZFUsMUb+9fLnvDNtPp/OWQrAeftsx9Fdm9Oq/lYpp5P+V3FOMx4K9ARuDyFsB3wMvB1jHLCB1+UDl8QYPwoh1ATGhxBeBfoDr8UYrw8hXA5cDvyZZJfk7Yt+9QAGF/2UJElScSz6Cu4/MBnXbgEXToAKOelmUlZ4/MNvufGVz1mwfA0AHZtvzZCTu9KoVtUNvFJKT3FOM34jhPA2sCuwD3AO0IFkFXV9r/sO+K5ovCyE8BnQFDiC5FY/AA8Ab5KU2SOAB2OMERgTQtg6hNC46H0kSZK0Pv++EMYPS8bHPQw7HgLBHWb162KMfDBjESPGzeTpj2YD0LdLU67vuwuVK1ZIOZ20YcU5zfg1YCuSTZ/eAXaNMc7bmA8JIbQEOgMfAI3WKajfk5yGDEnRnbnOy2YVHbPMSpIk/Zo1K2DwHvDDDKi6NRw5GHY8OO1UKuWWr86n3z1jmDRrCQA1q1bkxQt60ryu94pV9ijOacaTgK7ATsASYHEI4f0Y46rifEAIoQbwFHBRjHFpWOdfCGOMMYQQNyZwCOFs4GyAFi1abMxLJUmSypYVC2Dw7rB8LjTaGfo/D9W2TjuVSrEfVqzhpPs+WHtNbJuGNbjtuE60b1zLe8Uq6xTnNOM/AhRd99ofuB/YBqiyodeGECqRFNmHY4xPFx2e+9/Th0MIjUnuWwswm5/ev7ZZ0bGf5xkCDAHo1q3bRhVhSZKkMmPOxzCkVzJu0xv6jYAKnhqqX/bl/OU8MW4WD4/5hmWr89l9u3r03L4BZ+/VmhxLrLJUcU4zPp9kA6iuJLsPDyU53XhDrwvAfcBnMcZb1nnoOeBU4PqinyPXOX5+COExko2flni9rCRJ0s8UFsAb18I7NyfzDn3gmGGpRlLplZtXwFfzV3DMXe+xYk0BABf33oEL9ts+5WTS5ivOacZVgVuA8THG/I147z2Ak4HJIYQJRcf+SlJiR4QQzgC+AY4teuxFktvyTCe5Nc9pG/FZkiRJZd+yufDoccmqLMDxj3p9rH5VQWHkoAHvMGPBCgAu2LcNF+2/g6cTq8wozmnGN4UQ9iQppveHEBoANWKMMzbwutHAr/0/Zb9feH4EzttwZEmSpHJo7D3w4qXJOKcKXPo5VKuTbiaVSlO/X8qVIz/l428Xs6agkNP2aEmvtg3Ze4cGaUeTtqjinGZ8JdANaEtyvWwlYDjJyqskSZIyKXcpDDsYvp+czA+5GXY5HqrUSDeXSp3p85bx/pcL+fvITwFo37gWHZrU4v8ObU/wNk0qg4pzmnEfktvqfAQQY5xTtBmUJEmSMumrN+HBI5Jxgx3hjP9A1dqpRlLpU1gYeWf6Ak4dOnbtsT/s24ZLfts2xVRS5hWnzK5Z9xY6IYStMpxJkiSpfPtuIrx6JXz1RjLf/yrY4yJwdU0/c8/bX3Hzq5+Tm1cIwE3HdGS/HRtSZ6vKKSeTMq84ZXZECOFuYOsQwlnA6cA9mY0lSZJUzsQIU56FcUNhxtvJsRqN4Ljh0Lx7utlUquQXFPLWF/P5+7OfMGdJLgDH79qcIzs3ZbfW9VJOJ5Wc4m4A1RtYSnLd7P/FGF/NeDJJkqTyoiAf3vwnvHNTMq/fFg67DbbdPd1cKnVy8wo47PbRTJu3HIDdWtdlwPGdaVSrasrJpJK33jIbQsgBRsUY9wEssJIkSVtS/mr4+CF44ZJkXr0e/P69ZEXWU4q1jvyCQv7xwmcMe+9rALpuW4d/HLkT7RrXSjeYlKL1ltkYY0EIoTCEUDvGuKSkQkmSJJV5U56D/1wBi79N5r85H3qcAzW3STeXSp0R42YyYNQ0Zi9eRZ3qlTi+ewsuO6CtOxSr3CvONbPLgckhhFeBFf89GGO8IGOpJEmSyqoZ78BLf4Z5ye1TaN0LjrgDajdLM5VKoZmLVnLN81P4z5S5ABzWsQk3Hr0LVSvlpJxMKh2KU2afLvolSZKkTbXqBxh+NMwel8wbd4JjH4Q626abS6XS7a9N4+ZXvwCgdYOteOiMHjTdulrKqaTSpTgbQD1QEkEkSZLKrOXzYPDusGI+1N8BDhsI2/4m7VQqZQoLI3e+OZ1bR02joDACMPCEzhy80zZUzKmQcjqp9CnOyqwkSZI21ft3wqgroWANdDkVDh+YdiKVQm99MZ/LnpzI3KWrAfjDvm04c8/W1K5eKeVkUullmZUkScqEgjx44zoYfQtUqwu7ngn7XpF2KpUiBYWRCTMX8+T4WTw6NtkIbLfWdbnhqI60qFc95XRS6VfsMhtCqB5jXJnJMJIkSWXC6uXJacWLv4GK1eCiyVClRtqpVIrMXLSS/W55izX5hQA0qFmFYaftSocmtVNOJmWPDZbZEMLuwL1ADaBFCKEj8LsY47mZDidJkpR1fvgaBnRMxjscBH3vtshqrY+//YELHvuYmYtWAXBijxYcsktjurSo4y7F0kYqzsrsrcABwHMAMcaJIYS9MppKkiQp2+TlwtNnwmf/TuZ7XAj7XQUV3LhHMHnWEh4f9y3DxySnEx/esQm92jagbxdvySRtqmKdZhxjnPmzmzIXZCaOJElSFpo9Hp7oD4uTokKfu6Hj8alGUukwb1kuE2cu4U9PTmTxyjwq5QSuOrwDJ/bwlkzS5ipOmZ1ZdKpxDCFUAi4EPstsLEmSpCzxxnXw1r+ScZv94diHoLKb9wje+3IBJ977ATG5yw6/77Udfz5wx3RDSWVIccrsOcAAoCkwG/gPcF4mQ0mSJJVqMcKcj2HCI/DhPVCxKvQbAa33TjuZSoGJMxdz2ZOT+HzuMipWCNx47C7s1KQ2bRp67bS0JRWnzIYY44kZTyJJkpQNYoT//A3eH5TMazaB88dClZrp5lLqvpq/nGc/ns1db33FmoJCem5fn7P3ak3P7RukHU0qk4pTZt8NIXwNPA48FWNcnNlIkiRJpdTnL8Nzf4AV85J5vxHQYjeLbDm3YnU+Mxas4NDbR689NqhfZw7dpUmKqaSyb4NlNsa4QwihO3A8cEUIYQrwWIxxeMbTSZIklRZfvQmPHpeMW/aEo4dCjYapRlL6Pvr2B44e/B6FRdfF3nDULhzeqYm32ZFKQHF3Mx4LjA0hXAfcAjwAWGYlSVLZV5APQw+A2eOS+ZmvQbNu6WZS6q578TO+mLuMNz+fD8BlB7alXeNa7NPWf+CQSsoGy2wIoRbQh2RldjvgGaB7hnNJkiSlb/k8uLUDFKyBnY+Brv0tsuXcW1/MZ+6SXIa8/RXN6lSjY7PaHLdrC/r1aJF2NKncKc7K7ETgWeD/xRjfz3AeSZKk0uG5P8BHDybj9kdAnyFQoUK6mZSqt76Yz6lDx66dD+rXhU7Nt04xkVS+FafMto7xv3fHkiRJKuPyV8P9B8Hs8cl87z/DPn9NN5NSddVzn/LQmG8oKIw03boaD5y+K9UrV6TJ1tXSjiaVa79aZkMIt8UYLwKeCyH8T5mNMR6e0WSSJEklbcbb8MBhybhqbbhwElRz5a28evbj2Twy9lvGzlgEwO97bcf+7RrRpqG7V0ulwfpWZh8q+nlTSQSRJElKzdxP4ZOn4Z2i/+zZ/Q/Q+xoIId1cSsXMRSsZ9t7X3Dd6BgCt6m/Fg6d3p3nd6iknk7SuXy2zMcaic2voFGMcsO5jIYQLgbcyGUySJKlETH8NhvctmgQ46zVo2jXVSEpHXkEhA0ZNY9Ab09cee/minuy4Ta0UU0n6NcW5ZvZUYMDPjvX/hWOSJEnZoyAPRt8Gb/wjmR85GNr0hhoN0s2lVLz35QL63fPB2vnlB+1I3y5NaVizaoqpJK3P+q6ZPQHoB7QKITy3zkM1gUWZDiZJkpQxq5fBv1pBYV4yP/YhaO92IOVNYWFkwGvTeOXT75n6/TIATujenKsP34nKFd25Wirt1rcy+x7wHVAfuHmd48uASZkMJUmSlBExwtADYeaYZN5qLzj0Nqi3Xbq5VKJy8wqYsWAFt78+jRcnfw/ADo1qcOtxnejQpHbK6SQV1/qumf0G+Ab4TcnFkSRJypA1K2FgZ1j+PVTdGvb7O3Q7w02eyqFLnpjIC5O+A2DbetV589JeBP8cSFlng9fMhhB2A24H2gGVgRxgRYzRK+ElSVJ2+PJ1eKhPMt5uXzj+UajktZDlyaIVazh92Ifk5hUw9ftlHNutGfu1a8TeOzSwyEpZqjgbQA0CjgeeALoBpwA7ZDKUJEnSFvPWDfDGtcm45yWw3/+lm0clasHy1Tz90Sw++GoRE2YuZteWdejbuSlXHNKe2tUqpR1P0mYoTpklxjg9hJATYywA7g8hfAz8JbPRJEmSNkPeKnioL3z7XjI//hHY8ZB0M6nE/Pe62IMGvLP22Ik9WnBtn51TTCVpSypOmV0ZQqgMTAgh3ECyKZTbu0mSpNIpRvjwXnj5L8luxTUbw9lvQs1t0k6mEjJ62gJOuu/H2+ycv08bzt1nO6pVykkxlaQtrThl9mSS62TPB/4INAeOymQoSZKkTbJ0DrxwKXz+QjLf4UA44TE3eSonRoybybMfz+a9LxcCcG6v7di1VV32adsw5WSSMmGDZbZoV2OAVcDVmY0jSZK0iaY8ByNOTsZ1t0tWY6u6X2V5MH3eMq7+9xTembYAgI7NanPq7i3p26VZyskkZdKvltkQwmQg/trjMcZdMpJIkiRpYxQW/H/27jsuy3r/4/jrYoqIIiCgKMOFe+JMLXeOsmHLbO9dpzpZp9OyTnvP07DdsZ1ppTly5FbcAycoDjbKkHl/f39cBPZrkQIX3Lyfj0cP7++HG32f0yPkzXVd3y9s/Ay+ud5eD7gZRk4FDz0V5e72Z+Zz1+cbWLk3E4AAXy/m33UqoQHaqVqkPvizK7PjayyFiIiIyIkoLYF3RsDBdfb6hmUQ1tnZTFLt0nIK+c/32/h63QEA/H08efb87ozqFI6Hh24pF6kv/rDMHnd7sYiIiEjtYgwsfwV+vN9eh3aCSZ9CYKSzuaRaHSsq5Zb/xTNvWyoAXh4W/zm7K2f1jMDHS1fiReqbv3xm1rKsHCpuN/YBvIE8Y4weQhEREZGaV3AUZt0Bm7+w170uhTNe0iZPbiyvsITbP13P3K0pALQO8efaIa25oE8rLP17F6m3KrMBVMAvry37q8UEoH91hhIRERH5jeJjsPxVWDDVXge1gavmgn+ws7mk2rhcho9X7WPO5sP8vCudqOCGjO3anLtGxeKp24lF6r3KHM1TzhhjgG8sy3oQmFI9kURERET+n0Mb4P0zoSDbXsddBSMfBt+AP/88qZPyi0rIyi/mtKd/orjUvkHw7J4RPHNed5VYESlXmduMzzlu6QHEAQXVlkhERETkF8bAwsdh0ZP2uvtFcObL4OntbC6pNgXFpfR8ZC6FJS4AOoQH8MUNA2nk+7euwYhIPVCZrwpnHPe6BEjEvtVYREREpHoYY99SvOq/kL3Pnp31BvS4yNlcUm2W7Ezj1Z92sWpvJi4D47s1Z2SnMM7s3kLPxYrI76rMM7NX1EQQEREREYyBNe9A/Af2rcUAsePggg/Bw9PZbFItsvOL+DL+AFNnbQWgS0RjerZqytSzujicTERqu8rcZhwD3AJEH/9+Y8yZ1RdLRERE6p3CHPu52IPx9jpqEEz+Arz9nM0l1aKk1MXapCwueHNF+ezDq/oyuF0zB1OJSF1SmduMvwHeAWYCruqNIyIiIvXSxs/hq6vt135BcNNKaBTqbCapNh+tSOL+bzaXr+8d04EL+rQisKGPg6lEpK6pTJktMMa8VO1JREREpP4pzIGlL8Hip+x1j4vhzFfAw8PZXFLliktdTJ21lbScQn7YfBiAO0a059TYZvRoFehwOhGpiypTZl8sO4rnR6Dwl6ExJr7aUomIiIj7y02DV3pDwRF7ff3PEN7V2UxS5Vwuw5Jd6bw4bwfx+7Lx9LBoF9qIe07vwIhOYU7HE5E6rDJltitwCTCMituMTdlaRERE5O/b+i18don9OqwrXPEdNGjibCapcslZ+Vz45gqSs44B0DMykE+vHYCPl668i8jJq0yZPQ9obYwpqu4wIiIi4ubyM2H2vbBxur0e+wz0vgI8dYaouzDGsOXgUa58bzWpOfZNfUPaN+Oe02PpEN4YTw8dsyMiVaMyf3NsBgKB1GrOIiIiIu7KGFj3Ecy8FYzLvhp71mvQvJvTyaQK5RQUM+bFJeVXYsd0Cef8Pq0YGqvNvESk6lWmzAYC2y3LWs2vn5nV0TwiIiLy1zL3whdXwMF14OENZ70Onc4C7wZOJ5MqcqyolEe/28qM9QfJLSyhc4vGPDC+E/1aBzsdTUTcWGXK7IPVnkJERETcT/pOWPQkbPrcXnv6wi1rIbCVs7mkyrhchg+WJ/LQzK3lsxtPa8M/T+/gXCgRqTf+sswaYxbVRBARERFxEzkp8PNzsGYalBaBTyM4/QnoORksPS/pLlYnZnLxWyspKrX3B71paBtuGdaOBt6eDicTkfriL8usZVk52LsXA/gA3kCeMaZxdQYTERGROujgOnjzNPt1wxA45VYYeKtKrBvZmJzNTZ/Esz/Tfi52VKcwnprYjcCGPg4nE5H6pjJXZgN+eW1ZlgVMAPpXZygRERGpY3IOw8fnweGN9rrPNTDuGWczSZXKyC3kse+38VX8AQBObd+MW4e3o3dUU4eTiUh99bf2wTfGGOAby7IeBKb82Xsty5oGjAdSjTFdymYPAdcAaWVvu88Y833Zx+4FrgJKgVuNMXP+TjYRERFxyMbPYPEzkJ4AEXEw4iGIGex0KqkihSWlPPvjDt5cvAcALw+LaZf3YUj7Zg4nE5H6rjK3GZ9z3NIDiAMKKvF7vwe8Anzw/+bPG2N+9aNay7I6ARcCnYEWwDzLstobY0or8eeIiIiIE5KWwQ/3VFyN7TkZJrzqbCapUqsTM7n47ZUUldjPxU6d0JlJ/aJ0VqyI1AqVuTJ7xnGvS4BE7FuN/5QxZrFlWdGVzDEBmG6MKQT2Wpa1C+gLLK/k54uIiEhNmvcQ/Py8/bpZR7h8FviHOBpJqoYxhjlbUnhv2V5W7MkEYETHUF6+qBd+PtrcSURqj8o8M3tFFf+ZN1uWdSmwBrjTGJMFRAArjntPctlMREREagtjYOETsOETyN4HlgdcswBa9HQ6mVSRzQeOcOGbK8gtLAGgS0RjXrqwJ62bNXI4mYjIb1XmNuP3gduMJshBnQAAIABJREFUMdll66bAs8aYK0/gz3sdmIq9O/JU4Fngb/0+lmVdC1wLEBkZeQIRRERE5G9xlcLB9TDnPthf9rPnDuNh2P0Q2tHZbFIlVidm8th321i/Pxuwdyi+d2xHooMbYmknahGppSpzm3G3X4osgDEmy7KsE/oRrDEm5ZfXlmW9BcwqWx4Ajj9BvWXZ7Pd+jzeBNwHi4uLM771HREREqojLBd/eAus/rpj9KwW8GziXSapMRm4hn67Zz1OzEwDoHdWU64a0ZlTncIeTiYj8tcqUWQ/LspqW3Q6MZVlBlfy837Asq7kx5lDZ8mxgc9nrb4FPLMt6DnsDqHbAqhP5M0RERKSK7JwHM2+FowfAwwsmfwWt+qrIuollu9KZ9PbK8vXTE7txXlyrP/kMEZHapTKl9FlguWVZn5etzwMe+6tPsizrf8BpQIhlWcnAg8BplmX1wL7NOBG4DsAYs8WyrM+ArdibTN2knYxFREQcUpQHq9+GuQ/Y6+jBcO47EBDmbC6pEp+v2c/TcxJIzSkE4O7RsVw+MBp/3xO6ViEi4hjLPjr2L95kH50zrGy5wBiztVpTVVJcXJxZs2aN0zFERETcx5ED8Fp/KDxqr69ZABG9nc0kVSJ+XxavLNjFgu2pAJzeOZyrB8cQFx3kcDIRkT9mWdZaY0zc732sUj+CKyuvtaLAioiISDXISoIlz0L8+/a6y0QY/xw0aOJsLjlpuYUlzNxwkHu/2gRARKAfj53dhdNiQx1OJiJycnQ/iYiISH2WnwkH4uHzy6EoB/xDIe5KGHqv08nkJBUUl/LED9t5b1li+UzPxYqIO1GZFRERqa8Ob4I3BlWsh/4LTv2nc3mkShQUl/LS/J28vWQvRaUuAO4f15Hx3VoQ3kSbd4mI+1CZFRERqU+MgYQfYNbtkFt2Yt7AW6HrRGje3dlsclJKSl0s2J7KtKV7WbEnE4ARHcN4+aKe+Pl4OpxORKTqqcyKiIjUB6UlsOQZWPYyFOXas24XQPcLoc2wP/9cqfVSjxYw+oXFZOUXAzC2azjPntdDJVZE3JrKrIiIiLtb+x7MugOMC7waQI+L7ediW/7u5pBShxSVuJj01grWJGUB0C8miMfP6Up0sD8eHpbD6UREqpfKrIiIiLva+BmseRf2LQMPb+h6Pox5AvyaOp1MTlJJqYuXFuzipfk7AQgN8OXu0bFM7N0Sy1KJFZH6QWVWRETE3WTshvkPw9YZ9rp5d7hoOjRu4WwuqRKrEzO58t3V5BSWAHBWjxY8d34PXYkVkXpHZVZERMRdlBbDkudg4X/stYc33LoOAnUUiztIOVrAdR+uZf3+bADGdWvO8+f3wMfLw+FkIiLOUJkVERFxBwk/wP8urFif9z50mgC65bTOM8ZwxXurWZiQBkAjXy/emNybQe1CHE4mIuIslVkREZG6bN9KmHkbpG2z13FXwchHwLeRs7nkpGXkFvLWkr18tmY/mXlFRAY15NohrZnUN1K3FIuIoDIrIiJSNx09CIuegrXv2uvQznD2G9C8m7O55KRl5xeRmlPI1FlbWbIzHYBBbUN4+7I4GnjrqB0RkV+ozIqIiNQ1G6bD19dVrK/8ESL7OZdHqkxBcSlDn1lYfl7sZQOieHhCF4dTiYjUTiqzIiIidcWK12HVW5C5216P/g/0uhR8A5zNJSctLaeQaz5YU76505WnxNCvdRDDOoQ6nExEpPZSmRUREanNsvfD3Adg33LIOWTPOp4BIx6G4DbOZpOTNn9bCh+uSCrf3KlvdBBD2odw9eDWuqVYROQvqMyKiIjURjmHYc00WPSkvW4YDJ3OggmvanMnN5BwOIdZGw/y8oJdAPRoFcjITmHcNLStw8lEROoOlVkREZHaorQY1rwLq/4LGbsq5uOegz5XOZdLqtTL83fy7Nwd5euPruqnY3ZERE6AyqyIiIjT8jNh1u2wdUbFLHIA9LwEOo6HBk2cyyZVZtnudK58bzUFxS4A3r2iD/1jgvHz0e3EIiInQmVWRETEKcbAkmdgwaP22sMLhtwN/W9QgXUjn67ex7tLE9l+OAeAs3q04M5RsbQKauhwMhGRuk1lVkREpKblHIaNn8HyVyA3xZ6NeQr6XguW5Ww2qTLzt6WwbHcG7/y8F4B+MUFMGdOBnpFNHU4mIuIeVGZFRERq0qYv4Mvjnn/teCac+TL4BTqXSarU7rRc5mw5zFOzEwBo6OPJK5N6MqxDmMPJRETci8qsiIhITdgwHeY+CLmH7fXo/0CPi1Vi3UhJqYvrP1rLvG2p5bN5/ziVtqHafVpEpDqozIqIiFSnnXNh3kOQstledz4HBt0Bzbs5GkuqjjGGB2Zs4cMVSQAENPDiyXO7ERseQJtmKrIiItVFZVZERKQ6ZO6xr8Ru+9ZeRw6As/8LTaOczSVVauaGg7y1ZA8bk48Q5O/DhB4tuH9cJzw99OyziEh1U5kVERGpSsbA93fD6rfstacP3LwamkY7GkuqjstlSM0pZOp3W/lu4yEA2oc14tubB9HAW8fsiIjUFJVZERGRqmAMfHMDbJsJRbn2bOI06HQ2eHg4m02qTMrRAsa8uITMvCIAIgL9mHXLIJr6+zicTESk/lGZFRERORn5mRD/Pqx9H7L2QtQpEBgF454FH50j6i5yC0t44JvNfLXuAAD9Wwdxbq+WTOgRgY+XflghIuIElVkREZETkZsG8x6E9R9XzFr2hUtngKe3c7mkShWVuHhj0W6em7sDAE8Pi6kTujCpX6TDyURERGVWRETk7zAGvr0Z1n1kr739YeAt9g7F3g2czSZVxhjDtxsOctv09eWzawbHcM/pHfDy1JVYEZHaQGVWRESkMlwuWPk6zLmvYnbGi9DrMrC0c607WbornZs/iScrvxiA64a05tbh7fD31bdNIiK1ib4qi4iI/BmXC3bOgW9uhGOZ9qz7RTDhVfDQzrXuZFdqDi/M28mssh2KT23fjLtHx9IloonDyURE5PeozIqIiPye/ath7buw8TNw2Vfo6D7JvhrrpZ1r3UlOQTHvL0vkmR/t52JjQvx58txu9I0JcjiZiIj8GZVZERGR4yXMhq0zYMMn9rpBE+h0lv1cbEg7Z7NJlXtwxmbeX55Uvn7t4l6M7drcwUQiIlJZKrMiIiIuF2z9GjZ9CQnf2TNvf5j0KcQMdjabVDljDP9dvIfP1+xnd1oeAI9M6Mz4bi0I0nmxIiJ1hsqsiIjUX8bA3sX287BHk+2Zd0O45idoFquNndyMMYZDRwo457VlHD5aAMCgtiE8d353QhtrJ2oRkbpGZVZEROonY+DTybB9lr0OaAFXzobASJVYN3S0oJj+/5lPflEpAP1igph2eR/tUCwiUofpK7iIiNQvxsDX18PG6fbaq4FdYsO6gKe3s9mkyiVn5fPED9vLdyjuFxPEZQOj9VysiIgbUJkVEZH6IXU7fHkVZCVBUY49G3AzDLkb/AKdzSZVLvVoAS/O38nHK/cBEBHox+jO4fx7fEcsXXkXEXELKrMiIuLets2EFa9D0jLAQPRgaBwB458DH3+n00kVKypxsWB7Co/M3MrBIwX4enlwy7C23DxMO1GLiLgblVkREXFPGbth9hTY+aO9btELel0CcVc6m0uqzYLtKVz53pry9U1D23D36A4OJhIRkeqkMisiIu6lMBcWPw1LX7DXgZFw8Rf27sTilr7fdIin5ySwN90+ZueS/lFcPTiGqGBdeRcRcWcqsyIi4h4ObYAlz8LWGRWz8z+AThOcyyTVasnONBYlpPH2z3sBGN05jEv6RzOoXYjDyUREpCaozIqISN1mDCx7CeY+YK8bBELPyXDaFPANcDabVIuC4lI+X7Off8/YAoCXh8XMWwbRsXljh5OJiEhNUpkVEZG6yRj47h+w8fOy3YktuHwWRA9yOplUo6/ik/nHZxvK19/fOpiOzQO0Q7GISD2kMisiInVLaTGsmQbbv4O9i8DLD3pMhqH3QZMIp9NJNVm1N5MX5u1g2e4MAG44rQ3nx7UiJkTPxYqI1FcqsyIiUjcYA3sWwodnVczCusK1P4Gnt2OxpHodKyrl1unrmLs1BYDYsAAePbsLfaKDHE4mIiJOU5kVEZHa7ehBWPCofV5s4VF71ucaGDUVvBqAbi91SylHC3j0u23M3HAQgJBGvvz3kt70jmrqcDIREaktVGZFRKT2cblg+0zYMB0Svq+Yd58EA26E8K7OZZNqVeoyvPbTLp6duwMAb0+LW4e145bh7RxOJiIitY3KrIiI1B5HkqHgKLw9HIrz7VnjCBh8J3SdCA2aOJtPqk1BcSmzNh7irs8rNne6ZVhbbhveDi9PDweTiYhIbaUyKyIizivKgx2z4YsrK2ZtR8D45yEw0rlcUiN+3pnO5HdWlq+vGRzDP0/vgLdKrIiI/AmVWRERcU7mXvj8cji0vmI24VXwC4LYMXoe1s1tTM7m7s83kpCSA8Bpsc145MwuRAY3dDiZiIjUBSqzIiJS845lwzsjId1+LpJmHaHnxRA5AFrGOZtNqt2qvZk8PHMLWw7aG3r1jQni3+M60bWlbiMXEZHKU5kVEZGak7IFZt0BmXsgL82+lbjXpdDhDPDQLaXuLjkrn9umr2dtUhZgl9gbTm3D0A6hDicTEZG6SGVWRESq3+4FkLQMts2CtG0Q3g0G3wX9r3c6mdSAzLwirnp/Nev2ZQPg6+XBJ9f0o3eUzooVEZETpzIrIiLV58gBe1fir66DvFTw8IYxT0O/a51OJjXgSH4x//xyA3O2pAAQE+LPtUNac1FfbeolIiInT2VWRESqXmkJzLoN1n1UMTvnbeh2nnOZpMbkFBSzZGc6ry3cxeYDRwlr7MsZ3Vrwr3EdsbSpl4iIVBGVWRERqTrZ+2H5K7Dyv4ABnwAY8yT4BkDsWKfTSQ3ILyqhz2PzKCh2AXDpgCgemdDF4VQiIuKOVGZFRKRqrHoLvr+rYt3rUjj9CfDxdy6T1JiVezK49sO1HDlWDMCYLuHcNTqW1iH69y8iItVDZVZERE6cqxTWvgdLX4TsJHs2/nnoej74NnI0mtSML9cms3R3Ol/FHwBgXLfm9GwVyJWnxODhoVuKRUSk+lRbmbUsaxowHkg1xnQpmwUBnwLRQCJwvjEmy7IfoHkRGAvkA5cbY+KrK5uIiFSBte/Bd3eBy74SR5eJMPQ+CG7jaCypfkUlLjYkZ5OeU8idn28AoHmTBjx5bjeGtG/mcDoREakvqvPK7HvAK8AHx82mAPONMU9YljWlbH0PMAZoV/ZPP+D1sl9FRKS2SVoO398NKZvsdY+LYfiDEBDmbC6pEcYYrv5gDYt3pJXPvrpxIL0imzqYSkRE6qNqK7PGmMWWZUX/v/EE4LSy1+8DC7HL7ATgA2OMAVZYlhVoWVZzY8yh6sonIiJ/06GNMP9h2DXPXrfqD+OehXBt7lMfZOcXcf5/l7M7LY9SlyHY34eXJ/Uk2N+X2PAAp+OJiEg9VNPPzIYdV1APA7/8GD8C2H/c+5LLZiqzIiK1wZx/2bsUA/gFwdlvQPvRzmaSGrHt0FGen7uDudtSMAb6RgcRGx7AfWM74ufj6XQ8ERGpxxzbAMoYYyzLMn/38yzLuha4FiAyUoeui4hUq58eh8VPgykFDy+Y9Cm0HgoeKjHu7lhRKS/O38kbi3YD9jOxZ3ZvwZQxHXRWrIiI1Ao1XWZTfrl92LKs5kBq2fwA0Oq497Usm/2GMeZN4E2AuLi4v12GRUTkLxgDi560j9rJTwcPbzj1n9DvevALdDqdVLOC4lLW7cvmordWlM+ev6A7Z/ds6WAqERGR36rpMvstcBnwRNmvM46b32xZ1nTsjZ+O6HlZEZEalp8JSctgyTNwcJ09ix0H456Bxi2czSbVzhjDawt38/SchPLZBXGtmHpWF3y8PBxMJiIi8vuq82ie/2Fv9hRiWVYy8CB2if3MsqyrgCTg/LK3f499LM8u7KN5rqiuXCIi8juy98M7oyDnoL2O6A2Xfwfefs7mkmp35Fgx2w4d5aaP48nIKwLgwj6tOL9PK+1QLCIitVp17mZ80R98aPjvvNcAN1VXFhER+R0lRZC0FD6/HAqy7Vnr02DM09CsvYPBpKas2JPBJe+spLjUfmpneIdQXr24Fw289Uy0iIjUfo5tACUiIg7aMB2+uwuKcux17DjofDZ0OUebO7k5YwzTliby9pI9HDpSAMBNQ9swvGOYrsSKiEidojIrIlJfFBfA4Y2w4FHYu8ie9brULrFthjmbTWrEnrRcbvw4nu2H7R9iDO8Qyl2jY+nYvLHDyURERP4+lVkRkfqg4Ai8PRLSyzb3CesKF30CgTrirD44WlDMVe+tZnViFgDdWjbhvSv6EuTv43AyERGRE6cyKyLizkoK4ePzIPFn+6zYiDgY9ShEDXA6mdSArLwiHv9hG5+tSQagQ3gAtwxrx7huzR1OJiIicvJUZkVE3FFRHqx5F1a/BVmJ0KQV9L4MBv1Dz8TWA8eKSlmyM41rP1wLQICvF/eM6cCkvpF4eFgOpxMREakaKrMiIu6kuACy98Ebg6C00J61HgqTvwIPnRVaH3yz7gC3f7q+fH3f2A5cO6SNg4lERESqh8qsiIi7KMqD/7SoWMeOhXPeBJ9GYOlqnLv7Kj6Z6av2syoxE4ArT4nhvLiW2txJRETclsqsiEhdd3A9zLwNDpVdjWvVHwbdAbGnO5tLasSM9QeYOmsr6blFAHSNaMJrF/eiVVBDh5OJiIhUL5VZEZG6Ki0Bfn4eNvzPXod2hrgrIO5KPRfr5owxrNybybtL9zJnSwoAZ/eM4JEJnQlo4O1wOhERkZqhMisiUpcYA9u/g+/vhpyD9szygCt+gMj+zmaTGlHqMkx+eyXL92QAENLIlw+v6qvbiUVEpN5RmRURqSu+vQXiP6hYdzwDekyGtiPAU1/O3d3hIwX86+tNzN+eCtiPQX970yC6tmzicDIRERFn6LsfEZHazBiYPcW+pXjPT/bstHuhw3gI7+JsNqkRPyWk8v6yRBYmpAHQKsiPUZ3CuWtULH4+up1cRETqL5VZEZHaKD8TDsTD2ndh+yx7Ft4VLvwEAiOdzSY1Yn9mPl+sTebF+TsBiAnx5+7RsYzt2tzhZCIiIrWDyqyISG2RvgsWTIVjmbB38a8/9kCWzomtJ0pdhuW7M5j8zsry2f+u6c+ANsEOphIREal9VGZFRJzkcsGuefDNDZCfbs98AiByIEQPgnajoElLFdl6YumudC5+u6LEPnRGJ86La4W/r/66FhER+f/0t6OIiFOykuCtYRUlNnYc9LpU58PWQ6/+tIsZ6w+wIyUXgCtOiebcXi3pEqHNnURERP6IyqyISE3KSYHNX8CO2RW3EkcNgtGPQouezmaTGuVyGb7ffIh3ft7Lun3Z+Hh6MK5rc05pG8KkfnouWkRE5K+ozIqI1IT0nfDTY7Dl64pZ1CnQ5Rzoc7VzucQRR/KLufPz9czbZh+z4+PpwbJ7hxHSyNfhZCIiInWHyqyISHVyueyjdVb91143aQWDbocOZ0BAmLPZpMYdyD7GP7/YwNJdGeWzZVOGEd64AR4eloPJRERE6h6VWRGR6lCYC/MfhlVvVswu+Bjanw6e+tJb3xSWlPLaT7t5acFOjIGwxr5cf2obJvSIIMjfx+l4IiIidZK+oxIRqUq75sO6j2D3fCg4Ys8G3gqD7wS/QGeziSO+23iImz6JL19PndCZSwZEOxdIRETETajMioicLFcp/PQf2PgZHNlnzxqFQf+LYdRjOlannvpweSIvzt9Jem4RADcPbcstw9vi6+XpbDARERE3oTIrInKiSorgm+th85cVs45nwOC7oEUP53KJo3al5nLzJ/FsP5wDwJgu4dwxsj3twwIcTiYiIuJeVGZFRP6ugiOw+m2Y/4i9btwS2gyFsU+Dt5+z2cQxe9PzeHTWVuZvt3cojg0L4P0r+xLepIHDyURERNyTyqyISGWVlsCSZ2Dh4xWzQXfAsAd0K3E9VlLq4v5vNjN99X4Amvh58/TEbozqHO5wMhEREfemMisiUhnbZsKnkyvWp06BATdCgybOZRJHHTlWzH1fb+K7jYcAaOTrxbPnd2dkxzAdsyMiIlIDVGZFRP7MgXj45HzIS7PX3S6EMU9qZ+J67GhBMQ99u4Wv4g8AENDAiytPieGSAVGENPJ1OJ2IiEj9oTIrIvJ7Fj8N6/8HmbvBtwn0vxF6Toawzk4nE4cUlpTyzs97eWp2AgChAb5c1DeSO0a2dziZiIhI/aQyKyLyi5zD8M0NkL0PMnZBeDfoeh7EjoEu5zqdThxyJL+YVxfu4s3Fe8pnF8S14smJ3RxMJSIiIiqzIlK/GQMLHoX0HbBjDpQW2iW23Sg463XwD3E6oTikoLiUS6etYtXezPLZP0+P5YqBMfj56KxYERERp6nMikj9VFoMi56Eg+th11zw8oPgNtB1Igy+0+l04qC8whLeXLyHlxfsxGUgItCPawbHcGHfSBp4q8SKiIjUFiqzIlK/lJbAgTUw+144GA9+QRDSHq6cAw2DnE4nDtqVmsNTsxP4cWtK+ezmoW25c1R7LEu7E4uIiNQ2KrMiUn/kpsLX18HuBfY6ojdcPR9UVOotl8uwbn8Wby7ew5wtFSX2X2M7cnH/SBr66K9JERGR2kp/S4uI+3O54L2xsG+5vW7QBCZ/BS3jnM0ljikoLmXmhoM8PHMruYUl5fNpl8cxNDZUV2JFRETqAJVZEXFfGbvhu3/AvhVQUgCNwmDov6DbBeDdwOl04oD9mfnc/81mFu1IK5+N6BjKNYNb0yMyEF8vPRMrIiJSV6jMioh7yUuHpKWw+Bk4vNGeNY2ByAEw4VXw8HA2n9S45Kx8fkpIY/qqfWw5eBQAHy8P7hzZnrN6RhDWWD/YEBERqYtUZkXEfWz+Cr64omIdORAG3gwdxjmXSRyTerSAj1bu46X5O8tnfaODuGxgNKd3CcfTQ7cSi4iI1GUqsyJS9+34EX78l31WLMCYp6D1UGjW3tlc4ojMvCLu+HT9r24l/s/ZXTm9SzhB/j4OJhMREZGqpDIrInWTMbBhOqRshuWvQKNwaDsSht5r71Is9c68rSn888uNZOYVAdC4gRdTxnRkRKdQQgN0K7GIiIi7UZkVkbqnMAe+uws2Tq+Ynfs2xAx2LpM4whjDNR+sZfvhoyRnHQPg/LiWxEUFMbF3Szx0K7GIiIjbUpkVkbpl5m2w9j37tW8TuHk1+AWCl6+jsaRmrU3KZOqsbexOzSWnsIQmft6c26sllw2MolvLQKfjiYiISA1QmRWR2q8wB2ZPgdRtcGAtNG4JA26EHpPAr6nT6aQGfbp6H7M2HmLJznQABrYJJqSRL8+d3x0vT+1ULSIiUp+ozIpI7ZWWAMtehnUf2mufAGjVHyZ9al+NlXpj2s97+WJtMlsP2UfrdGremGuHtOasnhEOJxMRERGnqMyKSO2TmwpfXw+759trnwDodx0Mux8sPQNZX5S6DM/NTeDzNcmk5hQC0DMykKcndqdtaCOH04mIiIjTVGZFpPbIy4A59/16Y6dJn0H70c5lkhq3eEcaH65IYu7WlPLZ8A6hvHBhDwIaeDuYTERERGoTlVkRcd6RA/DeWMhKrJid+TL0utSxSFKzsvOLeObHBFKPFvJjWYmNCPRjbNdw7h3TUbsSi4iIyG+ozIqIczL3wvJXYfVb9rrdKPus2LgrwFNX4OqDHSk5fLv+IK8t3IXLgLenRWiALy9d1JP+rYOdjiciIiK1mMqsiNS84mOQtBQ+mggY8PCGwf+Aofc5nUxqyILtKczacIiv1h0A7BI7pnM4r1zUE0vPRYuIiEglqMyKSM3auwTeH1+xHnwnDH/AuTxSY4wxzN2awscr97FoRxoAAb5e3DysLded2sbhdCIiIlLXqMyKSM3YORcWPw37V9rrQf+AzmdD827O5pJql5yVz+rETO75chNFJS4AGvp4MuOmU2gb2khXYkVEROSEqMyKSPVKmA37lsHSF+11SHsY/Ti0G+FsLqlWLpdh04EjPD9vBwsT0srnY7qEc/foWGJC/FViRURE5KSozIpI9SjMgWUvw6In7bXlCVfPg4hezuaSalVU4uKTlUk89v02iksNAE0bevPv8Z3oHdWUqGB/hxOKiIiIu1CZFZGq9+XVsOnzivVNq6FpFHj5OpdJqlVWXhFTZ20t39AJYHy35kzuH6VdiUVERKRaqMyKSNVZ8BisfRfy0sA/FAbdDh3PgMBIp5NJNSh1GT5YnsiC7aks2ZkO2M/CXtwvkhtOa0uQv4+zAUVERMStqcyKyMnJz4TdC2D7LNjyNXg3hPZj4Lx3wdvP6XRSDQ4dOca8rSk8O3cH2fnFALQNbcSFfVpx9eDWDqcTERGR+kJlVkROXHEBPNcRSgrsdYNAuH0TNGjsbC6pFgezj/FTQir/+npz+axbyyZMv7Y/DX3014mIiIjULH33ISJ/38F1MO30ihLb7UIYcrf9XKynt7PZpErlFBSzP/MYL87fwZwtKeXz24a345IBUYQ00nPQIiIi4gyVWRGpvKRlEP8hbPjEXne/CMI6w4CbQcesuJWcgmI2JR9h0tsry2deHhZPTexGv9bBRATqFnIRERFxlsqsiPy13DRI+B5m3mqvm8bAoDug92XO5pIql51fxDmvL2NPWl757OyeEYzv1pyhsaF4eOiHFiIiIlI7OFJmLctKBHKAUqDEGBNnWVYQ8CkQDSQC5xtjspzIJyJl8jNh9hTY+GnFbOI06HKuc5mkWuxKzeXmT+LZfjgHgIhAP644JZoBbYLp3KKJw+lEREREfsvJK7NDjTHpx62nAPONMU9YljWlbH2PM9FE6rm178FPj0PuYXvt1xRGToU2w6BJhKPRpOoUFJcyZ8thXpy3kz3p9pXYmBB/bji1DRN7t9RVWBEREanVatPzupbTAAAcqElEQVRtxhOA08pevw8sRGVWpGZt/hJWvwNJS+11l3OhZR/odSn4+DubTarUxyuTfrUr8Sltg7m4XxRjuzZ3MJWIiIhI5TlVZg3wo2VZBvivMeZNIMwYc6js44eBMIeyidQ/KVvguzth33J7HdwWJn0GwW2czSVVKiuviDs+W8/ChLTy2bm9WnL9qa1pFxbgYDIRERGRv8+pMjvIGHPAsqxQYK5lWduP/6AxxpQV3d+wLOta4FqAyMjI6k8q4s6OJMObp0FeWbkJaAFX/QiBrRyNJVXH5TJ8t+kQz/yYQFJGPgDNAnw5p2cE153ahiB/H4cTioiIiJwYR8qsMeZA2a+plmV9DfQFUizLam6MOWRZVnMg9Q8+903gTYC4uLjfLbwi8hcOroc170D8B/Y6ord9TmzsGGdzSZUxxjBtaSJP/rCdolIXAEPaN6NHq0BuH95Oz8OKiIhInVfjZdayLH/AwxiTU/Z6FPAI8C1wGfBE2a8zajqbiNs7lg07ZsPX19lrnwAY+TD0ucrZXFJl8gpLeHjmFj5bk1w+u6hvK64e3Jo2zRo5mExERESkajlxZTYM+NqyrF/+/E+MMbMty1oNfGZZ1lVAEnC+A9lE3JMxcHgjTBsDxWXnh45/AeKucDaXVJmM3EJGPb+YjLyi8tntI9oxuX8UIY18HUwmIiIiUj1qvMwaY/YA3X9nngEMr+k8Im7v4Hp4d2xFiR14K/S9Vs/FuomVezL494zN7EjJBaB7yyaM6hzOhX1aEawSKyIiIm6sNh3NIyJVac8imHEzHNlnrzuMh/43QvQpzuaSk1ZQXMojs7ayMTmbzQeOAjCsQyintm/GZQOjnQ0nIiIiUkNUZkXczd4lsPQF2DXPXseOta/EthnqbC45aQmHc3hq9nbmb6/YH69vdBB3jmpPv9bBDiYTERERqXkqsyLuInkNJC2Duf+21yHtYcJr0KqPs7nkpC3ekcbLC3ayOjELgIhAPyb2bsktw9ri5enhcDoRERERZ6jMitR1xti7E2/8tGI26XNoP8q5THLSjDE8NSeBn7ansv1wDgBtmvlz39iODO8Y5nA6EREREeepzIrUVdn74dPJkLoNSgvB8oTLvoWwzuDX1Ol0coK2Hz7Kp6v3s3x3RnmJ7RcTxMMTOtMhvLHD6URERERqD5VZkbpm8TOw6XNI226vg9pApwlw2hTw0u61dZExhmW7M5i18SD/W7W/fD6obQhvXxZHA29PB9OJiIiI1E4qsyJ1gcsFs++BhNkVuxN3Ogs6ngFdzgX73GapY4pKXKzfn82D325h2yF7V2IfLw9eurAHIzqG6XlYERERkT+hMitSmxUcgf2r4fu7IGuvPWs7AsY9B02jnM0mJ6yguJQX5u1k2tK9FJW4AOgQHsALF/bQrcQiIiIilaQyK1LbGAN7F8HmryD+/Yp5SCxcv0S3Etdha5MymbnhEO8tSyyfndG9BVcNiqFHq0DngomIiIjUQSqzIrVF8TGYPxXWvgvF+RXzATdD94vsjZ10O3GdU1BcSnxSFq8u3MXSXRnl82uHtObOUe3x9dLzsCIiIiInQmVWxGmuUljxGqx+p+JW4i4T4ZTbILyrCmwdlVNQzPbDOVz/4Voy8orK51/eMIDuLQP1PKyIiIjISVKZFXGKywWJS+Dzy+BYlj2Lu9J+HlYFts5anZjJD5sOM23p3vLZiI5h3Dq8LV1aNMHDQ/9uRURERKqCyqxITXO5YNETsOQ5cBXbszbDYeI7Oh+2jjLG8MPmw7yxaDcbk4+Uz28f0Y7B7ZrRKzIQSz+gEBEREalSKrMiNcFVCgumwuppUFhRdug5GfpcDS16OpdNTlhRiYsHZmxm+uqKs2FbNGnAs+f3oG9MEJ66CisiIiJSbVRmRarTjh9h3kOQuqVi1udq+wrsafeChzb/qYtW7c3k/m82sSMlF4Bgfx+GtG/GP0+PpXkTP4fTiYiIiNQPKrMi1WHpS7DlazgYb6+jTrE3cxp2P/gGOJtNTtgjM7ey6UA2qxPtZ5z7tw6iR6um3HN6rG4jFhEREalhKrMiVcXlgqSl8PPzsHu+PYsZAsMfgpa9HY0mJy4pI49nftzB/sx81u/PpomfN6e0DeaqQTEM6xDmdDwRERGRektlVqQqFB+DV/tC9j577d8Mbl6tDZ3qKGMMC3ekMX9bCh+v3Icx0LShN31jgnjvij409NGXThERERGn6TsykZOx+ydY/TZsn2Wvw7vB+BfsDZ08dI5oXZNbWMLSXen85/ttJGXkA+Dn7cmtw9txw2ltHE4nIiIiIsdTmRX5u44egj0/wfxHIOeQPWsYAqfcCv1vAk/9Z1XX7MvI54X5O/gq/kD5rE0zf169uBcdwhs7mExERERE/oi+6xaprOx98PZIyD1cMWs7EkY8BOFdnEolJ+Gr+GTWJmXx8Ur79vBGvl6M6BjKLcPb0aZZI4fTiYiIiMifUZkV+SspW2DhE7DtW3vd+jToeh60HgpNIpxMJifoszX7WbornRnrDwIQ0siXc3tHMOX0DtqVWERERKSOUJkV+SNFefDDPbDuQ3vt3RDGPQs9JjmbS05Iqcvw3rJEvt90iLVJ9tE6Qf4+fHbdANqG6iqsiIiISF2jMivy/x1cB/EfwJpp9tq7IVz4MbQZ5mwuOSHHikp5/Idt/LwrnT1peQAMaB3MtMv74Ofj6XA6ERERETlRKrMix1vwGCx+qmI99H4YeAt4N3Auk/xtxhg+WbWPw0cKeHnBLgB8PD0Y0r4Zr13ci0a++tInIiIiUtfpOzoRY2DNO/D9P8GU2rNLvobIgSqxdUxGbiE/bD7MnC2HWbIzHQAvD4srB8Vw39iODqcTERERkaqkMiv127aZ8M2NUHgUmsZA57Ng8F3gq2co65I1iZkkZx3j9k/Xl898PD3Y/PBofLx03q+IiIiIO1KZlfrp4Dr4YQrsX2GvY8fB6Y9D0yhnc8nfsmhHGkt3pfPm4j3ls/N6t2TKmA4ENPBWkRURERFxYyqzUr8U5sIn50PSUnvdNBou+QaCYhyNJZWXU1DMjPUH+TI+mXX7sgGICPTjjcm9CWzoTaughg4nFBEREZGaoDIr9UNpMSx5FhY+bq8bt4Rz34Kogc7mkkpbm5TFtkNHeeKH7eQWlgAQGxbAqxf3pG1ogMPpRERERKSmqcyK+9s5Fz6eWLEe/TgMuNG5PFJpJaUuNiRn88ycHSzfk1E+n9w/kn+MjKVpQ28sy3IwoYiIiIg4RWVW3JPLBavfgtlTwLjsWfsxcNZr0DDI2Wzylw5kH+OjFUm8vnB3+ayRrxfvXtGHqOCGhAZol2kRERGR+k5lVtxP0nL46lo4ss9e97zEPiu2WayzueQvLdmZxvNzdxBf9iyshwXXndqGYR1C6R3ZFA8PXYUVEREREZvKrLiP7d/DgkchdYu9DmkPF3+hHYpruSPHinno2y2s2JPBoSMFAHSJaMz1p7ZhRMcwGnh7OpxQRERERGojlVmp+/atgFVvweYv7HVEbxjxMMQMdjaX/C6Xy5CYkUdeYSlTZ21lVWImYJ8LO6pTGPeM6UCbZjrnV0RERET+nMqs1F3HsmHhE7DydXvt3wyunq8rsbWUMYY96Xnc9HE82w/nlM/DGvtyx4j2XNg30sF0IiIiIlLXqMxK3WMMfHsLrPuwYnbpDIg5FbSzba2zYX82u1JzeWDGZvKKSgF7M6fHz+lKsL8PA9uGOJxQREREROoilVmpO0oK4atrYOuMitmYp6HXJeDt51wu+ZV9GfnsTM3hp4RUlu/OYHdaXvnHRncO4+yeEQzrEIaPl4eDKUVERESkrlOZldqvtASWvQh7FsHeRdAoHLpOhGH/Bm8d0eK0guJS5m9LpcTl4p2f97Ix+Uj5xzw9LLpGNOEfI9vTLqwRLZs2dDCpiIiIiLgTlVmp3Q5vhumTIDvJXnefBGe/7mwmAWD+thRWJ2ax9dBRFu9IK5+HNPLl3+M70iG8MW1DG+Gp43REREREpBqozErttOJ1mD2lYt1uFJz3Pvjoyp5TjDHM25bKxyuT2JmSy4HsYwD4eHlwYZ9WXDOkNV4eFpFBDbH07LKIiIiIVDOVWaldFj4J22fB4Y2ABf2uhw5jIXqwNndyQH5RCT/vTOeZHxPYk5ZHicsA0LlFYzo2b8z94zoSHeLvcEoRERERqY9UZqV22LcCvr+7rMRinxU78V0ds+OA4lIXby7ew4LtqaxNyiqfD2obQlRwQy4dEE1seICDCUVEREREVGbFaXkZMOt22PatvY7oDZfN0u3ENSi3sITNB47w45YUFiaksie9YvfhdqGNOLtXBGO6NCdGV2BFREREpBZRmRXnzL4PVrxqv7Y84Kp50LK3s5nqkW83HGRRQhpfxif/aj64XQijOoczuV+knn0VERERkVpLZVZqVn4mzP03HNwAKZvBrymcOgV6X6azYmvAloNHuPerTRQWu0hIyQGgVZAfvSKbcumAKLq3DMTLU+e/ioiIiEjtpzIrNcMYSPwZPjoXSgsBC7qcA2OeBv9gp9O5rcT0PBbtSOONRbs5dKSgfD68QyhtQv25c1QsbZo1cjChiIiIiMiJUZmV6ncsG/47pOKs2H7Xw5gnnc3kxjbsz+aLtcks2J5afnwOQP/WQfSNCaZ7yyYM7xjmYEIRERERkZOnMivVpzAXPrsEdi+w121HwogHIbyrs7ncUH5RCbdPX09CSg5JGfkAeHpYDGgdzCUDohjSvhmNfPWfu4iIiIi4D313K1XPVQqr3oKFj0NBNjSJhAE3Qb/rdFZsFSksKWV3ah5Pz9nOTwlp5fOIQD9Gdw7jsgHR9G8djIeH/v8WEREREfekMitVK2E2fH2dXWIB+lwD455xNpMbyC0sYf62FApLXHywPJHNB46Wf8zb0+KG09oSHdyQc3q1dC6kiIiIiEgNUpmVk2cM7FkIs++FtG32LHowXPgxNGjiaLS6yhhDQkoO7y1NJH5fFjtScn/18TbN/BnTpTk9IwMZGhuqK7AiIiIiUu+ozMqJKy6AZS/D+o8ha689a9kXzngBwjo7m60Oycor4uCRYxzJL+azNftJzMhn/f7sX71ndOcwekY2ZUyXcHy9PAlv0sChtCIiIiIitYPKrPw9rlLY+CmsfR/2r6iYtx4Kox6F8C7OZasDMnILySss5Yv4ZPZl5LEjJZeth47+6j1N/LyJi2pK35gg+sYE0b91MA28PR1KLCIiIiJSO6nMSuVkJcHyV2H122BK7Vl4N+h0Jgy8Dbx8nM1XC+UXlbA6MYslO9IwwA+bDnHwuLNeAaKDG9InuimjO4fTKqghYY0b0L1lEyxtlCUiIiIi8qdUZuWPuVyQuhX2LoI599mzBk2g2wXQ/wYIau1svlpkV2oOx4pcLN6ZxtJd6exIySU9t/BX7wnw9SIuqiln94ogoIE3IzuG4eejK64iIiIiIidCZVZ+q7gAfn7evhJblFMxH3yn/Y+Pv3PZaoGC4lKy84v5btMhNiZnsz8zn/h9v37GtVPzxvSNaUpcVBD9WgfRqXljXW0VEREREalCKrNiK8qHNe/A9u9h37KKeexY6H0FRA8Cn4bO5XOIy2XYnZbL0l3pJKTksiYxk52pv95ZOCbEnx6tArn+1Nb4ensysE0wvl664ioiIiIiUp1UZuur0hL7FuJDG+CnxyDnUMXHwrpC+1Ew4GZoGORcxhqWnlvIoewC1u3P4odNhykudbEmKetX7wnw9WJEx1DahwUQGdSQ07uEE9hQzwuLiIiIiNS0WldmLcs6HXgR8ATeNsY84XAk92CMXVjXTIPMPbBzLhQet4tuh/H21deel4BvI+dyVqMjx4rJLyrBGFi1N5OiEhfbDh8lPbeIo8eKWbQj7Vfv7xLRmD7RTekdFURcVFN6RTUlyF/FVURERESkNqhVZdayLE/gVWAkkAystizrW2PMVmeT1TGuUjAuyM+wr7wm/gyr3oKSY/bHPbygaTRED4Zel0CzDhAU42jkqmKMocRlSDlawI6UHPZl5JOUmU98UhYbko/84ee1DvGnTTN/JvWLIjq4IXFRQTRp6F2DyUVERERE5O+oVWUW6AvsMsbsAbAsazowAVCZ/SMlhfbtwnkZsPQFKMqFg+t+/709L4GoU6Dz2eDdoGZzVqH9mflk5RexLzOfpIx8Sl2G5bszcBnDun3ZFJW6fvM5Tfy86di8Mef2iqCRrxceHhYDWgfj4WERFuCLl6eHA/9LRERERETkRNW2MhsB7D9unQz0cyjLSVvz3LlYrpKT/n0sDO3z12FhfvMxf1fOr9bJvm045teNXX7dKbZ8yPYK4ZBvNNleoRzNC7J/LLC1lv9swMDKvZkU/04pdbkMOYW///9pbFgAvaICiQ72p2VTP1oFNSQ6+P/au/MYu8oyjuPfX1sWhUIFDFEKUrBKkEghiLgjRsAlYgwqahSVpC5gXKNgjMblD4yJuKISRFFRxCqKgixBjEoiVKTQFiRUllhciiwFNIKFxz/OO+U6dIq2nbn3zHw/yeTe8573nPPeuU/uO8993/POduy+02OdHixJkiRNM6OWzD6qJIuBxQB77LHHkFuzcTvdt4rZbH4yC3AHO7Bq1l6szdz/3jEb7sv23DB7IbdnZ/44u/3v13+3/Q8A/xyrfA99seNj5vCUXeey6w6PHEGuKhbtMY8dtt1qfZ1ZwdFVSZIkaQYZtWT2NmD3ge35rWy9qjoNOA3goIMOeuRQ5QjZ66PXbNHzPWmLnk2SJEmS+mvUhrKWAguTLEiyNXAMcN6Q2yRJkiRJGjEjNTJbVeuSnABcRPevec6oqpVDbpYkSZIkacSMVDILUFUXABcMux2SJEmSpNE1atOMJUmSJEl6VCazkiRJkqTeMZmVJEmSJPWOyawkSZIkqXdMZiVJkiRJvWMyK0mSJEnqHZNZSZIkSVLvmMxKkiRJknrHZFaSJEmS1Dsms5IkSZKk3jGZlSRJkiT1jsmsJEmSJKl3TGYlSZIkSb1jMitJkiRJ6h2TWUmSJElS75jMSpIkSZJ6x2RWkiRJktQ7JrOSJEmSpN4xmZUkSZIk9Y7JrCRJkiSpd1JVw27DJktyO3DrsNvxKHYB/j7sRmgkGRuaiLGhjTE+NBFjQxMxNrQxox4fT6qqx29oR6+T2T5I8ruqOmjY7dDoMTY0EWNDG2N8aCLGhiZibGhj+hwfTjOWJEmSJPWOyawkSZIkqXdMZiffacNugEaWsaGJGBvaGONDEzE2NBFjQxvT2/jwnllJkiRJUu84MitJkiRJ6h2T2UmS5MgkNyRZleTEYbdHUyPJGUnWJFkxULZTkkuS3NgeH9fKk+QLLUauTXLgwDHHtvo3Jjl2GK9FW1aS3ZNcluS6JCuTvLuVGx8zXJJtk1yZ5JoWGx9v5QuSXNFi4PtJtm7l27TtVW3/ngPnOqmV35DkiOG8Im1pSWYnuTrJz9q2sSEAktySZHmSZUl+18rsV0SSeUmWJPlDkuuTPGs6xobJ7CRIMhv4MvASYF/gdUn2HW6rNEW+CRw5ruxE4NKqWghc2rahi4+F7Wcx8BXoOiHgY8AzgYOBj4192KjX1gHvr6p9gUOA49vngvGh+4HDqmp/YBFwZJJDgE8Dp1TVk4G7gONa/eOAu1r5Ka0eLZ6OAZ5G9zl0auuP1H/vBq4f2DY2NOiFVbVo4F+r2K8I4PPAhVW1D7A/3WfItIsNk9nJcTCwqqpuqqoHgLOBo4bcJk2BqvoVcOe44qOAM9vzM4FXDpR/qzq/BeYleQJwBHBJVd1ZVXcBl/DIBFk9U1V/qarft+f30nUqu2F8zHjtPb6vbW7Vfgo4DFjSysfHxljMLAFelCSt/Oyqur+qbgZW0fVH6rEk84GXAae37WBsaOPsV2a4JDsCzwe+DlBVD1TV3UzD2DCZnRy7AX8a2F7dyjQz7VpVf2nP/wrs2p5PFCfGzzTXpv4dAFyB8SHWTyNdBqyh+2Phj8DdVbWuVRl8n9fHQNu/FtgZY2O6+hzwQeChtr0zxoYeVsDFSa5KsriV2a9oAXA78I12i8LpSbZjGsaGyaw0hapbPtwlxGewJNsDPwTeU1X3DO4zPmauqnqwqhYB8+lGzPYZcpM0ApK8HFhTVVcNuy0aWc+tqgPppoken+T5gzvtV2asOcCBwFeq6gDgHzw8pRiYPrFhMjs5bgN2H9ie38o0M/2tTdWgPa5p5RPFifEzTSXZii6RPauqftSKjQ+t16aBXQY8i26a15y2a/B9Xh8Dbf+OwB0YG9PRc4BXJLmF7palw+jugzM2BEBV3dYe1wDn0n0ZZr+i1cDqqrqibS+hS26nXWyYzE6OpcDCttrg1nSLLpw35DZpeM4DxlZ/Oxb4yUD5m9oKcocAa9vUj4uAw5M8rt1kf3grU4+1+9a+DlxfVZ8d2GV8zHBJHp9kXnv+GODFdPdUXwYc3aqNj42xmDka+EX7hv084Ji2ou0CuoU8rpyaV6HJUFUnVdX8qtqT7m+JX1TVGzA2BCTZLsncsed0/cEK7FdmvKr6K/CnJE9tRS8CrmMaxsacR6+i/1dVrUtyAt2bPRs4o6pWDrlZmgJJvgccCuySZDXdCnAnA+ckOQ64FXhNq34B8FK6hTj+CbwFoKruTPJJui9FAD5RVeMXlVL/PAd4I7C83RsJ8GGMD8ETgDPb6rKzgHOq6mdJrgPOTvIp4GraQh7t8dtJVtEtOHcMQFWtTHIO3R8s64Djq+rBKX4tmhofwthQd7/jud13pcwBvltVFyZZiv2K4F3AWW1g7Sa693sW0yw20n1hJ0mSJElSfzjNWJIkSZLUOyazkiRJkqTeMZmVJEmSJPWOyawkSZIkqXdMZiVJkiRJvWMyK0nSZkoyL8k7B7afmGTJFF17zySvn4prSZI0SkxmJUnafPOA9clsVf25qo6eomvvCZjMSpJmHJNZSZI238nA3kmWJflMGy1dAZDkzUl+nOSSJLckOSHJ+5JcneS3SXZq9fZOcmGSq5L8Osk+4y+S5AXtGsva8XPbtZ/Xyt6bZHZrw9Ik1yZ5Wzv20CS/SnJ+khuSfDXJrFb/m0lWJFme5L1T+HuTJGmTzRl2AyRJmgZOBParqkXQTf0dt38/4ABgW2AV8KGqOiDJKcCbgM8BpwFvr6obkzwTOBU4bNx5PgAcX1WXJ9ke+Fe79geq6uXt2ouBtVX1jCTbAJcnubgdfzCwL3ArcCHwKuBmYLeq2q8dP29L/EIkSZpsJrOSJE2+y6rqXuDeJGuBn7by5cDTW2L6bOAHScaO2WYD57kc+GySs4AfVdXqgfpjDm/nHJvmvCOwEHgAuLKqbgJI8j3gucClwF5JvgicD1w8/oSSJI0ik1lJkibf/QPPHxrYfoiuL54F3D02sjuRqjo5yfnAS+lGXI/YQLUA76qqi/6rMDkUqEeesu5Ksj9wBPB24DXAW/+nVyVJ0hB5z6wkSZvvXmDuph5cVfcANyd5NUA6+4+vl2TvqlpeVZ8GlgL7bODaFwHvSLJVO+YpSbZr+w5OsiDJLOC1wG+S7ALMqqofAh8BDtzU1yFJ0lQymZUkaTNV1R10I6UrknxmE0/zBuC4JNcAK4GjNlDnPe0a1wL/Bn4OXAs8mOSatnjT6cB1wO/bIlRf4+GZWEuBLwHX090rey6wG/DLJMuA7wAnbWL7JUmaUqkaP+NIkiRNN22a8fqFoiRJ6jtHZiVJkiRJvePIrCRJkiSpdxyZlSRJkiT1jsmsJEmSJKl3TGYlSZIkSb1jMitJkiRJ6h2TWUmSJElS75jMSpIkSZJ65z/gN2jlK32nwwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(16, 10))\n",
    "\n",
    "for i in range(len(dyna_q_params['methods'])):\n",
    "    plt.plot(rewards[i, :], label=dyna_q_params['methods'][i])\n",
    "\n",
    "plt.xlabel('time steps')\n",
    "plt.ylabel('cumulative reward')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "把policy畫出來看看"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "ACTION_FLAGS=['↑', '↓', '←', '→'] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib.table import Table\n",
    "\n",
    "def draw_policy(action_value_function, maze):\n",
    "    \"\"\"\n",
    "    繪製policy\n",
    "    因為自己被很多q_value搞混，所以試著以不同的名稱來命名\n",
    "    \n",
    "    action_value_function: action-value function\n",
    "    maze: Maze object\n",
    "    \"\"\"\n",
    "    fig, ax = plt.subplots()\n",
    "    ax.set_axis_off() # 隱藏圖表的軸線\n",
    "    tb = Table(ax) # new一個Table object\n",
    "    \n",
    "    # 設置表格大小\n",
    "    nrows, ncols = action_value_function.shape[0: 2]\n",
    "    width = 1.0 / ncols\n",
    "    height = 1.0 / nrows\n",
    "    \n",
    "    # loop action-value function\n",
    "    for i in range(nrows):\n",
    "        for j in range(ncols):\n",
    "            # 取得最佳的action index\n",
    "            best_action = np.argmax(action_value_function[i, j, :])\n",
    "            \n",
    "            action_flag = ACTION_FLAGS[best_action]\n",
    "            \n",
    "            if [i, j] == maze.START_STATE:\n",
    "                action_flag = 'S'\n",
    "            elif [i, j] == maze.GOAL_STATE:\n",
    "                action_flag = 'G'\n",
    "            elif [i, j] in maze.obstacles:\n",
    "                action_flag = 'W'\n",
    "            \n",
    "            tb.add_cell(i, j, width, height, text=action_flag, loc='center', facecolor='white')\n",
    "            \n",
    "    ax.add_table(tb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAANXUlEQVR4nO3dQWgcdRvH8WfehHRCI4Gi9kVqKtpc2sZKRZpOoA0ByTWnVoqQFgxYbA9qD15yFHpSDzFILybQQ0/JITZBROlh2cWuWbQabCkEXYx92xTMEm22B+f/Hvo22Nfd9U935pmn3e8HAkkGdr5s01/CsswEzjkBAOj4V9YBANBKGF0AUMToAoAiRhcAFDG6AKCovdHBzs7O/1Sr1e1aMT7CMIyr1aq5XxYWu2jyQ5M/i11Gm25ubGz8u9axoNFbxoIgcNbeUhYEgVhrErHZRZMfmvxZ7DLcFNQ6Zuq3AwA87hhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQBGjCwCKGF0AUMToAoAiRhcAFKmMbqVSke+++07jVN5o8kMTHnc3b96UY8eOyfPPPy8vv/yyHDx4UGZnZ1M7n8roXr9+XT788EONU3mjyQ9NeJw552RkZEQOHToky8vLsri4KBcuXJBffvkltXPy8gKQsCtXrsjy8nLWGQ+w2CSSfddXX30lHR0d8uabb25+b+fOnXL69OnUzsnoAgmrVqsyMjJiauQsNolk37W0tCT79+9XPWfDO0cAaOz8+fNy9uzZv33/xo0b8tprr8nly5dp+h+rXX/11ltvSS6Xk46ODikWi6mcI/W/dAuFgsRxLCIii4uLcvfu3bRP+Y9o8kPTP3v99dflhx9+eOBjfn5enn322cxed7bYZLVrz549UiqVNr/++OOP5csvv5TV1dXUzpn67Xree+89uXTpkvzxxx/S1tYmn3/+uWzf/vC3XUvi1hxJNyXRRdOj2/T/vvjiC9m6datEUfRYNYnY7GqmyTkn/f39cvz4cTl58qSIiJTLZTl06JD89NNPzTbVvF2POOfqftw73LwzZ864Xbt2uZWVlaYfy2KTc8l00eTHYlPSLDY5Z7Or2aZff/3VHT161D333HPulVdecYODg+7ChQtJNNXcVbUbU/7555/S1tbW9OMk+Zs2qSaR5Lpo8mOxKUkWm0RsdhluyvbGlEn9B0kSTX5oApLDW8YAQBGjCwCKGF0AUMToAoAiRhcAFDG6AKCI0QUARYwuAChidAFAEaMLAIoYXQBQxOgCgCJGFwAUNbxdTxiGcRAEpoY5DEMJgtrXBs6SxS6a/NDkz2KX0aa43jG16+kmxeK1M0VsdtHkhyZ/FrsMN2V7PV0AAKMLAKoYXQBQxOgCgCJGFwAUMboAoIjRBQBFjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXSEilUpFSqZR1xgMsNlmk+Typju6VK1dkeXlZ85T/yGKTiM0umuqrVCoyPDwsURTJwsJC1jkiYrMpjmOZm5vLOuMB2s+T6uhWq1UZGRkx8Z/kPotNIja7aKpvbGxMoiiSoaEhGR8fl3K5nGmPxaY4juXEiROSy+Uy7fh/2s9TatfTPX/+vJw9e/Zv379x44a88MILcvny5Yd63GaunZlWk9UumtJvum9jY0OWlpZkYmJCPvnkEwnDsKnHs9jUbNfk5KScOnVKdu/e/bdjvb29Mjs7q94kkurzVPvK6s65uh/3Difn559/dvv27XO5XO6hH8Nik3M2u2jyk1RTsVh0o6OjiTyWxSbnmutaX193hw8fdlNTU4n1OJfMc5XS81RzV1VfXrh27ZpMTk7KwMCA5mkbstgkYrOLJjSjq6tLLl68KKurq1mnZKrhPdKS9uqrr2qezovFJhGbXTShWVu3bpUzZ85knZEp3jIGAIq4MWVCLHbR5Icmfxa7DDdxY0oAyBqjCwCKGF0AUMToAoAiRhcAFDG6AKCI0QUARYwuAChidAFAEaMLAIoYXQBQxOgCgCJGFwAUNbyebhiGcRAEpoY5DEMJgtp3wciSxS6a/NDkz2KX0aa43jEu7ZgQi100+aHJn8Uuw01c2hEAssboAoAiRhcAFDG6AKCI0QUARYwuAChidAFAEaMLAIoYXQBQxOgCgCJGFwAUMboAoEhldCuVipRKJY1TeaPJD03+LHbR5Ee1yTlX9+Pe4easra25AwcOuC1btrj5+fmmH89ik3PNd9H06DY5x885TTWbau5q6pd2PHLkiOzYsUOuXr0qt27dkpmZGenp6Xnox0viMm5JNyXRRdOj25RGl8WmJLparKn2RX7rrbFL6C/dO3fuuGKx6EZHR93GxkbTj2exybnmu2h6dJuc4+ecpppNNXc19dd0Ozs7Nz8PwzDt03mhyQ9N/ix20eRHu4l3LwCAIkYXABQxugCgiBtTJsRiF01+aPJnsctwEzemBICsMboAoIjRBQBFjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQFF7o4NhGMZBEJga5jAMJQhq33ooSxa7aPJDkz+LXUab4nrHuLRjQix20eSHJn8Wuww3cWlHAMgaowsAihhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQBGjCwCKGF0AUMToAoAildGtVCpSKpU0TlXX22+/LR999NHm18PDw/LGG29sfv3uu+/KBx98QBdNNLVQ132aG5X66FYqFRkeHpYoimRhYSHt09U1MDAg+XxeRETiOJbbt2/L0tLS5vF8Pi9RFNFFE00t1CWiv1Gpj+7Y2JhEUSRDQ0MyPj4u5XI57VPWFEWRFAoFERFZWlqSvXv3yhNPPCG//fab3L17V3788UfZv38/XTTR1EJdIvob1fAi5kmYnp6WpaUlmZiYkJmZGQnDMO1T1vTMM89Ie3u7lMtlyefzcvDgQVlZWZFCoSDd3d3S19cnHR0ddNFEUwt1iehvVOqj29nZufl5VoN7XxRFks/nJZ/PyzvvvCMrKyuSz+elu7tbBgYG6KKJphbs0t6olnr3wv3Xlb7//nvZu3ev9Pf3S6FQyPT1JKtdNNHUKl3aWmp0oyiSzz77TLZt2yZtbW2ybds2WVtbk0KhkOk/usUummhqlS5tLTW6fX19cvv2benv73/ge93d3fLkk0/SRRNNLdiljRtTJsRiF01+aPJnsctwEzemBICsMboAoIjRBQBFjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQFHDO0eEYRgHQWBqmMMwlCCoefGeTFnsoskPTf4sdhltiusd49KOCbHYRZMfmvxZ7DLcxKUdASBrjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQBGjCwCKVEa3UqlIqVTSOBUAmJb66FYqFRkeHpYoimRhYSHt0wGAaamP7tjYmERRJENDQzI+Pi7lcjntUzYUx7HMzc1l2lCLxS6a/NDkz2KXdlPqozs9PS3Hjh2Tp59+WnK5nPT09KR9yrriOJYTJ05ILpfLrKEWi100+aHJn8WuLJpULmL+zTffyMTEhExNTTX9WM1csHhyclJOnTolu3fv/tux3t5emZ2dfay6aKIpySarXYabat/OwjlX9+Pe4eYVi0U3OjqayGM107S+vu4OHz7spqamEmn5K4tdNPmhyZ/FLsNNNXe1pd4y1tXVJRcvXpTV1dWsUx5gsYsmPzT5s9iVRVNLvbyQJotdNPmhyZ/FLsNNNV9e4MaUCbHYRZMfmvxZ7DLcxI0pASBrjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXABQxugCgiNEFAEWMLgAoYnQBQBGjCwCKGF0AUMToAoCi9kYHwzCMgyAwNcxhGEoQ1L4LRpYsdtHkhyZ/FruMNsX1jnE93YRY7KLJD03+LHYZbuJ6ugCQNUYXABQxugCgiNEFAEWMLgAoYnQBQBGjCwCKGF0AUMToAoAiRhcAFDG6AKCI0QUARSqjW6lUpFQqaZzKG01+3n//fdmzZ4+8+OKL8tJLL8nXX3+ddZLJ50nEZhdNflSbnHN1P+4dbs7a2po7cOCA27Jli5ufn2/68Sw2Odd8l8WmfD7v+vv7XbVadc45t7q66lZWVjJtsvg8OcfPOU01m2ruauqXdjxy5Ijs2LFDrl69Krdu3ZKZmRnp6el56MdL4jJuSTcl0WWxaWZmRj799FOZm5trqiPJJovPUxpdFpuS6GqxptoX+a23xi6hv3Tv3LnjisWiGx0ddRsbG00/nsUm55rvsti0vr7u9u3b53p7e93JkyfdpUuXMm+y+Dw5x885TTWbau5q6q/pdnZ2bn4ehmHap/NCk5+uri5ZXFyUc+fOyVNPPSVHjx6VqampTJssPk8iNrto8qPd1PB2PUBbW5sMDg7K4OCg9PX1yfT0tBw/fjzrLOCRxVvGUNe1a9fk+vXrm19/++23snPnzgyLgEcff+mirt9//11Onz4ta2tr0t7eLrt27ZJz585lnQU80rgxZUIsdtHkhyZ/FrsMN3FjSgDIGqMLAIoYXQBQxOgCgCJGFwAUMboAoIjRBQBFjC4AKGJ0AUARowsAihhdAFDE6AKAIkYXABQ1vLRjGIY3gyDYrhXjIwzDOAgCc78sLHbR5Icmfxa7jDbdrHes4aUdAQDJMvXbAQAed4wuAChidAFAEaMLAIoYXQBQ9F+/OVqqcfpvxAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# dyna-Q\n",
    "draw_policy(q_values[0], shortcut_maze)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAPvElEQVR4nO3df2hVhf/H8ddtomdsIYhpSClo+8c0w/7Rs9IYyf5K9lfKGI5Rg6AfYCn0z4KCYH9lf4w1+ieH+yOCXKFrREYJlw03HbkaNYSBl2Qfm5CXTG+Q93z/8NO+bm7jfLz3vM973ucDhLUL9zw55/YS5jgnE0WRAAA2Hko7AAAqCaMLAIYYXQAwxOgCgCFGFwAMrVjqxerq6v8UCoX1VjFxBEFQLBQK7v6y8NhFUzw0xeexy2nT1Vu3bj260GuZpX5lLJPJRN5+pSyTychbk+Szi6Z4aIrPY5fjpsxCr7n62wEAHnSMLgAYYnQBwBCjCwCGGF0AMMToAoAhRhcADDG6AGCI0QUAQ4wuABhidAHAEKMLAIYYXSw7+XxeY2NjaWfgAXH16lU1Nzdr8+bNeuaZZ7R792719/cndjxGF8tKPp9XY2OjwjDU4OBg2jlY5qIoUlNTk/bs2aOpqSlduHBBn332mX777bfEjml6a8fx8XHV1tZq8+bN9/0e5b6NWzmaJJ9dD2LTSy+9pMcee0y//vqrfv/9d508eVIbN2687/crR9N8Hs5TEk2Sz65Smr777ju9//77Onv27H0ff4mm9G/tWCgU1NTUpKmpKcvDLsljk+Szy0NTb2+vmpubtW7dOmWz2ZIHNwkeztN8Hpuk9LsmJia0c+dO02Mu+eSIUvT19amzs/Oe709PT+vgwYMaGRlJ6tDLqkny2eWxSZKqq6tnvw6CIJWGu3k8Tx6bJL9dd3vttdeUzWa1cuVKjY6OJnOQKIoW/XPn5fK5fPlytGPHjiibzd73e3hsiiKfXQ9q0+joaNTa2lry+/zrQT1Pd+NzvrAzZ85Ee/bsmfO9mZmZaNOmTff9nnc1Lbirpj9emJycVHd3t+rr6y0PuySPTZLPLo9NHnk8Tx6bpPS7GhoaVCgU9PHHH89+7+bNm4kek2eklYnHrge16fz58+rq6tLx48fdNJWbxybJZ1epTdPT0zp8+LDOnTunRx55RDU1NXr11Vd14MCBUpsW/Ic0RrdMPHbRFA9N8XnsctyU/m8vAEClY3QBwBCjCwCGGF0AMMToAoAhRhcADDG6AGCI0QUAQ4wuABhidAHAEKMLAIYYXQAwxOgCgCFGFwAMLfm4niAIiplMxtUwB0GgTGbBO6alymMXTfHQFJ/HLqdNxcVe4366ZeKxi6Z4aIrPY5fjJu6nCwBpY3QBwBCjCwCGGF0AMMToAoAhRhcADDG6AGCI0QUAQ4wuABhidAHAEKMLAIYYXQAwVLGjm8/nNTY2lnaGex7Pk8cmxOfx+lk2mY7u+Pi4pqamLA+5oHw+r8bGRoVhqMHBwbRzFuThXHk8Tx6b5vNw7ebz0uTx+lk3mY5uoVBQU1NT6he/vb1dYRiqoaFBHR0dyuVyqfYsxMO58niePDbN5+HazeelyeP1s25K7H66fX196uzsvOf709PT2rJli0ZGRu7rfctx78xbt25pYmJCXV1d6unpURAEJb1fqV1ez5W38+SxyeO1S6qp1C7J3/VLuGnB++ku+eSIUrS0tKilpWXO93K5nPbv369jx44lddhYqqurZ78uxwkulddz5e08Sf6aPF47j03/8nb9JPumxEZ3IZOTk+ru7lYYhpaHXZY4V8uXx2vnsalSmY7uvn37LA+3rHGuli+P185jU6Wq2F8ZA4A08GDKMvHYRVM8NMXnsctxEw+mBIC0MboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgiNEFAEOMLgAYYnQBwNCS99MNgqCYyWRcDXMQBMpkFrx5T6o8dtEUD03xeexy2lRc7DVu7VgmHrtoioem+Dx2OW7i1o4AkDZGFwAMMboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgiNEFAEMmo5vP53Xx4kWLQ8WWz+c1NjaWdoZ7Hs+TxyYsb5afKZPRvXTpko4dO2ZxqFjy+bwaGxsVhqEGBwfTznHL43ny2ITlzfozVZE/Xmhvb1cYhmpoaFBHR4dyuVzaSfcYHx/X1NRUqg0ez5PHpmKxqFOnTqWdMYfHpoVU4ue8Ike3t7dXzc3NWrdunbLZrDZu3Jh20j0KhYKamppS/UB6PE/emorFotra2pTNZlPtuJvHpsVU4ud8ySdHPKiqq6tnvw6CIMWSO/r6+tTZ2XnP96enp3Xw4EGNjIykUOXvPEn+mnp6enTixAlt3bpVAwMDc16rq6tTf38/Tf/F5/yOxEd3eHhYVVVVkqQLFy5o27ZtWrVqVdKHXVZaWlrU0tIy53u5XE779+939bNw3OvQoUP6/PPP1dbWptbW1rRzJPlskvic/yvxHy989dVXevPNN3XhwgW9/PLLun79etKHfCBMTk6qu7tb9fX1aadgCbW1tRoYGNDMzEzaKbM8Ni2mEj/niY9uZ2ennnvuORUKBX399ddav3590od8IOzbt09hGKadgRhqamp05MiRtDPm8Ni0kEr8nJs9mPL27duzP2YohceH0Ek+u2iKh6b4PHY5bkr3wZTlGFwAWO4q8lfGACAtjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgiNEFAEOMLgAYYnQBwBCjCwCGlryJeRAExUwm42qYgyBQJrPgzXtS5bGLpnhois9jl9Om4mKvmd3asVw83sZN8tlFUzw0xeexy3FTurd2BAAwugBgitEFAEOMLgAYYnQBwBCjCwCGGF0AMMToAoAhRhcADDG6AGCI0QUAQ4wuABiqmNE9fPiwPvroo9n/bmxs1CuvvDL732+//bY+/PBDumiiqYK6/pXP5zU2NmZyrIoZ3fr6eg0NDUmSisWirl27pomJidnXh4aGFIYhXTTRVEFd0p3BbWxsVBiGGhwcTPx4iY5usVjUqVOnkjxEbGEYanh4WJI0MTGhbdu26eGHH9Yff/yhv//+W7/88ot27txJF000VVCXJLW3tysMQzU0NKijo0O5XC7R4y15E/NSFItFtbW16dFHH9WLL76Y1GFi27Bhg1asWKFcLqehoSHt3r1bV65c0fDwsFavXq3t27dr5cqVdNFEUwV1SVJvb68mJibU1dWlkydPKgiCRI+X2Oj29PToxIkT2rp1qwYGBua8VldXp/7+/qQOvagwDDU0NKShoSG99dZbunLlioaGhrR69WrV19eb93juoommSumqrq6e/TrpwZUS/PHCoUOHtGfPHh09elQ///zznD9pDK70/z9X+umnn7Rt2zbt2rVLw8PDqf48yWsXTTRVSpe1xEa3trZWAwMDmpmZSeoQ/7MwDHX69GmtWbNGVVVVWrNmja5fv67h4eFUL7rHLppoqpQua4n+Q1pNTY2OHDmS5CH+J9u3b9e1a9e0a9euOd9bvXq11q5dSxdNNFVglzUeTFkmHrtoioem+Dx2OW7iwZQAkDZGFwAMMboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgiNEFAEOMLgAYWvLJEUEQFDOZjKthDoJAmcyCN+9JlccumuKhKT6PXU6biou9xq0dy8RjF03x0BSfxy7HTdzaEQDSxugCgCFGFwAMMboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgqGJHN5/P6+LFi2lnzOGxCagE+XxeY2NjJseq2NG9dOmSjh07lnbGHB6bgAddPp9XY2OjwjDU4OBg4sczHd3x8XFNTU1ZHhJl5PH60RSPxybJR1d7e7vCMFRDQ4M6OjqUy+USPZ7p6BYKBTU1NaV+knF/PF4/muLx2CT56Ort7VVzc7PWrVunbDarjRs3Jnq8JZ8cUYq+vj51dnbe8/3p6WkdPHhQIyMjSR0aZeDx+tG0fJskv13V1dWzXwdBkPwBoyha9M+dl8vn8uXL0Y4dO6JsNnvf71GOpqGhoejcuXNRa2trdP78+ahQKJT8nqV2eWyaz8v1uxtN8ZSjKYp8dpWjaXR0NGptbS35ff7136YFd9X0cT3ffvutampqFIbhfb9HOR7N8c477+iHH37QX3/9paqqKn3zzTdav359Se9ZapfHpvm8XD+a0mmSfHaVo+n8+fPq6urS8ePHS3qfeU0LPq6nYp+RdvToUX355Zc6e/asNmzY4KLLY1O50RSPxybJZxejm7ByXvTbt2+rqqqqLO9Vri6PTeVEUzwemySfXY6beDDlfOUat3Ly2ASgfCp6dAHAGqMLAIYYXQAwxOgCgCFGFwAMMboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwt+bieIAiKmUzG1TAHQaBMZsE7pqXKYxdN8dAUn8cup03FxV6r6PvplpPHLprioSk+j12Om7ifLgCkjdEFAEOMLgAYYnQBwBCjCwCGGF0AMMToAoAhRhcADDG6AGCI0QUAQ4wuABhidAHAkMno5vN5jY2NWRwqNo9NHn3wwQd68skn9dRTT+npp5/WuXPn0k4Cys5yDxIf3Xw+r8bGRoVhqMHBwaQPF4vHpmKxqFOnTqWdMcfw8LBOnz6tsbExjY+P68yZM3r88cfTzppjfHxcU1NTaWfMQVN8Hrqs9yDx0W1vb1cYhmpoaFBHR4dyuVzSh1x2TcViUW1tbcpms6l2zDc9Pa21a9dq1apVkqS1a9dqw4YNKVfNVSgU1NTUlPr/uHejKT4PXdZ7kPj9dG/duqWJiQl1dXWpp6dHQRCU9H7luHdmuZtK7eru7tbrr7+urVu33vNaXV2d+vv7zZsk6caNG3r22Wd18+ZNvfDCCzpw4ID27t173+9XalNfX586Ozvv+f709LS2bNmikZERmhJs8tpV6uc8wT1Y+M7qURQt+ufOy6UbHR2NWltby/JeHpuiqLSuP//8M9q7d290/PjxsvVEUXnO1T///BN9//330bvvvhutX78++vTTT1Nvutvly5ejHTt2RNls9r7fg6b4PHaVoymhPVhwV/ntBQdqa2s1MDCgmZmZtFPuUVVVpeeff17vvfeeurq69MUXX6SdNMfk5KS6u7tVX1+fdsosmuLz2pWkJZ+RBjs1NTU6cuRI2hlzTE5O6qGHHlJdXZ0k6ccff9SmTZtSrppr3759aSfcg6b4vHYlidHFom7cuKE33nhD169f14oVK/TEE0/ok08+STsLWNZ4MGWZeOyiKR6a4vPY5biJB1MCQNoYXQAwxOgCgCFGFwAMMboAYIjRBQBDjC4AGGJ0AcAQowsAhhhdADDE6AKAIUYXAAwxugBgaMlbOwZBcDWTyay3iokjCIJiJpNx95eFxy6a4qEpPo9dTpuuLvbakrd2BACUl6u/HQDgQcfoAoAhRhcADDG6AGCI0QUAQ/8HCdHTFWsFQtoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# dyna-Q+\n",
    "draw_policy(q_values[1], shortcut_maze)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面結果來看可以發現，dyna-Q的話根本沒有去注意到右邊還有一條新的路，但dyna-Q+是有發現的，就跟人生一樣，每天上班都走一樣的路，不願意去開發新的路線，即使有更短的路出現你也不會發現。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}