Breakout_explained.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Breakout explained",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/NicMaq/Reinforcement-Learning/blob/master/Breakout_explained.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d_W0XuDneAR9",
        "colab_type": "text"
      },
      "source": [
        "# Learning to play Breakout with a DQN \n",
        "\n",
        "\n",
        "Breakout was an arcade game developed and published by Atari on May 13, 1976. To play: a layer of bricks  lines the top third of the screen and the goal is to destroy them all! The following animated gif was captured during one of our eval sessions. Execute the following cells and you'll train a network to do the same. \n",
        "\n",
        "This Google Colab was written to support my post: Best practices for Reinforcement Learning XXX.\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/gymbreakout-20200617223846-7912_orig.gif)\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UAtnlQcr3Rl_",
        "colab_type": "text"
      },
      "source": [
        "# Imports and global constants\n",
        "\n",
        "First we import the required packages and create a few global constants (size of the image senf to the DQN, number of frames to process, number of actions,...). And the hyperparameters (learning rate, discounted ratio, epsilon for the e-greedy policy). These are the values to fine tune if you want to improve performance.\n",
        "\n",
        "\n",
        "I am running tensorflow 2.x and numpy 1.18.5.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ozVZJKTleBZj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "\n",
        "import time\n",
        "from datetime import datetime\n",
        "import gym\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import math\n",
        "import sys\n",
        "from skimage.transform import rescale, resize, downscale_local_mean\n",
        "import imageio"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "35PEkAeFd71V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Global constants\n",
        "MAX_STEPS = 2500000 \n",
        "EVAL_STEPS = 200000 # Evaluate the model every EVAL_STEPS frames\n",
        "EVAL_GAMES = 100     # For EVAL_GAMES games\n",
        "MINI_BATCH_SIZE = 32 \n",
        "MAX_SAMPLES = 1000000\n",
        "IMG_HEIGHT = 84\n",
        "IMG_WIDTH = 84\n",
        "\n",
        "# Policy\n",
        "# qlearning e-greedy = 0 ; expected sarsa e-greedy = 1 ; expected sarsa softmax = 2\n",
        "POLICY = 0\n",
        "\n",
        "# NUM_ACTIONS \n",
        "ACTIONS = {\n",
        "    0: \"NOOP\",\n",
        "    1: \"FIRE\",\n",
        "    2: \"RIGHT\",\n",
        "    3: \"LEFT\",\n",
        "}\n",
        "NUM_ACTIONS = len(ACTIONS)\n",
        "\n",
        "# Epsilon = Greedy Policy\n",
        "MIN_EPSILON = 0.1\n",
        "MAX_EPSILON = 1 \n",
        "EVAL_EPSILON = 0.0\n",
        "EXPLORE_STEPS = 300000\n",
        "ANNEALING_STEPS = 900000\n",
        "\n",
        "# Tau = Softmax Policy\n",
        "TAU = 0.00005 \n",
        "\n",
        "# Network update\n",
        "MODELUPDATE_TRAIN_STEPS = 5000\n",
        "START_LEARNING = 50000\n",
        "UPDATE_FREQ = 2\n",
        "REPEAT_ACTION = 1 \n",
        "NO_OP_MAX = 0\n",
        "\n",
        "# Save model\n",
        "SAVEMODEL_STEPS = 1000000 \n",
        "\n",
        "# Learning rate (alpha) and Discount factor (gamma) \n",
        "ALPHA = 0.00001\n",
        "GAMMA = 0.99\n",
        "\n",
        "# Epochs for training the DNN - How many mini batches will be sent at each steps for training. 2 = 2 gradient descents at each step\n",
        "EPOCHS = 1 \n",
        " \n",
        "# Directories\n",
        "SAVE_DIR = 'models/GymBreakout/ExpectedSarsa'\n",
        "ROOT_TF_LOG = 'tf_logs'\n",
        "\n",
        "#GPU CPU - Use Argparse to modify this \n",
        "USE_DEVICE = '/GPU:0' \n",
        "USE_CPU = '/CPU:0'\n",
        "RENDER = False"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2PEWiT084_aO",
        "colab_type": "text"
      },
      "source": [
        "# The agent and the environment\n",
        "\n",
        "One of the key ideas in Reinforcement Learning is that an agent learns from its interactions with an environment. The agent is the learner and the decision maker. For each action the agent takes, it retrieves from the environment the new state (a 194*160 RGB image) and the reward (how many bricks we removed). We will create the environment later in this notebook using the Open AI Gym environment: https://gym.openai.com/envs/Breakout-v0/\n",
        "\n",
        "<br><br>\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/screen-shot-2020-06-17-at-5-33-36-pm_orig.png)\n",
        "<br><br>\n",
        "\n",
        "In this notebook, we will use one of the central ideas in Reinforcement Learning: Temporal Difference learning. We will implement a special case of Expected Sarsa: QLearning. In the python version of this notebook (XXX), I implemented Expected Sarsa and QLearning and designed different policies: e-greedy and softmax. To keep it simple, in this notebook we'll use an e-greedy policy. \n",
        "\n",
        "We are implementing the same network and strategies as in the Deepmind paper: https://www.nature.com/articles/nature14236. \n",
        "\n",
        "The agent uses two networks. One for approximating the action-value function of the current state (model) and to take actions; the other one for calculating the td-error (target_model). The rationale is to support convergence by not chasing our own tail.\n",
        "\n",
        "We will also use an experience memory to feed random samples into the network at each step, but we'll discuss this in a later cell.\n",
        "\n",
        "The run_training function (defined in a few cells) orchestrates training and evaluation. It asks the agent to start a game (def play_game), to decide the best action under current policy (def choose_action); to calculate the TD error; and to back-propagate the gradients (def calculate_target_and_train). Then, every EVAL_STEPS frames, our run_training function  evaluates the network's performance.\n",
        "\n",
        "I implemented the following pseudo code as it can be found in the  Reinforcement Learning book by [Richard S. Sutton and Andrew G. Barto](https://www.amazon.com/Reinforcement-Learning-Introduction-Adaptive-Computation/dp/0262039249). I really really encourage you to read (and reread) this book.\n",
        "<br><br>\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/screen-shot-2020-06-18-at-3-56-38-pm_orig.png)\n",
        "\n",
        "<br>\n",
        "\n",
        "**The two critical parts: 1. The policies and 2. the Bellman equation.**\n",
        "\n",
        "In GYM_ACROBOT.py, I offer the option to use two policies: the e-greedy and the softmax policies. These policies are discussed in a separate colab notebook: [e-greedy and softmax explained](https://colab.research.google.com/drive/1--qFcl5QuTuudC-yYcE1odKx_htui4h6?usp=sharing). QLearning is pretty straightforward as it uses a greedy policy for the TD update and an e-greedy policy to select the best action. \n",
        "In XXX GYM_ACROBOT.py and XXX GYM_SPACE_INVADERS.py, I implemented ties management for the e-greedy policy. I often turn it off because I noticed that for ACROBOT and BREAKOUT, ties rarely happens (but they do happen). \n",
        "<br><br>\n",
        "**The Bellman equation**\n",
        "\n",
        "The [Bellman equation](https://en.wikipedia.org/wiki/Bellman_equation#:~:text=A%20Bellman%20equation%2C%20named%20after,method%20known%20as%20dynamic%20programming.&text=This%20breaks%20a%20dynamic%20optimization,%E2%80%9Cprinciple%20of%20optimality%E2%80%9D%20prescribes.)  for QLearning, named after Richard E. Bellman, writes the \"value\" of a decision problem at a certain point in time resulting from the payoff from some initial choices and the \"value\" of the remaining decision problem that come from those initial choices.\n",
        "\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/screen-shot-2020-06-18-at-3-18-06-pm_orig.png)\n",
        "\n",
        "As indicated in the pseudo code, this update is done after every transition from a nonterminal state; this update happens in calculate_target_and_train.\n",
        "<br>\n",
        "We start by calculating the next state's action-value with the target network for each element of the batch:\n",
        "\n",
        "```\n",
        "next_qsa = self.target_model((batch_next_history, batch_action_all_ones), training=True)\n",
        "```\n",
        "Then we instantiate the tensor representing the maximum value over a of each action-value.   \n",
        "```\n",
        "max_q = tf.math.reduce_max(next_qsa, axis=1, keepdims=True)\n",
        "```\n",
        "The Bellman equation is for non terminal states. For the terminal state, the update is simply the expected reward.\n",
        "```\n",
        "v_next_vect = batch_terminal * max_q\n",
        "```\n",
        "Then we calculate the TD error: \n",
        "```\n",
        "target_vec = batch_reward + GAMMA * v_next_vect\n",
        "```\n",
        "Our goal is to update the action_value of the action that was taken. Therefore we tf.mulitply the target_vec by the one-hot encoding of the batch of actions.\n",
        "```\n",
        "target_mat = tf.multiply(target_vec, batch_action_one_hot)\n",
        "```\n",
        "Now that we have the predicted action-value, we need the current output of the network.\n",
        "```\n",
        "qsa = self.model((batch_history, batch_action_one_hot), training=True)\n",
        "```\n",
        "The difference between the two will be used to calculate the loss.\n",
        "```\n",
        "qsa_mat = tf.multiply(qsa, batch_action_one_hot)\n",
        "delta_mat = target_mat - qsa_mat\n",
        "```\n",
        "\n",
        "We are using the following Huber loss implementation:\n",
        "\n",
        "```\n",
        "squared_loss = 0.5 * tf.square(delta_mat)\n",
        "linear_loss = tf.abs(delta_mat) -0.5\n",
        "ones = tf.ones_like(delta_mat)\n",
        "loss_mat = tf.where(tf.greater(linear_loss, ones), x = linear_loss, y = squared_loss)\n",
        "loss_train = tf.reduce_mean(loss_mat, axis=1, keepdims=True)\n",
        "```\n",
        "\n",
        "We use Tensorflow tf.GradientTape() to record all useful information from the forward propagation and apply the gradients:\n",
        "\n",
        "```\n",
        "grads = tape.gradient(loss_train, self.model.trainable_variables)\n",
        "self.optimizer.apply_gradients(zip(grads, self.model.trainable_variables))  \n",
        "```\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eO_TFXG6eaRp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class Agent:\n",
        "\n",
        "    def __init__(self, env, model, target_model, optimizer, exp_buffer):\n",
        "        self.env = env\n",
        "        self.exp_buffer = exp_buffer\n",
        "        self.model = model\n",
        "        self.target_model = target_model\n",
        "        self.optimizer = optimizer\n",
        "        \n",
        "        with tf.device(USE_DEVICE):\n",
        "            self.decay = (MAX_EPSILON-MIN_EPSILON) / ANNEALING_STEPS\n",
        "            self.epsilon = tf.constant(MAX_EPSILON)\n",
        "            self.epsilon = tf.cast(self.epsilon, dtype=tf.float32)\n",
        "            self.min_epsilon = tf.constant(MIN_EPSILON, dtype=tf.float16)\n",
        "            self.min_epsilon = tf.cast(self.min_epsilon, dtype=tf.float32)\n",
        "            self.epsilon_evaluation = tf.constant(EVAL_EPSILON, dtype=tf.float16)\n",
        "            self.epsilon_evaluation = tf.cast(self.epsilon_evaluation, dtype=tf.float32)\n",
        "\n",
        "            assert self.epsilon.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"epsilon not on : %s\" % USE_DEVICE\n",
        "            assert self.min_epsilon.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"min_epsilon not on : %s\" % USE_DEVICE\n",
        "            assert self.epsilon_evaluation.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"epsilon_evaluation not on : %s\" % USE_DEVICE\n",
        "        \n",
        "        self._reset()\n",
        "\n",
        "\n",
        "    def _reset(self):\n",
        "        self.image = self.env.reset()\n",
        "        self.state = preprocess(self.image)\n",
        "\n",
        "\n",
        "    def eval_game(self):\n",
        "\n",
        "        dead = False\n",
        "        steps = 0 \n",
        "        game_reward = 0\n",
        "        raw_images = []\n",
        "\n",
        "        self._reset()\n",
        "\n",
        "        raw_images.append(self.image)\n",
        "        \n",
        "        remaining_lives = 5 \n",
        "        history = np.repeat(self.state, 4, axis=2)\n",
        "        init_history = history \n",
        "\n",
        "        while True:\n",
        "\n",
        "            # Play next step\n",
        "            if RENDER: self.env.render()\n",
        "\n",
        "            if steps % REPEAT_ACTION == 0:\n",
        "                \n",
        "                history_foraction =  np.reshape(history, (1, IMG_HEIGHT, IMG_WIDTH,4))\n",
        "\n",
        "                with tf.device(USE_DEVICE):\n",
        "                    \n",
        "                    tf_history = tf.constant(history_foraction)\n",
        "                    tf_history = tf.cast(tf_history, dtype=tf.float32)\n",
        "\n",
        "                    assert tf_history.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"tf_history not on : %s\" % USE_DEVICE\n",
        "                    \n",
        "                    if POLICY == 2:\n",
        "                        action_probs = self.choose_action(tf_history)\n",
        "                        probs = action_probs.numpy()\n",
        "                        action = np.random.choice(NUM_ACTIONS, p=probs.squeeze())\n",
        "                    else:\n",
        "                        action = self.choose_action(tf_history)\n",
        "                        action = action.numpy()\n",
        "            \n",
        "            # Do the NO_OP actions then fire once \n",
        "            if np.all(np.equal(history, init_history)):\n",
        "                if NO_OP_MAX > 0:\n",
        "                    no_op = np.random.randint(0, NO_OP_MAX)\n",
        "                    for op in tf.range(no_op):\n",
        "                        action = np.random.randint(2, 4) # Select either right or left\n",
        "                        _, _, _, _ = self.env.step(action)\n",
        "                action = 1\n",
        "            \n",
        "            next_image, step_reward, done, info = self.env.step(action)\n",
        "\n",
        "            if steps > 2500:\n",
        "                print('Max steps reached')\n",
        "                done = True\n",
        "\n",
        "            game_reward += step_reward\n",
        "            raw_images.append(next_image)\n",
        "\n",
        "            next_state = preprocess(next_image)\n",
        "            next_history = np.append(history[:,:,-3:], next_state, axis=2)\n",
        "\n",
        "            if  remaining_lives > info['ale.lives']:\n",
        "                dead = True\n",
        "                if info['ale.lives'] == 0: done = True\n",
        "                print(\"Player is dead! game_reward is: %s\" % (game_reward))\n",
        "                remaining_lives = info['ale.lives']\n",
        "\n",
        "   \n",
        "            # if the game is done, break the loop\n",
        "            if done:\n",
        "                return game_reward, raw_images\n",
        "\n",
        "            # move the agent to the next state \n",
        "            if dead:\n",
        "                dead = False\n",
        "                history = init_history\n",
        "\n",
        "            else:\n",
        "                history = next_history\n",
        "\n",
        "            steps += 1\n",
        "\n",
        "\n",
        "    def play_game(self, global_steps):\n",
        "\n",
        "        loss = np.zeros((1,), dtype=np.float32) \n",
        "        dead = False\n",
        "\n",
        "        steps = 0 \n",
        "        game_reward = 0\n",
        "        process_time = 0\n",
        "        train_time = 0\n",
        "\n",
        "        data_images = []\n",
        "        data_actions = []\n",
        "        data_rewards = []\n",
        "        data_dones = []\n",
        "\n",
        "        self._reset()\n",
        "\n",
        "        remaining_lives = 5 \n",
        "        history = np.repeat(self.state, 4, axis=2)\n",
        "        init_history = history \n",
        "\n",
        "        while True:\n",
        "\n",
        "            # Play next step\n",
        "            if RENDER: self.env.render()\n",
        "            \n",
        "            if steps % REPEAT_ACTION == 0:\n",
        "                \n",
        "                history_foraction =  np.reshape(history, (1, IMG_HEIGHT, IMG_WIDTH,4))\n",
        "               \n",
        "                with tf.device(USE_DEVICE):\n",
        "                    \n",
        "                    tf_history = tf.constant(history_foraction)\n",
        "                    tf_history = tf.cast(tf_history, dtype=tf.float32)\n",
        "\n",
        "                    assert tf_history.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"tf_history not on : %s\" % USE_DEVICE\n",
        "                    \n",
        "                    if POLICY == 2:\n",
        "                        action_probs = self.choose_action(tf_history)\n",
        "                        probs = action_probs.numpy()\n",
        "                        action = np.random.choice(NUM_ACTIONS, p=probs.squeeze())\n",
        "                    else:\n",
        "                        action = self.choose_action(tf_history)\n",
        "                        action = action.numpy()\n",
        "            \n",
        "            # Fire once at start # if no fire required the action has to be initialized\n",
        "            if np.all(np.equal(history, init_history)):\n",
        "                action = 1\n",
        "\n",
        "            next_image, step_reward, done, info = self.env.step(action)\n",
        "\n",
        "            if steps > 2500:\n",
        "                print('Max steps reached')\n",
        "                step_reward = -1\n",
        "                done = True\n",
        "\n",
        "            game_reward += step_reward\n",
        "            \n",
        "            # If you need clipping\n",
        "            '''\n",
        "            if step_reward > 0:\n",
        "                step_reward = 1\n",
        "            elif step_reward == 0:\n",
        "                step_reward = 0\n",
        "            else:\n",
        "                step_reward = -1           \n",
        "            '''\n",
        "\n",
        "            lap_time = time.time()\n",
        "            next_state = preprocess(next_image)\n",
        "            process_time +=  time.time() - lap_time\n",
        "\n",
        "            next_history = np.append(history[:,:,-3:], next_state, axis=2)\n",
        "\n",
        "            # Decay epsilon\n",
        "            with tf.device(USE_DEVICE):\n",
        "                if self.epsilon > self.min_epsilon and global_steps > EXPLORE_STEPS:\n",
        "                        self.epsilon -= self.decay\n",
        "                        if self.epsilon < self.min_epsilon: \n",
        "                            self.epsilon = tf.constant(self.min_epsilon)\n",
        "                        assert self.epsilon.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"self.epsilon not updated on : %s\" % USE_DEVICE\n",
        "            \n",
        "            if  remaining_lives > info['ale.lives']:\n",
        "                dead = True\n",
        "                if info['ale.lives'] == 0: done = True\n",
        "                print(\"Player is dead! game_reward is: %s\" % (game_reward))\n",
        "                remaining_lives = info['ale.lives']\n",
        "                step_reward = -1\n",
        "\n",
        "            data_actions.append(action) \n",
        "            data_images.append(next_state[:,:,0].numpy()) \n",
        "            data_rewards.append(step_reward) \n",
        "            data_dones.append(int(dead)) \n",
        "            \n",
        "            if steps % UPDATE_FREQ == 0 :\n",
        "                \n",
        "                if global_steps > START_LEARNING:\n",
        "                    \n",
        "                    lap_time = time.time()\n",
        "                    \n",
        "                    with tf.device(USE_DEVICE):\n",
        "                        # Calculate target\n",
        "                        lossBatch = self.calculate_target_and_train() \n",
        "                        lossMean = tf.reduce_mean(lossBatch)\n",
        "                        loss += lossMean.numpy() \n",
        "                    train_time +=  time.time() - lap_time\n",
        "                \n",
        "            # if the game is done, break the loop\n",
        "            if done:\n",
        "\n",
        "                np_data_images = np.asarray(data_images, dtype=np.int16)\n",
        "                np_data_rewards = np.asarray(data_rewards, dtype=np.int16)\n",
        "                np_data_actions = np.asarray(data_actions, dtype=np.int16)\n",
        "                np_data_dones = np.asarray(data_dones, dtype=np.int16)\n",
        "                \n",
        "                data = (np_data_images, np_data_actions, np_data_rewards, np_data_dones)\n",
        "                \n",
        "                return data, steps, game_reward, loss, process_time, train_time\n",
        "\n",
        "            # move the agent to the next state \n",
        "            if dead:\n",
        "                dead = False\n",
        "                history = init_history\n",
        "\n",
        "            else:\n",
        "                history = next_history\n",
        "\n",
        "            steps += 1\n",
        " \n",
        "\n",
        "    #@tf.function    \n",
        "    def calculate_target_and_train(self):\n",
        "\n",
        "        loss = tf.constant(0)\n",
        "        loss = tf.cast(loss, dtype=tf.float32)\n",
        "\n",
        "        #yield history, next_history, action_one_hot, terminals, rewards\n",
        "        for batch_history, batch_next_history, batch_action_one_hot, batch_terminal, batch_reward in self.exp_buffer.dataset.take(EPOCHS):\n",
        "\n",
        "            batch_action_all_ones = tf.ones_like(batch_action_one_hot)\n",
        "            \n",
        "            # predict Q(s',a') for the Bellman equation\n",
        "            next_qsa = self.target_model((batch_next_history, batch_action_all_ones), training=True)\n",
        "            \n",
        "            if POLICY == 1:\n",
        "\n",
        "                # e-greedy policy - Expected Sarsa\n",
        "                sum_piq = egreedy_policy(next_qsa, self.epsilon)\n",
        "                v_next_vect = batch_terminal * sum_piq\n",
        "\n",
        "            elif POLICY == 2:\n",
        "\n",
        "                # Softmax policy - Expected Sarsa\n",
        "                action_probs = softmax_policy(next_qsa)\n",
        "                expectation = tf.multiply(action_probs, next_qsa)\n",
        "                sum_expectation = tf.reduce_sum(expectation, axis=1, keepdims=True)\n",
        "                v_next_vect = batch_terminal * sum_expectation\n",
        "\n",
        "            else:\n",
        "\n",
        "                # e-greedy policy - Q-Learning\n",
        "                max_q = tf.math.reduce_max(next_qsa, axis=1, keepdims=True)\n",
        "                v_next_vect = batch_terminal * max_q\n",
        "            \n",
        "            target_vec = batch_reward + GAMMA * v_next_vect\n",
        "            target_mat = tf.multiply(target_vec, batch_action_one_hot)\n",
        "\n",
        "            # Predict Q(s,a)\n",
        "            with tf.GradientTape() as tape:\n",
        "\n",
        "                qsa = self.model((batch_history, batch_action_one_hot), training=True)\n",
        "\n",
        "                qsa_mat = tf.multiply(qsa, batch_action_one_hot)\n",
        "                delta_mat = target_mat - qsa_mat\n",
        "                \n",
        "                # Huber loss\n",
        "                squared_loss = 0.5 * tf.square(delta_mat)\n",
        "                linear_loss = tf.abs(delta_mat) -0.5\n",
        "                ones = tf.ones_like(delta_mat)\n",
        "                loss_mat = tf.where(tf.greater(linear_loss, ones), x = linear_loss, y = squared_loss)\n",
        "                loss_train = tf.reduce_mean(loss_mat, axis=1, keepdims=True)\n",
        "\n",
        "                grads = tape.gradient(loss_train, self.model.trainable_variables)\n",
        "                self.optimizer.apply_gradients(zip(grads, self.model.trainable_variables))        \n",
        "                \n",
        "                loss = tf.add(loss_train,loss)\n",
        "\n",
        "        return loss\n",
        "\n",
        "\n",
        "    #@tf.function    \n",
        "    def choose_action(self, states):\n",
        "        \n",
        "        actions_all_ones = tf.ones((1,NUM_ACTIONS))\n",
        "\n",
        "        if POLICY == 2:\n",
        "            \n",
        "            # softmax\n",
        "            qsa = self.model((states, actions_all_ones), training=True)\n",
        "            action_probs = softmax_policy(qsa)\n",
        "\n",
        "            return action_probs \n",
        "        \n",
        "        else:\n",
        "\n",
        "            # e-greedy\n",
        "            randomNum = tf.random.uniform((), minval=0, maxval=1, dtype=tf.float32, seed=1)\n",
        "            if randomNum < self.epsilon:\n",
        "                random_action = tf.random.uniform((), minval=0, maxval=NUM_ACTIONS, dtype=tf.int32)\n",
        "                best_action = random_action \n",
        "\n",
        "            else:\n",
        "\n",
        "                qsa = self.model((states, actions_all_ones), training=True)\n",
        "                best_action = tf.math.argmax(qsa, axis=1, output_type=tf.dtypes.int32)\n",
        "                #best_action = argmax_ties(qsa)\n",
        "                best_action = best_action[0]\n",
        "            \n",
        "            return best_action "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n2QmFR2regFB",
        "colab_type": "text"
      },
      "source": [
        "# The experience buffer \n",
        "\n",
        "The experience buffer or the memory replay is a very simple class that stores all images, actions, rewards and terminal states. There are a few functions to append, count, and limit the maximum number of experiences (Exp).  \n",
        "\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/memory-replay_orig.png)\n",
        "\n",
        "At each step (represented here as one column), we're adding the action that was taken, the resulting state, reward, and terminal. Terminal indicates if the state is terminal (a red column is a terminal state).\n",
        "\n",
        "There are two important things to mention: \n",
        "\n",
        "1. The dtype you use to store the experiences is directly related to the size of the memory allocated to your process. If you want to run multiple training sessions in parallel I would recommend using np.int16 instead of np.float32.\n",
        "\n",
        "2. I added a tensorflow dataset to this class to feed the experience replay into the training loop. We will detail this data pipeline in one of the next cells.\n",
        "\n",
        "# The dataset\n",
        "\n",
        "As I detailed in the Medium post XXX, searching for the best performances is difficult, especially for Reinforcement Learning. As a matter of fact, each step of the process includes generating and storing data on the CPU and doing network related operations on the GPU. Therefore, it's not unusual to have better performances running exclusively on the CPU as it limits the data transfers between the two worlds. After trying many different options, I settled on a Tensorflow dataset. This data set will be the bridge between the CPU's memory and the GPU's memory. It will retrieve a batch of samples from the experience replay and feed it to the network. A batch of samples contains MINI_BATCH_SIZE elements. Each element is made of 5 experiences (5 columns). The network requires a stack of 4 states and the related action as inputs. Looking at only one image does not help you understand the motions involved by the game. Therefore, we need to feed the network a stack of images. Deepmind and many practitioners use a stack of 4. We'll do the same for this implementation. Having a history of 4 experiences means you need 5 consecutive experiences as we need to approximate the action-value function of both the current and the upcoming steps. \n",
        "<br><br>\n",
        "The only drawback is that we need to ensure the consistency of the histories in the data pipeline. For example, we can't have the following and we need to remove those samples. \n",
        "\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/memory-incorrect_orig.png)\n",
        "\n",
        "\n",
        "\n",
        "As detailed below, we'll do so by removing the stack of 5 samples which contains a terminal state in one of the first 4 experiences. \n",
        "\n",
        "\n",
        "```\n",
        "# Remove bad samples\n",
        "first4_dones = first5_dones[:,:-1]\n",
        "any_bad_samples = np.any(first4_dones, axis=1)\n",
        "indices_ok = np.logical_not(any_bad_samples)\n",
        "\n",
        "rewards_filtered = gathered_rewards[indices_ok,:]\n",
        "images_filtered = gathered_images[indices_ok,:,:,:]\n",
        "actions_filtered = gathered_actions[indices_ok,:]\n",
        "dones_filtered = gathered_dones[indices_ok,:]\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c0WPmeHSegUk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class ExperienceBuffer:\n",
        "    def __init__(self):\n",
        "\n",
        "        self.images = np.empty(shape=(1,IMG_HEIGHT,IMG_WIDTH), dtype=np.int16)\n",
        "        self.actions = np.empty(shape=(1,), dtype=np.int16)\n",
        "        self.rewards = np.empty(shape=(1,), dtype=np.int16)\n",
        "        self.dones = np.empty(shape=(1,), dtype=np.int16)\n",
        "\n",
        "        with tf.device(USE_DEVICE):\n",
        "            types = tf.float32, tf.float32, tf.float32, tf.float32,tf.float32 \n",
        "            shapes = (MINI_BATCH_SIZE,IMG_HEIGHT,IMG_WIDTH,4), \\\n",
        "                    (MINI_BATCH_SIZE,IMG_HEIGHT,IMG_WIDTH,4), \\\n",
        "                    (MINI_BATCH_SIZE,NUM_ACTIONS), \\\n",
        "                    (MINI_BATCH_SIZE,1), \\\n",
        "                    (MINI_BATCH_SIZE,1)  \n",
        "\n",
        "            fn_generate = lambda: self.generate_data()\n",
        "            self.dataset = tf.data.Dataset.from_generator(fn_generate, \\\n",
        "                                         output_types= types, \\\n",
        "                                         output_shapes = shapes)\n",
        "            #fn_map = lambda *args: args                             \n",
        "            #self.dataset = self.dataset.map(fn_map, num_parallel_calls=tf.data.experimental.AUTOTUNE)\n",
        "            self.dataset = self.dataset.prefetch(buffer_size=2*EPOCHS)\n",
        "\n",
        "    def count(self):\n",
        "        return self.images.shape[0]\n",
        "\n",
        "    def pop(self):\n",
        "        self.images = self.images[1:,:,:]\n",
        "        self.actions = self.actions[1:]\n",
        "        self.rewards = self.rewards[1:]\n",
        "        self.dones = self.dones[1:]\n",
        "\n",
        "    def append(self, experiences):\n",
        "        self.images = np.append(self.images, experiences[0], axis=0)\n",
        "        self.actions= np.append(self.actions, experiences[1], axis=0)\n",
        "        self.rewards = np.append(self.rewards, experiences[2], axis=0)\n",
        "        self.dones = np.append(self.dones, experiences[3], axis=0)\n",
        "\n",
        "        if self.images.shape[0] > MAX_SAMPLES:\n",
        "            self.images = self.images[-MAX_SAMPLES:,:,:] \n",
        "            self.actions = self.actions[-MAX_SAMPLES:] \n",
        "            self.rewards = self.rewards[-MAX_SAMPLES:] \n",
        "            self.dones = self.dones[-MAX_SAMPLES:] \n",
        "    \n",
        "    def generate_data(self):\n",
        "\n",
        "        mini_batch_size = MINI_BATCH_SIZE\n",
        "        mini_batch_size = float(mini_batch_size)\n",
        "        num_samples = mini_batch_size * 1.7 # We don't know how many samples we'll remove\n",
        "        num_samples = int(num_samples)\n",
        "\n",
        "        while True:\n",
        "           \n",
        "            replay_images = self.images\n",
        "            replay_actions = self.actions\n",
        "            replay_rewards = self.rewards\n",
        "            replay_dones = self.dones\n",
        "\n",
        "            indices4 = np.random.randint(low=0, high=self.count()-4, size=num_samples)\n",
        "            indices4 = indices4 + 4\n",
        "            indices3 = indices4 -1\n",
        "            indices2 = indices3 -1\n",
        "            indices1 = indices2 -1\n",
        "            indices0 = indices1 -1\n",
        "            indices = np.stack((indices0,indices1,indices2,indices3,indices4), axis=0)\n",
        "            indices = np.reshape(np.transpose(indices),(num_samples*5,))\n",
        "            reshaped_indices= np.reshape(indices,(-1,5))\n",
        "            reshaped_indices4 = np.reshape(indices4,(-1,1))\n",
        "\n",
        "            gathered_images = np.take(replay_images, reshaped_indices, axis=0)\n",
        "            gathered_actions = np.take(replay_actions, reshaped_indices4, axis=0)\n",
        "            gathered_rewards = np.take(replay_rewards, reshaped_indices4, axis=0)\n",
        "            gathered_dones = np.take(replay_dones, reshaped_indices4, axis=0)\n",
        "            first5_dones =  np.take(replay_dones, reshaped_indices, axis=0)\n",
        "\n",
        "            # Remove bad samples\n",
        "            first4_dones = first5_dones[:,:-1]\n",
        "            any_bad_samples = np.any(first4_dones, axis=1)\n",
        "            indices_ok = np.logical_not(any_bad_samples)\n",
        "\n",
        "            rewards_filtered = gathered_rewards[indices_ok,:]\n",
        "            images_filtered = gathered_images[indices_ok,:,:,:]\n",
        "            actions_filtered = gathered_actions[indices_ok,:]\n",
        "            dones_filtered = gathered_dones[indices_ok,:]\n",
        "\n",
        "            rewards = rewards_filtered[0:MINI_BATCH_SIZE,:]\n",
        "            images = images_filtered[0:MINI_BATCH_SIZE:,:,:]\n",
        "            actions = actions_filtered[0:MINI_BATCH_SIZE,:]\n",
        "            dones = dones_filtered[0:MINI_BATCH_SIZE,:]\n",
        "\n",
        "            raw_history = images[:,0:4,:,:]\n",
        "            history = np.transpose(raw_history,(0,2,3,1))\n",
        "            raw_next_history = images[:,1:5,:,:]\n",
        "            next_history = np.transpose(raw_next_history,(0,2,3,1))\n",
        "\n",
        "            actions = actions.astype(int)\n",
        "            actions = np.reshape(actions,(-1,))\n",
        "\n",
        "            action_one_hot = np.eye(NUM_ACTIONS)[actions]\n",
        "            action_one_hot = action_one_hot.astype(float)\n",
        "            action_one_hot\n",
        "\n",
        "            terminals = 1 - dones\n",
        "\n",
        "            history = history.astype(np.float32)\n",
        "            next_history = next_history.astype(np.float32)\n",
        "            action_one_hot = action_one_hot.astype(np.float32)\n",
        "            terminals = terminals.astype(np.float32)\n",
        "            rewards = rewards.astype(np.float32)\n",
        "            \n",
        "            yield history, next_history, action_one_hot, terminals, rewards\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tglyg5acep_y",
        "colab_type": "text"
      },
      "source": [
        "# The Tensorflow Keras Model\n",
        "\n",
        "The Keras model we're using is as close as it can be to the one used by Deepmind. My understanding (I'm not sure what the Torch default was in 2015) is that, except Keras, the only differences are the use of the Relu activation function and the kernel initializers.\n",
        "\n",
        "Although the number of dense layers may vary, I decided to keep 2 fully connected layers before the last visible layer. \n",
        "\n",
        "I tried many different kernel initializers, but I decided to use VarianceScaling for the Convolution layers and the default GlorotUniform for the dense layers. This gave me the best results. Especially for Expected Sarsa with Softmax, which does not converge if VarianceScaling is used on all layers.\n",
        "\n",
        "<br><br>\n",
        "![alt text](http://www.modelfit.us/uploads/7/6/0/6/76068583/screen-shot-2020-06-22-at-11-20-00-am_orig.png)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IX8jJNd5eqI3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def build_keras_Seq():\n",
        "    \n",
        "    frames = tf.keras.Input(shape=(IMG_HEIGHT,IMG_WIDTH, 4), name='frames')\n",
        "    actions = tf.keras.Input(shape=(NUM_ACTIONS,), name='actions')\n",
        "\n",
        "    normalized = tf.keras.layers.Lambda(lambda x: x / 255.0, name='normalization')(frames)\n",
        "    \n",
        "    init = tf.keras.initializers.VarianceScaling(scale=2.0, mode='fan_in', distribution='untruncated_normal', seed=None)\n",
        "    init0 = tf.keras.initializers.Zeros()\n",
        "    init1 = tf.keras.initializers.Ones()\n",
        "    init2 = tf.keras.initializers.GlorotUniform(seed=1) #[-limit, limit], where limit = sqrt(6 / (fan_in + fan_out))\n",
        "    init3 = tf.keras.initializers.he_uniform(seed=1)\n",
        "\n",
        "    x = tf.keras.layers.Conv2D(32, (8, 8), strides=(4, 4), kernel_initializer=init, padding='valid', use_bias=False)(normalized)\n",
        "    x = tf.keras.activations.relu(x) # , max_value=6)\n",
        "    \n",
        "    x = tf.keras.layers.Conv2D(64, (4, 4), strides=(2, 2), kernel_initializer=init, padding='valid', use_bias=False)(x)\n",
        "    x = tf.keras.activations.relu(x) #, max_value=6)\n",
        "\n",
        "    x = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), kernel_initializer=init, use_bias=False)(x)\n",
        "    x = tf.keras.activations.relu(x) #, max_value=6)\n",
        "\n",
        "    x = tf.keras.layers.Flatten()(x)\n",
        "\n",
        "    x = tf.keras.layers.Dense(512, kernel_initializer=init2)(x)\n",
        "    x = tf.keras.activations.relu(x) #, max_value=6)\n",
        "\n",
        "    #x = tf.keras.layers.Dense(128, kernel_initializer=init2)(x)\n",
        "    #x = tf.keras.activations.relu(x) #, max_value=6)\n",
        " \n",
        "    #x = tf.keras.layers.Dense(64, kernel_initializer=init2)(x)\n",
        "    #x = tf.keras.activations.relu(x) #, max_value=6)\n",
        "    \n",
        "    q_values = tf.keras.layers.Dense(NUM_ACTIONS, dtype='float32', name='q_values', kernel_initializer=init2, activation=None)(x)\n",
        "    output = tf.keras.layers.Multiply(dtype='float32', name='Qs')([q_values, actions])\n",
        "    \n",
        "    model = tf.keras.Model(inputs=[frames,actions], outputs=output)\n",
        "    return model     "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WYPvMez8e37a",
        "colab_type": "text"
      },
      "source": [
        "# Run training\n",
        "\n",
        "This cell orchestrates the training and the evaluation of the network. It  updates on a regular basis (MODELUPDATE_TRAIN_STEPS) the target network with: \n",
        "\n",
        "```\n",
        "agent.target_model.set_weights(agent.model.get_weights())\n",
        "```\n",
        "\n",
        "It registers the following tensorboard scalars and histograms:\n",
        "\n",
        "```\n",
        "tf.summary.scalar('loss', loss[0], step=global_steps)\n",
        "tf.summary.scalar('epsilon', agent.epsilon, step=global_steps)\n",
        "tf.summary.scalar('score', successFrame[-1], step=global_steps)\n",
        "tf.summary.scalar('steps', steps, step=global_steps)\n",
        "tf.summary.histogram('actions', agent.exp_buffer.actions[-steps:], step=global_steps)\n",
        "```\n",
        "\n",
        "I decided to use Tensorboard to display the loss (of course), epsilon (it is interesting to understand when the network starts to converge), the score, the number of steps for each game and the actions' distribution (to understand how it evolves with convergence). \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-0W8d3lGe4EL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def run_training(agent, now):\n",
        "\n",
        "    logdir = \"{}/run/{}/\".format(ROOT_TF_LOG, now)\n",
        "    \n",
        "    with tf.device(USE_DEVICE):\n",
        "        file_writer = tf.summary.create_file_writer(logdir)\n",
        "\n",
        "    modelId = now\n",
        "\n",
        "    with tf.device(USE_DEVICE):\n",
        "        agent.target_model.set_weights(agent.model.get_weights())\n",
        "\n",
        "    # Metrics - Should be a collections deque with max capacity set to more than last summary scalar successFrame.\n",
        "    successMemory = np.empty((1,0))\n",
        "    successFrame = np.empty((1,0))\n",
        "    previous_global_steps_tn = 0\n",
        "    previous_global_steps_eval = 0\n",
        "    \n",
        "    game_count = 1\n",
        "    global_steps = 0\n",
        "    loss =  np.zeros((1,),dtype=np.float32)\n",
        "    best_score = -1\n",
        "    \n",
        "    lap_time = time.time()\n",
        "   \n",
        "    try:\n",
        "\n",
        "        while global_steps <= MAX_STEPS: \n",
        "\n",
        "            print('\\nGame {} - Run {}'.format(game_count, now))\n",
        "\n",
        "            #if global_steps % SAVEMODEL_STEPS  > previous_global_steps % SAVEMODEL_STEPS:\n",
        "            #    save_theModel(model, modelId, game_count, samples)\n",
        "\n",
        "            # return steps, game_reward, loss, epsilon\n",
        "            data_game, steps, game_reward, loss, process_time, train_time = agent.play_game(global_steps)\n",
        "            loss /= steps + 1 # steps starts at 0 \n",
        "\n",
        "            buffer_previous_size = agent.exp_buffer.count()\n",
        "            agent.exp_buffer.append(data_game)\n",
        "\n",
        "            global_steps += steps + 1   \n",
        "            print('Global_steps is: %s' % global_steps)\n",
        "            \n",
        "            if buffer_previous_size == 1 :\n",
        "                print(\"Experience Replay buffer pop\")\n",
        "                agent.exp_buffer.pop()\n",
        "\n",
        "            # Update the target network \n",
        "            train_steps = (global_steps - previous_global_steps_tn)*EPOCHS*MINI_BATCH_SIZE/UPDATE_FREQ\n",
        "            if train_steps > MODELUPDATE_TRAIN_STEPS:\n",
        "                with tf.device(USE_DEVICE):\n",
        "                    agent.target_model.set_weights(agent.model.get_weights())\n",
        "                print('Updating target model **************************** Updating target model ****************')\n",
        "                previous_global_steps_tn = global_steps\n",
        "\n",
        "            if POLICY == 0 or POLICY == 1: print('Epsilon is: %s' % agent.epsilon)\n",
        "            \n",
        "            # Evaluate every EVAL_STEPS frames the performance \n",
        "            if (global_steps > EXPLORE_STEPS + ANNEALING_STEPS  and global_steps > previous_global_steps_eval + EVAL_STEPS) or global_steps > MAX_STEPS:\n",
        "                \n",
        "                successEval = np.empty((1,0))\n",
        "                agent.epsilon = agent.epsilon_evaluation \n",
        "                remaining_eval_games = EVAL_GAMES\n",
        "                previous_global_steps_eval = global_steps\n",
        "                \n",
        "                while remaining_eval_games > 0:\n",
        "                    \n",
        "                    print('Evaluation game %s' % remaining_eval_games)\n",
        "                    remaining_eval_games -= 1 \n",
        "\n",
        "                    game_reward, raw_frames = agent.eval_game()\n",
        "                    print('game_reward is: ', game_reward)\n",
        "                    successEval = np.append(successEval, game_reward)\n",
        "        \n",
        "                    if  game_reward > best_score:\n",
        "                        generate_gif(raw_frames, modelId, game_count, game_reward)\n",
        "                        best_score = game_reward\n",
        "                        print('Generating GIF  **************************** Generating Gif ****************')\n",
        "     \n",
        "                    if remaining_eval_games == 0:\n",
        "                        agent.epsilon = agent.min_epsilon \n",
        "\n",
        "                    assert agent.epsilon.device[-5:].lower() == USE_DEVICE[-5:].lower(), \"agent.epsilon not on : %s\" % USE_DEVICE\n",
        "                \n",
        "                with file_writer.as_default():\n",
        "                    with tf.device(USE_DEVICE):\n",
        "                        tf.summary.scalar('eval', np.mean(successEval), step=global_steps)\n",
        "                        tf.summary.scalar('eval-var', np.var(successEval), step=global_steps)\n",
        "                        tf.summary.histogram('scores', successEval, step=global_steps)\n",
        "                \n",
        "                print('Evaluation games average score is %s ' % np.mean(successEval)) \n",
        "                print('Evaluation games score variance is %s ' % np.var(successEval))\n",
        "\n",
        "\n",
        "            successMemory = np.append(successMemory,game_reward)\n",
        "            successFrame = np.append(successFrame,np.mean(successMemory[-10:successMemory.size]))\n",
        "           \n",
        "            actions_distrib = np.histogram(agent.exp_buffer.actions[-steps:], bins=[0,1,2,3,4,5,6], density=True)\n",
        "\n",
        "            print('Memory contains %s samples' % agent.exp_buffer.count())       \n",
        "            print('Reward over 10 games is: %s and loss is: %s' % (successFrame[-1],loss[0]))\n",
        "            print('Actions distribution (last game, %) is: ', (100 * actions_distrib[0]).astype(int))\n",
        "            print('Steps survived: %s' % (steps+1))\n",
        " \n",
        "            # Add user custom data to TensorBoard\n",
        "            with file_writer.as_default():\n",
        "                with tf.device(USE_DEVICE):\n",
        "                    tf.summary.scalar('loss', loss[0], step=global_steps)\n",
        "                    tf.summary.scalar('epsilon', agent.epsilon, step=global_steps)\n",
        "                    tf.summary.scalar('score', game_reward, step=global_steps)\n",
        "                    tf.summary.scalar('steps', steps, step=global_steps)\n",
        "                    tf.summary.histogram('actions', agent.exp_buffer.actions[-steps:], step=global_steps)\n",
        "\n",
        "            previous_time = lap_time\n",
        "            lap_time = time.time()\n",
        "            print(\"Image processing time for the last game: \", process_time)\n",
        "            print(\"Train time for the last game: \", train_time)\n",
        "            print(\"Elapsed time for the last game: \", lap_time - previous_time)\n",
        "            \n",
        "            #if game_count == 1:\n",
        "            #    break\n",
        "            \n",
        "            game_count += 1  \n",
        "            \n",
        "    except KeyboardInterrupt:\n",
        "\n",
        "        print('Save the model')\n",
        "        save_theModel(agent.model, modelId, game_count)\n",
        "        file_writer.close()    \n",
        "\n",
        "        raise\n",
        "\n",
        "    print('Save the model ', modelId)\n",
        "    save_theModel(agent.model, modelId, game_count)\n",
        "    file_writer.close()   "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2xg5vYDe8MQ",
        "colab_type": "text"
      },
      "source": [
        "# Utilities\n",
        "\n",
        "Here we will declare three new functions to:\n",
        " \n",
        "\n",
        "*   Preprocess the frames retrieved from the environment. We need to scale them down to process a reasonable amount of pixels,\n",
        "*   Generate an animated gif,\n",
        "*   Save the model.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1i5bHavze8V6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess(image):\n",
        "    \n",
        "    img_gray = tf.image.rgb_to_grayscale(image)\n",
        "    img_cropped = tf.image.crop_to_bounding_box(img_gray, 34, 0, 160, 160)\n",
        "    img_resized = tf.image.resize(img_cropped, [IMG_HEIGHT, IMG_WIDTH], method='nearest')\n",
        "\n",
        "    return img_resized    "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Krzw7LLje_Uq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generate_gif(frames, pathName, game_count, game_reward):\n",
        "\n",
        "    for idx, frame_idx in enumerate(frames): \n",
        "        frames[idx] = resize(frame_idx, (420, 320, 3), preserve_range=True, order=0).astype(np.uint8)\n",
        "        \n",
        "    imageio.mimsave(f'{SAVE_DIR}{\"/GymBreakout-{}-{}-{}.gif\".format(pathName, game_count, game_reward)}', frames, duration=1/30)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5lnh2sGHfCEO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def save_theModel(model, pathName, game_count):\n",
        "    \n",
        "    now_save = pathName + '_' + str(game_count)\n",
        "    modelPath = \"{}/GymBreakout-{}.h5\".format(SAVE_DIR, now_save)\n",
        "    model.save(modelPath)\n",
        "    print('Saved model: ', modelPath)\n",
        "    print(datetime.utcnow().strftime(\"%a, %d %b %Y %H:%M:%S +0000\"))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Uy7EunlofT39",
        "colab_type": "text"
      },
      "source": [
        "# Our main\n",
        "\n",
        "The last step, our \"main\", is to instantiate the different objects and seed the random number generators. Most importantly, we will create the keras optimizer. We will use Adam, a different optimizer than the one used by Deepmind (RMSProp). \n",
        "\n",
        "```\n",
        "optimizer = tf.keras.optimizers.Adam(ALPHA, epsilon=1e-8)\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fUeVGUlSbBOZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# If running in Collab, uncomment the following two lines to display tensorboard. \n",
        "# You should have no actve dashboards until your run the following cells\n",
        "# Be aware that you're going to crash your session around game 1800.\n",
        "# This process needs about 30Gb of RAM to store 1.000.000 experiences ;-)\n",
        "\n",
        "#%load_ext tensorboard\n",
        "#%tensorboard --logdir tf_logs"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-dNAmI70fUCJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "57fb6e72-e23d-4f2d-841e-9f23378115af"
      },
      "source": [
        "# Please allocate a GPU to this notebook.\n",
        "gpus = tf.config.list_physical_devices('GPU')\n",
        "\n",
        "assert len(gpus) > 0, \"Please allocate a GPU to this notebook.\"\n",
        "print('GPUS are: ', gpus)\n",
        "\n",
        "for gpu in gpus:\n",
        "    tf.config.experimental.set_memory_growth(gpu,True)\n",
        "\n",
        "# To be sure everything runs on GPU, set to True\n",
        "tf.debugging.set_log_device_placement(False)\n",
        "\n",
        "now = datetime.utcnow().strftime(\"%Y%m%d%H%M%S\")\n",
        "\n",
        "# Seeding the random\n",
        "# Don't forget to seed the network activation function if needed\n",
        "np.random.seed(seed=42)\n",
        "#random.seed(43)\n",
        "tf.random.set_seed(44)\n",
        "\n",
        "# Create env\n",
        "env = gym.make('BreakoutDeterministic-v0')\n",
        "env.seed = 45\n",
        "\n",
        "print(\"obs shape is: \", env.observation_space.shape)\n",
        "print(\"actions space is: \", env.action_space.n)\n",
        "actions = env.unwrapped.get_action_meanings()\n",
        "print('actions are: ', actions)\n",
        "\n",
        "# Build Model\n",
        "with tf.device(USE_DEVICE):\n",
        "\n",
        "    model = build_keras_Seq()\n",
        "    target_model = build_keras_Seq()\n",
        "    optimizer = tf.keras.optimizers.Adam(ALPHA, epsilon=1e-8)\n",
        " \n",
        "memory = ExperienceBuffer()\n",
        "agent = Agent(env, model, target_model, optimizer, memory)\n",
        "\n",
        "#Training\n",
        "run_training(agent, now)\n",
        "\n",
        "# Close env \n",
        "env.close()\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[1;30;43mStreaming output truncated to the last 5000 lines.\u001b[0m\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 269417\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 269417 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.011375404\n",
            "Actions distribution (last game, %) is:  [17 32 29 20  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.31781625747680664\n",
            "Train time for the last game:  2.8231728076934814\n",
            "Elapsed time for the last game:  6.19070029258728\n",
            "\n",
            "Game 1562 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 269569\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 269569 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.011514792\n",
            "Actions distribution (last game, %) is:  [26 29 17 26  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40149736404418945\n",
            "Train time for the last game:  3.466017961502075\n",
            "Elapsed time for the last game:  8.014976501464844\n",
            "\n",
            "Game 1563 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 269818\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 269818 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.011639054\n",
            "Actions distribution (last game, %) is:  [23 30 26 19  0  0]\n",
            "Steps survived: 249\n",
            "Image processing time for the last game:  0.7756483554840088\n",
            "Train time for the last game:  6.117579936981201\n",
            "Elapsed time for the last game:  10.426068544387817\n",
            "\n",
            "Game 1564 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 269988\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 269988 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.011926414\n",
            "Actions distribution (last game, %) is:  [24 26 20 28  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.45156383514404297\n",
            "Train time for the last game:  3.9185950756073\n",
            "Elapsed time for the last game:  7.562713384628296\n",
            "\n",
            "Game 1565 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 270113\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 270113 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012059875\n",
            "Actions distribution (last game, %) is:  [21 26 25 26  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32422900199890137\n",
            "Train time for the last game:  2.9050166606903076\n",
            "Elapsed time for the last game:  7.336198329925537\n",
            "\n",
            "Game 1566 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 270332\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 270332 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.011593615\n",
            "Actions distribution (last game, %) is:  [23 22 27 26  0  0]\n",
            "Steps survived: 219\n",
            "Image processing time for the last game:  0.5837781429290771\n",
            "Train time for the last game:  4.9922034740448\n",
            "Elapsed time for the last game:  8.89718770980835\n",
            "\n",
            "Game 1567 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 270503\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 270503 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.011461908\n",
            "Actions distribution (last game, %) is:  [20 26 28 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.4560375213623047\n",
            "Train time for the last game:  3.934593439102173\n",
            "Elapsed time for the last game:  7.590258598327637\n",
            "\n",
            "Game 1568 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 270627\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 270627 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.010556227\n",
            "Actions distribution (last game, %) is:  [24 27 23 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32373952865600586\n",
            "Train time for the last game:  2.858386993408203\n",
            "Elapsed time for the last game:  7.624205827713013\n",
            "\n",
            "Game 1569 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 270826\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 270826 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.011661342\n",
            "Actions distribution (last game, %) is:  [21 25 29 23  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5397896766662598\n",
            "Train time for the last game:  4.613198518753052\n",
            "Elapsed time for the last game:  8.456337451934814\n",
            "\n",
            "Game 1570 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 271024\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271024 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012482539\n",
            "Actions distribution (last game, %) is:  [25 27 26 19  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.535794734954834\n",
            "Train time for the last game:  4.575385570526123\n",
            "Elapsed time for the last game:  8.41172981262207\n",
            "\n",
            "Game 1571 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 271148\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271148 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.010536646\n",
            "Actions distribution (last game, %) is:  [22 31 17 27  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33388280868530273\n",
            "Train time for the last game:  2.8594822883605957\n",
            "Elapsed time for the last game:  7.391193389892578\n",
            "\n",
            "Game 1572 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 271322\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271322 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012738528\n",
            "Actions distribution (last game, %) is:  [19 23 28 29  0  0]\n",
            "Steps survived: 174\n",
            "Image processing time for the last game:  0.4571874141693115\n",
            "Train time for the last game:  3.961298704147339\n",
            "Elapsed time for the last game:  7.541697025299072\n",
            "\n",
            "Game 1573 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 271522\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271522 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.0119741885\n",
            "Actions distribution (last game, %) is:  [24 30 22 22  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5264730453491211\n",
            "Train time for the last game:  4.505585670471191\n",
            "Elapsed time for the last game:  8.344477415084839\n",
            "\n",
            "Game 1574 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 271647\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271647 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.01238481\n",
            "Actions distribution (last game, %) is:  [24 30 20 25  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3356330394744873\n",
            "Train time for the last game:  2.9213104248046875\n",
            "Elapsed time for the last game:  7.470252752304077\n",
            "\n",
            "Game 1575 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 271829\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271829 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.011840479\n",
            "Actions distribution (last game, %) is:  [30 23 18 27  0  0]\n",
            "Steps survived: 182\n",
            "Image processing time for the last game:  0.4797699451446533\n",
            "Train time for the last game:  4.152455568313599\n",
            "Elapsed time for the last game:  7.943694114685059\n",
            "\n",
            "Game 1576 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 271999\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 271999 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.011859372\n",
            "Actions distribution (last game, %) is:  [21 26 26 24  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4418954849243164\n",
            "Train time for the last game:  3.8755404949188232\n",
            "Elapsed time for the last game:  7.722672939300537\n",
            "\n",
            "Game 1577 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 272226\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 272226 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.012877302\n",
            "Actions distribution (last game, %) is:  [22 26 25 25  0  0]\n",
            "Steps survived: 227\n",
            "Image processing time for the last game:  0.6013534069061279\n",
            "Train time for the last game:  5.144536972045898\n",
            "Elapsed time for the last game:  10.155436515808105\n",
            "\n",
            "Game 1578 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 272424\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 272424 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.011465954\n",
            "Actions distribution (last game, %) is:  [23 25 21 28  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.530181884765625\n",
            "Train time for the last game:  4.579597234725952\n",
            "Elapsed time for the last game:  8.435170888900757\n",
            "\n",
            "Game 1579 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 272576\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 272576 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.012314214\n",
            "Actions distribution (last game, %) is:  [23 25 20 30  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40192461013793945\n",
            "Train time for the last game:  3.470642566680908\n",
            "Elapsed time for the last game:  7.092056751251221\n",
            "\n",
            "Game 1580 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 272700\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 272700 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.011475006\n",
            "Actions distribution (last game, %) is:  [30 25 21 23  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.34459996223449707\n",
            "Train time for the last game:  2.811295509338379\n",
            "Elapsed time for the last game:  7.328328371047974\n",
            "\n",
            "Game 1581 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 272824\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 272824 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.011832809\n",
            "Actions distribution (last game, %) is:  [26 28 23 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3243100643157959\n",
            "Train time for the last game:  2.851851463317871\n",
            "Elapsed time for the last game:  6.270069599151611\n",
            "\n",
            "Game 1582 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 273022\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273022 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.011590834\n",
            "Actions distribution (last game, %) is:  [27 28 24 18  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.514089822769165\n",
            "Train time for the last game:  4.491820335388184\n",
            "Elapsed time for the last game:  8.320234298706055\n",
            "\n",
            "Game 1583 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 273146\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273146 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.0126448\n",
            "Actions distribution (last game, %) is:  [25 22 27 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3300633430480957\n",
            "Train time for the last game:  2.821699619293213\n",
            "Elapsed time for the last game:  7.329159259796143\n",
            "\n",
            "Game 1584 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 273270\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273270 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.014140741\n",
            "Actions distribution (last game, %) is:  [17 24 30 28  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33129453659057617\n",
            "Train time for the last game:  2.8129348754882812\n",
            "Elapsed time for the last game:  6.232624769210815\n",
            "\n",
            "Game 1585 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 5.0\n",
            "Global_steps is: 273613\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273613 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.012219526\n",
            "Actions distribution (last game, %) is:  [19 28 25 26  0  0]\n",
            "Steps survived: 343\n",
            "Image processing time for the last game:  0.8991460800170898\n",
            "Train time for the last game:  7.7480628490448\n",
            "Elapsed time for the last game:  12.334818363189697\n",
            "\n",
            "Game 1586 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 273813\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273813 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.013049933\n",
            "Actions distribution (last game, %) is:  [24 31 20 24  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5179715156555176\n",
            "Train time for the last game:  4.49121618270874\n",
            "Elapsed time for the last game:  9.346062660217285\n",
            "\n",
            "Game 1587 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 273938\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 273938 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012287962\n",
            "Actions distribution (last game, %) is:  [20 29 27 22  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32942652702331543\n",
            "Train time for the last game:  2.8639321327209473\n",
            "Elapsed time for the last game:  6.4081807136535645\n",
            "\n",
            "Game 1588 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 274063\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274063 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.012306117\n",
            "Actions distribution (last game, %) is:  [26 29 24 19  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3313453197479248\n",
            "Train time for the last game:  2.8765549659729004\n",
            "Elapsed time for the last game:  6.40760064125061\n",
            "\n",
            "Game 1589 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 274236\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274236 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013072726\n",
            "Actions distribution (last game, %) is:  [22 25 24 27  0  0]\n",
            "Steps survived: 173\n",
            "Image processing time for the last game:  0.46696996688842773\n",
            "Train time for the last game:  4.036667346954346\n",
            "Elapsed time for the last game:  8.858391284942627\n",
            "\n",
            "Game 1590 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 274435\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274435 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012557603\n",
            "Actions distribution (last game, %) is:  [19 28 28 23  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.525611400604248\n",
            "Train time for the last game:  4.567718982696533\n",
            "Elapsed time for the last game:  8.420197248458862\n",
            "\n",
            "Game 1591 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 274682\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274682 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.012755219\n",
            "Actions distribution (last game, %) is:  [27 23 21 27  0  0]\n",
            "Steps survived: 247\n",
            "Image processing time for the last game:  0.6409926414489746\n",
            "Train time for the last game:  5.607564449310303\n",
            "Elapsed time for the last game:  9.874497175216675\n",
            "\n",
            "Game 1592 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 274807\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274807 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.012250076\n",
            "Actions distribution (last game, %) is:  [25 28 26 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3200831413269043\n",
            "Train time for the last game:  2.8677120208740234\n",
            "Elapsed time for the last game:  7.346582889556885\n",
            "\n",
            "Game 1593 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 274977\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 274977 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.013288864\n",
            "Actions distribution (last game, %) is:  [23 24 26 25  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4355900287628174\n",
            "Train time for the last game:  3.8267807960510254\n",
            "Elapsed time for the last game:  7.596372127532959\n",
            "\n",
            "Game 1594 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 275162\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 275162 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.012812415\n",
            "Actions distribution (last game, %) is:  [22 28 26 22  0  0]\n",
            "Steps survived: 185\n",
            "Image processing time for the last game:  0.4816930294036865\n",
            "Train time for the last game:  4.2449564933776855\n",
            "Elapsed time for the last game:  8.267547607421875\n",
            "\n",
            "Game 1595 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 275409\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 275409 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.013218859\n",
            "Actions distribution (last game, %) is:  [22 27 32 17  0  0]\n",
            "Steps survived: 247\n",
            "Image processing time for the last game:  0.6637969017028809\n",
            "Train time for the last game:  5.60320782661438\n",
            "Elapsed time for the last game:  10.726272344589233\n",
            "\n",
            "Game 1596 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 275578\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 275578 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.013147403\n",
            "Actions distribution (last game, %) is:  [24 29 23 22  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.4369010925292969\n",
            "Train time for the last game:  3.8269481658935547\n",
            "Elapsed time for the last game:  7.663027286529541\n",
            "\n",
            "Game 1597 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 275780\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 275780 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.011531287\n",
            "Actions distribution (last game, %) is:  [26 29 19 23  0  0]\n",
            "Steps survived: 202\n",
            "Image processing time for the last game:  0.5237481594085693\n",
            "Train time for the last game:  4.568331003189087\n",
            "Elapsed time for the last game:  8.512979745864868\n",
            "\n",
            "Game 1598 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 275904\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 275904 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.012451666\n",
            "Actions distribution (last game, %) is:  [23 31 21 23  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32288360595703125\n",
            "Train time for the last game:  2.8093581199645996\n",
            "Elapsed time for the last game:  7.348409414291382\n",
            "\n",
            "Game 1599 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 276056\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276056 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.012675879\n",
            "Actions distribution (last game, %) is:  [24 31 24 19  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40402960777282715\n",
            "Train time for the last game:  3.5121874809265137\n",
            "Elapsed time for the last game:  7.171745777130127\n",
            "\n",
            "Game 1600 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 276282\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276282 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.012669317\n",
            "Actions distribution (last game, %) is:  [25 24 26 23  0  0]\n",
            "Steps survived: 226\n",
            "Image processing time for the last game:  0.6021111011505127\n",
            "Train time for the last game:  5.184024095535278\n",
            "Elapsed time for the last game:  9.269934892654419\n",
            "\n",
            "Game 1601 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 276407\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276407 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.013385071\n",
            "Actions distribution (last game, %) is:  [20 27 31 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.36785459518432617\n",
            "Train time for the last game:  2.9844448566436768\n",
            "Elapsed time for the last game:  7.6223626136779785\n",
            "\n",
            "Game 1602 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 276578\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276578 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.014623202\n",
            "Actions distribution (last game, %) is:  [26 24 27 21  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.5184471607208252\n",
            "Train time for the last game:  4.105808973312378\n",
            "Elapsed time for the last game:  8.092472314834595\n",
            "\n",
            "Game 1603 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 276701\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276701 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.013253961\n",
            "Actions distribution (last game, %) is:  [27 25 26 20  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.3209521770477295\n",
            "Train time for the last game:  2.8500640392303467\n",
            "Elapsed time for the last game:  6.507250070571899\n",
            "\n",
            "Game 1604 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 276824\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 276824 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.014401572\n",
            "Actions distribution (last game, %) is:  [23 28 23 23  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.33001160621643066\n",
            "Train time for the last game:  2.8663594722747803\n",
            "Elapsed time for the last game:  7.416454076766968\n",
            "\n",
            "Game 1605 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 277024\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277024 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013532469\n",
            "Actions distribution (last game, %) is:  [22 21 32 24  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5295100212097168\n",
            "Train time for the last game:  4.5487072467803955\n",
            "Elapsed time for the last game:  8.485687494277954\n",
            "\n",
            "Game 1606 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 277176\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277176 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013257682\n",
            "Actions distribution (last game, %) is:  [20 29 19 31  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40421342849731445\n",
            "Train time for the last game:  3.4748048782348633\n",
            "Elapsed time for the last game:  7.248218536376953\n",
            "\n",
            "Game 1607 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 277377\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277377 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.012897614\n",
            "Actions distribution (last game, %) is:  [25 28 23 23  0  0]\n",
            "Steps survived: 201\n",
            "Image processing time for the last game:  0.5372650623321533\n",
            "Train time for the last game:  4.622088193893433\n",
            "Elapsed time for the last game:  9.57193899154663\n",
            "\n",
            "Game 1608 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 277548\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277548 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.012909171\n",
            "Actions distribution (last game, %) is:  [27 24 24 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.4527134895324707\n",
            "Train time for the last game:  3.8961079120635986\n",
            "Elapsed time for the last game:  7.802360534667969\n",
            "\n",
            "Game 1609 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 277701\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277701 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.013585593\n",
            "Actions distribution (last game, %) is:  [23 28 20 27  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.40714335441589355\n",
            "Train time for the last game:  3.5449838638305664\n",
            "Elapsed time for the last game:  7.375719785690308\n",
            "\n",
            "Game 1610 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 277977\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 277977 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012631892\n",
            "Actions distribution (last game, %) is:  [21 26 25 26  0  0]\n",
            "Steps survived: 276\n",
            "Image processing time for the last game:  0.7420341968536377\n",
            "Train time for the last game:  6.341752529144287\n",
            "Elapsed time for the last game:  11.69580864906311\n",
            "\n",
            "Game 1611 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 278207\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 278207 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.013216639\n",
            "Actions distribution (last game, %) is:  [26 23 25 24  0  0]\n",
            "Steps survived: 230\n",
            "Image processing time for the last game:  0.6158711910247803\n",
            "Train time for the last game:  5.2336297035217285\n",
            "Elapsed time for the last game:  9.450817584991455\n",
            "\n",
            "Game 1612 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 278426\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 278426 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.013748118\n",
            "Actions distribution (last game, %) is:  [20 22 31 25  0  0]\n",
            "Steps survived: 219\n",
            "Image processing time for the last game:  0.5750820636749268\n",
            "Train time for the last game:  4.9912145137786865\n",
            "Elapsed time for the last game:  9.270028591156006\n",
            "\n",
            "Game 1613 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 278580\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 278580 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.0125599615\n",
            "Actions distribution (last game, %) is:  [27 28 19 24  0  0]\n",
            "Steps survived: 154\n",
            "Image processing time for the last game:  0.3971748352050781\n",
            "Train time for the last game:  3.512697696685791\n",
            "Elapsed time for the last game:  8.192774772644043\n",
            "\n",
            "Game 1614 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 278800\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 278800 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.0144447815\n",
            "Actions distribution (last game, %) is:  [21 26 26 26  0  0]\n",
            "Steps survived: 220\n",
            "Image processing time for the last game:  0.5761592388153076\n",
            "Train time for the last game:  5.020500421524048\n",
            "Elapsed time for the last game:  9.111176490783691\n",
            "\n",
            "Game 1615 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 278952\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 278952 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.012887412\n",
            "Actions distribution (last game, %) is:  [25 20 28 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3923046588897705\n",
            "Train time for the last game:  3.458858013153076\n",
            "Elapsed time for the last game:  7.303613662719727\n",
            "\n",
            "Game 1616 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 279201\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279201 samples\n",
            "Reward over 10 games is: 2.0 and loss is: 0.0148378005\n",
            "Actions distribution (last game, %) is:  [27 22 27 22  0  0]\n",
            "Steps survived: 249\n",
            "Image processing time for the last game:  0.653925895690918\n",
            "Train time for the last game:  5.732290506362915\n",
            "Elapsed time for the last game:  10.877414226531982\n",
            "\n",
            "Game 1617 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 279354\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279354 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.014009856\n",
            "Actions distribution (last game, %) is:  [25 30 24 20  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.39977335929870605\n",
            "Train time for the last game:  3.5083224773406982\n",
            "Elapsed time for the last game:  7.49352240562439\n",
            "\n",
            "Game 1618 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 279479\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279479 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.012940679\n",
            "Actions distribution (last game, %) is:  [29 21 28 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.33760571479797363\n",
            "Train time for the last game:  2.9257586002349854\n",
            "Elapsed time for the last game:  6.6928300857543945\n",
            "\n",
            "Game 1619 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 279631\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279631 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.013965426\n",
            "Actions distribution (last game, %) is:  [20 30 18 30  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.4009840488433838\n",
            "Train time for the last game:  3.5623536109924316\n",
            "Elapsed time for the last game:  8.268641233444214\n",
            "\n",
            "Game 1620 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 279755\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279755 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.013211067\n",
            "Actions distribution (last game, %) is:  [23 31 26 18  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.31968188285827637\n",
            "Train time for the last game:  2.8416171073913574\n",
            "Elapsed time for the last game:  6.590651273727417\n",
            "\n",
            "Game 1621 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 279880\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 279880 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.014507459\n",
            "Actions distribution (last game, %) is:  [24 24 24 27  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.331906795501709\n",
            "Train time for the last game:  2.8491921424865723\n",
            "Elapsed time for the last game:  6.718889236450195\n",
            "\n",
            "Game 1622 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 280080\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280080 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.012578452\n",
            "Actions distribution (last game, %) is:  [24 28 24 22  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5233304500579834\n",
            "Train time for the last game:  4.564618825912476\n",
            "Elapsed time for the last game:  9.516093015670776\n",
            "\n",
            "Game 1623 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 280280\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280280 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.014994855\n",
            "Actions distribution (last game, %) is:  [26 27 28 18  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5221927165985107\n",
            "Train time for the last game:  4.527543306350708\n",
            "Elapsed time for the last game:  8.493034601211548\n",
            "\n",
            "Game 1624 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 280450\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280450 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.013709524\n",
            "Actions distribution (last game, %) is:  [24 24 27 24  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4470663070678711\n",
            "Train time for the last game:  3.8686370849609375\n",
            "Elapsed time for the last game:  7.8677308559417725\n",
            "\n",
            "Game 1625 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 280575\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280575 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013769573\n",
            "Actions distribution (last game, %) is:  [25 25 26 23  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3347437381744385\n",
            "Train time for the last game:  2.8906054496765137\n",
            "Elapsed time for the last game:  7.480390310287476\n",
            "\n",
            "Game 1626 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 280698\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280698 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.012984858\n",
            "Actions distribution (last game, %) is:  [26 28 26 18  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.3428213596343994\n",
            "Train time for the last game:  2.882296562194824\n",
            "Elapsed time for the last game:  6.7737486362457275\n",
            "\n",
            "Game 1627 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 280872\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 280872 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.013053934\n",
            "Actions distribution (last game, %) is:  [25 25 25 23  0  0]\n",
            "Steps survived: 174\n",
            "Image processing time for the last game:  0.4489147663116455\n",
            "Train time for the last game:  3.9559438228607178\n",
            "Elapsed time for the last game:  7.938962936401367\n",
            "\n",
            "Game 1628 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 281040\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281040 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.013753398\n",
            "Actions distribution (last game, %) is:  [19 24 28 28  0  0]\n",
            "Steps survived: 168\n",
            "Image processing time for the last game:  0.44626283645629883\n",
            "Train time for the last game:  3.8289473056793213\n",
            "Elapsed time for the last game:  8.674090385437012\n",
            "\n",
            "Game 1629 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 281164\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281164 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.0131520815\n",
            "Actions distribution (last game, %) is:  [23 34 17 23  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33509016036987305\n",
            "Train time for the last game:  2.87469744682312\n",
            "Elapsed time for the last game:  6.560480356216431\n",
            "\n",
            "Game 1630 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 281414\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281414 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013838249\n",
            "Actions distribution (last game, %) is:  [26 23 23 26  0  0]\n",
            "Steps survived: 250\n",
            "Image processing time for the last game:  0.678215742111206\n",
            "Train time for the last game:  5.653562307357788\n",
            "Elapsed time for the last game:  10.081088781356812\n",
            "\n",
            "Game 1631 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 281538\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281538 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.014738078\n",
            "Actions distribution (last game, %) is:  [21 26 25 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32759642601013184\n",
            "Train time for the last game:  2.810328722000122\n",
            "Elapsed time for the last game:  7.419554710388184\n",
            "\n",
            "Game 1632 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 281662\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281662 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.014208949\n",
            "Actions distribution (last game, %) is:  [33 22 21 22  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33290648460388184\n",
            "Train time for the last game:  2.846859931945801\n",
            "Elapsed time for the last game:  6.5351033210754395\n",
            "\n",
            "Game 1633 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 281859\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 281859 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.014196025\n",
            "Actions distribution (last game, %) is:  [27 23 23 25  0  0]\n",
            "Steps survived: 197\n",
            "Image processing time for the last game:  0.5175449848175049\n",
            "Train time for the last game:  4.485012054443359\n",
            "Elapsed time for the last game:  8.682981967926025\n",
            "\n",
            "Game 1634 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 282057\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282057 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.014137852\n",
            "Actions distribution (last game, %) is:  [23 24 22 28  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5209159851074219\n",
            "Train time for the last game:  4.506000518798828\n",
            "Elapsed time for the last game:  9.530052661895752\n",
            "\n",
            "Game 1635 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 282208\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282208 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.014321724\n",
            "Actions distribution (last game, %) is:  [23 31 21 24  0  0]\n",
            "Steps survived: 151\n",
            "Image processing time for the last game:  0.38851094245910645\n",
            "Train time for the last game:  3.504558801651001\n",
            "Elapsed time for the last game:  7.481394290924072\n",
            "\n",
            "Game 1636 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 282427\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282427 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.013306266\n",
            "Actions distribution (last game, %) is:  [25 23 22 27  0  0]\n",
            "Steps survived: 219\n",
            "Image processing time for the last game:  0.5832705497741699\n",
            "Train time for the last game:  4.984607696533203\n",
            "Elapsed time for the last game:  9.254898309707642\n",
            "\n",
            "Game 1637 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 282551\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282551 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.012301198\n",
            "Actions distribution (last game, %) is:  [23 26 23 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3314807415008545\n",
            "Train time for the last game:  2.862837791442871\n",
            "Elapsed time for the last game:  7.529016017913818\n",
            "\n",
            "Game 1638 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 282722\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282722 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.014744638\n",
            "Actions distribution (last game, %) is:  [25 30 18 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.44588398933410645\n",
            "Train time for the last game:  3.975701093673706\n",
            "Elapsed time for the last game:  8.07233452796936\n",
            "\n",
            "Game 1639 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 282847\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 282847 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.014364281\n",
            "Actions distribution (last game, %) is:  [31 20 24 23  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.35897183418273926\n",
            "Train time for the last game:  3.041682243347168\n",
            "Elapsed time for the last game:  6.95081901550293\n",
            "\n",
            "Game 1640 - Run 20200911003313\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 283141\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283141 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0143055525\n",
            "Actions distribution (last game, %) is:  [23 25 22 28  0  0]\n",
            "Steps survived: 294\n",
            "Image processing time for the last game:  0.88234543800354\n",
            "Train time for the last game:  6.939437627792358\n",
            "Elapsed time for the last game:  12.601443529129028\n",
            "\n",
            "Game 1641 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 283264\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283264 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.013556273\n",
            "Actions distribution (last game, %) is:  [22 29 23 24  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32311534881591797\n",
            "Train time for the last game:  2.8415708541870117\n",
            "Elapsed time for the last game:  6.710163593292236\n",
            "\n",
            "Game 1642 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 283389\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283389 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.013318656\n",
            "Actions distribution (last game, %) is:  [22 20 27 29  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3271656036376953\n",
            "Train time for the last game:  2.8838701248168945\n",
            "Elapsed time for the last game:  6.723494052886963\n",
            "\n",
            "Game 1643 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 283570\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283570 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0135274185\n",
            "Actions distribution (last game, %) is:  [22 24 24 28  0  0]\n",
            "Steps survived: 181\n",
            "Image processing time for the last game:  0.47730278968811035\n",
            "Train time for the last game:  4.147144794464111\n",
            "Elapsed time for the last game:  9.032408475875854\n",
            "\n",
            "Game 1644 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 283695\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283695 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.013826306\n",
            "Actions distribution (last game, %) is:  [23 28 29 18  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32714247703552246\n",
            "Train time for the last game:  2.884190082550049\n",
            "Elapsed time for the last game:  6.658751487731934\n",
            "\n",
            "Game 1645 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 283847\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 283847 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.0141719505\n",
            "Actions distribution (last game, %) is:  [29 26 23 20  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.4077422618865967\n",
            "Train time for the last game:  3.472644805908203\n",
            "Elapsed time for the last game:  7.593807935714722\n",
            "\n",
            "Game 1646 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 284105\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284105 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.014781077\n",
            "Actions distribution (last game, %) is:  [22 25 25 26  0  0]\n",
            "Steps survived: 258\n",
            "Image processing time for the last game:  0.6685843467712402\n",
            "Train time for the last game:  5.867065668106079\n",
            "Elapsed time for the last game:  11.125482320785522\n",
            "\n",
            "Game 1647 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 284229\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284229 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0154553875\n",
            "Actions distribution (last game, %) is:  [17 29 24 28  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32493019104003906\n",
            "Train time for the last game:  2.8328137397766113\n",
            "Elapsed time for the last game:  6.748178243637085\n",
            "\n",
            "Game 1648 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 5.0\n",
            "Player is dead! game_reward is: 5.0\n",
            "Player is dead! game_reward is: 5.0\n",
            "Player is dead! game_reward is: 5.0\n",
            "Global_steps is: 284529\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284529 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.01482248\n",
            "Actions distribution (last game, %) is:  [22 28 25 24  0  0]\n",
            "Steps survived: 300\n",
            "Image processing time for the last game:  0.8034327030181885\n",
            "Train time for the last game:  6.881736516952515\n",
            "Elapsed time for the last game:  11.861646175384521\n",
            "\n",
            "Game 1649 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 284654\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284654 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.015701614\n",
            "Actions distribution (last game, %) is:  [25 27 25 21  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3407413959503174\n",
            "Train time for the last game:  2.8774325847625732\n",
            "Elapsed time for the last game:  7.497637510299683\n",
            "\n",
            "Game 1650 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 284779\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284779 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.012910668\n",
            "Actions distribution (last game, %) is:  [22 23 27 26  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.33156728744506836\n",
            "Train time for the last game:  2.8850340843200684\n",
            "Elapsed time for the last game:  6.667083263397217\n",
            "\n",
            "Game 1651 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 284931\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 284931 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.0143544795\n",
            "Actions distribution (last game, %) is:  [27 23 22 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40201854705810547\n",
            "Train time for the last game:  3.437213182449341\n",
            "Elapsed time for the last game:  7.582465410232544\n",
            "\n",
            "Game 1652 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 285118\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 285118 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.014980832\n",
            "Actions distribution (last game, %) is:  [23 21 24 30  0  0]\n",
            "Steps survived: 187\n",
            "Image processing time for the last game:  0.49062347412109375\n",
            "Train time for the last game:  4.256166934967041\n",
            "Elapsed time for the last game:  9.188369035720825\n",
            "\n",
            "Game 1653 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 285348\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 285348 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.014439598\n",
            "Actions distribution (last game, %) is:  [20 27 26 25  0  0]\n",
            "Steps survived: 230\n",
            "Image processing time for the last game:  0.6001598834991455\n",
            "Train time for the last game:  5.197162866592407\n",
            "Elapsed time for the last game:  9.508928775787354\n",
            "\n",
            "Game 1654 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 285472\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 285472 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.015489674\n",
            "Actions distribution (last game, %) is:  [34 24 19 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3272826671600342\n",
            "Train time for the last game:  2.8374135494232178\n",
            "Elapsed time for the last game:  6.753492116928101\n",
            "\n",
            "Game 1655 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 285690\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 285690 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.015310921\n",
            "Actions distribution (last game, %) is:  [20 30 29 19  0  0]\n",
            "Steps survived: 218\n",
            "Image processing time for the last game:  0.5815327167510986\n",
            "Train time for the last game:  4.944793701171875\n",
            "Elapsed time for the last game:  10.040253400802612\n",
            "\n",
            "Game 1656 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 285873\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 285873 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.01509263\n",
            "Actions distribution (last game, %) is:  [26 25 23 24  0  0]\n",
            "Steps survived: 183\n",
            "Image processing time for the last game:  0.4750821590423584\n",
            "Train time for the last game:  4.180670499801636\n",
            "Elapsed time for the last game:  8.300517797470093\n",
            "\n",
            "Game 1657 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 286072\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 286072 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.0138959205\n",
            "Actions distribution (last game, %) is:  [24 27 27 21  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.516556978225708\n",
            "Train time for the last game:  4.501212120056152\n",
            "Elapsed time for the last game:  8.953924179077148\n",
            "\n",
            "Game 1658 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 286269\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 286269 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.015470948\n",
            "Actions distribution (last game, %) is:  [23 31 23 21  0  0]\n",
            "Steps survived: 197\n",
            "Image processing time for the last game:  0.522059440612793\n",
            "Train time for the last game:  4.5643837451934814\n",
            "Elapsed time for the last game:  9.600494623184204\n",
            "\n",
            "Game 1659 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 286452\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 286452 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.015486088\n",
            "Actions distribution (last game, %) is:  [22 30 20 26  0  0]\n",
            "Steps survived: 183\n",
            "Image processing time for the last game:  0.4895484447479248\n",
            "Train time for the last game:  4.181905031204224\n",
            "Elapsed time for the last game:  8.44865083694458\n",
            "\n",
            "Game 1660 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 286577\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 286577 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.017177373\n",
            "Actions distribution (last game, %) is:  [19 21 38 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3224775791168213\n",
            "Train time for the last game:  2.879509687423706\n",
            "Elapsed time for the last game:  6.8203511238098145\n",
            "\n",
            "Game 1661 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 286842\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 286842 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.014852979\n",
            "Actions distribution (last game, %) is:  [26 24 23 26  0  0]\n",
            "Steps survived: 265\n",
            "Image processing time for the last game:  0.6948142051696777\n",
            "Train time for the last game:  6.036391258239746\n",
            "Elapsed time for the last game:  11.412861585617065\n",
            "\n",
            "Game 1662 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 287092\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287092 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.015664216\n",
            "Actions distribution (last game, %) is:  [26 23 24 26  0  0]\n",
            "Steps survived: 250\n",
            "Image processing time for the last game:  0.6648766994476318\n",
            "Train time for the last game:  5.764450550079346\n",
            "Elapsed time for the last game:  10.355046272277832\n",
            "\n",
            "Game 1663 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 287245\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287245 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.015166628\n",
            "Actions distribution (last game, %) is:  [21 27 29 21  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.4132704734802246\n",
            "Train time for the last game:  3.563575267791748\n",
            "Elapsed time for the last game:  7.6652445793151855\n",
            "\n",
            "Game 1664 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 287472\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287472 samples\n",
            "Reward over 10 games is: 2.0 and loss is: 0.0167041\n",
            "Actions distribution (last game, %) is:  [27 22 24 24  0  0]\n",
            "Steps survived: 227\n",
            "Image processing time for the last game:  0.5969059467315674\n",
            "Train time for the last game:  5.194781541824341\n",
            "Elapsed time for the last game:  10.319003820419312\n",
            "\n",
            "Game 1665 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 287595\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287595 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.0138027845\n",
            "Actions distribution (last game, %) is:  [26 29 20 23  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32277488708496094\n",
            "Train time for the last game:  2.838167428970337\n",
            "Elapsed time for the last game:  6.708911180496216\n",
            "\n",
            "Game 1666 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 287822\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287822 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.014739956\n",
            "Actions distribution (last game, %) is:  [26 26 21 25  0  0]\n",
            "Steps survived: 227\n",
            "Image processing time for the last game:  0.6011064052581787\n",
            "Train time for the last game:  5.187735557556152\n",
            "Elapsed time for the last game:  9.840202331542969\n",
            "\n",
            "Game 1667 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 287946\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 287946 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.015080466\n",
            "Actions distribution (last game, %) is:  [21 28 24 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3264272212982178\n",
            "Train time for the last game:  2.9013211727142334\n",
            "Elapsed time for the last game:  7.567204475402832\n",
            "\n",
            "Game 1668 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 288069\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288069 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.0152834365\n",
            "Actions distribution (last game, %) is:  [24 32 22 20  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32938218116760254\n",
            "Train time for the last game:  2.864699602127075\n",
            "Elapsed time for the last game:  6.799898624420166\n",
            "\n",
            "Game 1669 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 288346\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288346 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.0151006235\n",
            "Actions distribution (last game, %) is:  [22 25 17 34  0  0]\n",
            "Steps survived: 277\n",
            "Image processing time for the last game:  0.7127418518066406\n",
            "Train time for the last game:  6.3049705028533936\n",
            "Elapsed time for the last game:  11.18207836151123\n",
            "\n",
            "Game 1670 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 288566\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288566 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.014930675\n",
            "Actions distribution (last game, %) is:  [26 26 23 24  0  0]\n",
            "Steps survived: 220\n",
            "Image processing time for the last game:  0.5805919170379639\n",
            "Train time for the last game:  5.028484106063843\n",
            "Elapsed time for the last game:  10.160241842269897\n",
            "\n",
            "Game 1671 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 288691\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288691 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.01501789\n",
            "Actions distribution (last game, %) is:  [26 25 26 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.33141374588012695\n",
            "Train time for the last game:  2.891709089279175\n",
            "Elapsed time for the last game:  6.814138412475586\n",
            "\n",
            "Game 1672 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 288842\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288842 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.015840184\n",
            "Actions distribution (last game, %) is:  [22 22 24 31  0  0]\n",
            "Steps survived: 151\n",
            "Image processing time for the last game:  0.3953123092651367\n",
            "Train time for the last game:  3.498953104019165\n",
            "Elapsed time for the last game:  7.767919540405273\n",
            "\n",
            "Game 1673 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 288967\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 288967 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.015034086\n",
            "Actions distribution (last game, %) is:  [26 22 25 25  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32149767875671387\n",
            "Train time for the last game:  2.888514518737793\n",
            "Elapsed time for the last game:  7.533643484115601\n",
            "\n",
            "Game 1674 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 289119\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289119 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015475198\n",
            "Actions distribution (last game, %) is:  [23 24 19 31  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.39609766006469727\n",
            "Train time for the last game:  3.5160884857177734\n",
            "Elapsed time for the last game:  7.758238315582275\n",
            "\n",
            "Game 1675 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 289318\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289318 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.015072842\n",
            "Actions distribution (last game, %) is:  [24 22 30 21  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5158393383026123\n",
            "Train time for the last game:  4.5430872440338135\n",
            "Elapsed time for the last game:  9.179846286773682\n",
            "\n",
            "Game 1676 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 289470\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289470 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015958885\n",
            "Actions distribution (last game, %) is:  [25 25 19 29  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.48195481300354004\n",
            "Train time for the last game:  3.677459955215454\n",
            "Elapsed time for the last game:  8.745736837387085\n",
            "\n",
            "Game 1677 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 289641\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289641 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.016174125\n",
            "Actions distribution (last game, %) is:  [24 28 25 21  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.4541664123535156\n",
            "Train time for the last game:  3.9550728797912598\n",
            "Elapsed time for the last game:  8.135749101638794\n",
            "\n",
            "Game 1678 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 289766\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289766 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0146678155\n",
            "Actions distribution (last game, %) is:  [19 27 24 29  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3271369934082031\n",
            "Train time for the last game:  2.914564371109009\n",
            "Elapsed time for the last game:  7.095111608505249\n",
            "\n",
            "Game 1679 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 289935\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 289935 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.015701773\n",
            "Actions distribution (last game, %) is:  [21 31 24 22  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.439694881439209\n",
            "Train time for the last game:  3.890146493911743\n",
            "Elapsed time for the last game:  8.819178581237793\n",
            "\n",
            "Game 1680 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 290059\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 290059 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.014656301\n",
            "Actions distribution (last game, %) is:  [31 25 17 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3288249969482422\n",
            "Train time for the last game:  2.8486032485961914\n",
            "Elapsed time for the last game:  6.935958623886108\n",
            "\n",
            "Game 1681 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 290260\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 290260 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017109416\n",
            "Actions distribution (last game, %) is:  [26 28 20 24  0  0]\n",
            "Steps survived: 201\n",
            "Image processing time for the last game:  0.5367550849914551\n",
            "Train time for the last game:  4.5969038009643555\n",
            "Elapsed time for the last game:  9.173683881759644\n",
            "\n",
            "Game 1682 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 290459\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 290459 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.01509024\n",
            "Actions distribution (last game, %) is:  [20 27 26 26  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5273082256317139\n",
            "Train time for the last game:  4.619345664978027\n",
            "Elapsed time for the last game:  9.706688642501831\n",
            "\n",
            "Game 1683 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 290661\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 290661 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0158518\n",
            "Actions distribution (last game, %) is:  [20 21 31 26  0  0]\n",
            "Steps survived: 202\n",
            "Image processing time for the last game:  0.5427553653717041\n",
            "Train time for the last game:  4.589287996292114\n",
            "Elapsed time for the last game:  9.133768796920776\n",
            "\n",
            "Game 1684 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 290883\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 290883 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.015648292\n",
            "Actions distribution (last game, %) is:  [30 24 23 21  0  0]\n",
            "Steps survived: 222\n",
            "Image processing time for the last game:  0.5810775756835938\n",
            "Train time for the last game:  5.05761456489563\n",
            "Elapsed time for the last game:  9.735373735427856\n",
            "\n",
            "Game 1685 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 291054\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291054 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.016233625\n",
            "Actions distribution (last game, %) is:  [20 29 26 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.44737792015075684\n",
            "Train time for the last game:  3.9272468090057373\n",
            "Elapsed time for the last game:  8.892321586608887\n",
            "\n",
            "Game 1686 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 291179\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291179 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015545434\n",
            "Actions distribution (last game, %) is:  [21 29 23 25  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32927393913269043\n",
            "Train time for the last game:  2.963569402694702\n",
            "Elapsed time for the last game:  7.000742197036743\n",
            "\n",
            "Game 1687 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 291349\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291349 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.016789962\n",
            "Actions distribution (last game, %) is:  [28 28 21 20  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4577028751373291\n",
            "Train time for the last game:  3.9113481044769287\n",
            "Elapsed time for the last game:  8.376514911651611\n",
            "\n",
            "Game 1688 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 291473\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291473 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015983086\n",
            "Actions distribution (last game, %) is:  [21 34 22 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3290882110595703\n",
            "Train time for the last game:  2.8540029525756836\n",
            "Elapsed time for the last game:  7.552217245101929\n",
            "\n",
            "Game 1689 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 291625\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291625 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015461053\n",
            "Actions distribution (last game, %) is:  [29 26 20 23  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.40468764305114746\n",
            "Train time for the last game:  3.4986839294433594\n",
            "Elapsed time for the last game:  7.818437337875366\n",
            "\n",
            "Game 1690 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 291749\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291749 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.016168\n",
            "Actions distribution (last game, %) is:  [24 22 26 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3298373222351074\n",
            "Train time for the last game:  2.8848063945770264\n",
            "Elapsed time for the last game:  7.135661840438843\n",
            "\n",
            "Game 1691 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 291962\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 291962 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.016534396\n",
            "Actions distribution (last game, %) is:  [21 29 21 27  0  0]\n",
            "Steps survived: 213\n",
            "Image processing time for the last game:  0.5617477893829346\n",
            "Train time for the last game:  4.93647837638855\n",
            "Elapsed time for the last game:  10.108593702316284\n",
            "\n",
            "Game 1692 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 292086\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 292086 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.015428811\n",
            "Actions distribution (last game, %) is:  [25 29 23 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3201887607574463\n",
            "Train time for the last game:  2.849128007888794\n",
            "Elapsed time for the last game:  6.877668142318726\n",
            "\n",
            "Game 1693 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 292257\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 292257 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.016686816\n",
            "Actions distribution (last game, %) is:  [25 28 17 28  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.4509704113006592\n",
            "Train time for the last game:  3.903488874435425\n",
            "Elapsed time for the last game:  8.242711544036865\n",
            "\n",
            "Game 1694 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 292409\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 292409 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.014618915\n",
            "Actions distribution (last game, %) is:  [26 23 20 29  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3997008800506592\n",
            "Train time for the last game:  3.466984510421753\n",
            "Elapsed time for the last game:  8.312875986099243\n",
            "\n",
            "Game 1695 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 292706\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 292706 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.015935136\n",
            "Actions distribution (last game, %) is:  [30 25 24 19  0  0]\n",
            "Steps survived: 297\n",
            "Image processing time for the last game:  0.7825305461883545\n",
            "Train time for the last game:  6.8270909786224365\n",
            "Elapsed time for the last game:  11.901009321212769\n",
            "\n",
            "Game 1696 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 292917\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 292917 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.015604222\n",
            "Actions distribution (last game, %) is:  [24 29 24 21  0  0]\n",
            "Steps survived: 211\n",
            "Image processing time for the last game:  0.571561336517334\n",
            "Train time for the last game:  4.945843935012817\n",
            "Elapsed time for the last game:  9.563366889953613\n",
            "\n",
            "Game 1697 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 293091\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293091 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.016381161\n",
            "Actions distribution (last game, %) is:  [20 28 26 24  0  0]\n",
            "Steps survived: 174\n",
            "Image processing time for the last game:  0.4546346664428711\n",
            "Train time for the last game:  3.9979405403137207\n",
            "Elapsed time for the last game:  8.941718339920044\n",
            "\n",
            "Game 1698 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 293318\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293318 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.015471087\n",
            "Actions distribution (last game, %) is:  [21 23 25 29  0  0]\n",
            "Steps survived: 227\n",
            "Image processing time for the last game:  0.6315288543701172\n",
            "Train time for the last game:  5.390931844711304\n",
            "Elapsed time for the last game:  10.196795225143433\n",
            "\n",
            "Game 1699 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 293442\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293442 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.016832003\n",
            "Actions distribution (last game, %) is:  [26 23 30 19  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3270225524902344\n",
            "Train time for the last game:  2.860896587371826\n",
            "Elapsed time for the last game:  7.001922130584717\n",
            "\n",
            "Game 1700 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 293692\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293692 samples\n",
            "Reward over 10 games is: 1.9 and loss is: 0.016960928\n",
            "Actions distribution (last game, %) is:  [22 24 30 22  0  0]\n",
            "Steps survived: 250\n",
            "Image processing time for the last game:  0.6485583782196045\n",
            "Train time for the last game:  5.714100360870361\n",
            "Elapsed time for the last game:  11.063783884048462\n",
            "\n",
            "Game 1701 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 293817\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293817 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.016491871\n",
            "Actions distribution (last game, %) is:  [26 24 25 23  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.33138585090637207\n",
            "Train time for the last game:  2.962991714477539\n",
            "Elapsed time for the last game:  7.062451124191284\n",
            "\n",
            "Game 1702 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 293970\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 293970 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.016105937\n",
            "Actions distribution (last game, %) is:  [29 26 19 23  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.40967273712158203\n",
            "Train time for the last game:  3.5227530002593994\n",
            "Elapsed time for the last game:  7.971473693847656\n",
            "\n",
            "Game 1703 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 294168\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294168 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.015617793\n",
            "Actions distribution (last game, %) is:  [21 27 26 24  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5230655670166016\n",
            "Train time for the last game:  4.6250975131988525\n",
            "Elapsed time for the last game:  9.763603925704956\n",
            "\n",
            "Game 1704 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 294291\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294291 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.016504768\n",
            "Actions distribution (last game, %) is:  [27 27 25 19  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.3288149833679199\n",
            "Train time for the last game:  2.9072813987731934\n",
            "Elapsed time for the last game:  7.059694290161133\n",
            "\n",
            "Game 1705 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 294414\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294414 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.01610254\n",
            "Actions distribution (last game, %) is:  [22 22 27 27  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32712221145629883\n",
            "Train time for the last game:  2.9872546195983887\n",
            "Elapsed time for the last game:  7.341168165206909\n",
            "\n",
            "Game 1706 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 294586\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294586 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.01700513\n",
            "Actions distribution (last game, %) is:  [27 25 23 23  0  0]\n",
            "Steps survived: 172\n",
            "Image processing time for the last game:  0.4557456970214844\n",
            "Train time for the last game:  4.005669355392456\n",
            "Elapsed time for the last game:  8.979487895965576\n",
            "\n",
            "Game 1707 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 294785\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294785 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.016423104\n",
            "Actions distribution (last game, %) is:  [24 22 32 20  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5339055061340332\n",
            "Train time for the last game:  4.639146089553833\n",
            "Elapsed time for the last game:  9.1859769821167\n",
            "\n",
            "Game 1708 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 294909\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 294909 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.015828231\n",
            "Actions distribution (last game, %) is:  [20 26 28 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3178091049194336\n",
            "Train time for the last game:  2.8975515365600586\n",
            "Elapsed time for the last game:  7.219804763793945\n",
            "\n",
            "Game 1709 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 295033\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295033 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017429827\n",
            "Actions distribution (last game, %) is:  [21 34 19 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32494068145751953\n",
            "Train time for the last game:  2.8491921424865723\n",
            "Elapsed time for the last game:  7.55338716506958\n",
            "\n",
            "Game 1710 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 295206\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295206 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.018188432\n",
            "Actions distribution (last game, %) is:  [24 21 29 25  0  0]\n",
            "Steps survived: 173\n",
            "Image processing time for the last game:  0.45862698554992676\n",
            "Train time for the last game:  4.000027418136597\n",
            "Elapsed time for the last game:  8.506917476654053\n",
            "\n",
            "Game 1711 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 295426\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295426 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.016245104\n",
            "Actions distribution (last game, %) is:  [24 27 25 22  0  0]\n",
            "Steps survived: 220\n",
            "Image processing time for the last game:  0.5837368965148926\n",
            "Train time for the last game:  5.0332396030426025\n",
            "Elapsed time for the last game:  9.709367513656616\n",
            "\n",
            "Game 1712 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 295550\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295550 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.016134141\n",
            "Actions distribution (last game, %) is:  [25 22 23 28  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.376629114151001\n",
            "Train time for the last game:  2.998077630996704\n",
            "Elapsed time for the last game:  7.872365236282349\n",
            "\n",
            "Game 1713 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 295703\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295703 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.015465817\n",
            "Actions distribution (last game, %) is:  [22 32 22 22  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.47672462463378906\n",
            "Train time for the last game:  3.7050960063934326\n",
            "Elapsed time for the last game:  8.122934818267822\n",
            "\n",
            "Game 1714 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 295827\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295827 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.016231304\n",
            "Actions distribution (last game, %) is:  [30 28 17 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33194899559020996\n",
            "Train time for the last game:  2.8866474628448486\n",
            "Elapsed time for the last game:  7.295026540756226\n",
            "\n",
            "Game 1715 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 295951\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 295951 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.017597616\n",
            "Actions distribution (last game, %) is:  [18 31 25 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32985353469848633\n",
            "Train time for the last game:  2.8976075649261475\n",
            "Elapsed time for the last game:  7.647628307342529\n",
            "\n",
            "Game 1716 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 296075\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296075 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.016386759\n",
            "Actions distribution (last game, %) is:  [22 30 22 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3430821895599365\n",
            "Train time for the last game:  2.857046604156494\n",
            "Elapsed time for the last game:  7.206933975219727\n",
            "\n",
            "Game 1717 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 296277\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296277 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.016387124\n",
            "Actions distribution (last game, %) is:  [21 29 22 26  0  0]\n",
            "Steps survived: 202\n",
            "Image processing time for the last game:  0.5372295379638672\n",
            "Train time for the last game:  4.578677415847778\n",
            "Elapsed time for the last game:  9.369694948196411\n",
            "\n",
            "Game 1718 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 296463\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296463 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.016487496\n",
            "Actions distribution (last game, %) is:  [30 19 24 24  0  0]\n",
            "Steps survived: 186\n",
            "Image processing time for the last game:  0.4912104606628418\n",
            "Train time for the last game:  4.248649835586548\n",
            "Elapsed time for the last game:  9.34850001335144\n",
            "\n",
            "Game 1719 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 296662\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296662 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.016372144\n",
            "Actions distribution (last game, %) is:  [24 30 25 20  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5175800323486328\n",
            "Train time for the last game:  4.569220304489136\n",
            "Elapsed time for the last game:  9.307737827301025\n",
            "\n",
            "Game 1720 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 296787\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296787 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.016569288\n",
            "Actions distribution (last game, %) is:  [21 29 28 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32404661178588867\n",
            "Train time for the last game:  2.910557508468628\n",
            "Elapsed time for the last game:  7.134600877761841\n",
            "\n",
            "Game 1721 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 296986\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 296986 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.015041635\n",
            "Actions distribution (last game, %) is:  [22 30 24 23  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5151963233947754\n",
            "Train time for the last game:  4.5319859981536865\n",
            "Elapsed time for the last game:  9.653210878372192\n",
            "\n",
            "Game 1722 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 297254\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297254 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.016285798\n",
            "Actions distribution (last game, %) is:  [25 24 24 25  0  0]\n",
            "Steps survived: 268\n",
            "Image processing time for the last game:  0.7022340297698975\n",
            "Train time for the last game:  6.064980506896973\n",
            "Elapsed time for the last game:  10.96092176437378\n",
            "\n",
            "Game 1723 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 297385\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297385 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.016311962\n",
            "Actions distribution (last game, %) is:  [22 23 26 27  0  0]\n",
            "Steps survived: 131\n",
            "Image processing time for the last game:  0.33586597442626953\n",
            "Train time for the last game:  3.026543140411377\n",
            "Elapsed time for the last game:  7.380175828933716\n",
            "\n",
            "Game 1724 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 297509\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297509 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.017345553\n",
            "Actions distribution (last game, %) is:  [21 27 25 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3265347480773926\n",
            "Train time for the last game:  2.890562057495117\n",
            "Elapsed time for the last game:  7.668133497238159\n",
            "\n",
            "Game 1725 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 297709\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297709 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.016500222\n",
            "Actions distribution (last game, %) is:  [29 27 22 20  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5404942035675049\n",
            "Train time for the last game:  4.542200326919556\n",
            "Elapsed time for the last game:  9.312579154968262\n",
            "\n",
            "Game 1726 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 297834\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297834 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.016505219\n",
            "Actions distribution (last game, %) is:  [18 18 32 30  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3303408622741699\n",
            "Train time for the last game:  2.8796627521514893\n",
            "Elapsed time for the last game:  7.292978286743164\n",
            "\n",
            "Game 1727 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 297958\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 297958 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.016015567\n",
            "Actions distribution (last game, %) is:  [24 30 24 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3221738338470459\n",
            "Train time for the last game:  2.8505959510803223\n",
            "Elapsed time for the last game:  7.610398769378662\n",
            "\n",
            "Game 1728 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 298083\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298083 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017065294\n",
            "Actions distribution (last game, %) is:  [24 24 33 17  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.33951425552368164\n",
            "Train time for the last game:  2.896035671234131\n",
            "Elapsed time for the last game:  7.129109859466553\n",
            "\n",
            "Game 1729 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 298208\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298208 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.01843968\n",
            "Actions distribution (last game, %) is:  [29 20 25 23  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32399535179138184\n",
            "Train time for the last game:  2.892488479614258\n",
            "Elapsed time for the last game:  7.300786256790161\n",
            "\n",
            "Game 1730 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 298406\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298406 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017749315\n",
            "Actions distribution (last game, %) is:  [21 28 24 24  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5294625759124756\n",
            "Train time for the last game:  4.495826959609985\n",
            "Elapsed time for the last game:  9.65148115158081\n",
            "\n",
            "Game 1731 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 298606\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298606 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.018099114\n",
            "Actions distribution (last game, %) is:  [27 26 24 22  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5203096866607666\n",
            "Train time for the last game:  4.589847564697266\n",
            "Elapsed time for the last game:  9.185082197189331\n",
            "\n",
            "Game 1732 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 298776\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298776 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.01806208\n",
            "Actions distribution (last game, %) is:  [24 26 24 24  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.43737077713012695\n",
            "Train time for the last game:  3.8688364028930664\n",
            "Elapsed time for the last game:  8.452878952026367\n",
            "\n",
            "Game 1733 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 298900\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 298900 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.01615443\n",
            "Actions distribution (last game, %) is:  [23 24 23 28  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3219611644744873\n",
            "Train time for the last game:  2.8314645290374756\n",
            "Elapsed time for the last game:  7.652367830276489\n",
            "\n",
            "Game 1734 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 299072\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299072 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.017506227\n",
            "Actions distribution (last game, %) is:  [19 29 24 26  0  0]\n",
            "Steps survived: 172\n",
            "Image processing time for the last game:  0.4647643566131592\n",
            "Train time for the last game:  4.012057542800903\n",
            "Elapsed time for the last game:  8.582137107849121\n",
            "\n",
            "Game 1735 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 299196\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299196 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.017597187\n",
            "Actions distribution (last game, %) is:  [24 30 21 24  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3583827018737793\n",
            "Train time for the last game:  2.902726173400879\n",
            "Elapsed time for the last game:  7.3124542236328125\n",
            "\n",
            "Game 1736 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 299318\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299318 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.017179316\n",
            "Actions distribution (last game, %) is:  [27 31 21 19  0  0]\n",
            "Steps survived: 122\n",
            "Image processing time for the last game:  0.3134925365447998\n",
            "Train time for the last game:  2.809685707092285\n",
            "Elapsed time for the last game:  7.549443244934082\n",
            "\n",
            "Game 1737 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 6.0\n",
            "Global_steps is: 299711\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299711 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.018132461\n",
            "Actions distribution (last game, %) is:  [22 25 28 23  0  0]\n",
            "Steps survived: 393\n",
            "Image processing time for the last game:  1.0215225219726562\n",
            "Train time for the last game:  8.887990236282349\n",
            "Elapsed time for the last game:  14.484005212783813\n",
            "\n",
            "Game 1738 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 299835\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299835 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.017977063\n",
            "Actions distribution (last game, %) is:  [19 27 25 27  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32529211044311523\n",
            "Train time for the last game:  2.860200881958008\n",
            "Elapsed time for the last game:  7.328317880630493\n",
            "\n",
            "Game 1739 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 299960\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 299960 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.019835498\n",
            "Actions distribution (last game, %) is:  [24 23 29 23  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3422682285308838\n",
            "Train time for the last game:  2.875905752182007\n",
            "Elapsed time for the last game:  7.687100648880005\n",
            "\n",
            "Game 1740 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 300216\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(1.0, shape=(), dtype=float32)\n",
            "Memory contains 300216 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.017228566\n",
            "Actions distribution (last game, %) is:  [26 27 21 23  0  0]\n",
            "Steps survived: 256\n",
            "Image processing time for the last game:  0.6767711639404297\n",
            "Train time for the last game:  5.843280076980591\n",
            "Elapsed time for the last game:  10.772649765014648\n",
            "\n",
            "Game 1741 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 300340\n",
            "Epsilon is: tf.Tensor(0.99987435, shape=(), dtype=float32)\n",
            "Memory contains 300340 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.01894195\n",
            "Actions distribution (last game, %) is:  [29 25 23 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3232717514038086\n",
            "Train time for the last game:  2.856935739517212\n",
            "Elapsed time for the last game:  7.261128902435303\n",
            "\n",
            "Game 1742 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 300538\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9996737, shape=(), dtype=float32)\n",
            "Memory contains 300538 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.017390177\n",
            "Actions distribution (last game, %) is:  [24 27 26 21  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5196976661682129\n",
            "Train time for the last game:  4.539788484573364\n",
            "Elapsed time for the last game:  9.802043914794922\n",
            "\n",
            "Game 1743 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 300690\n",
            "Epsilon is: tf.Tensor(0.9995197, shape=(), dtype=float32)\n",
            "Memory contains 300690 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.017173806\n",
            "Actions distribution (last game, %) is:  [25 29 19 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.4028315544128418\n",
            "Train time for the last game:  3.5393166542053223\n",
            "Elapsed time for the last game:  8.047385692596436\n",
            "\n",
            "Game 1744 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 300888\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9993191, shape=(), dtype=float32)\n",
            "Memory contains 300888 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.017358176\n",
            "Actions distribution (last game, %) is:  [27 27 22 22  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5280725955963135\n",
            "Train time for the last game:  4.521947383880615\n",
            "Elapsed time for the last game:  9.33763599395752\n",
            "\n",
            "Game 1745 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 301058\n",
            "Epsilon is: tf.Tensor(0.9991468, shape=(), dtype=float32)\n",
            "Memory contains 301058 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.01972381\n",
            "Actions distribution (last game, %) is:  [23 30 24 21  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4575061798095703\n",
            "Train time for the last game:  3.9036853313446045\n",
            "Elapsed time for the last game:  9.028394222259521\n",
            "\n",
            "Game 1746 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 301182\n",
            "Epsilon is: tf.Tensor(0.9990212, shape=(), dtype=float32)\n",
            "Memory contains 301182 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.017084302\n",
            "Actions distribution (last game, %) is:  [26 28 18 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32357263565063477\n",
            "Train time for the last game:  2.862131118774414\n",
            "Elapsed time for the last game:  7.255330324172974\n",
            "\n",
            "Game 1747 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 301478\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99872124, shape=(), dtype=float32)\n",
            "Memory contains 301478 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.017943848\n",
            "Actions distribution (last game, %) is:  [27 26 26 19  0  0]\n",
            "Steps survived: 296\n",
            "Image processing time for the last game:  0.7717077732086182\n",
            "Train time for the last game:  6.7938103675842285\n",
            "Elapsed time for the last game:  12.104270935058594\n",
            "\n",
            "Game 1748 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 301601\n",
            "Epsilon is: tf.Tensor(0.9985966, shape=(), dtype=float32)\n",
            "Memory contains 301601 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.018170878\n",
            "Actions distribution (last game, %) is:  [22 27 19 30  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.37111997604370117\n",
            "Train time for the last game:  2.968144416809082\n",
            "Elapsed time for the last game:  8.33660078048706\n",
            "\n",
            "Game 1749 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 301771\n",
            "Epsilon is: tf.Tensor(0.99842435, shape=(), dtype=float32)\n",
            "Memory contains 301771 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.019669533\n",
            "Actions distribution (last game, %) is:  [26 28 23 20  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.5213594436645508\n",
            "Train time for the last game:  4.106266736984253\n",
            "Elapsed time for the last game:  9.023263931274414\n",
            "\n",
            "Game 1750 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 301940\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9982531, shape=(), dtype=float32)\n",
            "Memory contains 301940 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.018625557\n",
            "Actions distribution (last game, %) is:  [26 22 27 24  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.45080089569091797\n",
            "Train time for the last game:  3.867833375930786\n",
            "Elapsed time for the last game:  8.717523574829102\n",
            "\n",
            "Game 1751 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 302065\n",
            "Epsilon is: tf.Tensor(0.99812645, shape=(), dtype=float32)\n",
            "Memory contains 302065 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.017592102\n",
            "Actions distribution (last game, %) is:  [20 22 27 29  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32396364212036133\n",
            "Train time for the last game:  2.926867961883545\n",
            "Elapsed time for the last game:  7.817360877990723\n",
            "\n",
            "Game 1752 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 302187\n",
            "Epsilon is: tf.Tensor(0.9980028, shape=(), dtype=float32)\n",
            "Memory contains 302187 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.017210182\n",
            "Actions distribution (last game, %) is:  [21 29 23 24  0  0]\n",
            "Steps survived: 122\n",
            "Image processing time for the last game:  0.33321142196655273\n",
            "Train time for the last game:  2.828627347946167\n",
            "Elapsed time for the last game:  7.4088311195373535\n",
            "\n",
            "Game 1753 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 302310\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9978782, shape=(), dtype=float32)\n",
            "Memory contains 302310 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017607877\n",
            "Actions distribution (last game, %) is:  [24 23 22 28  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32662057876586914\n",
            "Train time for the last game:  2.8815219402313232\n",
            "Elapsed time for the last game:  7.508041858673096\n",
            "\n",
            "Game 1754 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 302463\n",
            "Epsilon is: tf.Tensor(0.99772316, shape=(), dtype=float32)\n",
            "Memory contains 302463 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.017951228\n",
            "Actions distribution (last game, %) is:  [26 23 26 23  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.4033999443054199\n",
            "Train time for the last game:  3.503647804260254\n",
            "Elapsed time for the last game:  8.53571343421936\n",
            "\n",
            "Game 1755 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 302665\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9975185, shape=(), dtype=float32)\n",
            "Memory contains 302665 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.017685866\n",
            "Actions distribution (last game, %) is:  [22 27 30 19  0  0]\n",
            "Steps survived: 202\n",
            "Image processing time for the last game:  0.5318939685821533\n",
            "Train time for the last game:  4.642370223999023\n",
            "Elapsed time for the last game:  9.637287855148315\n",
            "\n",
            "Game 1756 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 302817\n",
            "Epsilon is: tf.Tensor(0.99736446, shape=(), dtype=float32)\n",
            "Memory contains 302817 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.019553272\n",
            "Actions distribution (last game, %) is:  [23 23 25 27  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3964574337005615\n",
            "Train time for the last game:  3.4858620166778564\n",
            "Elapsed time for the last game:  8.237404346466064\n",
            "\n",
            "Game 1757 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 302989\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9971902, shape=(), dtype=float32)\n",
            "Memory contains 302989 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.016663589\n",
            "Actions distribution (last game, %) is:  [28 27 25 18  0  0]\n",
            "Steps survived: 172\n",
            "Image processing time for the last game:  0.4507181644439697\n",
            "Train time for the last game:  3.942079544067383\n",
            "Elapsed time for the last game:  9.06108570098877\n",
            "\n",
            "Game 1758 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 303114\n",
            "Epsilon is: tf.Tensor(0.9970635, shape=(), dtype=float32)\n",
            "Memory contains 303114 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.018418698\n",
            "Actions distribution (last game, %) is:  [32 25 22 19  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3252406120300293\n",
            "Train time for the last game:  2.9585041999816895\n",
            "Elapsed time for the last game:  7.529950857162476\n",
            "\n",
            "Game 1759 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 303266\n",
            "Epsilon is: tf.Tensor(0.9969095, shape=(), dtype=float32)\n",
            "Memory contains 303266 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.01799731\n",
            "Actions distribution (last game, %) is:  [23 29 16 29  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3956313133239746\n",
            "Train time for the last game:  3.5224874019622803\n",
            "Elapsed time for the last game:  8.201024532318115\n",
            "\n",
            "Game 1760 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 303464\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99670887, shape=(), dtype=float32)\n",
            "Memory contains 303464 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.019025126\n",
            "Actions distribution (last game, %) is:  [26 28 25 19  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5188016891479492\n",
            "Train time for the last game:  4.519083261489868\n",
            "Elapsed time for the last game:  9.81841492652893\n",
            "\n",
            "Game 1761 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 303681\n",
            "Epsilon is: tf.Tensor(0.996489, shape=(), dtype=float32)\n",
            "Memory contains 303681 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.017497469\n",
            "Actions distribution (last game, %) is:  [23 21 26 28  0  0]\n",
            "Steps survived: 217\n",
            "Image processing time for the last game:  0.5790576934814453\n",
            "Train time for the last game:  5.04855751991272\n",
            "Elapsed time for the last game:  10.026439905166626\n",
            "\n",
            "Game 1762 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 303833\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99633497, shape=(), dtype=float32)\n",
            "Memory contains 303833 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.018531706\n",
            "Actions distribution (last game, %) is:  [27 24 24 23  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.39868879318237305\n",
            "Train time for the last game:  3.4894073009490967\n",
            "Elapsed time for the last game:  8.32853364944458\n",
            "\n",
            "Game 1763 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 303985\n",
            "Epsilon is: tf.Tensor(0.99618095, shape=(), dtype=float32)\n",
            "Memory contains 303985 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.017827993\n",
            "Actions distribution (last game, %) is:  [26 30 17 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3977079391479492\n",
            "Train time for the last game:  3.4591293334960938\n",
            "Elapsed time for the last game:  8.491030216217041\n",
            "\n",
            "Game 1764 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 304183\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9959803, shape=(), dtype=float32)\n",
            "Memory contains 304183 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.018064544\n",
            "Actions distribution (last game, %) is:  [24 21 23 30  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5165164470672607\n",
            "Train time for the last game:  4.546940088272095\n",
            "Elapsed time for the last game:  9.547043085098267\n",
            "\n",
            "Game 1765 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 304383\n",
            "Epsilon is: tf.Tensor(0.99577767, shape=(), dtype=float32)\n",
            "Memory contains 304383 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.018217718\n",
            "Actions distribution (last game, %) is:  [26 26 24 23  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5267090797424316\n",
            "Train time for the last game:  4.549148797988892\n",
            "Elapsed time for the last game:  9.627354621887207\n",
            "\n",
            "Game 1766 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 304535\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99562365, shape=(), dtype=float32)\n",
            "Memory contains 304535 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.016992744\n",
            "Actions distribution (last game, %) is:  [22 26 19 31  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.39856815338134766\n",
            "Train time for the last game:  3.506669521331787\n",
            "Elapsed time for the last game:  8.620453357696533\n",
            "\n",
            "Game 1767 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 304659\n",
            "Epsilon is: tf.Tensor(0.995498, shape=(), dtype=float32)\n",
            "Memory contains 304659 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.019113297\n",
            "Actions distribution (last game, %) is:  [28 28 21 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32668209075927734\n",
            "Train time for the last game:  2.834305763244629\n",
            "Elapsed time for the last game:  7.485787630081177\n",
            "\n",
            "Game 1768 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 304927\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99522644, shape=(), dtype=float32)\n",
            "Memory contains 304927 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.018796844\n",
            "Actions distribution (last game, %) is:  [26 28 23 21  0  0]\n",
            "Steps survived: 268\n",
            "Image processing time for the last game:  0.69693922996521\n",
            "Train time for the last game:  6.123680591583252\n",
            "Elapsed time for the last game:  11.605053424835205\n",
            "\n",
            "Game 1769 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 305196\n",
            "Epsilon is: tf.Tensor(0.9949539, shape=(), dtype=float32)\n",
            "Memory contains 305196 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.02040711\n",
            "Actions distribution (last game, %) is:  [24 29 27 19  0  0]\n",
            "Steps survived: 269\n",
            "Image processing time for the last game:  0.7695560455322266\n",
            "Train time for the last game:  6.51708984375\n",
            "Elapsed time for the last game:  12.371381998062134\n",
            "\n",
            "Game 1770 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 305319\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99482924, shape=(), dtype=float32)\n",
            "Memory contains 305319 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.020888763\n",
            "Actions distribution (last game, %) is:  [22 30 26 21  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.3416323661804199\n",
            "Train time for the last game:  2.9111080169677734\n",
            "Elapsed time for the last game:  7.475515365600586\n",
            "\n",
            "Game 1771 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 305442\n",
            "Epsilon is: tf.Tensor(0.9947046, shape=(), dtype=float32)\n",
            "Memory contains 305442 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.01958704\n",
            "Actions distribution (last game, %) is:  [27 24 21 26  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.31705403327941895\n",
            "Train time for the last game:  2.880125045776367\n",
            "Elapsed time for the last game:  7.617333173751831\n",
            "\n",
            "Game 1772 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 305691\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9944523, shape=(), dtype=float32)\n",
            "Memory contains 305691 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.019627461\n",
            "Actions distribution (last game, %) is:  [23 24 27 25  0  0]\n",
            "Steps survived: 249\n",
            "Image processing time for the last game:  0.6580994129180908\n",
            "Train time for the last game:  5.704529523849487\n",
            "Elapsed time for the last game:  11.338428258895874\n",
            "\n",
            "Game 1773 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 305890\n",
            "Epsilon is: tf.Tensor(0.99425066, shape=(), dtype=float32)\n",
            "Memory contains 305890 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.01849826\n",
            "Actions distribution (last game, %) is:  [18 27 27 26  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5176563262939453\n",
            "Train time for the last game:  4.67941951751709\n",
            "Elapsed time for the last game:  9.829833030700684\n",
            "\n",
            "Game 1774 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 306042\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99409664, shape=(), dtype=float32)\n",
            "Memory contains 306042 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.019137377\n",
            "Actions distribution (last game, %) is:  [26 29 17 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.4054255485534668\n",
            "Train time for the last game:  3.541433095932007\n",
            "Elapsed time for the last game:  8.316378355026245\n",
            "\n",
            "Game 1775 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 306195\n",
            "Epsilon is: tf.Tensor(0.9939416, shape=(), dtype=float32)\n",
            "Memory contains 306195 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.019359143\n",
            "Actions distribution (last game, %) is:  [19 26 25 28  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.40479302406311035\n",
            "Train time for the last game:  3.562277317047119\n",
            "Elapsed time for the last game:  8.623396873474121\n",
            "\n",
            "Game 1776 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 306347\n",
            "Epsilon is: tf.Tensor(0.9937876, shape=(), dtype=float32)\n",
            "Memory contains 306347 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.019352503\n",
            "Actions distribution (last game, %) is:  [21 33 16 29  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.39281272888183594\n",
            "Train time for the last game:  3.4875879287719727\n",
            "Elapsed time for the last game:  8.32619333267212\n",
            "\n",
            "Game 1777 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 306501\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99363154, shape=(), dtype=float32)\n",
            "Memory contains 306501 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.019791555\n",
            "Actions distribution (last game, %) is:  [22 33 21 21  0  0]\n",
            "Steps survived: 154\n",
            "Image processing time for the last game:  0.39803314208984375\n",
            "Train time for the last game:  3.546032428741455\n",
            "Elapsed time for the last game:  8.430814027786255\n",
            "\n",
            "Game 1778 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 306747\n",
            "Epsilon is: tf.Tensor(0.9933823, shape=(), dtype=float32)\n",
            "Memory contains 306747 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.019773996\n",
            "Actions distribution (last game, %) is:  [29 25 24 20  0  0]\n",
            "Steps survived: 246\n",
            "Image processing time for the last game:  0.652571439743042\n",
            "Train time for the last game:  5.6153788566589355\n",
            "Elapsed time for the last game:  11.236868619918823\n",
            "\n",
            "Game 1779 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 306929\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99319786, shape=(), dtype=float32)\n",
            "Memory contains 306929 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.020331912\n",
            "Actions distribution (last game, %) is:  [25 24 18 31  0  0]\n",
            "Steps survived: 182\n",
            "Image processing time for the last game:  0.4899411201477051\n",
            "Train time for the last game:  4.222285032272339\n",
            "Elapsed time for the last game:  9.277574300765991\n",
            "\n",
            "Game 1780 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 307156\n",
            "Epsilon is: tf.Tensor(0.99296784, shape=(), dtype=float32)\n",
            "Memory contains 307156 samples\n",
            "Reward over 10 games is: 1.7 and loss is: 0.020171918\n",
            "Actions distribution (last game, %) is:  [26 27 25 21  0  0]\n",
            "Steps survived: 227\n",
            "Image processing time for the last game:  0.5925271511077881\n",
            "Train time for the last game:  5.241144895553589\n",
            "Elapsed time for the last game:  10.57382845878601\n",
            "\n",
            "Game 1781 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 307384\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9927368, shape=(), dtype=float32)\n",
            "Memory contains 307384 samples\n",
            "Reward over 10 games is: 2.0 and loss is: 0.02005757\n",
            "Actions distribution (last game, %) is:  [21 29 24 24  0  0]\n",
            "Steps survived: 228\n",
            "Image processing time for the last game:  0.596447229385376\n",
            "Train time for the last game:  5.195728063583374\n",
            "Elapsed time for the last game:  10.654017686843872\n",
            "\n",
            "Game 1782 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 307554\n",
            "Epsilon is: tf.Tensor(0.99256456, shape=(), dtype=float32)\n",
            "Memory contains 307554 samples\n",
            "Reward over 10 games is: 1.8 and loss is: 0.020395413\n",
            "Actions distribution (last game, %) is:  [28 28 21 21  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.4467146396636963\n",
            "Train time for the last game:  3.867539882659912\n",
            "Elapsed time for the last game:  8.73979926109314\n",
            "\n",
            "Game 1783 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 307677\n",
            "Epsilon is: tf.Tensor(0.9924399, shape=(), dtype=float32)\n",
            "Memory contains 307677 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.01963147\n",
            "Actions distribution (last game, %) is:  [18 24 29 27  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.3774557113647461\n",
            "Train time for the last game:  2.9881792068481445\n",
            "Elapsed time for the last game:  7.792200565338135\n",
            "\n",
            "Game 1784 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 307801\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9923143, shape=(), dtype=float32)\n",
            "Memory contains 307801 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.021571623\n",
            "Actions distribution (last game, %) is:  [32 23 22 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33196282386779785\n",
            "Train time for the last game:  2.8448657989501953\n",
            "Elapsed time for the last game:  7.769838094711304\n",
            "\n",
            "Game 1785 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 308000\n",
            "Epsilon is: tf.Tensor(0.99211264, shape=(), dtype=float32)\n",
            "Memory contains 308000 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.020951122\n",
            "Actions distribution (last game, %) is:  [20 25 26 27  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5075995922088623\n",
            "Train time for the last game:  4.590563774108887\n",
            "Elapsed time for the last game:  9.759944438934326\n",
            "\n",
            "Game 1786 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 308125\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.991986, shape=(), dtype=float32)\n",
            "Memory contains 308125 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.0192027\n",
            "Actions distribution (last game, %) is:  [20 27 25 25  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3342292308807373\n",
            "Train time for the last game:  2.945439338684082\n",
            "Elapsed time for the last game:  7.991495847702026\n",
            "\n",
            "Game 1787 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 308323\n",
            "Epsilon is: tf.Tensor(0.99178535, shape=(), dtype=float32)\n",
            "Memory contains 308323 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.020208819\n",
            "Actions distribution (last game, %) is:  [19 30 29 20  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5196599960327148\n",
            "Train time for the last game:  4.556278705596924\n",
            "Elapsed time for the last game:  9.927306175231934\n",
            "\n",
            "Game 1788 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 308521\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9915847, shape=(), dtype=float32)\n",
            "Memory contains 308521 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.020251954\n",
            "Actions distribution (last game, %) is:  [23 24 25 25  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5290229320526123\n",
            "Train time for the last game:  4.522266864776611\n",
            "Elapsed time for the last game:  9.686108112335205\n",
            "\n",
            "Game 1789 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 308720\n",
            "Epsilon is: tf.Tensor(0.9913831, shape=(), dtype=float32)\n",
            "Memory contains 308720 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.021089649\n",
            "Actions distribution (last game, %) is:  [19 24 26 28  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5377769470214844\n",
            "Train time for the last game:  4.5165793895721436\n",
            "Elapsed time for the last game:  9.740771770477295\n",
            "\n",
            "Game 1790 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 308843\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99125844, shape=(), dtype=float32)\n",
            "Memory contains 308843 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.019829819\n",
            "Actions distribution (last game, %) is:  [21 28 27 22  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32446885108947754\n",
            "Train time for the last game:  2.8381574153900146\n",
            "Elapsed time for the last game:  7.758044481277466\n",
            "\n",
            "Game 1791 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 308967\n",
            "Epsilon is: tf.Tensor(0.9911328, shape=(), dtype=float32)\n",
            "Memory contains 308967 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.020176282\n",
            "Actions distribution (last game, %) is:  [22 33 21 22  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3186204433441162\n",
            "Train time for the last game:  2.8545000553131104\n",
            "Elapsed time for the last game:  7.602620363235474\n",
            "\n",
            "Game 1792 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 309166\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.99093115, shape=(), dtype=float32)\n",
            "Memory contains 309166 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.019676816\n",
            "Actions distribution (last game, %) is:  [30 25 25 18  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5192582607269287\n",
            "Train time for the last game:  4.512711048126221\n",
            "Elapsed time for the last game:  9.736215114593506\n",
            "\n",
            "Game 1793 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 309364\n",
            "Epsilon is: tf.Tensor(0.9907305, shape=(), dtype=float32)\n",
            "Memory contains 309364 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.019576268\n",
            "Actions distribution (last game, %) is:  [24 20 30 23  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5127890110015869\n",
            "Train time for the last game:  4.5198798179626465\n",
            "Elapsed time for the last game:  9.807640075683594\n",
            "\n",
            "Game 1794 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 309488\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9906049, shape=(), dtype=float32)\n",
            "Memory contains 309488 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.019432787\n",
            "Actions distribution (last game, %) is:  [24 29 27 18  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.32309794425964355\n",
            "Train time for the last game:  2.838937759399414\n",
            "Elapsed time for the last game:  7.526416540145874\n",
            "\n",
            "Game 1795 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 309688\n",
            "Epsilon is: tf.Tensor(0.9904022, shape=(), dtype=float32)\n",
            "Memory contains 309688 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.020062134\n",
            "Actions distribution (last game, %) is:  [21 25 25 28  0  0]\n",
            "Steps survived: 200\n",
            "Image processing time for the last game:  0.5092682838439941\n",
            "Train time for the last game:  4.594431161880493\n",
            "Elapsed time for the last game:  9.647952318191528\n",
            "\n",
            "Game 1796 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 309861\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9902269, shape=(), dtype=float32)\n",
            "Memory contains 309861 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.02090228\n",
            "Actions distribution (last game, %) is:  [25 27 20 27  0  0]\n",
            "Steps survived: 173\n",
            "Image processing time for the last game:  0.4630718231201172\n",
            "Train time for the last game:  4.002798080444336\n",
            "Elapsed time for the last game:  9.286699295043945\n",
            "\n",
            "Game 1797 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 309986\n",
            "Epsilon is: tf.Tensor(0.99010026, shape=(), dtype=float32)\n",
            "Memory contains 309986 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.021434728\n",
            "Actions distribution (last game, %) is:  [19 25 36 18  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.32590723037719727\n",
            "Train time for the last game:  2.94789981842041\n",
            "Elapsed time for the last game:  7.817496299743652\n",
            "\n",
            "Game 1798 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 310168\n",
            "Epsilon is: tf.Tensor(0.98991585, shape=(), dtype=float32)\n",
            "Memory contains 310168 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.019425238\n",
            "Actions distribution (last game, %) is:  [25 30 18 25  0  0]\n",
            "Steps survived: 182\n",
            "Image processing time for the last game:  0.47708773612976074\n",
            "Train time for the last game:  4.220153093338013\n",
            "Elapsed time for the last game:  9.255872964859009\n",
            "\n",
            "Game 1799 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 310366\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9897152, shape=(), dtype=float32)\n",
            "Memory contains 310366 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.019870486\n",
            "Actions distribution (last game, %) is:  [21 31 24 22  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5238082408905029\n",
            "Train time for the last game:  4.682630300521851\n",
            "Elapsed time for the last game:  10.071507215499878\n",
            "\n",
            "Game 1800 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 310535\n",
            "Epsilon is: tf.Tensor(0.989544, shape=(), dtype=float32)\n",
            "Memory contains 310535 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.021347828\n",
            "Actions distribution (last game, %) is:  [20 29 24 25  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.45485496520996094\n",
            "Train time for the last game:  3.960376262664795\n",
            "Elapsed time for the last game:  8.948599338531494\n",
            "\n",
            "Game 1801 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 310688\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98938894, shape=(), dtype=float32)\n",
            "Memory contains 310688 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.020616068\n",
            "Actions distribution (last game, %) is:  [23 32 19 24  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.3937816619873047\n",
            "Train time for the last game:  3.534661054611206\n",
            "Elapsed time for the last game:  8.548955917358398\n",
            "\n",
            "Game 1802 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 310933\n",
            "Epsilon is: tf.Tensor(0.9891407, shape=(), dtype=float32)\n",
            "Memory contains 310933 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.020863524\n",
            "Actions distribution (last game, %) is:  [33 23 24 18  0  0]\n",
            "Steps survived: 245\n",
            "Image processing time for the last game:  0.6488125324249268\n",
            "Train time for the last game:  5.664924144744873\n",
            "Elapsed time for the last game:  11.290709257125854\n",
            "\n",
            "Game 1803 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 311132\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98893905, shape=(), dtype=float32)\n",
            "Memory contains 311132 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.019555332\n",
            "Actions distribution (last game, %) is:  [26 25 26 20  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5120046138763428\n",
            "Train time for the last game:  4.624229669570923\n",
            "Elapsed time for the last game:  9.858553647994995\n",
            "\n",
            "Game 1804 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 311331\n",
            "Epsilon is: tf.Tensor(0.9887374, shape=(), dtype=float32)\n",
            "Memory contains 311331 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.021614434\n",
            "Actions distribution (last game, %) is:  [19 31 23 25  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5244512557983398\n",
            "Train time for the last game:  4.5795578956604\n",
            "Elapsed time for the last game:  9.75676679611206\n",
            "\n",
            "Game 1805 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 311454\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9886128, shape=(), dtype=float32)\n",
            "Memory contains 311454 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.0199971\n",
            "Actions distribution (last game, %) is:  [22 29 23 24  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.33083415031433105\n",
            "Train time for the last game:  2.9136927127838135\n",
            "Elapsed time for the last game:  7.893188238143921\n",
            "\n",
            "Game 1806 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 311625\n",
            "Epsilon is: tf.Tensor(0.9884395, shape=(), dtype=float32)\n",
            "Memory contains 311625 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.022450875\n",
            "Actions distribution (last game, %) is:  [19 34 22 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.44713783264160156\n",
            "Train time for the last game:  3.909681797027588\n",
            "Elapsed time for the last game:  8.845893859863281\n",
            "\n",
            "Game 1807 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 311748\n",
            "Epsilon is: tf.Tensor(0.98831487, shape=(), dtype=float32)\n",
            "Memory contains 311748 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.019542415\n",
            "Actions distribution (last game, %) is:  [23 31 22 21  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.31879615783691406\n",
            "Train time for the last game:  2.855449914932251\n",
            "Elapsed time for the last game:  7.746577978134155\n",
            "\n",
            "Game 1808 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 311946\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98811424, shape=(), dtype=float32)\n",
            "Memory contains 311946 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.019466363\n",
            "Actions distribution (last game, %) is:  [19 26 28 25  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5155889987945557\n",
            "Train time for the last game:  4.593987226486206\n",
            "Elapsed time for the last game:  9.926353693008423\n",
            "\n",
            "Game 1809 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 312116\n",
            "Epsilon is: tf.Tensor(0.987942, shape=(), dtype=float32)\n",
            "Memory contains 312116 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.02283141\n",
            "Actions distribution (last game, %) is:  [30 30 23 16  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.44059276580810547\n",
            "Train time for the last game:  3.9107062816619873\n",
            "Elapsed time for the last game:  8.898548364639282\n",
            "\n",
            "Game 1810 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 312267\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.987789, shape=(), dtype=float32)\n",
            "Memory contains 312267 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.020188162\n",
            "Actions distribution (last game, %) is:  [24 30 22 23  0  0]\n",
            "Steps survived: 151\n",
            "Image processing time for the last game:  0.402449369430542\n",
            "Train time for the last game:  3.497547149658203\n",
            "Elapsed time for the last game:  8.39844560623169\n",
            "\n",
            "Game 1811 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 312449\n",
            "Epsilon is: tf.Tensor(0.98760456, shape=(), dtype=float32)\n",
            "Memory contains 312449 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.02138678\n",
            "Actions distribution (last game, %) is:  [20 24 25 29  0  0]\n",
            "Steps survived: 182\n",
            "Image processing time for the last game:  0.4690091609954834\n",
            "Train time for the last game:  4.2206830978393555\n",
            "Elapsed time for the last game:  9.471337795257568\n",
            "\n",
            "Game 1812 - Run 20200911003313\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 312668\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98738265, shape=(), dtype=float32)\n",
            "Memory contains 312668 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.02048758\n",
            "Actions distribution (last game, %) is:  [17 28 27 26  0  0]\n",
            "Steps survived: 219\n",
            "Image processing time for the last game:  0.5728740692138672\n",
            "Train time for the last game:  5.0880022048950195\n",
            "Elapsed time for the last game:  10.544721603393555\n",
            "\n",
            "Game 1813 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 312839\n",
            "Epsilon is: tf.Tensor(0.9872094, shape=(), dtype=float32)\n",
            "Memory contains 312839 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.018618964\n",
            "Actions distribution (last game, %) is:  [22 22 26 28  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.43735647201538086\n",
            "Train time for the last game:  4.026925563812256\n",
            "Elapsed time for the last game:  9.119022130966187\n",
            "\n",
            "Game 1814 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 312962\n",
            "Epsilon is: tf.Tensor(0.98708475, shape=(), dtype=float32)\n",
            "Memory contains 312962 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.021370191\n",
            "Actions distribution (last game, %) is:  [27 22 23 25  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32911133766174316\n",
            "Train time for the last game:  2.9230968952178955\n",
            "Elapsed time for the last game:  7.911020994186401\n",
            "\n",
            "Game 1815 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 313135\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98690945, shape=(), dtype=float32)\n",
            "Memory contains 313135 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.020804817\n",
            "Actions distribution (last game, %) is:  [18 25 26 29  0  0]\n",
            "Steps survived: 173\n",
            "Image processing time for the last game:  0.44667887687683105\n",
            "Train time for the last game:  4.037518739700317\n",
            "Elapsed time for the last game:  9.07194995880127\n",
            "\n",
            "Game 1816 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 313287\n",
            "Epsilon is: tf.Tensor(0.98675543, shape=(), dtype=float32)\n",
            "Memory contains 313287 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.02012352\n",
            "Actions distribution (last game, %) is:  [25 25 24 25  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.3972153663635254\n",
            "Train time for the last game:  3.5098209381103516\n",
            "Elapsed time for the last game:  8.644191980361938\n",
            "\n",
            "Game 1817 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 313410\n",
            "Epsilon is: tf.Tensor(0.9866308, shape=(), dtype=float32)\n",
            "Memory contains 313410 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.019884327\n",
            "Actions distribution (last game, %) is:  [31 26 18 23  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.36934804916381836\n",
            "Train time for the last game:  3.021512508392334\n",
            "Elapsed time for the last game:  8.143088579177856\n",
            "\n",
            "Game 1818 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 313659\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9863785, shape=(), dtype=float32)\n",
            "Memory contains 313659 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.021014873\n",
            "Actions distribution (last game, %) is:  [22 27 27 22  0  0]\n",
            "Steps survived: 249\n",
            "Image processing time for the last game:  0.7137444019317627\n",
            "Train time for the last game:  5.978292465209961\n",
            "Elapsed time for the last game:  11.545418977737427\n",
            "\n",
            "Game 1819 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 313828\n",
            "Epsilon is: tf.Tensor(0.98620725, shape=(), dtype=float32)\n",
            "Memory contains 313828 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.021770397\n",
            "Actions distribution (last game, %) is:  [22 29 25 22  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.45232248306274414\n",
            "Train time for the last game:  4.078167200088501\n",
            "Elapsed time for the last game:  9.476784467697144\n",
            "\n",
            "Game 1820 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 313981\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9860522, shape=(), dtype=float32)\n",
            "Memory contains 313981 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.020508485\n",
            "Actions distribution (last game, %) is:  [25 30 19 24  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.40640783309936523\n",
            "Train time for the last game:  3.653096914291382\n",
            "Elapsed time for the last game:  8.888619899749756\n",
            "\n",
            "Game 1821 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 314180\n",
            "Epsilon is: tf.Tensor(0.9858506, shape=(), dtype=float32)\n",
            "Memory contains 314180 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.02186107\n",
            "Actions distribution (last game, %) is:  [22 25 27 24  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5353806018829346\n",
            "Train time for the last game:  4.746216535568237\n",
            "Elapsed time for the last game:  10.219146728515625\n",
            "\n",
            "Game 1822 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 314333\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98569554, shape=(), dtype=float32)\n",
            "Memory contains 314333 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.021207703\n",
            "Actions distribution (last game, %) is:  [26 24 27 21  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.42354750633239746\n",
            "Train time for the last game:  3.716902017593384\n",
            "Elapsed time for the last game:  8.906028032302856\n",
            "\n",
            "Game 1823 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 314554\n",
            "Epsilon is: tf.Tensor(0.9854716, shape=(), dtype=float32)\n",
            "Memory contains 314554 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.02091999\n",
            "Actions distribution (last game, %) is:  [25 23 27 24  0  0]\n",
            "Steps survived: 221\n",
            "Image processing time for the last game:  0.5916769504547119\n",
            "Train time for the last game:  5.4538655281066895\n",
            "Elapsed time for the last game:  11.046537160873413\n",
            "\n",
            "Game 1824 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 314753\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98526996, shape=(), dtype=float32)\n",
            "Memory contains 314753 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.023368645\n",
            "Actions distribution (last game, %) is:  [29 25 17 27  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5490918159484863\n",
            "Train time for the last game:  4.824873924255371\n",
            "Elapsed time for the last game:  10.31208348274231\n",
            "\n",
            "Game 1825 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 314952\n",
            "Epsilon is: tf.Tensor(0.9850683, shape=(), dtype=float32)\n",
            "Memory contains 314952 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.022189131\n",
            "Actions distribution (last game, %) is:  [25 30 24 19  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5369060039520264\n",
            "Train time for the last game:  4.807525157928467\n",
            "Elapsed time for the last game:  10.391335010528564\n",
            "\n",
            "Game 1826 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 315077\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98494166, shape=(), dtype=float32)\n",
            "Memory contains 315077 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.02190868\n",
            "Actions distribution (last game, %) is:  [25 16 25 31  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3389780521392822\n",
            "Train time for the last game:  3.0728213787078857\n",
            "Elapsed time for the last game:  8.24158263206482\n",
            "\n",
            "Game 1827 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 315229\n",
            "Epsilon is: tf.Tensor(0.98478764, shape=(), dtype=float32)\n",
            "Memory contains 315229 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.020332789\n",
            "Actions distribution (last game, %) is:  [21 26 24 27  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.4040544033050537\n",
            "Train time for the last game:  3.654129981994629\n",
            "Elapsed time for the last game:  8.91719937324524\n",
            "\n",
            "Game 1828 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 315473\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9845404, shape=(), dtype=float32)\n",
            "Memory contains 315473 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.021337558\n",
            "Actions distribution (last game, %) is:  [27 25 22 23  0  0]\n",
            "Steps survived: 244\n",
            "Image processing time for the last game:  0.656606912612915\n",
            "Train time for the last game:  5.866702079772949\n",
            "Elapsed time for the last game:  11.651242733001709\n",
            "\n",
            "Game 1829 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 315644\n",
            "Epsilon is: tf.Tensor(0.98436713, shape=(), dtype=float32)\n",
            "Memory contains 315644 samples\n",
            "Reward over 10 games is: 1.6 and loss is: 0.019577842\n",
            "Actions distribution (last game, %) is:  [22 26 25 25  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.4612865447998047\n",
            "Train time for the last game:  4.1083824634552\n",
            "Elapsed time for the last game:  9.479741096496582\n",
            "\n",
            "Game 1830 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 315768\n",
            "Epsilon is: tf.Tensor(0.9842415, shape=(), dtype=float32)\n",
            "Memory contains 315768 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.021245837\n",
            "Actions distribution (last game, %) is:  [26 28 22 21  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3359487056732178\n",
            "Train time for the last game:  3.0158464908599854\n",
            "Elapsed time for the last game:  8.04191517829895\n",
            "\n",
            "Game 1831 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 315937\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98407024, shape=(), dtype=float32)\n",
            "Memory contains 315937 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.022928348\n",
            "Actions distribution (last game, %) is:  [27 23 23 24  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.479323148727417\n",
            "Train time for the last game:  4.209762811660767\n",
            "Elapsed time for the last game:  9.532776832580566\n",
            "\n",
            "Game 1832 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 316090\n",
            "Epsilon is: tf.Tensor(0.9839152, shape=(), dtype=float32)\n",
            "Memory contains 316090 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.022155412\n",
            "Actions distribution (last game, %) is:  [23 26 25 25  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.40654516220092773\n",
            "Train time for the last game:  3.7113099098205566\n",
            "Elapsed time for the last game:  8.944342374801636\n",
            "\n",
            "Game 1833 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 316262\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9837409, shape=(), dtype=float32)\n",
            "Memory contains 316262 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.021144064\n",
            "Actions distribution (last game, %) is:  [26 25 18 28  0  0]\n",
            "Steps survived: 172\n",
            "Image processing time for the last game:  0.4690723419189453\n",
            "Train time for the last game:  4.148905277252197\n",
            "Elapsed time for the last game:  9.409906148910522\n",
            "\n",
            "Game 1834 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 316414\n",
            "Epsilon is: tf.Tensor(0.9835869, shape=(), dtype=float32)\n",
            "Memory contains 316414 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.02216594\n",
            "Actions distribution (last game, %) is:  [31 27 16 23  0  0]\n",
            "Steps survived: 152\n",
            "Image processing time for the last game:  0.41320323944091797\n",
            "Train time for the last game:  3.690915822982788\n",
            "Elapsed time for the last game:  8.845934629440308\n",
            "\n",
            "Game 1835 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 316612\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9833863, shape=(), dtype=float32)\n",
            "Memory contains 316612 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.022568842\n",
            "Actions distribution (last game, %) is:  [26 20 29 22  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5654106140136719\n",
            "Train time for the last game:  4.840960502624512\n",
            "Elapsed time for the last game:  10.46163010597229\n",
            "\n",
            "Game 1836 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 316810\n",
            "Epsilon is: tf.Tensor(0.98318565, shape=(), dtype=float32)\n",
            "Memory contains 316810 samples\n",
            "Reward over 10 games is: 1.4 and loss is: 0.021427821\n",
            "Actions distribution (last game, %) is:  [25 27 23 23  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5322167873382568\n",
            "Train time for the last game:  4.796859264373779\n",
            "Elapsed time for the last game:  10.206648349761963\n",
            "\n",
            "Game 1837 - Run 20200911003313\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 317009\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.982984, shape=(), dtype=float32)\n",
            "Memory contains 317009 samples\n",
            "Reward over 10 games is: 1.5 and loss is: 0.02139448\n",
            "Actions distribution (last game, %) is:  [26 24 19 29  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5485925674438477\n",
            "Train time for the last game:  4.839220762252808\n",
            "Elapsed time for the last game:  10.411818504333496\n",
            "\n",
            "Game 1838 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 317132\n",
            "Epsilon is: tf.Tensor(0.9828594, shape=(), dtype=float32)\n",
            "Memory contains 317132 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.023755291\n",
            "Actions distribution (last game, %) is:  [22 24 23 29  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.33706235885620117\n",
            "Train time for the last game:  3.0262982845306396\n",
            "Elapsed time for the last game:  8.190761089324951\n",
            "\n",
            "Game 1839 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 317381\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98260707, shape=(), dtype=float32)\n",
            "Memory contains 317381 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.022105489\n",
            "Actions distribution (last game, %) is:  [27 27 25 18  0  0]\n",
            "Steps survived: 249\n",
            "Image processing time for the last game:  0.6969881057739258\n",
            "Train time for the last game:  6.145136594772339\n",
            "Elapsed time for the last game:  11.9683518409729\n",
            "\n",
            "Game 1840 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 317506\n",
            "Epsilon is: tf.Tensor(0.9824804, shape=(), dtype=float32)\n",
            "Memory contains 317506 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.023689898\n",
            "Actions distribution (last game, %) is:  [24 27 26 21  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3473179340362549\n",
            "Train time for the last game:  3.0612881183624268\n",
            "Elapsed time for the last game:  8.253010034561157\n",
            "\n",
            "Game 1841 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 317675\n",
            "Epsilon is: tf.Tensor(0.98230916, shape=(), dtype=float32)\n",
            "Memory contains 317675 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.02463152\n",
            "Actions distribution (last game, %) is:  [23 27 27 20  0  0]\n",
            "Steps survived: 169\n",
            "Image processing time for the last game:  0.45625948905944824\n",
            "Train time for the last game:  4.099863290786743\n",
            "Elapsed time for the last game:  9.444988012313843\n",
            "\n",
            "Game 1842 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 317848\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98213387, shape=(), dtype=float32)\n",
            "Memory contains 317848 samples\n",
            "Reward over 10 games is: 1.3 and loss is: 0.022317726\n",
            "Actions distribution (last game, %) is:  [21 27 26 25  0  0]\n",
            "Steps survived: 173\n",
            "Image processing time for the last game:  0.46050047874450684\n",
            "Train time for the last game:  4.19508695602417\n",
            "Elapsed time for the last game:  9.61785101890564\n",
            "\n",
            "Game 1843 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 317970\n",
            "Epsilon is: tf.Tensor(0.98201025, shape=(), dtype=float32)\n",
            "Memory contains 317970 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.021908613\n",
            "Actions distribution (last game, %) is:  [20 24 26 28  0  0]\n",
            "Steps survived: 122\n",
            "Image processing time for the last game:  0.33203697204589844\n",
            "Train time for the last game:  2.9376001358032227\n",
            "Elapsed time for the last game:  7.9766223430633545\n",
            "\n",
            "Game 1844 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318095\n",
            "Epsilon is: tf.Tensor(0.9818836, shape=(), dtype=float32)\n",
            "Memory contains 318095 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.022482\n",
            "Actions distribution (last game, %) is:  [25 25 25 24  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3300619125366211\n",
            "Train time for the last game:  3.0698933601379395\n",
            "Elapsed time for the last game:  8.173331260681152\n",
            "\n",
            "Game 1845 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318218\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98175895, shape=(), dtype=float32)\n",
            "Memory contains 318218 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.02412174\n",
            "Actions distribution (last game, %) is:  [19 33 22 23  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.32714056968688965\n",
            "Train time for the last game:  3.022676706314087\n",
            "Elapsed time for the last game:  8.202412366867065\n",
            "\n",
            "Game 1846 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 318390\n",
            "Epsilon is: tf.Tensor(0.98158467, shape=(), dtype=float32)\n",
            "Memory contains 318390 samples\n",
            "Reward over 10 games is: 0.8 and loss is: 0.022626208\n",
            "Actions distribution (last game, %) is:  [22 26 25 25  0  0]\n",
            "Steps survived: 172\n",
            "Image processing time for the last game:  0.45635390281677246\n",
            "Train time for the last game:  4.228549003601074\n",
            "Elapsed time for the last game:  9.569444417953491\n",
            "\n",
            "Game 1847 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318514\n",
            "Epsilon is: tf.Tensor(0.981459, shape=(), dtype=float32)\n",
            "Memory contains 318514 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.023006197\n",
            "Actions distribution (last game, %) is:  [28 21 24 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33655333518981934\n",
            "Train time for the last game:  3.024364709854126\n",
            "Elapsed time for the last game:  8.179073572158813\n",
            "\n",
            "Game 1848 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318638\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9813334, shape=(), dtype=float32)\n",
            "Memory contains 318638 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.021500634\n",
            "Actions distribution (last game, %) is:  [16 31 25 26  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3437776565551758\n",
            "Train time for the last game:  3.0221986770629883\n",
            "Elapsed time for the last game:  8.209592580795288\n",
            "\n",
            "Game 1849 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318762\n",
            "Epsilon is: tf.Tensor(0.9812077, shape=(), dtype=float32)\n",
            "Memory contains 318762 samples\n",
            "Reward over 10 games is: 0.3 and loss is: 0.022529403\n",
            "Actions distribution (last game, %) is:  [20 22 31 25  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.34203100204467773\n",
            "Train time for the last game:  3.016686201095581\n",
            "Elapsed time for the last game:  8.118896245956421\n",
            "\n",
            "Game 1850 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 318886\n",
            "Epsilon is: tf.Tensor(0.9810821, shape=(), dtype=float32)\n",
            "Memory contains 318886 samples\n",
            "Reward over 10 games is: 0.3 and loss is: 0.023978792\n",
            "Actions distribution (last game, %) is:  [17 29 23 29  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.3455810546875\n",
            "Train time for the last game:  3.0468332767486572\n",
            "Elapsed time for the last game:  8.351256608963013\n",
            "\n",
            "Game 1851 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 319057\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9809088, shape=(), dtype=float32)\n",
            "Memory contains 319057 samples\n",
            "Reward over 10 games is: 0.3 and loss is: 0.02235013\n",
            "Actions distribution (last game, %) is:  [24 27 23 24  0  0]\n",
            "Steps survived: 171\n",
            "Image processing time for the last game:  0.5469150543212891\n",
            "Train time for the last game:  4.381279706954956\n",
            "Elapsed time for the last game:  9.997253179550171\n",
            "\n",
            "Game 1852 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Player is dead! game_reward is: 4.0\n",
            "Global_steps is: 319338\n",
            "Epsilon is: tf.Tensor(0.9806241, shape=(), dtype=float32)\n",
            "Memory contains 319338 samples\n",
            "Reward over 10 games is: 0.6 and loss is: 0.023456309\n",
            "Actions distribution (last game, %) is:  [22 26 26 24  0  0]\n",
            "Steps survived: 281\n",
            "Image processing time for the last game:  0.7718710899353027\n",
            "Train time for the last game:  6.884900331497192\n",
            "Elapsed time for the last game:  13.126277446746826\n",
            "\n",
            "Game 1853 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 319491\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.98046905, shape=(), dtype=float32)\n",
            "Memory contains 319491 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.022626616\n",
            "Actions distribution (last game, %) is:  [23 24 25 26  0  0]\n",
            "Steps survived: 153\n",
            "Image processing time for the last game:  0.4137992858886719\n",
            "Train time for the last game:  3.833324909210205\n",
            "Elapsed time for the last game:  9.26824426651001\n",
            "\n",
            "Game 1854 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 319615\n",
            "Epsilon is: tf.Tensor(0.9803434, shape=(), dtype=float32)\n",
            "Memory contains 319615 samples\n",
            "Reward over 10 games is: 0.7 and loss is: 0.022107424\n",
            "Actions distribution (last game, %) is:  [15 33 27 23  0  0]\n",
            "Steps survived: 124\n",
            "Image processing time for the last game:  0.33536577224731445\n",
            "Train time for the last game:  3.06134295463562\n",
            "Elapsed time for the last game:  8.433222532272339\n",
            "\n",
            "Game 1855 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Player is dead! game_reward is: 3.0\n",
            "Global_steps is: 319862\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9800931, shape=(), dtype=float32)\n",
            "Memory contains 319862 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.02275537\n",
            "Actions distribution (last game, %) is:  [24 25 29 20  0  0]\n",
            "Steps survived: 247\n",
            "Image processing time for the last game:  0.6806049346923828\n",
            "Train time for the last game:  6.137909412384033\n",
            "Elapsed time for the last game:  12.213981866836548\n",
            "\n",
            "Game 1856 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 319985\n",
            "Epsilon is: tf.Tensor(0.9799685, shape=(), dtype=float32)\n",
            "Memory contains 319985 samples\n",
            "Reward over 10 games is: 0.9 and loss is: 0.02256755\n",
            "Actions distribution (last game, %) is:  [16 27 26 29  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.33672475814819336\n",
            "Train time for the last game:  3.088592052459717\n",
            "Elapsed time for the last game:  8.408603191375732\n",
            "\n",
            "Game 1857 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 320189\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9797618, shape=(), dtype=float32)\n",
            "Memory contains 320189 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.022264086\n",
            "Actions distribution (last game, %) is:  [23 26 25 24  0  0]\n",
            "Steps survived: 204\n",
            "Image processing time for the last game:  0.5935232639312744\n",
            "Train time for the last game:  5.090543508529663\n",
            "Elapsed time for the last game:  10.966902732849121\n",
            "\n",
            "Game 1858 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 320314\n",
            "Epsilon is: tf.Tensor(0.9796351, shape=(), dtype=float32)\n",
            "Memory contains 320314 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.025323467\n",
            "Actions distribution (last game, %) is:  [21 29 30 18  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.356032133102417\n",
            "Train time for the last game:  3.111435651779175\n",
            "Elapsed time for the last game:  8.510764122009277\n",
            "\n",
            "Game 1859 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 320437\n",
            "Epsilon is: tf.Tensor(0.9795105, shape=(), dtype=float32)\n",
            "Memory contains 320437 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.022248361\n",
            "Actions distribution (last game, %) is:  [21 31 22 24  0  0]\n",
            "Steps survived: 123\n",
            "Image processing time for the last game:  0.33905935287475586\n",
            "Train time for the last game:  3.0358052253723145\n",
            "Elapsed time for the last game:  8.308658123016357\n",
            "\n",
            "Game 1860 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Global_steps is: 320562\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.9793838, shape=(), dtype=float32)\n",
            "Memory contains 320562 samples\n",
            "Reward over 10 games is: 1.1 and loss is: 0.02512084\n",
            "Actions distribution (last game, %) is:  [26 26 26 20  0  0]\n",
            "Steps survived: 125\n",
            "Image processing time for the last game:  0.3453536033630371\n",
            "Train time for the last game:  3.1199140548706055\n",
            "Elapsed time for the last game:  8.53987741470337\n",
            "\n",
            "Game 1861 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 320761\n",
            "Epsilon is: tf.Tensor(0.9791822, shape=(), dtype=float32)\n",
            "Memory contains 320761 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.0223299\n",
            "Actions distribution (last game, %) is:  [26 27 25 20  0  0]\n",
            "Steps survived: 199\n",
            "Image processing time for the last game:  0.5417284965515137\n",
            "Train time for the last game:  4.867079496383667\n",
            "Elapsed time for the last game:  10.49718427658081\n",
            "\n",
            "Game 1862 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 320959\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.97898155, shape=(), dtype=float32)\n",
            "Memory contains 320959 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.023717215\n",
            "Actions distribution (last game, %) is:  [23 27 28 20  0  0]\n",
            "Steps survived: 198\n",
            "Image processing time for the last game:  0.5272567272186279\n",
            "Train time for the last game:  4.765732526779175\n",
            "Elapsed time for the last game:  10.37953519821167\n",
            "\n",
            "Game 1863 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 1.0\n",
            "Global_steps is: 321129\n",
            "Epsilon is: tf.Tensor(0.9788093, shape=(), dtype=float32)\n",
            "Memory contains 321129 samples\n",
            "Reward over 10 games is: 1.0 and loss is: 0.023960775\n",
            "Actions distribution (last game, %) is:  [28 26 23 21  0  0]\n",
            "Steps survived: 170\n",
            "Image processing time for the last game:  0.45772838592529297\n",
            "Train time for the last game:  4.11941385269165\n",
            "Elapsed time for the last game:  9.724055528640747\n",
            "\n",
            "Game 1864 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Global_steps is: 321351\n",
            "Updating target model **************************** Updating target model ****************\n",
            "Epsilon is: tf.Tensor(0.97858435, shape=(), dtype=float32)\n",
            "Memory contains 321351 samples\n",
            "Reward over 10 games is: 1.2 and loss is: 0.023275139\n",
            "Actions distribution (last game, %) is:  [26 24 26 22  0  0]\n",
            "Steps survived: 222\n",
            "Image processing time for the last game:  0.6025362014770508\n",
            "Train time for the last game:  5.298831462860107\n",
            "Elapsed time for the last game:  11.29943060874939\n",
            "\n",
            "Game 1865 - Run 20200911003313\n",
            "Player is dead! game_reward is: 0.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n",
            "Player is dead! game_reward is: 2.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZFumPCVWiC4G",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}