CNN_S_D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-1cmZnQOm517"
   },
   "source": [
    "## P&O ISSP: Brain-computer interface voor sturing van een directionele akoestische zoom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "15TgxRRrm52A"
   },
   "source": [
    "In this notebook, we will develop a basic deep convolutional network for classifying the EEG and Stimuli signal into **Left** or **Right**.  \n",
    "\n",
    "One of the ways to process the EEG data is to find specific patterns in the signal. Based on the presence or absence of these patterns we will decide where is the auditory attention. But handcrafting these pattern might be difficult, so we will convolutional neural network to learn filters which can detect those patterns.\n",
    "\n",
    "The paper (pre-print version) proposes a CNN model to predict the auditory attention. We will try to create the model given in the paper: **Deckers, Lucas, et al. \"EEG-based detection of the attended speaker and the locus of auditory attention with convolutional neural networks.\" bioRxiv (2018): 475673.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iAJUj1LWm52C"
   },
   "source": [
    "**Note**: If keras is  not already installed, execute: !pip install keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "v_xoiZWUm52E"
   },
   "outputs": [],
   "source": [
    "# Load required libraries\n",
    "import keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Conv2D, Flatten, Activation, GlobalAveragePooling2D\n",
    "from keras.utils import to_categorical\n",
    "from keras import regularizers\n",
    "from keras.callbacks import LearningRateScheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WRUN1nnem52N"
   },
   "source": [
    "![fig_1](fig_1.png)\n",
    "\n",
    "* The EEG data preprocessing has been explained in another tutorial.\n",
    "* After preprocessing we get 64 channel EEG data (shown in <span style=\"color:black\">**black**</span>) and two 1-channel audio signal (stimuli) (shown in <span style=\"color:blue\">**blue**</span>).\n",
    "* The first step in the model is a convolutional layer, indicated in red. A (66 x 9) spatio-temporal filter is shifted over the input matrix, containing the audio envelopes and the EEG.\n",
    "* A rectifying linear unit (ReLu) activation function is used after the convolution step.\n",
    "* In the next step, average pooling, i.e., averaging the values of the resulting (parallel) data over the time dimension is done. \n",
    "* Following the pooling, there is a two-layer fully connected network (FCN), containing 5 neurons in the first layer and 2 neurons in the second layer, with a sigmoidal activation function.\n",
    "* The network was initialized with random weights between −0.1 and 0.1.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4hmbrPpkm52O"
   },
   "outputs": [],
   "source": [
    "initz = keras.initializers.RandomUniform(minval=-0.1, maxval=0.1, seed=None) #intializer\n",
    "\n",
    "model = Sequential()#add model layers\n",
    "\n",
    "## ---- add your code ----here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lvReB1npm52U"
   },
   "outputs": [],
   "source": [
    "# To check the model summary:\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8lHZ476_m52a"
   },
   "source": [
    "As the learning rate was halved after respectively 10, 25, and 40 training epochs. We have to write a learning rate schedular."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1TXvgHZ7m52c"
   },
   "outputs": [],
   "source": [
    "# Step decay function to reduce the learning rate over the epochs\n",
    "def step_decay(epoch):\n",
    "    if epoch <10:\n",
    "        lrate = 0.1\n",
    "    elif epoch>=10 and epoch<25:\n",
    "        lrate = 0.05\n",
    "    elif epoch>=25 and epoch<40:\n",
    "        lrate = 0.025\n",
    "    else:\n",
    "        lrate = 0.0125\n",
    "    return lrate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "G3mzuy-jm52i"
   },
   "source": [
    "* Now we prepare our data to train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3X0Vn9JWm52k"
   },
   "outputs": [],
   "source": [
    "import h5py\n",
    "import numpy as np\n",
    "\n",
    "def load_large_mat(filepath):\n",
    "    arrays = {}\n",
    "    f = h5py.File(filepath)\n",
    "    for k, v in f.items():\n",
    "        arrays[k] = np.array(v)\n",
    "    f.close()\n",
    "    return arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DuPGMqTKm52p"
   },
   "outputs": [],
   "source": [
    "# Preprocessing\n",
    "def fn_all(fnarrays1,fnarrays2):\n",
    "    #------add your prerprocessing steps here\n",
    "    fnxtr_all = fnarrays1\n",
    "    fny_tr_all = fnarrays2\n",
    "    x = np.expand_dims(fnxtr_all,-1)\n",
    "    y = to_categorical(fny_tr_all-1)\n",
    "    return x,y"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "CNN_S_D.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}