DAE2_DesigningExperiments.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gzIXWFwMJiSn"
   },
   "source": [
    "# DAE 2 Designing an experiment\n",
    "\n",
    "Author:\n",
    "- |\n",
    "    Sofia Dahl, sof@create.aau.dk\\\n",
    "    Dept. Architecture, Design and Media Technology, Aalborg University Copenhagen\n",
    "---\n",
    "\n",
    "## Learning Goals\n",
    "After working your way through this notebook you should be able to..\n",
    "\n",
    "- Discuss pros and cons of \n",
    "  + Between-group designs\n",
    "  + Within-group designs\n",
    "\n",
    "- Discuss when it is appropriate to perform an experimental study  \n",
    "\n",
    "## Table of Contents\n",
    "\n",
    "- [1. Concerns when designing a study](#Concerns)\n",
    "    - [1.1 Confounding Variables](#Confounds)\n",
    "    - [1.2 Control Groups](#Control)\n",
    "- [2. Good experimental designs](#GoodDesigns)    \n",
    "    - [2.1 Between Group](#Between)\n",
    "    - [2.2 Within Groups](#Within)\n",
    "- [3. When not to use experiments](#WhenNot) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "executionInfo": {
     "elapsed": 527,
     "status": "ok",
     "timestamp": 1620195233310,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "eQk8Iy6uJiSt"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-TZ1l8N2JiSu"
   },
   "source": [
    "<a id= 'Concerns'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v8YfcsJSJiSv"
   },
   "source": [
    "## 1 Concerns when Designing a study\n",
    "\n",
    "When we have a research question and/or would like to test the performance of a technical solution versus another, we have to consider how best to design and set up the study.\n",
    "\n",
    "We perform experiments to gain knowledge about whether the *independent* variable has a (measurable) effect on the *dependent* variable. \n",
    "\n",
    "![](variablemanipulation.png \"Title\")\n",
    "\n",
    "\n",
    "There are a number of problems here. \n",
    "\n",
    "Firstly, to detect a change from our intervention, we naturally need to know what value we had before. \n",
    "\n",
    "\n",
    "...but even if we measure a change in the dependent variable, how can we be sure that this is due to our manipulation of the independent variable? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 580
    },
    "executionInfo": {
     "elapsed": 666,
     "status": "ok",
     "timestamp": 1620195303747,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "PI4jL4kyJiSv",
    "outputId": "15249d48-5d8d-480c-83fb-cb73ccc91273"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case_A</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>participant_0</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_1</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_2</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_3</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_4</th>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_5</th>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_6</th>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_7</th>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_8</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_9</th>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_10</th>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_11</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_12</th>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_13</th>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_14</th>\n",
       "      <td>64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_15</th>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_16</th>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>participant_17</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                Case_A\n",
       "participant_0       40\n",
       "participant_1       30\n",
       "participant_2       30\n",
       "participant_3       40\n",
       "participant_4       20\n",
       "participant_5       50\n",
       "participant_6       65\n",
       "participant_7       35\n",
       "participant_8       30\n",
       "participant_9       65\n",
       "participant_10      60\n",
       "participant_11      30\n",
       "participant_12      27\n",
       "participant_13      42\n",
       "participant_14      64\n",
       "participant_15      50\n",
       "participant_16      32\n",
       "participant_17      30"
      ]
     },
     "execution_count": 2,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#TRY OUT\n",
    "#Let's say we have some measured data from a group of participants (Group_A)\n",
    "\n",
    "A=np.array([40,30,30,40,20,50,65,35,30,65,60,30,27,42,64,50,32,30])\n",
    "\n",
    "#And to make it convenient we put this in a data frame with index for the participants\n",
    "index = [f'participant_{num}' for num in range(A.shape[0])] #generate row index for all entries\n",
    "\n",
    "data = pd.DataFrame(A, columns=([\"Case_A\"]), index=index)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LDQO67NTJiSw"
   },
   "source": [
    "Let's say that all these are game scores and that none of the participants had played this particular game before. Why are the scores so different? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OSxmwnweJiSw"
   },
   "source": [
    "<a id='Confounds'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KSNtWshxJiSx"
   },
   "source": [
    "### 1.2 Confounding Variables\n",
    "There are, in fact, many other things other than your manipulation of the independent variable that can influence your measured data (dependent variable).\n",
    "\n",
    "For one thing, your participants may have different prior knowledge, experiences, and preferences. There can also be factors in the actual test that have an influence, such as the place where the testing takes place, or by whom and in what way the instructions for the test are given.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SAlB6rvqJiSy"
   },
   "source": [
    "A more correct illustration then looks more like this:\n",
    "\n",
    "![](confoundingvariables.png \"Title\")\n",
    "\n",
    "Many other *confounding* variables (arrows in the figure) could influence the outcome of the experiment. To be able to draw the right conclusions on the effect of the indpendent variable, a proper experiment needs control for (or randomize) *ALL* confounding variables. That is, to avoid any systematic differences that can influence the interpretation of the manipulation, ALL the possile confounding influences need either to be kept: \n",
    "\n",
    "*Constant* (e.g. instructions and testing set up is as far as possible done the same way for all participants)\n",
    "\n",
    "or\n",
    "\n",
    "*Random* (e.g. test conditions are randomly assigned to avoid any systematic effects due to presentation order).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JD9OhXtlJiSy"
   },
   "source": [
    "<a id='Control'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Snyck4dbJiSz"
   },
   "source": [
    "### 1.2 Control group\n",
    "\n",
    "Let's say we are testing a complex interactive situation (e.g. application with music that changes interactively). Here, there can be several confounding variables, and thus we should *compare* the interaction to an equivalent situation without any interaction (i.e. manipulation of the independent variable).\n",
    "\n",
    "In this example application with music that changes interactively, we hypothesize that the interactivity will have an effect on some dependent variable (e.g. user experience). So a good control case would be the exact same application but with music that is non-interactive (i.e. prerecorded music). Since everything except the interactivity is identical between the two cases, we would assume that any change in user experience is due to the interactivity. Or?\n",
    "\n",
    "\n",
    "Also, even if we control for the confounding variables we can foresee in the situation,  there can still be a lot of outcome variability due to participants just being...people. And people are different.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KzdK1ljQJiSz"
   },
   "source": [
    "### Try out 1: \n",
    "We can illustrate this by adding another variable with data (Group_B) based on the original data but with some added random variability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 284
    },
    "executionInfo": {
     "elapsed": 631,
     "status": "ok",
     "timestamp": 1620196238710,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "wubSJGdWJiSz",
    "outputId": "a1af57f8-b5c1-403e-de71-b5d7cf2bf554"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Case_A</th>\n",
       "      <th>Case_B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>18.000000</td>\n",
       "      <td>18.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>41.111111</td>\n",
       "      <td>41.691710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>14.482805</td>\n",
       "      <td>14.194665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>20.000000</td>\n",
       "      <td>22.032634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>30.000000</td>\n",
       "      <td>31.250782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.500000</td>\n",
       "      <td>38.236262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>49.011344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>65.000000</td>\n",
       "      <td>66.651939</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Case_A     Case_B\n",
       "count  18.000000  18.000000\n",
       "mean   41.111111  41.691710\n",
       "std    14.482805  14.194665\n",
       "min    20.000000  22.032634\n",
       "25%    30.000000  31.250782\n",
       "50%    37.500000  38.236262\n",
       "75%    50.000000  49.011344\n",
       "max    65.000000  66.651939"
      ]
     },
     "execution_count": 27,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can illustrate this by adding another variable with data (Group_B). \n",
    "# We base this data on that of Group_A, but with some random variability added \n",
    "\n",
    "data['Case_B']=data.Case_A+np.random.normal(0, 1.5, data.shape[0])\n",
    "\n",
    "# We can use describe to get an overview and see that the values are a bit different\n",
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "executionInfo": {
     "elapsed": 509,
     "status": "ok",
     "timestamp": 1620196230603,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "vxffT7PaJiS0",
    "outputId": "c19e516f-316d-4adf-c91d-269927def158"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMZ0lEQVR4nO3db4hl9X3H8fcnu4rJRrTRiyxZp2NpMEipqww2okizYtBG7JMQNE1JQ2Ce2KKQEkwfaaHQPkmTByWwmKSBalJrXBoMNZHE0AbaTWd10+iuSxO7wZWYHUmtUaiy9tsH96xONrPOGZ0z9zf3vl8wzP1z9u73x1zfezxzzkyqCklSu9426QEkSW/MUEtS4wy1JDXOUEtS4wy1JDVu+xAvev7559f8/PwQLy1JU+nAgQPPVdVotecGCfX8/DxLS0tDvLQkTaUkPzndcx76kKTGGWpJapyhlqTGGWpJapyhlqTGGWpJatyaoU5ycZKDKz5eSHL7ZgwnSepxHnVVHQF2AyTZBjwD7Bt4LklSZ72HPq4FflxVpz0xW5K0sdZ7ZeLNwFdWeyLJIrAIMDc39xbHmjF3njPpCTbd/P/eC8DRv/zghCeR2td7jzrJmcBNwD+s9nxV7a2qhapaGI1WvVxdkvQmrOfQxw3Ao1X1s6GGkST9qvWE+hZOc9hDkjScXqFOsgO4Dnhg2HEkSafq9c3EqnoJOG/gWSRJq/DKRElqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMYZaklqnKGWpMb1/S3k5ya5P8mTSQ4nuXLowSRJY71+CznwOeChqvpQkjOBdww4kyRphTVDneQc4BrgjwCq6hXglWHHkiSd1OfQx0XAMvClJI8luTvJjlM3SrKYZCnJ0vLy8oYPKkmzqk+otwOXA5+vqsuAl4A7Tt2oqvZW1UJVLYxGow0eU5JmV59QHwOOVdX+7v79jMMtSdoEa4a6qp4Fnk5ycffQtcChQaeSJL2m71kffwLc053x8RTw8eFGkiSt1CvUVXUQWBh4FknSKrwyUZIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIaZ6glqXGGWpIa1+u3kCc5CvwCeBU4UVX+RnJJ2iS9Qt15f1U9N9gkkqRVeehDkhrXN9QFfCvJgSSLq22QZDHJUpKl5eXljZtQkmZc31BfXVWXAzcAtya55tQNqmpvVS1U1cJoNNrQISVplvUKdVU9030+DuwDrhhyKEnS69YMdZIdSc4+eRv4APD40INJksb6nPVxAbAvycnt762qhwadSpL0mjVDXVVPAZduwiySpFV4ep4kNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1Ljeoc6ybYkjyV5cMiBJEm/bD171LcBh4caRJK0ul6hTrIL+CBw97DjSJJOtb3ndp8FPgWcfboNkiwCiwBzc3NvfTJNtaNnfWR8486JjrF57vyfSU+woebv+Maa27z2NZ4lA32d19yjTnIjcLyqDrzRdlW1t6oWqmphNBpt2ICSNOv6HPq4CrgpyVHgq8CeJH836FSSpNesGeqq+nRV7aqqeeBm4DtV9dHBJ5MkAZ5HLUnN6/vNRACq6rvAdweZRJK0KveoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxhlqSGmeoJalxa4Y6yVlJvp/kB0meSHLXZgwmSRrb3mObl4E9VfVikjOA7yX5p6r6t4FnkyTRI9RVVcCL3d0zuo8acihJ0ut6HaNOsi3JQeA48HBV7V9lm8UkS0mWlpeXN3pOSZpZvUJdVa9W1W5gF3BFkt9aZZu9VbVQVQuj0Wij55SkmbWusz6q6nngEeD6YcaRJJ2qz1kfoyTndrffDlwHPDn0YJKksT5nfewEvpxkG+Ow31dVDw47liTppD5nffwHcNkmzCJJWoVXJkpS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDVuzVAnuTDJI0kOJXkiyW2bMZgkaWzN30IOnAA+WVWPJjkbOJDk4ao6NPBskiR67FFX1U+r6tHu9i+Aw8C7hx5MkjS2rmPUSeaBy4D9QwwjSfpVfQ59AJDkncDXgNur6oVVnl8EFgHm5ube/ER3nvPm/6zUqPk7vjHpEbSF9dqjTnIG40jfU1UPrLZNVe2tqoWqWhiNRhs5oyTNtD5nfQT4AnC4qj4z/EiSpJX67FFfBfwhsCfJwe7j9waeS5LUWfMYdVV9D8gmzCJJWoVXJkpS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS4wy1JDXOUEtS49YMdZIvJjme5PHNGEiS9Mv67FH/LXD9wHNIkk5jzVBX1T8DP9+EWSRJq9i+US+UZBFYBJibm9uol5WmwtGzPjLpEbSFbdg3E6tqb1UtVNXCaDTaqJeVpJnnWR+S1DhDLUmN63N63leAfwUuTnIsySeGH0uSdNKa30ysqls2YxBJ0uo89CFJjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjesV6iTXJzmS5EdJ7hh6KEnS69YMdZJtwN8ANwCXALckuWTowSRJY332qK8AflRVT1XVK8BXgd8fdixJ0knbe2zzbuDpFfePAb9z6kZJFoHF7u6LSY689fF6Ox94bhP/vhbM4prBdc+Srbfmu/JW/vSvn+6JPqHupar2Ans36vXWI8lSVS1M4u+elFlcM7juSc+xmWZxzafT59DHM8CFK+7v6h6TJG2CPqH+d+A9SS5KciZwM/D1YceSJJ205qGPqjqR5I+BbwLbgC9W1RODT7Y+EznkMmGzuGZw3bNkFte8qlTVpGeQJL0Br0yUpMYZaklq3JYKdZILkzyS5FCSJ5Lc1j3+riQPJ/nP7vOvTXrWjZTkrCTfT/KDbt13dY9flGR/d2n/33ff7J0qSbYleSzJg939WVjz0SQ/THIwyVL32LS/x89Ncn+SJ5McTnLltK95PbZUqIETwCer6hLgfcCt3eXsdwDfrqr3AN/u7k+Tl4E9VXUpsBu4Psn7gL8C/rqqfhP4b+ATE5xxKLcBh1fcn4U1A7y/qnavOI942t/jnwMeqqr3Apcy/ppP+5r7q6ot+wH8I3AdcATY2T22Ezgy6dkGXPM7gEcZXx36HLC9e/xK4JuTnm+D17qL8X+ge4AHgUz7mrt1HQXOP+WxqX2PA+cA/0V3csMsrHm9H1ttj/o1SeaBy4D9wAVV9dPuqWeBCyY01mC6QwAHgePAw8CPgeer6kS3yTHGl/tPk88CnwL+r7t/HtO/ZoACvpXkQPejGWC63+MXAcvAl7rDXHcn2cF0r3ldtmSok7wT+Bpwe1W9sPK5Gv/zO3XnHFbVq1W1m/Fe5hXAeyc80qCS3Agcr6oDk55lAq6uqssZ/8TKW5Ncs/LJKXyPbwcuBz5fVZcBL3HKYY4pXPO6bLlQJzmDcaTvqaoHuod/lmRn9/xOxnudU6mqngceYfy//ecmOXnR0rRd2n8VcFOSo4x/YuMexscxp3nNAFTVM93n48A+xv8wT/N7/BhwrKr2d/fvZxzuaV7zumypUCcJ8AXgcFV9ZsVTXwc+1t3+GONj11MjySjJud3ttzM+Ln+YcbA/1G02Veuuqk9X1a6qmmf8Ywu+U1V/wBSvGSDJjiRnn7wNfAB4nCl+j1fVs8DTSS7uHroWOMQUr3m9ttSViUmuBv4F+CGvH7f8M8bHqe8D5oCfAB+uqp9PZMgBJPlt4MuML+F/G3BfVf15kt9gvLf5LuAx4KNV9fLkJh1Gkt8F/rSqbpz2NXfr29fd3Q7cW1V/keQ8pvs9vhu4GzgTeAr4ON17nSld83psqVBL0izaUoc+JGkWGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTG/T9ExLoTMYkqZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# We can plot the data in histograms\n",
    "plt.hist(data.Case_A, bins=5)\n",
    "plt.hist(data.Case_B, bins=5)\n",
    "plt.show()\n",
    "# Note that the data can look very similar for the two groups. \n",
    "# Because we're adding random numbers based on a normal distribution with mean 0, the values might not change much. \n",
    "# If you're lucky. \n",
    "# But, being random, it can also vary a lot more.\n",
    "# Re-run the randomization code above and plot again to convince yourself that the values change slightly between runs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AIvBNZdhJiS0"
   },
   "source": [
    "### NOTE: \n",
    "With this number of participants, you can see that even small random variations can lead to differences in the outcome data. More on how what conclusions we can draw from the data in later chapters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zsCFDaAKJiS0"
   },
   "source": [
    "<a id= 'GoodDesigns'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "isg-kAE5JiS1"
   },
   "source": [
    "## 2. Good experimental designs\n",
    "\n",
    "There are basically two type of designs for well-conducted experiments:\n",
    "\n",
    "*Between group* where each level of the manipulated independent variable is presented to a different group of participants. \n",
    "\n",
    "and\n",
    "\n",
    "*Within group* all participants are exposed to the different levels of the manipulated independent variable.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a204BTSrJiS1"
   },
   "source": [
    "<a id= 'Between'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R2z_pRiZJiS1"
   },
   "source": [
    "### 2. Between-Group Design\n",
    "\n",
    "In the between-group design, you randomly assign participants to one of the experimental conditions. The random assignment is helps minimizes the influence of any *systematic* differences between the groups that might affect data. However, you might also wish to keep the sample sizes for the two groups approximately equal, so sometimes semi-random is necessary. This can, for instance, mean that you monitor the number of participants assigned to any one condition and, if already many more than the other condition,  reduce the probability of the remaining participants being assigned to this condition. \n",
    "\n",
    "<img src=\"https://github.com/sofiadahl/Design_and_Analysis_of_Experiments/blob/main/BetweenGroupDesign.png?raw=1\" width=\"600\">\n",
    "\n",
    "Since participants are only exposed to ONE condition (e.g. interface with OR without interactive music), there is no learning or carry-over effect between conditions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t-8s9dnxJiS1"
   },
   "source": [
    "### Try out 2: \n",
    "Let's revisit our Case_B and now insert some SYSTEMATIC variability, that we (hope) is from our manipulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "executionInfo": {
     "elapsed": 1371,
     "status": "ok",
     "timestamp": 1620196497530,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "upIWra2sJiS1",
    "outputId": "10387e1f-71e8-46c6-b1e4-3e169bd51595"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMzklEQVR4nO3db4hl9X3H8ffHXWWTjWijF5Gs0zEkbJBSVzvYiCLNikEbsU9C0TQlDYF5YotCStj00VootE/S5EEIXUzSQI2pMS4NSk0kMbQp7aa7ahp1XZrYTV2J7kqqRgvK2m8f3DPuZDLrnF3vmfvz7vsFw94/Z67fH3N5e+bMOTOpKiRJ7Tpt2gNIkt6YoZakxhlqSWqcoZakxhlqSWrcxiFe9Nxzz635+fkhXlqSZtK+ffueq6rRas8NEur5+Xn27t07xEtL0kxK8tPjPeehD0lqnKGWpMYZaklqnKGWpMYZaklqnKGWpMatGeokW5M8suzjxSS3rsdwkqQe51FX1QFgG0CSDcDTwO6B55IkdU700MfVwE+q6rgnZkuSJutEr0y8EbhztSeSLAKLAHNzc29yrNkyv+O+aY8wcQf/8kPTHkE6ZfTeo05yBnAD8PXVnq+qXVW1UFULo9Gql6tLkk7CiRz6uA54qKqeHWoYSdKvOpFQ38RxDntIkobTK9RJNgPXAPcMO44kaaVeP0ysqpeBcwaeRZK0Cq9MlKTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJapyhlqTGGWpJalzfv0J+dpK7kzyRZH+Sy4ceTJI01uuvkAOfA+6vqg8nOQN4+4AzSZKWWTPUSc4CrgL+CKCqXgVeHXYsSdKSPoc+LgSOAF9O8nCS25NsXrlRksUke5PsPXLkyMQHlaRTVZ9QbwQuBb5QVZcALwM7Vm5UVbuqaqGqFkaj0YTHlKRTV59QHwIOVdWe7v7djMMtSVoHa4a6qp4BnkqytXvoauDxQaeSJL2u71kffwLc0Z3x8STw8eFGkiQt1yvUVfUIsDDwLJKkVXhloiQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1rtdfIU9yEPgF8BpwtKr8i+SStE56hbrzgap6brBJJEmr8tCHJDWu7x51Ad9OUsDfVNWulRskWQQWAebm5iY3oQZ3cNNHTvyTdk58jMna+cK0J5Ampu8e9ZVVdSlwHXBzkqtWblBVu6pqoaoWRqPRRIeUpFNZr1BX1dPdv4eB3cBlQw4lSTpmzVAn2ZzkzKXbwAeBR4ceTJI01ucY9XnA7iRL23+1qu4fdCpJ0uvWDHVVPQlcvA6zSJJW4el5ktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1JjTPUktQ4Qy1Jjesd6iQbkjyc5N4hB5Ik/bIT2aO+Bdg/1CCSpNX1CnWSLcCHgNuHHUeStNLGntt9FvgUcObxNkiyCCwCzM3NvfnJWrXzrBP+lIObBphD6mF+x33THuG4Dm76yLRHmLydLwzysmvuUSe5HjhcVfveaLuq2lVVC1W1MBqNJjagJJ3q+hz6uAK4IclB4GvA9iR/N+hUkqTXrRnqqvp0VW2pqnngRuC7VfXRwSeTJAGeRy1Jzev7w0QAqup7wPcGmUSStCr3qCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcYZakhpnqCWpcWuGOsmmJD9I8sMkjyW5bT0GkySNbeyxzSvA9qp6KcnpwPeT/GNV/dvAs0mS6BHqqirgpe7u6d1HDTmUJOmYPnvUJNkA7APeA3y+qvasss0isAgwNzc3yRmlE7fzrGlPMFk7X5j2BJqiXj9MrKrXqmobsAW4LMlvrLLNrqpaqKqF0Wg06Tkl6ZR1Qmd9VNXzwIPAtcOMI0laqc9ZH6MkZ3e33wZcAzwx9GCSpLE+x6jPB77SHac+Dbirqu4ddixJ0pI+Z338B3DJOswiSVqFVyZKUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1zlBLUuMMtSQ1bs1QJ7kgyYNJHk/yWJJb1mMwSdLYmn+FHDgKfLKqHkpyJrAvyQNV9fjAs0mS6LFHXVU/q6qHutu/APYD7xp6MEnS2Akdo04yD1wC7BliGEnSr+pz6AOAJO8AvgHcWlUvrvL8IrAIMDc3d9IDze+476Q/dz0c3DTtCXRK2nnWSX2a79fZ0GuPOsnpjCN9R1Xds9o2VbWrqhaqamE0Gk1yRkk6pfU56yPAF4H9VfWZ4UeSJC3XZ4/6CuAPge1JHuk+fnfguSRJnTWPUVfV94GswyySpFV4ZaIkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNc5QS1LjDLUkNW7NUCf5UpLDSR5dj4EkSb+szx713wLXDjyHJOk41gx1Vf0T8PN1mEWStIqJHaNOsphkb5K9R44cmdTLStIpb2KhrqpdVbVQVQuj0WhSLytJpzzP+pCkxhlqSWpcn9Pz7gT+Fdia5FCSTww/liRpyca1Nqiqm9ZjEEnS6jz0IUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmNM9SS1DhDLUmN6xXqJNcmOZDkx0l2DD2UJOmYNUOdZAPweeA64CLgpiQXDT2YJGmszx71ZcCPq+rJqnoV+Brwe8OOJUlasrHHNu8Cnlp2/xDw2ys3SrIILHZ3X0py4CRnOhd47iQ/d3CZ7Ms1vdYJc62zybUud9ubKsSvH++JPqHupap2Abve7Osk2VtVCxMYqXmudTa51tk0zbX2OfTxNHDBsvtbusckSeugT6j/HXhvkguTnAHcCHxz2LEkSUvWPPRRVUeT/DHwLWAD8KWqemzAmd704ZO3ENc6m1zrbJraWlNV0/pvS5J68MpESWqcoZakxk0t1EkuSPJgkseTPJbklu7xdyZ5IMl/dv/+2rRmnJQkm5L8IMkPu7Xe1j1+YZI93aX5f9/9sHYmJNmQ5OEk93b3Z3KtSQ4m+VGSR5Ls7R6bufcwQJKzk9yd5Ikk+5NcPotrTbK1+3oufbyY5NZprnWae9RHgU9W1UXA+4Gbu0vTdwDfqar3At/p7r/VvQJsr6qLgW3AtUneD/wV8NdV9R7gf4BPTHHGSbsF2L/s/iyv9QNVtW3ZObaz+B4G+Bxwf1W9D7iY8dd35tZaVQe6r+c24LeA/wV2M821VlUTH8A/ANcAB4Dzu8fOBw5Me7YJr/PtwEOMr+58DtjYPX458K1pzzehNW7p3sjbgXsZX9A5q2s9CJy74rGZew8DZwH/RXcCwiyvdcX6Pgj8y7TX2sQx6iTzwCXAHuC8qvpZ99QzwHlTGmuiukMBjwCHgQeAnwDPV9XRbpNDjC/XnwWfBT4F/F93/xxmd60FfDvJvu7XKMBsvocvBI4AX+4Oad2eZDOzudblbgTu7G5Pba1TD3WSdwDfAG6tqheXP1fj/3XNxPmDVfVajb+V2sL4F129b8ojDSLJ9cDhqto37VnWyZVVdSnj3y55c5Krlj85Q+/hjcClwBeq6hLgZVZ86z9DawWg+znKDcDXVz633mudaqiTnM440ndU1T3dw88mOb97/nzGe6Azo6qeBx5k/O3/2UmWLjqalUvzrwBuSHKQ8W9a3M742OYsrpWqerr79zDj45iXMZvv4UPAoara092/m3G4Z3GtS64DHqqqZ7v7U1vrNM/6CPBFYH9VfWbZU98EPtbd/hjjY9dvaUlGSc7ubr+N8bH4/YyD/eFus5lYa1V9uqq2VNU8428bv1tVf8AMrjXJ5iRnLt1mfDzzUWbwPVxVzwBPJdnaPXQ18DgzuNZlbuLYYQ+Y4lqndmVikiuBfwZ+xLFjmX/G+Dj1XcAc8FPg96vq51MZckKS/CbwFcaX4J8G3FVVf57k3Yz3Ot8JPAx8tKpemd6kk5Xkd4A/rarrZ3Gt3Zp2d3c3Al+tqr9Icg4z9h4GSLINuB04A3gS+Djd+5nZW+tm4L+Bd1fVC91jU/u6egm5JDVu6j9MlCS9MUMtSY0z1JLUOEMtSY0z1JLUOEMtSY0z1JLUuP8HuY7sX4gXFakAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#now insert some SYSTEMATIC variability, that we (hope) is from our manipulation\n",
    "\n",
    "data['Case_B']=data.Case_A+np.random.normal(6, 1.5, data.shape[0])\n",
    "\n",
    "plt.hist(data.Case_A, bins=5)\n",
    "plt.hist(data.Case_B, bins=5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GIC6RHZ8JiS2"
   },
   "source": [
    "The systematic variability that we simulated (mean=6) shifts the histogram of Case_B to the right. When we plan the experiment, this is something we hope to see. A statistical test for *between-group* design tests whether the outcome scores for the different cases have means that differ *significantly* from each other.\n",
    "\n",
    "But there is still a lot of variability among the participants. In fact, the individual differences can overshadow the effect we want to measure. In this regard, it would be better if participants were the same for the two conditions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E5j5NV7fJiS2"
   },
   "source": [
    "<a id= 'Within'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BhsMmHQUJiS2"
   },
   "source": [
    "### 2. Within-Group Design\n",
    "\n",
    "One way to minimize the risk that individual variability overshadows any possible effect, is to let participants take part in both the experimental and the control condition. The benefit is that the variability due to individual differences is reduced, and this typically means that fewer participants are needed in order to spot an effect. \n",
    "But there are many other things that might spoil it. Again, it is important that there is no *systematic* effect of the order in which the conditions are presented, as well as and how participants are assigned. Participants can get tired, bored, more skilled, etc as they go through the test. Any such *order effects* need to be accounted for. \n",
    "\n",
    "<img src=\"https://github.com/sofiadahl/Design_and_Analysis_of_Experiments/blob/main/WithinGroupDesign.png?raw=1\" width=\"700\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x8KR-b4VJiS2"
   },
   "source": [
    "### Try out 3: \n",
    "Plot the difference between the different cases but for the same participants."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "executionInfo": {
     "elapsed": 514,
     "status": "ok",
     "timestamp": 1620196790261,
     "user": {
      "displayName": "",
      "photoUrl": "",
      "userId": ""
     },
     "user_tz": -120
    },
    "id": "9Og2gbqkJiS3",
    "outputId": "0d52bfa0-37ac-46b9-8921-f1e5117e0ea9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMWElEQVR4nO3cb4zkhV3H8ffHW6oWERLZ1IZjuyQ2JNVIwQ2V0BALaQOFwANJhKQqpGbVtJUmJs31iYk+ap+YVmPQC22tlv7RUwxyLZakbbSJRe8orcBBguQajrTe0VooaEqufn2wc8feZO7mB+zMfJd7v5INuzu/Gz75Bd43O/ObTVUhSerrxxY9QJJ0aoZakpoz1JLUnKGWpOYMtSQ1tzSLOz333HNrdXV1FnctSa9K+/fvf7qqlifdNpNQr66usm/fvlnctSS9KiX51slu86kPSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1NyjUSc5JsifJo0kOJLls1sMkSRuGXkf9UeDeqroxyWuA185wkyRpk6mhTnI2cAVwC0BVvQC8MNtZkqRjhjyivgA4AnwiyUXAfuC2qnp+80FJ1oF1gJWVla3eqVeZ1V17Fz1hrg5+6NpFT9A2NuQ56iXgEuD2qroYeB7YNX5QVe2uqrWqWltenvh2dUnSyzAk1IeAQ1V1/+jrPWyEW5I0B1NDXVXfAZ5McuHoW1cBj8x0lSTpuKFXfbwPuHN0xccTwK2zmyRJ2mxQqKvqQWBtxlskSRP4zkRJas5QS1JzhlqSmjPUktScoZak5gy1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJam5pyEFJDgI/AH4EHK2qtVmOkiS9aFCoR95WVU/PbIkkaSKf+pCk5oY+oi7gi0kK+Iuq2j1+QJJ1YB1gZWVl6xZK2pZWd+1d9IS5O/iha2dyv0MfUb+1qi4BrgHek+SK8QOqandVrVXV2vLy8paOlKTT2aBQV9VTo38eBu4CLp3lKEnSi6aGOsmZSc469jnwDuChWQ+TJG0Y8hz164C7khw7/tNVde9MV0mSjpsa6qp6ArhoDlskSRN4eZ4kNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJas5QS1JzhlqSmjPUktScoZak5gy1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJam5waFOsiPJ15PcM8tBkqQTvZRH1LcBB2Y1RJI02aBQJ9kJXAvcMds5kqRxSwOP+wjwAeCskx2QZB1YB1hZWXnly6RXkdVdexc9QdvY1EfUSa4DDlfV/lMdV1W7q2qtqtaWl5e3bKAkne6GPPVxOXB9koPAZ4Erk3xqpqskScdNDXVVfbCqdlbVKnAT8KWqetfMl0mSAK+jlqT2hr6YCEBVfQX4ykyWSJIm8hG1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJas5QS1JzhlqSmjPUktScoZak5gy1JDU3NdRJfiLJvyX5RpKHk/zhPIZJkjYsDTjmh8CVVfVckjOAryb5QlV9bcbbJEkMCHVVFfDc6MszRh81y1GSpBcNeo46yY4kDwKHgfuq6v4Jx6wn2Zdk35EjR7Z6pySdtgaFuqp+VFVvBnYClyb5hQnH7K6qtapaW15e3uqdknTaeklXfVTV94EvA1fPZo4kadyQqz6Wk5wz+vwngbcDj856mCRpw5CrPl4PfDLJDjbC/jdVdc9sZ0mSjhly1cc3gYvnsEWSNIHvTJSk5gy1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJas5QS1JzhlqSmjPUktScoZak5gy1JDU3NdRJzk/y5SSPJHk4yW3zGCZJ2rA04JijwO9X1QNJzgL2J7mvqh6Z8TZJEgMeUVfVt6vqgdHnPwAOAOfNepgkacOQR9THJVkFLgbun3DbOrAOsLKysgXTTh+ru/YueoKkxga/mJjkp4C/A95fVc+O315Vu6tqrarWlpeXt3KjJJ3WBoU6yRlsRPrOqvr72U6SJG025KqPAB8DDlTVH89+kiRpsyGPqC8Hfh24MsmDo493zniXJGlk6ouJVfVVIHPYIkmawHcmSlJzhlqSmjPUktScoZak5gy1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJas5QS1JzU0Od5ONJDid5aB6DJEknGvKI+i+Bq2e8Q5J0ElNDXVX/DHxvDlskSRMsbdUdJVkH1gFWVlZe9v2s7tq7VZMk6VVhy15MrKrdVbVWVWvLy8tbdbeSdNrzqg9Jas5QS1JzQy7P+wzwr8CFSQ4leffsZ0mSjpn6YmJV3TyPIZKkyXzqQ5KaM9SS1JyhlqTmDLUkNWeoJak5Qy1JzRlqSWrOUEtSc4Zakpoz1JLUnKGWpOYMtSQ1Z6glqTlDLUnNGWpJas5QS1JzhlqSmjPUktScoZak5gy1JDVnqCWpOUMtSc0ZaklqzlBLUnOGWpKaM9SS1NygUCe5OsljSR5PsmvWoyRJL5oa6iQ7gD8DrgHeBNyc5E2zHiZJ2jDkEfWlwONV9URVvQB8FrhhtrMkSccsDTjmPODJTV8fAt4yflCSdWB99OVzSR57BbvOBZ5+BX9+EbbjZnD3vLl7vua6Ox9+RX/8DSe7YUioB6mq3cDurbivJPuqam0r7mtetuNmcPe8uXu+tuvucUOe+ngKOH/T1ztH35MkzcGQUP878MYkFyR5DXATcPdsZ0mSjpn61EdVHU3yXuCfgB3Ax6vq4Rnv2pKnUOZsO24Gd8+bu+dru+4+Qapq0RskSafgOxMlqTlDLUnNLTTUSXYk+XqSeybc9uNJPjd62/r9SVbnv3CyKbtvSXIkyYOjj99axMZxSQ4m+Y/Rpn0Tbk+SPxmd728muWQRO8cN2P0rSZ7ZdL7/YBE7xyU5J8meJI8mOZDksrHbu57vabvbne8kF27a82CSZ5O8f+yYlud7qC27jvplug04APz0hNveDfx3Vf1ckpuADwO/Ns9xp3Cq3QCfq6r3znHPUG+rqpNd/H8N8MbRx1uA25nwxqYFOdVugH+pquvmtmaYjwL3VtWNo6ulXjt2e9fzPW03NDvfVfUY8GY4/isvngLuGjus6/keZGGPqJPsBK4F7jjJITcAnxx9vge4Kknmse1UBuzerm4A/qo2fA04J8nrFz1qO0pyNnAF8DGAqnqhqr4/dli78z1wd3dXAf9ZVd8a+3678/1SLPKpj48AHwD+7yS3H3/relUdBZ4BfmY+005p2m6AXx39eLUnyfmnOG6eCvhikv2jt/uPm/SrAs6by7JTm7Yb4LIk30jyhSQ/P89xJ3EBcAT4xOgpsjuSnDl2TMfzPWQ39Dvfm90EfGbC9zue78EWEuok1wGHq2r/Iv79L9fA3f8IrFbVLwL38eJPBYv21qq6hI0fAd+T5IpFDxpo2u4HgDdU1UXAnwL/MO+BEywBlwC3V9XFwPPAdvj1wEN2dzzfAIyeqrke+NtFb9lqi3pEfTlwfZKDbPw2viuTfGrsmONvXU+yBJwNfHeeIyeYuruqvltVPxx9eQfwS/OdOFlVPTX652E2nr+7dOyQlr8qYNruqnq2qp4bff554Iwk58596IkOAYeq6v7R13vYCOBmHc/31N1Nz/cx1wAPVNV/Tbit4/kebCGhrqoPVtXOqlpl40eVL1XVu8YOuxv4zdHnN46OWei7c4bsHnve63o2XnRcqCRnJjnr2OfAO4CHxg67G/iN0avjvww8U1XfnvPUEwzZneRnj712keRSNv6bXuhf6FX1HeDJJBeOvnUV8MjYYe3O95DdHc/3Jjcz+WkPaHi+X4pFX/VxgiR/BOyrqrvZeEHjr5M8DnyPjTC2NLb795JcDxxlY/cti9w28jrgrtH/X0vAp6vq3iS/A1BVfw58Hngn8DjwP8CtC9q62ZDdNwK/m+Qo8L/ATYv+C33kfcCdox/HnwBu3QbnG6bvbnm+R3+Rvx347U3f2w7nexDfQi5JzfnORElqzlBLUnOGWpKaM9SS1JyhlqTmDLUkNWeoJam5/wfwKl7VxFFNPQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# When participants are the same, we are interest in the difference WITHIN them BETWEEN the two conditions. \n",
    "\n",
    "data['Case_C']=data.Case_B-data.Case_A\n",
    "\n",
    "plt.hist(data.Case_C, bins=5)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wNXMEpLjJiS3"
   },
   "source": [
    "So the histogram showes the *difference* in outcome scores between two cases. What do we compare that with you wonder? Well, if there had been NO difference, the plotted values would be (or be close to) zero. A statistical test for *within group* design, tests whether the difference between outcome scores has a mean significantly different from 0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9GD5rTCHJiS3"
   },
   "source": [
    "<a id= 'WhenNot'> </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QJv4jGgXJiS3"
   },
   "source": [
    "## 3. When not to use experiments\n",
    "\n",
    "We have talked a lot about how we need to *control* a lot of variables to be able to interpret our data in a way that can confirm whether a manipulation has had an effect.\n",
    "\n",
    "There are many situations where it is not possible, or ethical, to do full experimental control. In other cases, controlling all variables results in an enviroment that has little in common with reality (e.g. the experience of going to a concert vs. hearing music alone in a sound-proof listening booth.)\n",
    "\n",
    "In other cases, a full-fledged experiment is simply overkill. In order to find out if your design and implementation is on the right track, user testing with a handful of participants is much more economical because you want to identify potential flaws, not measure an overall effect.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "Copy of DAE2_DesigningExperiments.ipynb",
   "provenance": [
    {
     "file_id": "https://github.com/sofiadahl/Design_and_Analysis_of_Experiments/blob/main/DAE2_DesigningExperiments.ipynb",
     "timestamp": 1620197191170
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}