analyze.results.example.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4dc74341-0f88-42df-8dc8-92a0aeb1096d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.stats import linregress\n",
    "import requests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66570816-b9cd-4b39-8145-0f49f0cfc15f",
   "metadata": {},
   "source": [
    "# Load and analyze bare benchmark results\n",
    "\n",
    "ASV web display is rich but does not do everything. When you need to run some analysis it does not provide, you can load the raw benchmark data, which are stored in the `results` folder of the `gh-pages` branch of the repository.\n",
    "\n",
    "Consider an example: let's check how the matrix-multiply routine `gemm` scales with the array size and compare to the expected cubic scaling, $\\sim N^3$.\n",
    "\n",
    "First note that the results are stored as json files in the `results/machine-name/` directory. We load the json for some commit hash:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e5fbd14c-a07f-41a9-8e56-85458e4c787e",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://raw.githubusercontent.com/OpenMathLib/BLAS-Benchmarks/gh-pages/results/gha-aws-graviton/fb8ab261-virtualenv-py3.12.json\"\n",
    "results = requests.get(url).json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c1fadbf-2c83-4a61-b32c-4dd1aaeff849",
   "metadata": {},
   "source": [
    "The format of the results file is documented in the [ASV developer docs](https://asv.readthedocs.io/en/stable/dev.html#benchmark-suite-layout-and-file-formats).\n",
    "\n",
    "In short, we are looking for a key `\"results\"` and a subkey with the benchmark name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "95629c08-4a0c-4a2e-bfe9-d0ecff0deb3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "gemm_results = results[\"results\"][\"benchmarks.gemm.time_gemm\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a1a58e9-33b6-40df-a5e8-2371d9cd7a52",
   "metadata": {},
   "source": [
    "Now, `gemm_results` is a list of lists and we are interested in the two first elements, which hold the benchmark results (in seconds, for `time_` benchmarks) and parameter lists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1ed8c749-2ec0-41e5-82fe-12c00965d5de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('100', \"'s'\") 3.094066903409531e-05\n",
      "('100', \"'d'\") 5.297513349516733e-05\n",
      "('100', \"'c'\") 0.0001136026793479741\n",
      "('100', \"'z'\") 0.00020522390999985873\n",
      "('200', \"'s'\") 0.000190337368421143\n",
      "('200', \"'d'\") 0.0003644822413794349\n",
      "('200', \"'c'\") 0.0007786495769218722\n",
      "('200', \"'z'\") 0.0015165394285706463\n",
      "('400', \"'s'\") 0.0013642758749998762\n",
      "('400', \"'d'\") 0.002745753000002793\n",
      "('400', \"'c'\") 0.0060967010000041455\n",
      "('400', \"'z'\") 0.012055932499990263\n",
      "('600', \"'s'\") 0.0046867961666663405\n",
      "('600', \"'d'\") 0.009184491749998358\n",
      "('600', \"'c'\") 0.02009252350001134\n",
      "('600', \"'z'\") 0.040977206500002694\n",
      "('800', \"'s'\") 0.01060090300001093\n",
      "('800', \"'d'\") 0.021474112999996464\n",
      "('800', \"'c'\") 0.0471736915000065\n",
      "('800', \"'z'\") 0.09596087750000493\n",
      "('1000', \"'s'\") 0.02110663850000094\n",
      "('1000', \"'d'\") 0.04267209399999672\n",
      "('1000', \"'c'\") 0.09408231599999795\n",
      "('1000', \"'z'\") 0.18838757800000394\n"
     ]
    }
   ],
   "source": [
    "import itertools\n",
    "\n",
    "# json format is documented \n",
    "# https://asv.readthedocs.io/en/stable/dev.html#benchmark-suite-layout-and-file-formats\n",
    "# In short, for multiple parameters the `results` list corresponds to an outer product of\n",
    "# parameters.\n",
    "#\n",
    "param_combs = itertools.product(*gemm_results[1])\n",
    "for res, parm in zip(gemm_results[0], param_combs):\n",
    "    print(parm, res)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf54c38c-292c-4ae7-ab6c-c5baa07948a5",
   "metadata": {},
   "source": [
    "Let's now quickly read in the double-precision, `'d'`, results vs the matrix size, run a linear regression in the logspace (since we expect a power-law scaling), and plot everything:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "024c977d-8980-4842-87ba-93acf7766338",
   "metadata": {},
   "outputs": [],
   "source": [
    "sizes, times = [], []\n",
    "\n",
    "param_combs = itertools.product(*gemm_results[1])\n",
    "for res, parm in zip(gemm_results[0], param_combs):\n",
    "    size, var = parm\n",
    "    if var == \"'s'\":\n",
    "        sizes.append(int(size))\n",
    "        times.append(res)\n",
    "\n",
    "times = np.asarray(times)\n",
    "sizes = np.asarray(sizes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "af3a368e-0193-484d-8727-22bf43aaae4f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinregressResult(slope=2.843086680878505, intercept=-23.556217088225388, rvalue=0.9996655473355168, pvalue=1.6776917146898127e-07, stderr=0.036774890861551744, intercept_stderr=0.22192218793752486)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_sizes = np.log(sizes)\n",
    "log_times = np.log(times)\n",
    "\n",
    "regr = linregress(log_sizes, log_times)\n",
    "regr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "17d44ba5-7ff0-407d-b9aa-d897899684ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f61adf5cd50>"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAG1CAYAAAAV2Js8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABMS0lEQVR4nO3dd3hUddrG8e/MpBHSCIGEQEgAaaEk1ChNQKSoKCIWRAFx0d2luQgr7q66ugpipWVFfVexLAoW0BVFEURAEBBCkSItdEio6aTMnPePaDRCIAMzOZPM/bmuXDpnzsx5hgTOnV+1GIZhICIiIuKFrGYXICIiImIWBSERERHxWgpCIiIi4rUUhERERMRrKQiJiIiI11IQEhEREa+lICQiIiJeS0FIREREvJaP2QV4OofDwdGjRwkODsZisZhdjoiIiJSDYRhkZWURHR2N1Vp2u4+C0CUcPXqUmJgYs8sQERGRy3Do0CHq1atX5vMKQpcQHBwMFP9BhoSEmFyNiIiIlEdmZiYxMTEl9/GyKAhdwi/dYSEhIQpCIiIilcylhrVosLSIiIh4LQUhERER8VrqGnMBh8NBQUGB2WWICXx9fbHZbGaXISIil0lB6AoVFBSQmpqKw+EwuxQxSVhYGFFRUVpeQUSkElIQugKGYXDs2DFsNhsxMTEXXadAqh7DMMjNzSU9PR2AOnXqmFyRiIg4S0HoChQVFZGbm0t0dDSBgYFmlyMmqFatGgDp6enUrl1b3WQiIpWMmjCugN1uB8DPz8/kSsRMv4TgwsJCkysRERFnKQi5gMaGeDd9/0VEKi91jYmIiEjFc9jhwGrIToOgSIjtBNaKH16gICQiIiIVa/unsPgRyDz667GQaOg7FeJvrtBS1DUmIiIiFWf7pzB/aOkQBJB5rPj49k8rtBwFIREREakYDntxSxDGBZ78+djiScXnVRAFITlP9+7deeihh8wuQ0REqpoDq89vCSrFgMwjxedVEAUhqbKmTJlChw4dCA4Opnbt2gwYMICffvrpoq+x2+089thjNGjQgGrVqtGoUSP+9a9/YRgX+u1FRESckp3m2vNcQEFIqqxvv/2WUaNG8f3337NkyRIKCwvp3bs3OTk5Zb5m6tSpvPLKK8yaNYsdO3YwdepUnnvuOWbOnFmBlYuIVFFBka49zwUUhNwgt6CozK9zhXaXn3s5PvzwQ1q1akW1atWoWbMmvXr1KjMg5OfnM3bsWGrXrk1AQABdunRh/fr1Jc93796d0aNHM3r0aEJDQ4mIiOCxxx4r1YricDiYMmVKSUtLQkICH3744WXVXl6LFy9m+PDhtGjRgoSEBObMmcPBgwfZsGFDma9ZvXo1t9xyCzfeeCNxcXEMGjSI3r17s27dOrfWKiLiFWI7Fc8Oo6z11ywQUrf4vAqi6fNlSE5OJjk5uWT1aGfEP/5lmc/1aFqLN+/rWPK43b++Jq/wwtdIahDOvAevKXncZeo3nM45f5f7/c/e6FR9x44dY/DgwTz33HPceuutZGVlsXLlyjK7f/7617/y0Ucf8dZbbxEbG8tzzz1Hnz592LNnD+Hh4QC89dZb3H///axbt44ffviBBx54gPr16zNy5EiguJvq3XffZfbs2TRu3JgVK1Zwzz33UKtWLa699toLXnfy5MlMnjz5op9l+/bt1K9fv1yfOyMjA6Ck5gvp1KkTr732Grt27aJJkyZs3ryZVatW8dJLL5XrGiIichFWG/SdijF/KAa/b435ORz1fbZC1xNSECrDqFGjGDVqFJmZmYSGhppdjksdO3aMoqIiBg4cSGxsLACtWrW64Lk5OTm88sorzJkzh379+gHw+uuvs2TJEv7zn/8wceJEAGJiYnj55ZexWCw0bdqUrVu38vLLLzNy5Ejy8/OZPHkyX3/9NddcUxzsGjZsyKpVq3j11VfLDEJ//OMfueOOOy76WaKjo8v1mR0OBw899BCdO3emZcuWZZ43adIkMjMzadasGTabDbvdzjPPPMOQIUPKdR0REbm4Nf6dWegzgXGF/yHacvrXJ0Kii0NQBa8jpCDkBtuf6lPmc9bfbcew4bFe5T531SM9rqywnyUkJHDdddfRqlUr+vTpQ+/evRk0aBA1atQ479y9e/dSWFhI586dS475+vrSsWNHduzYUXLs6quvLrXVxDXXXMOLL76I3W5nz5495Obmcv3115d674KCAtq0aVNmneHh4RdtvXHGqFGj+PHHH1m1atVFz5s/fz7//e9/mTt3Li1atGDTpk089NBDREdHM2zYMJfUIiLijewOgxlLdzNz2W4cRhs2Rvwf/+lRRH2/LK0sXdUE+pX/j9Vd516MzWZjyZIlrF69mq+++oqZM2fy97//nbVr19KgQQOXXOO3srOzAVi0aBF169Yt9Zy/v3+Zr3NV19jo0aP57LPPWLFiBfXq1bvouRMnTmTSpEncddddQHFL2YEDB5gyZYqCkIjIZTqecY5x76ewNrW4Bej2dvV48pYWLruvXQnzKxBTWCwWOnfuTOfOnXn88ceJjY1lwYIFjB8/vtR5jRo1ws/Pj++++66kG62wsJD169eXWmto7dq1pV73/fff07hxY2w2G/Hx8fj7+3Pw4MEyu8Eu5Eq7xgzDYMyYMSxYsIDly5eXK+Tl5uZitZbutbbZbDgcjvIVLSIipeTkF9F/1ipOZOVT3c/GM7e2YkCbupd+YQVREPJCa9euZenSpfTu3ZvatWuzdu1aTpw4QfPmzc87t3r16vzpT39i4sSJhIeHU79+fZ577jlyc3O5//77S847ePAg48eP58EHH2Tjxo3MnDmTF198EYDg4GAmTJjAX/7yFxwOB126dCEjI4PvvvuOkJCQMltarrRrbNSoUcydO5dPPvmE4OBgjh8/DkBoaCjVqlUDYNasWSxYsIClS5cC0L9/f5555hnq169PixYtSElJ4aWXXmLEiBGXXYeIiDer7u/DfZ3jWLTlGLPubkuDiOpml1SKgpAXCgkJYcWKFUybNo3MzExiY2N58cUXSwZD/96zzz6Lw+Hg3nvvJSsri/bt2/Pll1+WGlM0dOhQ8vLy6NixIzabjXHjxvHAAw+UPP+vf/2LWrVqMWXKFPbt20dYWBht27blb3/7m9s+5yuvvAIUT+//rTfffJPhw4cDcPLkSfbu3Vvy3MyZM3nsscf485//THp6OtHR0Tz44IM8/vjjbqtTRKSqOXQ6l/wiB1fVDgLgj90acX+XBvj7VPwYoEuxGFoy96J+mTWWkZFBSEhIqefOnTtHamoqDRo0ICAgwKQKzde9e3cSExOZNm2a2aWYQj8HIiK/WrTlGJM+2kJUaACfju5CNT9zws/F7t+/pRYhERERuWLnCu3867Pt/HftQQAaB/iQlV9oWhAqLwUhERERuSJ70rMYPTeFncezAPhz90b85fom+No8fwMLBSG5YsuXLze7BBERMYFhGHy44TCPf7KNvEI7EUF+vHRHIt2a1DK7tHJTEBIREZHLYhjw4YbD5BXa6XxVTV6+M5HawZVrrKSCkIiIiFwWq9XC9Lva8MmmI/yha0Ns1rI2U/Vcnt95JyIiIh7BMAzeWr2fZxZtLzkWFRrAg9c2qpQhCNQiJCIiIuWQkVvIXz/azJfb0gDo0yKK9nGu2Q/STApCIiIiclEbDpxh7HspHDmbh6/NwqP9mtMu9vyNuisjBSERERG5IIfD4NUV+3jhq5+wOwxiawYya3BbWtULNbs0l1EQEhERkQsaN28T/9t8FID+CdFMvrUlwQG+JlflWhosLUDxWkAWi4WzZ8+aXYqIiHiIG1tFEeBr5dmBrZhxV2KVC0GgFiHP4LDDgdWQnQZBkRDbCazuXZL89/uDderUiWPHjhEaWnWaO0VExDlFdgf7T+WWbJbat2UdVsTWqHRrAzlDQchs2z+FxY9A5tFfj4VEQ9+pEH9zhZXh5+dHVFRUhV1PREQ8y7GMPMa+l8LeEzl8Ma4rkSHF4acqhyBQ15i5tn8K84eWDkEAmceKj2//1C2XHT58ON9++y3Tp0/HYrFgsViYM2dOqa6xOXPmEBYWxmeffUbTpk0JDAxk0KBB5Obm8tZbbxEXF0eNGjUYO3Ysdru95L3z8/OZMGECdevWpXr16iQlJWkLDhERD/f19jT6TV/J+v1nKChysDst2+ySKoxahMzisBe3BGFc4EkDsMDiSdDsRpd3k02fPp1du3bRsmVLnnrqKQC2bdt23nm5ubnMmDGD999/n6ysLAYOHMitt95KWFgYn3/+Ofv27eO2226jc+fO3HnnnQCMHj2a7du38/777xMdHc2CBQvo27cvW7dupXHjxi79HCIicmXyi+xM/eIn3vguFYBWdUOZObgNcRHVTa6s4igImeXA6vNbgkoxIPNI8XkNurr00qGhofj5+REYGFjSHbZz587zzissLOSVV16hUaNGAAwaNIh33nmHtLQ0goKCiI+Pp0ePHnzzzTfceeedHDx4kDfffJODBw8SHR0NwIQJE1i8eDFvvvkmkydPdunnEBGRy3fgVA6j56aw9UgGACM6N+CRfk3x93HvGFVPoyBkluw0157nBoGBgSUhCCAyMpK4uDiCgoJKHUtPTwdg69at2O12mjRpUup98vPzqVmzZsUULSIi5fKfValsPZJBWKAvLwxKoFd8pNklmUJByCxB5fyBK+95buDrW3qapMViueAxh8MBQHZ2NjabjQ0bNmCzlf6N4rfhSUREzDepXzPyCuz85fomRIdVM7sc0ygImSW2U/HssMxjXHickKX4+dhObrm8n59fqUHOrtCmTRvsdjvp6el07era7jwREbkyu9KymLv2II/fFI/VaiHQz4fnb08wuyzTKQiZxWorniI/fyhgoXQY+nkH377Pum09obi4ONauXcv+/fsJCgoqadW5Ek2aNGHIkCEMHTqUF198kTZt2nDixAmWLl1K69atufHGG11QuYiIlOkC69IZFivz1h/in//bxrlCB/XDAxnRpYHZlXoMBSEzxd8Md7xdxjpCz7p1HaEJEyYwbNgw4uPjycvL480333TJ+7755ps8/fTTPPzwwxw5coSIiAiuvvpqbrrpJpe8v4iIlOEC69I5gqN5I/iPPL3vKgC6No6gf0K0WRV6JIthGBfql5GfZWZmEhoaSkZGBiEhIaWeO3fuHKmpqTRo0ICAgCtYcMqElaXFdVz2cyAicrl+WZfud0MtHBQfGlX0F1pffy8PdmuI1Woxo8IKd7H7929V+QUVDx06RPfu3YmPj6d169Z88MEHZpd0PquteIp8q0HF/1UIEhGR8rrIunRWAAtMD3ufP3WL85oQ5IwqH4R8fHyYNm0a27dv56uvvuKhhx4iJyfH7LJERERc4xLr0lkBv5xjxefJear8GKE6depQp04dAKKiooiIiOD06dNUr+49q2aKiEgVVgnWpfNkprcIrVixgv79+xMdHY3FYmHhwoXnnZOcnExcXBwBAQEkJSWxbt26y7rWhg0bsNvtxMTEXGHVIiIinsFevXb5TjRxXTpPZnoQysnJISEhgeTk5As+P2/ePMaPH88TTzzBxo0bSUhIoE+fPiWrGQMkJibSsmXL876OHv21qfD06dMMHTqU11577aL15Ofnk5mZWepLRETEE6VnnWPYUhtHjXAcZU59skBIXbetS1fZmd411q9fP/r161fm8y+99BIjR47kvvvuA2D27NksWrSIN954g0mTJgGwadOmi14jPz+fAQMGMGnSJDp1uvgPwpQpU3jyySed+gyaeOfd9P0XETOs2HWC8fM3cTK7gGd9hzPd9vLPW3ZX7Lp0lZ3pLUIXU1BQwIYNG+jVq1fJMavVSq9evVizZk253sMwDIYPH07Pnj259957L3n+o48+SkZGRsnXoUOHyjz3l20kCgoKylWLVE25ubnA+VuSiIi4Q6HdwdTFOxn6xjpOZhfQLCqYsWPGY7njbSwhdUqfHBJdvF6dG9elq+xMbxG6mJMnT2K324mMLN2vGRkZecHd0i/ku+++Y968ebRu3bpk/NE777xDq1atLni+v78//v7+5XpvHx8fAgMDOXHiBL6+vlitHp0rxcUMwyA3N5f09HTCwsLO219NRMQdFqYc4ZXlewEYklSfx26KJ8DXBrVvhmY3al06J3l0EHKFLl26uGT7iAuxWCzUqVOH1NRUDhw44JZriOcLCwsjKirK7DJExEvc1rYey3ed4IaWdbix9e9agH5Zl07KzaODUEREBDabjbS00lP+0tLSPObG4+fnR+PGjdU95qV8fX3VEiQibpVfZOf1Ffu4v0tDqvnZsFotJN/d1uyyqgyPDkJ+fn60a9eOpUuXMmDAAAAcDgdLly5l9OjR5hb3G1arVVsriIiIy6WezGH03I1sO5rJ4TN5PHtba7NLqnJMD0LZ2dns2bOn5HFqaiqbNm0iPDyc+vXrM378eIYNG0b79u3p2LEj06ZNIycnp2QWmbskJyeTnJyM3W5363VEREQuZGHKEf6+YCs5BXZqBPrSu4XWAXIH0zddXb58OT169Djv+LBhw5gzZw4As2bN4vnnn+f48eMkJiYyY8YMkpKSKqS+8m7aJiIi4gq5BUU88ck2PthwGICkBuFMv6sNUaHqeXBGee/fpgchT6cgJCIiFWVPejZ/fHcDe9KzsVpgTM/GjL2uMTZtluq08t6/Te8aExERkWJB/j6czikgMsSfaXe24ZpGNc0uqcpTEBIRETFRfpEdf5/i2adRoQG8PrQ9cTUDqRlUvjXt5MpoBUARERGTbDp0lutfWsHiH4+XHGsXW0MhqAIpCJUhOTmZ+Ph4OnToYHYpIiJSxTgcBq+v2MegV1Zz8HQus77ZjaPsXVPFjTRY+hI0WFpERFzpdE4BD8/fxDc/nQDghlZRTBnYmtBq2q/QlTRYWkRExMN8v+8U495PIS0zHz8fK4/fFM+QpPpYLJoVZhYFIRERkQqw70Q2d7/+PQ4DGtWqzqy729K8jnoazKYgJCIiUgEa1grinqtjyS2w89QtLQj00y3YE+i7ICIi4ibLf0qneZ0QIkOKV4V+on8LLY7oYTRrrAyaNSYiIperoMjB5M93MPzN9Tz0/ibsP88IUwjyPJo1dgmaNSYiIs44dDqX0e+lsPnQWQCGXRPL32+Mx89HbQ8VSbPGREREKtjnW4/xyEdbyDpXREiAD88NSqBvyyizy5KLUBASERG5QucK7Ty9aDvvfn8QgLb1w5gxuA31agSaXJlcioKQiIjIFXIYBt/vOw3An7o3Yvz1TfC1qSusMlAQEhERuUyGYWCxWAj08yH57rYczzzHtU1qmV2WOEFxVURExEk5+UWMn7eJV1fsKznWNCpYIagSUouQiIiIE7YdzWDM3BT2nczB38fKwLZ1qR0cYHZZcpkUhMqQnJxMcnIydrvd7FJERMQDGIbBO98f4OlFOygoclAnNIAZg9soBFVyWkfoErSOkIiIZOQW8shHW1i87TgA1zWrzQu3J1Cjup/JlUlZtI6QiIiICxQUObj139+x72QOvjYLk/o1Z0TnOO0YX0VosLSIiMhF+PlYuefqWOqHB/LRnzpxf5cGCkFViFqEREREfudkdj5ncwu4qnYwAPd1juPODjFU99dts6pRi5CIiMhvrN5zkhumr+QPb/1Adn4RABaLRSGoilIQEhERAYrsDl766ieG/Gct6Vn5+NqsnM4uMLsscTPFWxER8XrHMvIY9/4m1qUWb5NxZ/sY/nlzC6r52UyuTNxNQUhERLyDww4HVkN2GgRFQmwnsNpYuiONCR9s5kxuIdX9bEwe2IpbEuuaXa1UEAUhERGp+rZ/Cosfgcyjvx4Licbo+yxvf1+HM7mFtKwbwqzBbYmLqG5enVLhFITKoJWlRUSqiO2fwvyhwO/WD848hmX+MGbe/B9eq9uSMdddhb+PusK8jVaWvgStLC0iUok57DCtZemWoFIsEBIND20Fq0JQVVLe+7dmjYmISNV1YPVFQhCAAZlHis8Tr6QgJCIiVVd2mmvPkypHQUhERKosI6h2+U4MinRvIeKxFIRERKTKempLGEeNcBxljoa1QEjd4qn04pUUhEREpMrq1iyKp4uGYbGAwe83Sv35cd9nNVDaiykIiYhIlWEYBgdO5ZQ87tG0Nn+f+AiWO97BElKn9Mkh0XDH2xB/cwVXKZ5E6wiJiEiVcDa3gIkfbmHtvlN8Pq4r9WoEAlA3rBqE3QzNbrzgytLi3RSERESk0vth/2nGvpfC0Yxz+NmsbD2cURKESlht0KCrOQWKx1IQEhGRSsvhMHjl2728tGQXdodBXM1AZt3dlpZ1Q80uTSoJBaEyaIsNERHPdiIrn/HzN7Fy90kABiRG8/StrQjy161Nyk9bbFyCttgQEfFMz36xk9nf7qWar40nb2nB7e3qYbH8fmaYeKvy3r8Vm0VEpFJ6qFdjjmXkMbrHVTSODDa7HKmkNH1eREQqhSNn8/jXZ9ux/7w6YoCvjel3tVEIkiuiFiEREfF4X207zsQPt5CRV0iNQF9G92xsdklSRSgIiYiIx8ovsjPl853MWb0fgIR6odycUNfcoqRKURASERGPlHoyhzHvbeTHI5kAjOzagIl9muHno1Ed4joKQiIi4nGW7khj7Hsp5BTYqRHoy4t3JNCzmXaIF9dTEBIREY9Tr0YgRQ6Djg3CmX5XInVCq5ldklRRCkIiIuIRMvIKCa3mC0DTqGA++OM1xNcJwcemrjBxH/10iYiIqQzD4L11B+n87DI2HjxTcrx1vTCFIHE7tQiJiIhpMs8V8rePt/LZlmMAzF9/iLb1a5hclXgTBSERETHF5kNnGfNeCgdP5+JjtTCxT1NGdm1odlniZRSERESkQhmGwX9WpTJ18U4K7Qb1alRjxuA2agkSUygIiYhIhfpyWxpPL9oBQL+WUTx7W+uSQdIiFU1BqAzJyckkJydjt9vNLkVEpErp0yKSG1vX4eqGNbknqb52jBdTWQzDMMwuwpNlZmYSGhpKRkYGISEhZpcjIlLp2B0Gb6/Zz+3tYwjyL/792zAMBSBxq/Lev9UiJCIibpOeeY6H5m1i9d5TbDp0lml3JmKxWBSCxGMoCImIiFss/ymdh+dv5lROAYF+Nro1rqUAJB5HQUhERFyq0O7gha9+4tVv9wHQvE4Is+5uQ6NaQSZXJnI+BSEREXGZo2fzGDV3IykHzwIw9JpY/nZDcwJ8beYWJlIGBSEREXEZPx8rh8/kERLgw3ODWtO3ZR2zSxK5KAUhERG5IoV2B74/7wkWEeTPq/e2o1aQPzHhgSZXJnJp2s1OREQu294T2dw86zs+2XSk5Fjb+jUUgqTSUBASEZHL8vHGw/SfuYodxzJ5ackuCu0Os0sScZq6xkRExCk5+UU8/sk2Ptp4GIBrGtZk2l2JJd1jIpWJgpCIiJTbjmOZjJq7kX0ncrBa4KFeTRjV4ypsVq0PJJWTgpCIiJTL8YxzDEj+jvwiB5Eh/sy4qw1JDWuaXZbIFVEQEhGRcokKDWDoNbHsPZHDC7cnEF7dz+ySRK6Y00EoNTWVoqIiGjduXOr47t278fX1JS4uzlW1iYiIyVIOnqFWsD/1ahTPAnukbzOsFgtWdYVJFeH0yLbhw4ezevXq846vXbuW4cOHu6ImERExmcNh8Oq3e7l99hrGvpdSMiPMx2ZVCJIqxekglJKSQufOnc87fvXVV7Np0yZX1CQiIiY6lZ3PiLfWM+WLnRQ5DOqEVaOgSFPjpWpyumvMYrGQlZV13vGMjAzsdrtLihIREXOs2XuKce+nkJ6Vj7+PlSf6t2BwxxjtGi9VltMtQt26dWPKlCmlQo/dbmfKlCl06dLFpcWJiEjFsDsMXlqyi7v/73vSs/K5qnYQn4zuzN1J9RWCpEpzukVo6tSpdOvWjaZNm9K1a1cAVq5cSWZmJsuWLXN5gSIi4n6Fdgdfb0/DMOCO9vX4580tCPTTxGKp+iyGYRjOvujo0aPMmjWLzZs3U61aNVq3bs3o0aMJDw93R42mSE5OJjk5Gbvdzq5du8jIyCAkJMTsskREXMowjJIWn30nstl6JINbEuuaXJXIlcvMzCQ0NPSS9+/LCkLepLx/kCIilUlBkYPnv9xJkL8v43o1vvQLRCqZ8t6/L2tjmJUrV3LPPffQqVMnjhwp3nH4nXfeYdWqVZdXrYiIVJiDp3K5ffZqXl+ZyvSluzhwKsfskkRM43QQ+uijj+jTpw/VqlVj48aN5OfnA8WzxiZPnuzyAkVExHU+23KUG2esZPPhDEKr+TL7nnbE1qxudlkipnE6CD399NPMnj2b119/HV9f35LjnTt3ZuPGjS4tTkREXONcoZ2/LdjK6LkpZOUX0T62Bp+P60rvFlFmlyZiKqenBPz0009069btvOOhoaGcPXvWFTWJiIgLORwGd772PZsPncVigT93b8RfejXBx3ZZoyNEqhSng1BUVBR79uw5b0+xVatW0bBhQ1fVJSIiLmK1WrizfQxHzuTx8p0JdG1cy+ySRDyG00Fo5MiRjBs3jjfeeAOLxcLRo0dZs2YNEyZM4LHHHnNHjSIi4qTs/CKOZ+RxVe1gAAZ3jOHGVnUIDfS9xCtFvIvTQWjSpEk4HA6uu+46cnNz6datG/7+/kyYMIExY8a4o0YREXHCj0cyGPNeCgVFDj4f25XQQF8sFotCkMgFXPY6QgUFBezZs4fs7Gzi4+MJCgpydW0eQesIiUhlYRgGb63ez+TPd1JgdxAdGsAb93WgWZT+7RLvU97792Wvn+7n50d8fDyZmZl8/fXXNG3alObNm1/u24mIyBU4m1vAxA+3sGR7GgC9mkfywu2tCQv0M7kyEc/mdBC644476NatG6NHjyYvL48OHTqQmpqKYRi8//773Hbbbe6oU0REyrDhwGnGzE3haMY5/GxWHr2hGcM7xWmzVJFycHru5IoVK0o2W12wYAEOh4OzZ88yY8YMnn76aZcXKCIiF/d/K1M5mnGOuJqBfPznTtzXuYFCkEg5OR2EMjIySjZXXbx4MbfddhuBgYHceOON7N692+UFiojIxU0Z2IrhneL4bGxXWtYNNbsckUrF6SAUExPDmjVryMnJYfHixfTu3RuAM2fOEBAQ4PICRUSktFW7T/Lk/7bxy1yXsEA//nlzC4L8L3vYp4jXcvpvzUMPPcSQIUMICgoiNjaW7t27A8VdZq1atXJ1fSIi8rMiu4PpS3ayfsUianOWNdU60Klnf7DazC5NpNK6rOnzGzZs4ODBg1x//fUl0+YXLVpEWFgYnTt3dnmRZtL0eRHxBEfP5vHfObMYcubfRFtO//pESDT0nQrxN5tXnIgHKu/9+7LXEfIWCkIiYrYl29P44oPXeMHxAgDWUuOgf35wx9sKQyK/Ud77t3bcExHxYDOW7ubBt9cxwfEGFsvvQxDAz7/LLp4EDntFlydS6SkIiYh4sHaxNUiy7iTacpqyJ8QbkHkEDqyuwMpEqgYFIRERD3P0bF7J/3e+KoLpN9Up3wuz09xUkUjVpSAkIuIh8grsPPLhFnq/vIL9J3NKjteuE1u+NwiKdFNlIlWXS4PQwYMHsdvVRy0i4qxdaVncPGsV8344RE5BEWv2nfr1ydhOxbPDyuwcs0BI3eLzRMQpLg1CcXFxxMfH8/HHH7vybUVEqizDMHhv3UFunrWK3enZ1Ar257/3JzG4Y/1fT7LaiqfIA+eHoZ8f931W6wmJXAaXBqFvvvmGSZMmMW/ePFe+rYhIlZR1rpCx72/i0Y+3cq7QQbcmtfhiXFc6XRVx/snxNxdPkQ/53XihkGhNnRe5AlpH6BK0jpCIuEvyN3t4/sufsFktTOzTlAe6NsR6/vz40hz24tlh2WnFY4JiO6klSOQCynv/1sY0IiImGdm1IduPZjKiSwPaxdYo34usNmjQ1b2FiXgRp7vGTp06xahRo4iPjyciIoLw8PBSXyIicmFncgqYungnhXYHAH4+VpKHtC1/CBIRl3O6Rejee+9lz5493H///URGRmKxXKIZV0REWJd6mrHvpXA88xxWC0zs08zskkSEywhCK1euZNWqVSQkJLijHhGRKsXuMPj3N3t4+etdOAxoGFGdG1qVc4FEEXE7p4NQs2bNyMvLu/SJHuLs2bP06tWLoqIiioqKGDduHCNHjjS7LBHxAumZ53ho3iZW7y1eE2hgm7r8a0BLqvtreKaIp3B61tj69euZNGkSjz/+OC1btsTX17fU8542s8put5Ofn09gYCA5OTm0bNmSH374gZo1a5br9Zo1JiKXY13qaf707gZO5RQQ6GfjX7e05LZ29cwuS8RruG3WWFhYGJmZmfTs2bPUccMwsFgsHreytM1mIzAwEID8/HwMw0ArBoiIu0UE+ZFXaKdZVDCz7m7LVbWDzC5JRC7A6VljQ4YMwdfXl7lz57J06VKWLVvGsmXL+Oabb1i2bJnTBaxYsYL+/fsTHR2NxWJh4cKF552TnJxMXFwcAQEBJCUlsW7dOqeucfbsWRISEqhXrx4TJ04kIuICi5WJiFyhnPyikv9vWCuId/+QxMJRnRWCRDyY0y1CP/74IykpKTRt2tQlBeTk5JCQkMCIESMYOHDgec/PmzeP8ePHM3v2bJKSkpg2bRp9+vThp59+onbt2gAkJiZSVFR03mu/+uoroqOjCQsLY/PmzaSlpTFw4EAGDRpEZOSFNyfMz88nPz+/5HFmZqZLPqeIVG2LfzzOox9v4d9D2nFNo+Ku97b1NS1exNM5HYTat2/PoUOHXBaE+vXrR79+/cp8/qWXXmLkyJHcd999AMyePZtFixbxxhtvMGnSJAA2bdpUrmtFRkaSkJDAypUrGTRo0AXPmTJlCk8++aRzH0JEvNa5QjtTPt/BW2sOADBndWpJEBIRz+d019iYMWMYN24cc+bMYcOGDWzZsqXUlysVFBSwYcMGevXq9WvBViu9evVizZo15XqPtLQ0srKyAMjIyGDFihUXDXGPPvooGRkZJV+HDh26sg8hIlXWvhPZDPz36pIQ9GC3hsy6u63JVYmIM5xuEbrzzjsBGDFiRMkxi8XilsHSJ0+exG63n9eNFRkZyc6dO8v1HgcOHOCBBx4oGSQ9ZswYWrVqVeb5/v7++Pv7X1HdIlL1LUg5zN8X/EhugZ3w6n68eEcCPZrWNrssEXGS00EoNTXVHXW4TceOHcvddSYiUh6r957kL/M2A3B1w3Cm39WGyJAAk6sSkcvhdBCKjY11Rx0XFBERgc1mIy0trdTxtLQ0oqKiKqwOEZHfuqZhTW5JjKZBRHXG9GyM7VI7xouIx7qs5U2PHj3KqlWrSE9Px+FwlHpu7NixLikMwM/Pj3bt2rF06VIGDBgAgMPhYOnSpYwePdpl1xERuRjDMPhww2F6x0cRGuiLxWJh2p2J2mtRpApwOgjNmTOHBx98ED8/P2rWrFnqHwKLxeJ0EMrOzmbPnj0lj1NTU9m0aRPh4eHUr1+f8ePHM2zYMNq3b0/Hjh2ZNm0aOTk5JbPI3CU5OZnk5GSPWyBSRCpWRl4hf/t4K4u2HmNpi3ReuactFotFIUikinB6i42YmBj++Mc/8uijj2K1Oj3p7DzLly+nR48e5x0fNmwYc+bMAWDWrFk8//zzHD9+nMTERGbMmEFSUtIVX7s8tMWGiPfadOgso+du5PCZPHysFh7p24w/dG2gECRSCZT3/u10EKpZsybr1q2jUaNGV1xkZaAgJOJ9HA6D/6xKZerinRQ5DGLCqzFzcFsSY8LMLk1Eyqm892+nm3Tuv/9+PvjggysqTkTEU53JKeD+t9bzzOc7KHIY3NiqDp+N6aoQJFJFOT1GaMqUKdx0000sXryYVq1anbf7/EsvveSy4kREKprFAj8dz8Lfx8rj/eO5u2N9dYWJVGGXFYS+/PLLktWZfz9YWkSksnE4DCyW4n/DwgL9SB7Slmp+NppFqTtcpKpzOgi9+OKLvPHGGwwfPtwN5XgOzRoT8Q5pmecY934KA9vU444OMQC00WapIl7D6cHSUVFRrFy5ksaNG7urJo+iwdIiVdc3P6Xz8PzNnM4pICLIj5V/7Uk1P5vZZYmIC7htsPS4ceOYOXPmFRUnImKmgiIHkz/fwX1vrud0TgHxdUKY/+A1CkEiXsjprrF169axbNkyPvvsM1q0aHHeYOmPP/7YZcWJiLjaodO5jH4vhc2HzgIw7JpYHr2hOQG+CkEi3sjpIBQWFsbAgQPdUYuIiFtl5BZy08xVZOQVEhLgw3ODEujbUvsWingzp4PQm2++6Y46RETcLjTQl2HXxLJqz0lmDG5DvRqBZpckIiZzerA0QFFREcuXL2fv3r3cfffdBAcHc/ToUUJCQggKCnJHnabRYGmRym1Peja+NguxNasDUGR3YAC+tivfIkhEPFd5799OtwgdOHCAvn37cvDgQfLz87n++usJDg5m6tSp5OfnM3v27Csq3FNo+rxI5fbLjvGPf7KNRrWr89GfOuHvY8NHAUhEfuOyZo21b9+eM2fOUK1atZLjt956K0uXLnVpcWYaNWoU27dvZ/369WaXIiJOys4vYvz8zUz8cAt5hXZCq/mSm69fakTkfE63CK1cuZLVq1fj5+dX6nhcXBxHjhxxWWEiIpdj29EMRs9NIfVkDlYLjL++CX/qfhU2q1a+F5HzOR2EHA7HBbuLDh8+THBwsEuKEhFxlmEYvL3mAM8s2kGB3UGd0ABmDG5Dh7hws0sTEQ/mdNdY7969mTZtWslji8VCdnY2TzzxBDfccIMraxMRKbcih8HHKUcosDvo1TySz8d2VQgSkUtyetbY4cOH6dOnD4ZhsHv3btq3b8/u3buJiIhgxYoV1K5d2121mkKzxkQqj0Onc1m6I41hneK0CbSIlyvv/fuyp8+///77bNmyhezsbNq2bcuQIUNKDZ6uKhSERDyTw2Hw6op95OQXMaFPU7PLEREP47bp8wA+Pj7cc889l11cZaDp8yKe62R2PuPnb2bFrhMA9GsVRYvoUJOrEpHKyOkWoU8//fTCb2SxEBAQwFVXXUWDBg1cUpwnUIuQiGdZveck4+Zt4kRWPgG+Vp68uQV3tI9RV5iIlOK2FqEBAwZgsVj4fX765ZjFYqFLly4sXLiQGjVqOF+5iMgFFNkdTF+6m1nf7MEwoElkELPubkuTSM1WFZHL5/SssSVLltChQweWLFlCRkYGGRkZLFmyhKSkJD777DNWrFjBqVOnmDBhgjvqFREvZBgGI9/+gZnLikPQ4I4xfDKqi0KQiFwxp1uExo0bx2uvvUanTp1Kjl133XUEBATwwAMPsG3bNqZNm8aIESNcWqiIeC+LxUL/hGjW7z/D5IGtuDkh2uySRKSKcDoI7d2794J9bSEhIezbtw+Axo0bc/LkySuvTkS8VkGRg0NncmlUq3gj54Ft69G1cS1qBfubXJmIVCVOd421a9eOiRMncuLEiZJjJ06c4K9//SsdOnQAYPfu3cTExLiuShHxKgdO5TBo9mrufv17TmXnlxxXCBIRV3O6Reg///kPt9xyC/Xq1SsJO4cOHaJhw4Z88sknAGRnZ/OPf/zDtZWKiFf43+ajPPrxVrLziwgL9CX1ZA41gxSARMQ9LmtBRYfDwVdffcWuXbsAaNq0Kddffz1Wq9MNTB5P0+dFKkZegZ2nPtvGe+sOAdAhrgbT72pDdFjVW6hVRNzPrStLexMFIRH325WWxei5G9mVlo3FAqN7XMW46xrjY6t6v1yJSMVw68rS3kArS4tUnNnf7mVXWjYRQf5MvyuRzldFmF2SiHgJtQhdglqERNwv81whz3y2gwl9mmpAtIi4RHnv32p3FpEKt/VwBk/9b3vJCvUhAb5MHdRaIUhEKpy6xkSkwhiGwZvf7WfKFzsotBs0jQrizg71zS5LRLzYZbUI7d27l3/84x8MHjyY9PR0AL744gu2bdvm0uJEpOo4k1PAyLc38NRn2ym0G/SOj6RPiyizyxIRL+d0EPr2229p1aoVa9eu5eOPPyY7OxuAzZs388QTT7i8QBGp/NbvP80NM1by9Y40/GzFO8a/em87wgL9zC5NRLyc00Fo0qRJPP300yxZsgQ/v1//EevZsyfff/+9S4sTkcrvne8PcNdr33Ms4xwNIqrz8Z87MaxTHBaLxezSREScHyO0detW5s6de97x2rVra38xETlP86jiHeJvbVOXfw1oSZC/hiaKiOdw+l+ksLAwjh07RoMGDUodT0lJoW7dui4rTEQqrxNZ+SUzwNrHhfP52K40iQxSK5CIeBynu8buuusuHnnkEY4fP47FYsHhcPDdd98xYcIEhg4d6o4aRaSSKLQ7mLp4J9c+/w2707JKjjeNClYIEhGP5HQQmjx5Ms2aNSMmJobs7Gzi4+Pp1q0bnTp10karIl7s8Jlc7nrte15ZvpfcAjtLdqSZXZKIyCVd9srSBw8e5McffyQ7O5s2bdrQuHFjV9dmqt9usbFr1y6tLC1yEV9uO87EDzaTea6IYH8fnr2tNTe2rmN2WSLixbTpqotoiw2Rsp0rtDPl8x28teYAAAkxYcwa3IaY8ECTKxMRb+e2TVcNw+DDDz/km2++IT09HYfDUer5jz/+2PlqRaRSmv/DoZIQ9EC3hkzo3RQ/H+3cIyKVh9NB6KGHHuLVV1+lR48eREZGagCkiBe7u2N9Vu85xZ0dYujRrLbZ5YiIOM3prrHw8HDeffddbrjhBnfV5FHUNSbyq9yCIl5fkcofuzfE38dmdjkiImVyW9dYaGgoDRs2vKLiRKTy2Xk8k9FzU9iTns3ZvAKe6N/C7JJERK6Y0535//znP3nyySfJy8tzRz0i4mEMw+C/aw9wy6zv2JOeTWSIP73jtVmqiFQNTrcI3XHHHbz33nvUrl2buLg4fH19Sz2/ceNGlxUnIubKPFfIox9tZdHWYwB0b1qLF29PoGaQv8mViYi4htNBaNiwYWzYsIF77rlHg6VFqrDtRzN58N0fOHQ6Dx+rhb/2bcofujTEatXfeRGpOpwOQosWLeLLL7+kS5cu7qhHRDxEcIAPZ3MLqVejGjMHt6FN/RpmlyQi4nJOB6GYmBjNnhKpos4V2gnwLZ4NFhMeyJvDO9A4MpjQar6XeKWISOXk9GDpF198kb/+9a/s37/fDeWIiFnW7jtFjxeW8+2uEyXH2seFKwSJSJXmdIvQPffcQ25uLo0aNSIwMPC8wdKnT592WXEi4n52h8GsZXuYvnQXDgOSl+2hW+MIjf8TEa/gdBCaNm2aG8oQETOkZZ7jofc3sWbfKQBua1uPp25poRAkIl7jsmaNiUjlt/yndB6ev5lTOQUE+tl4ekBLBratZ3ZZIiIVqlxBKDMzs2SAdGZm5kXPrSoDqZOTk0lOTsZut5tdiojL/Xgkg+FvrgegeZ0QZt3dhka1gkyuSkSk4pVrrzGbzcaxY8eoXbs2Vqv1gs3mhmFgsViqXHDQXmNSVU34YDPV/Ww8ekPzkpliIiJVhUv3Glu2bBnh4eEAfPPNN66pUEQq1JfbjtM+tkbJqtDP3dZaiyOKiNcrVxC69tprS/6/QYMGxMTEnNcqZBgGhw4dcm11InLFzhXaeXrRdt79/iDdm9bijWEdsFotCkEiIlzGYOkGDRqUdJP91unTp2nQoEGV6xoTqcz2nshm9NwUdhwrHtvXLCoEh2FgRSFIRAQuIwj9Mhbo97KzswkICHBJUSJy5T7ccJjHP/mR3AI7Nav78dKdiVzbpJbZZYmIeJRyB6Hx48cDYLFYeOyxxwgMDCx5zm63s3btWhITE11eoIg4Jye/iMc++ZGPNx4BoFOjmky7M5HaIfpFRUTk98odhFJSUoDiFqGtW7fi5+dX8pyfnx8JCQlMmDDB9RWKiFMchsEP+89gtcBfejXhzz2uwqbxQCIiF1Su6fO/dd999zF9+nSvmUqu6fNSGfzy1/iXbuuthzPIK7TTsUG4mWWJiJimvPdvp4OQt1EQEk+XkVfIIx9uodNVNRl6TZzZ5YiIeASXriMkIp5p48EzjJmbwpGzeazac5JbEuoSGqjd4kVEyktBSKQScjgMXlu5jxe+/Ikih0H98EBm3d1GIUhExEkKQiKVzKnsfMbP38y3u04AcFPrOkwe2IqQAIUgERFnKQiJVCJ5BXb6z1zF0Yxz+PtY+efNLbirw/krvYuISPkoCIlUItX8bAy5OpYFKUeYdXcbmkVpAL+IyJXQrLFL0KwxMduxjDzyCuw0rBUEFI8Pyi9yUM1PO8aLiJSlvPdvawXWJCJOWrYzjRumr+RP727kXGHxPn5Wq0UhSETERdQ1JuKBCoocPLd4J/+3KhWAujWqcTa3kKhQBSAREVdSEBLxMAdPZPHqO++QdfIIV1vDaJ7Uh0k3tsDfRyFIRMTVFIREPMgPX8yh3vdP8YzlFPyynd/eN2DXVIi/2dTaRESqIo0RKkNycjLx8fF06NDB7FLESzi2fUK7teOozanST2Qeg/lDYfun5hQmIlKFadbYJWjWmFQIhx2mtcTIPMqFVwSyQEg0PLQVrOoiExG5FM0aE/FwhmEw/4dDTF28Ew6shjJDEIABmUeKzxMREZfRGCERE2TnF/H3BVv5ZNNRAG7zPcFV5XphmlvrEhHxNgpCIhXsxyMZjJ67kf2ncrFZLTzcuwkN6wfDynK8OCjS7fWJiHgTBSGRCmIYBnNW72fK5zspsDuIDg1gxuA2tI8LB0eD4jFAmceACw3b+3mMUGynii5bRKRK0xghkQoy4YMtPPm/7RTYHVwfH8nn47oWhyAoHgDdd+rPZ/5+pNDPj/s+q4HSIiIupiAkUkGua14bP5uVf/aP57V72xEW6Ff6hPib4Y63IaRO6eMh0cXHtY6QiIjLafr8JWj6vFwuh8Pg4Olc4iKqlxw7ejaP6LBql3ihvXh2WHZa8Zig2E5qCRIRcVJ5798aIyTiBiey8hk/fxPbj2by+biuRIYEAFw6BEFx6GnQ1c0ViogIKAiJuNzK3Sf4y7zNnMzOJ8DXyvZjmSVBSEREPIuCkIiLFNkdvLRkF698uxfDgKaRwSQPacNVtYPNLk1ERMqgICTiAkfO5jH2vRQ2HDgDwN1J9Xn8pngCfDW2R0TEkykIibjAq9/uZcOBMwT7+zDltlbc1Dra7JJERKQcFIREXGBSv2Zk5BXy8PVNqV8z0OxyRESknLSOkMhlSD2ZwzOLtuNwFK8+Eejnw/S72igEiYhUMmoREnHSJ5uO8LePt5JTYKdOaDVGdGlgdkkiInKZFIREyim3oIh/frqN+T8cBqBjg3D6tYoyuSoREbkSCkIi5fDT8SxGz93I7vRsLBYY07MxY3tehY9NvcsiIpWZgpDIJXy25SgPz99MfpGD2sH+TLsrkU6NIswuS0REXEBBSOQS6ocH4jAMrm1SixfvSCAiyN/skkRExEUUhEQuICO3kNBAXwBa1wvjoz91omV0KFarxeTKRETElTTAQeQ3DMPg/1buo/PUZWw7mlFyvHW9MIUgEZEqSC1CIj87k1PAhA82s3RnOgALNh6hRXSoyVWJiIg7KQiJAOtSTzP2vRSOZ57Dz8fKYzc2556rY80uS0RE3ExBSLya3WGQ/M0epn29C4cBDSOqM/PuNmoJEhHxEgpC4tU+3XyEl5bsAmBg27r865aWVPfXXwsREW+hf/HFq92SUJfFPx6nd3wUt7WrZ3Y5IiJSwTRrTLxKod3B6yv2ca7QDoDVamH2Pe0UgkREvJRahMRrHDqdy9j3U0g5eJZ9J3OYMrAVABaLpsWLiHgrr2kRys3NJTY2lgkTJphdiphg8Y/HuHHGSlIOniU4wIdujbVFhoiIeFGL0DPPPMPVV19tdhlSwc4V2pn8+Q7eXnMAgMSYMGYObkNMeKDJlYmIiCfwiiC0e/dudu7cSf/+/fnxxx/NLkcqyIFTOfzx3Y3sOJYJwIPXNmRC76b4asd4ERH5mel3hBUrVtC/f3+io6OxWCwsXLjwvHOSk5OJi4sjICCApKQk1q1b59Q1JkyYwJQpU1xUsVQWfj5WjmfkUbO6H3Pu68Cj/ZorBImISCmmtwjl5OSQkJDAiBEjGDhw4HnPz5s3j/HjxzN79mySkpKYNm0affr04aeffqJ27doAJCYmUlRUdN5rv/rqK9avX0+TJk1o0qQJq1evdvvnEXMV2h0lYadOaDVeG9qe+uGBRIYEmFyZiIh4IothGIbZRfzCYrGwYMECBgwYUHIsKSmJDh06MGvWLAAcDgcxMTGMGTOGSZMmXfI9H330Ud59911sNhvZ2dkUFhby8MMP8/jjj1/w/Pz8fPLz80seZ2ZmEhMTQ0ZGBiEhIVf2AcWtdhzLZPTcjfy1bzP6tIgyuxwRETFRZmYmoaGhl7x/e3Q/QUFBARs2bKBXr14lx6xWK7169WLNmjXleo8pU6Zw6NAh9u/fzwsvvMDIkSPLDEG/nB8aGlryFRMTc8WfQ9zLMAze+f4AtyR/x94TOby8ZBcOh8fkexER8WAeHYROnjyJ3W4nMjKy1PHIyEiOHz/ulms++uijZGRklHwdOnTILdcR18jIK+TP/93IYwt/pKDIQc9mtZk78mqsVq0NJCIil2b6GKGKNHz48Eue4+/vj7+/v/uLkSuWcvAMY95L4fCZPHxtFh7p24wRnRsoBImISLl5dBCKiIjAZrORlpZW6nhaWhpRURoD4s32n8zh9tlrKHIYxIRXY9bgtiTEhJldloiIVDIe3TXm5+dHu3btWLp0ackxh8PB0qVLueaaa0ysTMwWF1GdOzvEcGPrOiwa21UhSERELovpLULZ2dns2bOn5HFqaiqbNm0iPDyc+vXrM378eIYNG0b79u3p2LEj06ZNIycnh/vuu8+tdSUnJ5OcnIzdbnfrdaT81uw9RcNa1Uumwj95cwtsVov2ChMRkctm+vT55cuX06NHj/OODxs2jDlz5gAwa9Ysnn/+eY4fP05iYiIzZswgKSmpQuor7/Q7cR+7w2D60t3MXLabqxvU5N0/JGHTOCAREbmI8t6/TQ9Cnk5ByFzHM84x7v0U1qaeBuCO9vV46paWBPjaTK5MREQ8WXnv36Z3jYmU5Zud6Tz8wWZO5xRQ3c/G5IGtuCWxrtlliYhIFaIgJB6noMjB81/u5PWVqQC0iA5h1t1taRBR3eTKRESkqlEQEo9T5HCwbGc6AMM7xfHoDc3w91FXmIiIuJ6CUBk0a6ziGYaBxWIh0M+HWXe35eDpXO0ZJiIibqXB0pegwdLud67QzlOfbSeuZiAPdGtkdjkiIlIFaLC0VAp70rMYPTeFncez8LNZuSWxbsk6QSIiIu6mICSmMAyDDzYc5olPtpFXaCciyI+X70xUCBIRkQqlICQVLju/iH8s2MrCTUcB6HxVTV6+M5HawQpBIiJSsRSEpEIV2h3c9u/V/JSWhc1qYfz1TfjjtY20UrSIiJjCozddlarH12bljg4xRIcG8P4DVzOqx1UKQSIiYhq1CJVB0+dd52xuAadyCmhUKwiAEZ3juL19PUICfE2uTEREvJ2mz1+Cps9fmQ0HTjP2vU34+1j535guVPdX9hYREfcr7/1bXWPiFg6Hwb+X7+GOV7/nyNk87IZBWuY5s8sSEREpRb+ei8udyMpn/PxNrNx9EoBbEqN5ekBLgtUVJiIiHkZBSFxq1e6TPDRvEyez8wnwtfLUzS25vX09LBYNiBYREc+jICQuYxgGr6/cx8nsfJpEBjHr7rY0iQw2uywREZEyKQiJy1gsFl64PYHXVuxl/PVNqeanHeNFRMSzabC0XJEl29OY8vmOkse1gv35+43xCkEiIlIpqEWoDFpH6OLyi+w8+8VO3vxuPwBJDcPp2SzS3KJEREScpHWELkHrCJ1v/8kcxryXwtYjGQDc36UBj/Rthp+PGhhFRMQzlPf+rRYhccqnm4/yt4+3kp1fRFigLy/ensB1zdUSJCIilZOCkJTblM938OqKfQB0jAtn+uBE6oRWM7kqERGRy6e+DCm3jg3CsVpgTM+rmDsySSFIREQqPbUISZkMw+BoxjnqhhUHnuuaR7Ls4e7ERVQ3uTIRERHXUIuQXFDWuULGvr+JG6av5MjZvJLjCkEiIlKVqEVIzrP1cAaj39vIgVO52KwWfth/mrqJdc0uS0RExOUUhKSEYRi88d1+nv1iB4V2g7ph1ZgxuA3tYmuYXZqIiIhbKAiVwdsWVDyTU8DEDzfz9Y50APq0iOS52xIIDdSO8SIiUnVpQcVL8JYFFacu3skry/fiZ7Pyj5uac+/VsdoxXkREKi0tqChOGXddY/afzGF0z6toER1qdjkiIiIVQrPGvFR61jmmLt6J3VHcIBjga+OVe9opBImIiFdRi5AXWrHrBOPnb+JkdgHV/WyM7tnY7JJERERMoSDkRQrtDl5asotXlu8FoFlUMH1bRplclYiIiHkUhLzE4TO5jH0vhY0HzwJwz9X1+ceN8QT42swtTERExEQKQl5gxa4TjJ67kcxzRQQH+DD1ttbc0KqO2WWJiIiYTkHIC9QK9udckYPEmDBmDm5DTHig2SWJiIh4BAWhKio7v4gg/+Jvb/M6Ibw38mpa1wvF16aJgiIiIr/QXbEKWpBymM7PLiPl4JmSY+1iaygEiYiI/I7ujGVITk4mPj6eDh06mF1KueUWFDHhg838Zd5mMvIKeef7A2aXJCIi4tG0xcYlVJYtNnYcy2T03I3sPZGD1QJjr2vMmJ6NsVm1TYaIiHgfbbHhJQzD4L9rD/LUZ9spKHIQGeLP9LvacHXDmmaXJiIi4vEUhCq5pTvS+cfCHwHo0bQWL9yeQM0gf5OrEhERqRwUhCq565rXpl/LKNrWr8H9XRpgVVeYiIhIuSkIVTIOh8F76w9yS2Jdgvx9sFgs/HtIWywWBSARERFnKQhVIqey83n4g80s/+kEP+w/w0t3JGCxWBSCRERELpOCUCXx/b5TjHs/hbTMfPx9rLSPq2F2SSIiIpWegpAZHHY4sBqy0yAoEmI7gfXCm5/aHQYzl+1mxtLdOAxoVKs6yUPa0izKc6fyi4iIVBYKQhVt+6ew+BHIPPrrsZBo6DsV4m8udWp65jnGvp/C9/tOAzCoXT2euqUFgX76tomIiLiCVpauSNs/hflDS4cggMxjxce3f1rqsMViYU96DoF+Nl6+M4EXbk9QCBIREXEh3VUrisNe3BLEhRbyNgALLJ6EvckN2HyKvy21gv2ZfU9bwqv70bBWUEVWKyIi4hXUIlRRDqw+vyWoFAMyj/DEzNf53+Zfz2sfF64QJCIi4iYKQhUlO61cp2WeOMyzX+yk0O5wc0EiIiKirrGKEhRZrtNCa9dj3rCr8bUpo4qIiLib7rZlSE5OJj4+ng4dOrjmDWM7Fc8O48KLHzoMyPSL5PFRf6BejUDXXFNEREQuymIYxoVG78rPMjMzCQ0NJSMjg5CQK1y755dZY8BvB007AAsWLHe8fd4UehEREXFeee/fahGqSPE3wx1vQ0idUoeN4GiFIBERERNojFBFi78Zmt0IB1ZTlHkMS3AUtrjOZa4sLSIiIu6jIGQGqw0adNUfvoiIiMnUNSYiIiJeS0FIREREvJaCkIiIiHgtBSERERHxWgpCIiIi4rUUhERERMRrKQiJiIiI11IQEhEREa+lICQiIiJeS0FIREREvJaCkIiIiHgtBSERERHxWgpCIiIi4rW0AfolGIYBQGZmpsmViIiISHn9ct/+5T5eFgWhS8jKygIgJibG5EpERETEWVlZWYSGhpb5vMW4VFTycg6Hg6NHjxIcHEzHjh1Zv369S943MzOTmJgYDh06REhIiEveU6qWDh06uOznzdt4w59dZfqMnlarWfVU5HXdeS1Xvrc774WGYZCVlUV0dDRWa9kjgdQidAlWq5V69eoBYLPZXP6NCgkJURCSC3LHz5u38IY/u8r0GT2tVrPqqcjruvNaleleeLGWoF9osLQTRo0aZXYJ4kX083b5vOHPrjJ9Rk+r1ax6KvK67ryWp30/r5S6xkySmZlJaGgoGRkZHvWbkoiISEXxhHuhWoRM4u/vzxNPPIG/v7/ZpYiIiJjCE+6FahESERERr6UWIREREfFaCkIiIiLitRSERERExGspCImIiIjXUhASERERr6Ug5IEOHTpE9+7diY+Pp3Xr1nzwwQdmlyQiIlKhzp49S/v27UlMTKRly5a8/vrrbrmOps97oGPHjpGWlkZiYiLHjx+nXbt27Nq1i+rVq5tdmoiISIWw2+3k5+cTGBhITk4OLVu25IcffqBmzZouvY72GvNAderUoU6dOgBERUURERHB6dOnFYRERMRr2Gw2AgMDAcjPz8cwDNzRdqOuMTdYsWIF/fv3Jzo6GovFwsKFC887Jzk5mbi4OAICAkhKSmLdunUXfK8NGzZgt9uJiYlxc9UiIiKu44p74dmzZ0lISKBevXpMnDiRiIgIl9epIOQGOTk5JCQkkJycfMHn582bx/jx43niiSfYuHEjCQkJ9OnTh/T09FLnnT59mqFDh/Laa69VRNkiIiIu44p7YVhYGJs3byY1NZW5c+eSlpbm8jo1RsjNLBYLCxYsYMCAASXHkpKS6NChA7NmzQLA4XAQExPDmDFjmDRpElDcDHj99dczcuRI7r33XjNKFxERcYnLvRf+1p///Gd69uzJoEGDXFqbWoQqWEFBARs2bKBXr14lx6xWK7169WLNmjUAGIbB8OHD6dmzp0KQiIhUOeW5F6alpZGVlQVARkYGK1asoGnTpi6vRUGogp08eRK73U5kZGSp45GRkRw/fhyA7777jnnz5rFw4UISExNJTExk69atZpQrIiLicuW5Fx44cICuXbuSkJBA165dGTNmDK1atXJ5LZo15oG6dOmCw+EwuwwRERHTdOzYkU2bNrn9OmoRqmARERHYbLbzBnylpaURFRVlUlUiIiIVx5PuhQpCFczPz4927dqxdOnSkmMOh4OlS5dyzTXXmFiZiIhIxfCke6G6xtwgOzubPXv2lDxOTU1l06ZNhIeHU79+fcaPH8+wYcNo3749HTt2ZNq0aeTk5HDfffeZWLWIiIjrVJZ7oabPu8Hy5cvp0aPHeceHDRvGnDlzAJg1axbPP/88x48fJzExkRkzZpCUlFTBlYqIiLhHZbkXKgiJiIiI19IYIREREfFaCkIiIiLitRSERERExGspCImIiIjXUhASERERr6UgJCIiIl5LQUhERES8loKQiIiIeC0FIRHxCsOHD2fAgAFmlyEiHkYrS4uIV8jIyMAwDMLCwswuRUQ8iIKQiIiIeC11jYlIlfLhhx/SqlUrqlWrRs2aNenVqxc5OTmlusb279+PxWI576t79+4l77Nq1Sq6du1KtWrViImJYezYseTk5JjzoUTEbRSERKTKOHbsGIMHD2bEiBHs2LGD5cuXM3DgQH7f8B0TE8OxY8dKvlJSUqhZsybdunUDYO/evfTt25fbbruNLVu2MG/ePFatWsXo0aPN+Fgi4kbqGhORKmPjxo20a9eO/fv3ExsbW+q54cOHc/bsWRYuXFjq+Llz5+jevTu1atXik08+wWq18oc//AGbzcarr75act6qVau49tprycnJISAgoCI+johUAB+zCxARcZWEhASuu+46WrVqRZ8+fejduzeDBg2iRo0aZb5mxIgRZGVlsWTJEqzW4kbyzZs3s2XLFv773/+WnGcYBg6Hg9TUVJo3b+72zyIiFUNBSESqDJvNxpIlS1i9ejVfffUVM2fO5O9//ztr16694PlPP/00X375JevWrSM4OLjkeHZ2Ng8++CBjx4497zX169d3W/0iUvHUNSYiVZbdbic2Npbx48ezZcuWUl1jH330EYMHD+aLL77guuuuK/W6IUOGkJaWxtdff21C1SJSkTRYWkSqjLVr1zJ58mR++OEHDh48yMcff8yJEyfO68r68ccfGTp0KI888ggtWrTg+PHjHD9+nNOnTwPwyCOPsHr1akaPHs2mTZvYvXs3n3zyiQZLi1RBCkIiUmWEhISwYsUKbrjhBpo0acI//vEPXnzxRfr161fqvB9++IHc3Fyefvpp6tSpU/I1cOBAAFq3bs23337Lrl276Nq1K23atOHxxx8nOjrajI8lIm6krjERERHxWmoREhEREa+lICQiIiJeS0FIREREvJaCkIiIiHgtBSERERHxWgpCIiIi4rUUhERERMRrKQiJiIiI11IQEhEREa+lICQiIiJeS0FIREREvJaCkIiIiHit/wc7iW/C25IdVwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.loglog(sizes, np.exp(regr.intercept) * sizes**regr.slope, '--', label=r\"slope = $%.3s$\" % regr.slope)\n",
    "plt.loglog(sizes, times, 'o', label='time')\n",
    "\n",
    "plt.xlabel('size')\n",
    "plt.ylabel('time gemm, sec')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a242aff3-2109-4bd1-9fd1-a7b2494c276f",
   "metadata": {},
   "source": [
    "So indeed, the time complexity is about $O(N^{2.8})$, which reasonably close to the ideal result of $O(N^3)$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a32fa259-8957-4c45-a70b-bd44fee0ee30",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}