04_로지스틱_회귀.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "04.로지스틱 회귀.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyM4w2YAtNHr0km1x7iVUkz/",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/threegenie/studying_pytorch/blob/main/04_%EB%A1%9C%EC%A7%80%EC%8A%A4%ED%8B%B1_%ED%9A%8C%EA%B7%80.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fQtlIafURwey"
      },
      "source": [
        "> PyTorch로 시작하는 딥러닝 입문 - <nn.Module로 구현하는 로지스틱 회귀, 클래스로 파이토치 모델 구현하기>를 참고하여 직접 logistic regression model을 만들어보기(sklearn과 pytorch 비교)\n",
        "- Kaggle Titanic 데이터를 사용하여 생존자 예측(https://www.kaggle.com/c/titanic/data)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rT-ndyLlRrDw"
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "from torch.autograd import Variable\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import os, sys\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "F7Z_cYkkZduP",
        "outputId": "94c45d00-cd57-4883-d051-58248f4f83ee"
      },
      "source": [
        "# random seed 고정\n",
        "torch.manual_seed(1)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7fbdd4102d90>"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FUSRG5bRW6kX"
      },
      "source": [
        "### Data import"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k6x7T_N3SKh8"
      },
      "source": [
        "train = pd.read_csv('https://ds-lecture-data.s3.ap-northeast-2.amazonaws.com/titanic/train.csv')\n",
        "test = pd.read_csv('https://ds-lecture-data.s3.ap-northeast-2.amazonaws.com/titanic/test.csv')"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "MvSa7MW0W98m",
        "outputId": "67c5a985-500e-4224-fb97-f13472ca78c2"
      },
      "source": [
        "train.head()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>PassengerId</th>\n",
              "      <th>Survived</th>\n",
              "      <th>Pclass</th>\n",
              "      <th>Name</th>\n",
              "      <th>Sex</th>\n",
              "      <th>Age</th>\n",
              "      <th>SibSp</th>\n",
              "      <th>Parch</th>\n",
              "      <th>Ticket</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Cabin</th>\n",
              "      <th>Embarked</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Braund, Mr. Owen Harris</td>\n",
              "      <td>male</td>\n",
              "      <td>22.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>A/5 21171</td>\n",
              "      <td>7.2500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
              "      <td>female</td>\n",
              "      <td>38.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>PC 17599</td>\n",
              "      <td>71.2833</td>\n",
              "      <td>C85</td>\n",
              "      <td>C</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>Heikkinen, Miss. Laina</td>\n",
              "      <td>female</td>\n",
              "      <td>26.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>STON/O2. 3101282</td>\n",
              "      <td>7.9250</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
              "      <td>female</td>\n",
              "      <td>35.0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>113803</td>\n",
              "      <td>53.1000</td>\n",
              "      <td>C123</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>Allen, Mr. William Henry</td>\n",
              "      <td>male</td>\n",
              "      <td>35.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>373450</td>\n",
              "      <td>8.0500</td>\n",
              "      <td>NaN</td>\n",
              "      <td>S</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   PassengerId  Survived  Pclass  ...     Fare Cabin  Embarked\n",
              "0            1         0       3  ...   7.2500   NaN         S\n",
              "1            2         1       1  ...  71.2833   C85         C\n",
              "2            3         1       3  ...   7.9250   NaN         S\n",
              "3            4         1       1  ...  53.1000  C123         S\n",
              "4            5         0       3  ...   8.0500   NaN         S\n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "20XhzmmTXAj8"
      },
      "source": [
        "### Scikit-Learn으로 Logistic Regression 모델 만들기"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PhSXIUFVWtFd"
      },
      "source": [
        "# train, val split\n",
        "from sklearn.model_selection import train_test_split\n",
        "train, val = train_test_split(train, random_state=2)"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X7jnlw5bXLhq"
      },
      "source": [
        "target = 'Survived'\n",
        "features = ['Pclass','SibSp','Parch','Fare'] #scaling하기 귀찮아서 numeric한 특성만 사용.."
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iLS1eGbQXNVK"
      },
      "source": [
        "y_train = train[target]\n",
        "x_train = train[features]"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7HJ7G71dYz1J"
      },
      "source": [
        "x_val = val[features]\n",
        "y_val = val[target]"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TOCypo5hXQIL",
        "outputId": "be7ee764-5f6a-4f44-9f2a-d9ed1021267c"
      },
      "source": [
        "# logistic regression model import\n",
        "\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "\n",
        "logistic = LogisticRegression()\n",
        "logistic.fit(x_train, y_train)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
              "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tVfQ_rlvY8mI",
        "outputId": "eb2d3f61-e610-4163-abac-270586b66041"
      },
      "source": [
        "print('검증세트 정확도', logistic.score(x_val, y_val))"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "검증세트 정확도 0.6860986547085202\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TV0fDH2-YMqm",
        "outputId": "df99e1a2-9a2b-4804-d98e-00bd75175244"
      },
      "source": [
        "pred = logistic.predict(x_val)\n",
        "pred"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
              "       0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
              "       0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
              "       1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0,\n",
              "       0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0,\n",
              "       1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,\n",
              "       0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0,\n",
              "       0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
              "       0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0,\n",
              "       0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0,\n",
              "       0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qFmqsiRFZJvq"
      },
      "source": [
        "### Pytorch로 Logistic Regression 모델 만들기"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rqBEKReAcoGk"
      },
      "source": [
        "#1차원으로 만들기 위해 타겟 원소 따로 빼서 numpy array로 만듦\n",
        "y_train = [[x] for x in train['Survived']]\n",
        "y_val = [[x] for x in val['Survived']]"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vfBJSH9MdUGX"
      },
      "source": [
        "# 타겟 numpy to tensor\n",
        "y_train_ts = torch.Tensor(y_train)\n",
        "y_val_ts = torch.Tensor(y_val)"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zRz0_AERbLS5"
      },
      "source": [
        "# convert pandas dataframe to numpy array\n",
        "trainX = torch.tensor(x_train.values)\n",
        "valX = torch.tensor(x_val.values)"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F-Je3dUvZ5Hl"
      },
      "source": [
        "# numpy array to Torch tensor\n",
        "X_train_ts = trainX.float()\n",
        "X_val_ts = valX.float()"
      ],
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qap__d3xaAz0"
      },
      "source": [
        "# logistic regression model built\n",
        "class BinaryClassifier(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.linear = nn.Linear(4, 1)\n",
        "        self.sigmoid = nn.Sigmoid()\n",
        "\n",
        "    def forward(self, x):\n",
        "        return self.sigmoid(self.linear(x))"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mno-yOjnaIVz"
      },
      "source": [
        "model = BinaryClassifier()"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BFeEArWVaJuZ",
        "outputId": "9b5472b7-a92b-45c7-a178-75919f12e662"
      },
      "source": [
        "# optimizer 설정\n",
        "optimizer = optim.SGD(model.parameters(), lr=1)\n",
        "\n",
        "nb_epochs = 2000\n",
        "for epoch in range(nb_epochs + 1):\n",
        "\n",
        "    # H(x) 계산\n",
        "    hypothesis = model(X_train_ts)\n",
        "\n",
        "    # cost 계산\n",
        "    cost = F.binary_cross_entropy(hypothesis, y_train_ts)\n",
        "\n",
        "    # cost로 H(x) 개선\n",
        "    optimizer.zero_grad()\n",
        "    cost.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    # 20번마다 로그 출력\n",
        "    if epoch % 10 == 0:\n",
        "        prediction = hypothesis >= torch.FloatTensor([0.5]) # 예측값이 0.5를 넘으면 True로 간주\n",
        "        correct_prediction = prediction.float() == y_train_ts # 실제값과 일치하는 경우만 True로 간주\n",
        "        accuracy = correct_prediction.sum().item() / len(correct_prediction) # 정확도를 계산\n",
        "        print('Epoch {:4d}/{} Cost: {:.6f} Accuracy {:2.2f}%'.format( # 각 에포크마다 정확도를 출력\n",
        "            epoch, nb_epochs, cost.item(), accuracy * 100,\n",
        "        ))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch    0/2000 Cost: 5.968364 Accuracy 38.02%\n",
            "Epoch   10/2000 Cost: 35.747925 Accuracy 62.57%\n",
            "Epoch   20/2000 Cost: 35.741570 Accuracy 62.57%\n",
            "Epoch   30/2000 Cost: 35.735371 Accuracy 62.57%\n",
            "Epoch   40/2000 Cost: 35.729332 Accuracy 62.57%\n",
            "Epoch   50/2000 Cost: 35.723465 Accuracy 62.57%\n",
            "Epoch   60/2000 Cost: 35.717766 Accuracy 62.57%\n",
            "Epoch   70/2000 Cost: 35.712238 Accuracy 62.57%\n",
            "Epoch   80/2000 Cost: 35.706886 Accuracy 62.57%\n",
            "Epoch   90/2000 Cost: 35.701717 Accuracy 62.57%\n",
            "Epoch  100/2000 Cost: 35.679829 Accuracy 62.57%\n",
            "Epoch  110/2000 Cost: 35.674900 Accuracy 62.57%\n",
            "Epoch  120/2000 Cost: 35.670158 Accuracy 62.57%\n",
            "Epoch  130/2000 Cost: 35.665604 Accuracy 62.57%\n",
            "Epoch  140/2000 Cost: 35.661228 Accuracy 62.57%\n",
            "Epoch  150/2000 Cost: 35.657040 Accuracy 62.57%\n",
            "Epoch  160/2000 Cost: 35.653030 Accuracy 62.57%\n",
            "Epoch  170/2000 Cost: 35.649200 Accuracy 62.57%\n",
            "Epoch  180/2000 Cost: 35.645542 Accuracy 62.57%\n",
            "Epoch  190/2000 Cost: 35.642063 Accuracy 62.57%\n",
            "Epoch  200/2000 Cost: 35.638744 Accuracy 62.57%\n",
            "Epoch  210/2000 Cost: 35.635593 Accuracy 62.57%\n",
            "Epoch  220/2000 Cost: 35.632599 Accuracy 62.57%\n",
            "Epoch  230/2000 Cost: 35.629761 Accuracy 62.57%\n",
            "Epoch  240/2000 Cost: 35.627064 Accuracy 62.57%\n",
            "Epoch  250/2000 Cost: 35.624516 Accuracy 62.57%\n",
            "Epoch  260/2000 Cost: 35.622101 Accuracy 62.57%\n",
            "Epoch  270/2000 Cost: 35.619812 Accuracy 62.57%\n",
            "Epoch  280/2000 Cost: 35.617653 Accuracy 62.57%\n",
            "Epoch  290/2000 Cost: 35.615608 Accuracy 62.57%\n",
            "Epoch  300/2000 Cost: 35.613682 Accuracy 62.57%\n",
            "Epoch  310/2000 Cost: 35.611851 Accuracy 62.57%\n",
            "Epoch  320/2000 Cost: 35.576302 Accuracy 62.57%\n",
            "Epoch  330/2000 Cost: 35.574600 Accuracy 62.57%\n",
            "Epoch  340/2000 Cost: 35.572990 Accuracy 62.57%\n",
            "Epoch  350/2000 Cost: 35.571472 Accuracy 62.57%\n",
            "Epoch  360/2000 Cost: 35.570030 Accuracy 62.57%\n",
            "Epoch  370/2000 Cost: 35.568676 Accuracy 62.57%\n",
            "Epoch  380/2000 Cost: 35.567390 Accuracy 62.57%\n",
            "Epoch  390/2000 Cost: 35.566174 Accuracy 62.57%\n",
            "Epoch  400/2000 Cost: 35.565022 Accuracy 62.57%\n",
            "Epoch  410/2000 Cost: 35.563927 Accuracy 62.57%\n",
            "Epoch  420/2000 Cost: 35.562893 Accuracy 62.57%\n",
            "Epoch  430/2000 Cost: 35.561905 Accuracy 62.57%\n",
            "Epoch  440/2000 Cost: 35.560974 Accuracy 62.57%\n",
            "Epoch  450/2000 Cost: 35.560089 Accuracy 62.57%\n",
            "Epoch  460/2000 Cost: 35.559242 Accuracy 62.57%\n",
            "Epoch  470/2000 Cost: 35.558434 Accuracy 62.57%\n",
            "Epoch  480/2000 Cost: 35.557663 Accuracy 62.57%\n",
            "Epoch  490/2000 Cost: 35.556923 Accuracy 62.57%\n",
            "Epoch  500/2000 Cost: 35.556221 Accuracy 62.57%\n",
            "Epoch  510/2000 Cost: 35.555546 Accuracy 62.57%\n",
            "Epoch  520/2000 Cost: 35.554897 Accuracy 62.57%\n",
            "Epoch  530/2000 Cost: 35.554276 Accuracy 62.57%\n",
            "Epoch  540/2000 Cost: 35.553680 Accuracy 62.57%\n",
            "Epoch  550/2000 Cost: 35.553101 Accuracy 62.57%\n",
            "Epoch  560/2000 Cost: 35.552544 Accuracy 62.57%\n",
            "Epoch  570/2000 Cost: 35.552006 Accuracy 62.57%\n",
            "Epoch  580/2000 Cost: 35.551487 Accuracy 62.57%\n",
            "Epoch  590/2000 Cost: 35.550980 Accuracy 62.57%\n",
            "Epoch  600/2000 Cost: 35.550488 Accuracy 62.57%\n",
            "Epoch  610/2000 Cost: 35.550022 Accuracy 62.57%\n",
            "Epoch  620/2000 Cost: 35.549557 Accuracy 62.57%\n",
            "Epoch  630/2000 Cost: 35.549107 Accuracy 62.57%\n",
            "Epoch  640/2000 Cost: 35.548668 Accuracy 62.57%\n",
            "Epoch  650/2000 Cost: 35.548241 Accuracy 62.57%\n",
            "Epoch  660/2000 Cost: 35.547821 Accuracy 62.57%\n",
            "Epoch  670/2000 Cost: 35.547413 Accuracy 62.57%\n",
            "Epoch  680/2000 Cost: 35.547012 Accuracy 62.57%\n",
            "Epoch  690/2000 Cost: 35.546623 Accuracy 62.57%\n",
            "Epoch  700/2000 Cost: 35.546230 Accuracy 62.57%\n",
            "Epoch  710/2000 Cost: 35.545856 Accuracy 62.57%\n",
            "Epoch  720/2000 Cost: 35.545486 Accuracy 62.57%\n",
            "Epoch  730/2000 Cost: 35.545120 Accuracy 62.57%\n",
            "Epoch  740/2000 Cost: 35.544762 Accuracy 62.57%\n",
            "Epoch  750/2000 Cost: 35.544407 Accuracy 62.57%\n",
            "Epoch  760/2000 Cost: 35.544060 Accuracy 62.57%\n",
            "Epoch  770/2000 Cost: 35.543716 Accuracy 62.57%\n",
            "Epoch  780/2000 Cost: 35.543373 Accuracy 62.57%\n",
            "Epoch  790/2000 Cost: 35.543037 Accuracy 62.57%\n",
            "Epoch  800/2000 Cost: 35.542706 Accuracy 62.57%\n",
            "Epoch  810/2000 Cost: 35.542377 Accuracy 62.57%\n",
            "Epoch  820/2000 Cost: 35.542053 Accuracy 62.57%\n",
            "Epoch  830/2000 Cost: 35.541733 Accuracy 62.57%\n",
            "Epoch  840/2000 Cost: 35.541416 Accuracy 62.57%\n",
            "Epoch  850/2000 Cost: 35.541100 Accuracy 62.57%\n",
            "Epoch  860/2000 Cost: 35.540787 Accuracy 62.57%\n",
            "Epoch  870/2000 Cost: 35.540478 Accuracy 62.57%\n",
            "Epoch  880/2000 Cost: 35.540169 Accuracy 62.57%\n",
            "Epoch  890/2000 Cost: 35.539871 Accuracy 62.57%\n",
            "Epoch  900/2000 Cost: 35.539566 Accuracy 62.57%\n",
            "Epoch  910/2000 Cost: 35.539268 Accuracy 62.57%\n",
            "Epoch  920/2000 Cost: 35.538971 Accuracy 62.57%\n",
            "Epoch  930/2000 Cost: 35.538677 Accuracy 62.57%\n",
            "Epoch  940/2000 Cost: 35.538383 Accuracy 62.57%\n",
            "Epoch  950/2000 Cost: 35.538094 Accuracy 62.57%\n",
            "Epoch  960/2000 Cost: 35.537804 Accuracy 62.57%\n",
            "Epoch  970/2000 Cost: 35.537518 Accuracy 62.57%\n",
            "Epoch  980/2000 Cost: 35.537231 Accuracy 62.57%\n",
            "Epoch  990/2000 Cost: 35.536949 Accuracy 62.57%\n",
            "Epoch 1000/2000 Cost: 35.536667 Accuracy 62.57%\n",
            "Epoch 1010/2000 Cost: 35.536388 Accuracy 62.57%\n",
            "Epoch 1020/2000 Cost: 35.536110 Accuracy 62.57%\n",
            "Epoch 1030/2000 Cost: 35.535839 Accuracy 62.57%\n",
            "Epoch 1040/2000 Cost: 35.535561 Accuracy 62.57%\n",
            "Epoch 1050/2000 Cost: 35.535286 Accuracy 62.57%\n",
            "Epoch 1060/2000 Cost: 35.535015 Accuracy 62.57%\n",
            "Epoch 1070/2000 Cost: 35.534744 Accuracy 62.57%\n",
            "Epoch 1080/2000 Cost: 35.534477 Accuracy 62.57%\n",
            "Epoch 1090/2000 Cost: 35.534210 Accuracy 62.57%\n",
            "Epoch 1100/2000 Cost: 35.533939 Accuracy 62.57%\n",
            "Epoch 1110/2000 Cost: 35.533684 Accuracy 62.57%\n",
            "Epoch 1120/2000 Cost: 35.533417 Accuracy 62.57%\n",
            "Epoch 1130/2000 Cost: 35.533154 Accuracy 62.57%\n",
            "Epoch 1140/2000 Cost: 35.532890 Accuracy 62.57%\n",
            "Epoch 1150/2000 Cost: 35.532635 Accuracy 62.57%\n",
            "Epoch 1160/2000 Cost: 35.532375 Accuracy 62.57%\n",
            "Epoch 1170/2000 Cost: 35.532120 Accuracy 62.57%\n",
            "Epoch 1180/2000 Cost: 35.531864 Accuracy 62.57%\n",
            "Epoch 1190/2000 Cost: 35.531612 Accuracy 62.57%\n",
            "Epoch 1200/2000 Cost: 35.531357 Accuracy 62.57%\n",
            "Epoch 1210/2000 Cost: 35.531105 Accuracy 62.57%\n",
            "Epoch 1220/2000 Cost: 35.530857 Accuracy 62.57%\n",
            "Epoch 1230/2000 Cost: 35.530605 Accuracy 62.57%\n",
            "Epoch 1240/2000 Cost: 35.530357 Accuracy 62.57%\n",
            "Epoch 1250/2000 Cost: 35.530117 Accuracy 62.57%\n",
            "Epoch 1260/2000 Cost: 35.529865 Accuracy 62.57%\n",
            "Epoch 1270/2000 Cost: 35.529625 Accuracy 62.57%\n",
            "Epoch 1280/2000 Cost: 35.529377 Accuracy 62.57%\n",
            "Epoch 1290/2000 Cost: 35.529140 Accuracy 62.57%\n",
            "Epoch 1300/2000 Cost: 35.528896 Accuracy 62.57%\n",
            "Epoch 1310/2000 Cost: 35.528656 Accuracy 62.57%\n",
            "Epoch 1320/2000 Cost: 35.528419 Accuracy 62.57%\n",
            "Epoch 1330/2000 Cost: 35.528179 Accuracy 62.57%\n",
            "Epoch 1340/2000 Cost: 35.527946 Accuracy 62.57%\n",
            "Epoch 1350/2000 Cost: 35.527714 Accuracy 62.57%\n",
            "Epoch 1360/2000 Cost: 35.527473 Accuracy 62.57%\n",
            "Epoch 1370/2000 Cost: 35.527237 Accuracy 62.57%\n",
            "Epoch 1380/2000 Cost: 35.527012 Accuracy 62.57%\n",
            "Epoch 1390/2000 Cost: 35.526779 Accuracy 62.57%\n",
            "Epoch 1400/2000 Cost: 35.526546 Accuracy 62.57%\n",
            "Epoch 1410/2000 Cost: 35.526314 Accuracy 62.57%\n",
            "Epoch 1420/2000 Cost: 35.526089 Accuracy 62.57%\n",
            "Epoch 1430/2000 Cost: 35.525860 Accuracy 62.57%\n",
            "Epoch 1440/2000 Cost: 35.525639 Accuracy 62.57%\n",
            "Epoch 1450/2000 Cost: 35.525414 Accuracy 62.57%\n",
            "Epoch 1460/2000 Cost: 35.525185 Accuracy 62.57%\n",
            "Epoch 1470/2000 Cost: 35.524960 Accuracy 62.57%\n",
            "Epoch 1480/2000 Cost: 35.524738 Accuracy 62.57%\n",
            "Epoch 1490/2000 Cost: 35.524525 Accuracy 62.57%\n",
            "Epoch 1500/2000 Cost: 35.507420 Accuracy 62.57%\n",
            "Epoch 1510/2000 Cost: 35.507195 Accuracy 62.57%\n",
            "Epoch 1520/2000 Cost: 35.506973 Accuracy 62.57%\n",
            "Epoch 1530/2000 Cost: 35.506748 Accuracy 62.57%\n",
            "Epoch 1540/2000 Cost: 35.506527 Accuracy 62.57%\n",
            "Epoch 1550/2000 Cost: 35.506306 Accuracy 62.57%\n",
            "Epoch 1560/2000 Cost: 35.506088 Accuracy 62.57%\n",
            "Epoch 1570/2000 Cost: 35.505871 Accuracy 62.57%\n",
            "Epoch 1580/2000 Cost: 35.505653 Accuracy 62.57%\n",
            "Epoch 1590/2000 Cost: 35.505440 Accuracy 62.57%\n",
            "Epoch 1600/2000 Cost: 35.505219 Accuracy 62.57%\n",
            "Epoch 1610/2000 Cost: 35.505005 Accuracy 62.57%\n",
            "Epoch 1620/2000 Cost: 35.504795 Accuracy 62.57%\n",
            "Epoch 1630/2000 Cost: 35.504578 Accuracy 62.57%\n",
            "Epoch 1640/2000 Cost: 35.504368 Accuracy 62.57%\n",
            "Epoch 1650/2000 Cost: 35.504158 Accuracy 62.57%\n",
            "Epoch 1660/2000 Cost: 35.503948 Accuracy 62.57%\n",
            "Epoch 1670/2000 Cost: 35.503738 Accuracy 62.57%\n",
            "Epoch 1680/2000 Cost: 35.503532 Accuracy 62.57%\n",
            "Epoch 1690/2000 Cost: 35.503323 Accuracy 62.57%\n",
            "Epoch 1700/2000 Cost: 35.503120 Accuracy 62.57%\n",
            "Epoch 1710/2000 Cost: 35.502911 Accuracy 62.57%\n",
            "Epoch 1720/2000 Cost: 35.502708 Accuracy 62.57%\n",
            "Epoch 1730/2000 Cost: 35.502499 Accuracy 62.57%\n",
            "Epoch 1740/2000 Cost: 35.502296 Accuracy 62.57%\n",
            "Epoch 1750/2000 Cost: 35.502098 Accuracy 62.57%\n",
            "Epoch 1760/2000 Cost: 35.501896 Accuracy 62.57%\n",
            "Epoch 1770/2000 Cost: 35.501698 Accuracy 62.57%\n",
            "Epoch 1780/2000 Cost: 35.501495 Accuracy 62.57%\n",
            "Epoch 1790/2000 Cost: 35.501297 Accuracy 62.57%\n",
            "Epoch 1800/2000 Cost: 35.501102 Accuracy 62.57%\n",
            "Epoch 1810/2000 Cost: 35.500900 Accuracy 62.57%\n",
            "Epoch 1820/2000 Cost: 35.500706 Accuracy 62.57%\n",
            "Epoch 1830/2000 Cost: 35.500507 Accuracy 62.57%\n",
            "Epoch 1840/2000 Cost: 35.500313 Accuracy 62.57%\n",
            "Epoch 1850/2000 Cost: 35.500118 Accuracy 62.57%\n",
            "Epoch 1860/2000 Cost: 35.499931 Accuracy 62.57%\n",
            "Epoch 1870/2000 Cost: 35.499733 Accuracy 62.57%\n",
            "Epoch 1880/2000 Cost: 35.499542 Accuracy 62.57%\n",
            "Epoch 1890/2000 Cost: 35.499355 Accuracy 62.57%\n",
            "Epoch 1900/2000 Cost: 35.499161 Accuracy 62.57%\n",
            "Epoch 1910/2000 Cost: 35.498974 Accuracy 62.57%\n",
            "Epoch 1920/2000 Cost: 35.498783 Accuracy 62.57%\n",
            "Epoch 1930/2000 Cost: 35.498596 Accuracy 62.57%\n",
            "Epoch 1940/2000 Cost: 35.498409 Accuracy 62.57%\n",
            "Epoch 1950/2000 Cost: 35.498222 Accuracy 62.57%\n",
            "Epoch 1960/2000 Cost: 35.498039 Accuracy 62.57%\n",
            "Epoch 1970/2000 Cost: 35.497852 Accuracy 62.57%\n",
            "Epoch 1980/2000 Cost: 35.497669 Accuracy 62.57%\n",
            "Epoch 1990/2000 Cost: 35.497486 Accuracy 62.57%\n",
            "Epoch 2000/2000 Cost: 35.497303 Accuracy 62.57%\n"
          ]
        }
      ]
    }
  ]
}