docs/CONDOR_TensorFlow_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vcT_VpWpT1Kf"
   },
   "source": [
    "# CONDOR ordinal classification/regression in Tensorflow Keras \n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/GarrettJenkinson/condor_tensorflow/blob/main/docs/CONDOR_TensorFlow_demo.ipynb)\n",
    "\n",
    "\n",
    "This notebook uses MNIST hand-written digits and Amazon reviews as examples of ordinal classification, using the condor_tensorflow package for Tensorflow Keras.\n",
    "For more details on the method, read our [paper.](https://arxiv.org/abs/2110.07470)\n",
    "\n",
    "**Acknowledgments**: This notebook is based in part on PyTorch source code written by Sebastian Rashka [in this notebook](https://github.com/Raschka-research-group/coral-cnn/blob/master/coral-implementation-recipe.ipynb) and the CORAL ordinal notebook written by [Chris Kennedy and Stephen Matthews](https://github.com/ck37/coral-ordinal)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9QsCIIgoFOkr"
   },
   "source": [
    "## Installation for Google Colab\n",
    "\n",
    "With pip you can either install the latest source code from GitHub or the stable version of the module on pypi.org"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#upgrade sklearn...only needed for advanced ordinalEncoder behaviours\n",
    "if 'google.colab' in str(get_ipython()):\n",
    "    !pip install scikit-learn==0.24.2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pO9cwoJ33G2q",
    "outputId": "9307fd1e-d4ce-4c23-f5f8-427f8713b579"
   },
   "outputs": [],
   "source": [
    "if 'google.colab' in str(get_ipython()):\n",
    "    GITHUB_AUTH = \"GarrettJenkinson:<APIaccessTOKEN>\"\n",
    "    !git clone https://$GITHUB_AUTH@github.com/GarrettJenkinson/condor_tensorflow.git"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vWwEuq1E1gql",
    "outputId": "3d3d74d9-791d-4b88-8b8d-16973b1fe119"
   },
   "outputs": [],
   "source": [
    "# Install source package from GitHub\n",
    "if 'google.colab' in str(get_ipython()):\n",
    "    !pip install --force-reinstall --no-deps --use-feature=in-tree-build condor_tensorflow/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6Xemf4TAtrJC",
    "outputId": "059ed231-319a-4c49-aae1-abe26289fcf3"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import sklearn\n",
    "from sklearn import model_selection\n",
    "from sklearn.model_selection import train_test_split\n",
    "import pandas as pd\n",
    "from scipy import special\n",
    "import tensorflow_hub as hub\n",
    "import os\n",
    "import json\n",
    "import gzip\n",
    "from urllib.request import urlopen\n",
    "\n",
    "import tensorflow as tf\n",
    "print(\"Tensorflow version\", tf.__version__)\n",
    "\n",
    "import condor_tensorflow as condor\n",
    "print(\"CONDOR Ordinal version:\", condor.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rq0mT2yYucrx"
   },
   "source": [
    "## MNIST toy example\n",
    "\n",
    "MNIST is a database of handwritten digits extracted from handwriting sample forms and widely utilized in image classification tasks.\n",
    "\n",
    "The originally intended use of the dataset is categorical prediction (recognition of digits), without any ordinal component.  However, since the data are numerical, one could imagine a scenario where ordinal proximity of incorrect predictions to the correct prediction might be beneficial e.g. handwritten map coordinates.  Hence we utilize the MNIST dataset and enforce ordinal predictions to demonstrate the improved performance of CONDOR on the ordinal problem, while acknowledging that MNIST is usually more suited to categorical prediction.\n",
    "\n",
    "We begin by setting some core variables required for model building."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fSOcGJBJG1Tr"
   },
   "outputs": [],
   "source": [
    "##########################\n",
    "### SETTINGS\n",
    "##########################\n",
    "\n",
    "# Hyperparameters\n",
    "random_seed = 1 # Not yet used\n",
    "learning_rate = 0.05\n",
    "batch_size = 128\n",
    "num_epochs = 2\n",
    "\n",
    "# Architecture\n",
    "NUM_CLASSES = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we load the MNIST data and create training, test and validation datasets in a suitable format.  Finally we check the shapes of the data structures containing our MNIST data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NooIWGJbGR2u",
    "outputId": "6d75e93d-f2b4-41dd-aef1-85064ee1de97"
   },
   "outputs": [],
   "source": [
    "# Fetch and format the mnist data\n",
    "(mnist_images, mnist_labels), (mnist_images_test, mnist_labels_test) = tf.keras.datasets.mnist.load_data()\n",
    "\n",
    "# Split off a validation dataset for early stopping\n",
    "mnist_images, mnist_images_val, mnist_labels, mnist_labels_val = \\\n",
    "  model_selection.train_test_split(mnist_images, mnist_labels, test_size = 5000, random_state = 1)\n",
    "\n",
    "print(\"Shape of training images:\", mnist_images.shape)\n",
    "print(\"Shape of training labels:\", mnist_labels.shape)\n",
    "\n",
    "print(\"Shape of test images:\", mnist_images_test.shape)\n",
    "print(\"Shape of test labels:\", mnist_labels_test.shape)\n",
    "\n",
    "print(\"Shape of validation images:\", mnist_images_val.shape)\n",
    "print(\"Shape of validation labels:\", mnist_labels_val.shape)\n",
    "\n",
    "# Also rescales to 0-1 range.\n",
    "dataset = tf.data.Dataset.from_tensor_slices(\n",
    "  (tf.cast(mnist_images[..., tf.newaxis] / 255, tf.float32),\n",
    "   tf.cast(mnist_labels, tf.int64)))\n",
    "dataset = dataset.shuffle(1000).batch(batch_size)\n",
    "\n",
    "test_dataset = tf.data.Dataset.from_tensor_slices(\n",
    "  (tf.cast(mnist_images_test[..., tf.newaxis] / 255, tf.float32),\n",
    "   tf.cast(mnist_labels_test, tf.int64)))\n",
    "#test_dataset = test_dataset.shuffle(1000).batch(batch_size)\n",
    "# Here we do not shuffle the test dataset.\n",
    "test_dataset = test_dataset.batch(batch_size)\n",
    "\n",
    "val_dataset = tf.data.Dataset.from_tensor_slices(\n",
    "  (tf.cast(mnist_images_val[..., tf.newaxis] / 255, tf.float32),\n",
    "   tf.cast(mnist_labels_val, tf.int64)))\n",
    "val_dataset = val_dataset.shuffle(1000).batch(batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PAg-gWZED1mR"
   },
   "source": [
    "### Simple MLP model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_9MF55Xz53rx"
   },
   "source": [
    "Now we create a simple multi-layer perceptron model and apply the ordinal output layer required by CONDOR (i.e. a dense layer with 1 unit less than the number of output classes).  Note while we use the example of an MLP model, any categorical neural network architecture could be used.  The version below uses the Sequential API to create the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "p-ozv44W52Ti",
    "outputId": "b8751ccb-e887-44e5-ef71-54e4578d2096"
   },
   "outputs": [],
   "source": [
    "def create_model(num_classes):\n",
    "  model = tf.keras.Sequential()\n",
    "  model.add(tf.keras.layers.Flatten(input_shape = (28, 28, )))\n",
    "  model.add(tf.keras.layers.Dense(128, activation = \"relu\"))\n",
    "  model.add(tf.keras.layers.Dropout(0.2))\n",
    "  model.add(tf.keras.layers.Dense(32, activation = \"relu\"))\n",
    "  model.add(tf.keras.layers.Dropout(0.1))\n",
    "  # No activation function specified so this will output cumulative logits.\n",
    "  model.add(tf.keras.layers.Dense(num_classes-1))\n",
    "  return model\n",
    "\n",
    "model = create_model(NUM_CLASSES)\n",
    "\n",
    "# Note that the model generates 1 fewer outputs than the number of classes. \n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we could build the model using the Functional API as demonstrated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1YAftcr9wxTu",
    "outputId": "24548743-3ac5-4dc0-a8c2-3ea83c576477"
   },
   "outputs": [],
   "source": [
    "# Or a functional API version\n",
    "def create_model2(num_classes):\n",
    "  inputs = tf.keras.Input(shape = (28, 28, ))\n",
    "\n",
    "  x = tf.keras.layers.Flatten()(inputs)\n",
    "  x = tf.keras.layers.Dense(128, activation = \"relu\")(x)\n",
    "  x = tf.keras.layers.Dropout(0.2)(x)\n",
    "  x = tf.keras.layers.Dense(32, activation = \"relu\")(x)\n",
    "  x = tf.keras.layers.Dropout(0.1)(x)\n",
    "  # No activation function specified so this will output cumulative logits.\n",
    "  outputs = tf.keras.layers.Dense(num_classes-1)(x)\n",
    "\n",
    "  model = tf.keras.Model(inputs = inputs, outputs = outputs)\n",
    "\n",
    "  return model\n",
    "\n",
    "model = create_model2(NUM_CLASSES)\n",
    "\n",
    "# Note that the model generates 1 fewer outputs than the number of classes. \n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We compile the model using CONDOR's SparseCondorOrdinalCrossEntropy as the loss function.  This is the key component of the CONDOR method, which enables ordinal prediction with rank consistency.  The other metrics provided by CONDOR enable assessment of CONDOR's performance on the ordinal prediction problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AUIADdPeF2w6"
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer = tf.keras.optimizers.Adam(learning_rate = learning_rate),\n",
    "              loss = condor.SparseCondorOrdinalCrossEntropy(),\n",
    "              metrics = [condor.SparseOrdinalEarthMoversDistance(),\n",
    "                         condor.SparseOrdinalMeanAbsoluteError()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "as_rVDyAJurK",
    "outputId": "01ec32f4-d1b4-424c-bf6e-8524643d6c14"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "# This takes about 5 minutes on CPU, 2.5 minutes on GPU.\n",
    "history = model.fit(dataset, epochs = 5, validation_data = val_dataset,\n",
    "                    callbacks = [tf.keras.callbacks.EarlyStopping(patience = 3, restore_best_weights = True)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i--Xhzu8D6Mb"
   },
   "source": [
    "### Test set evaluation\n",
    "Now we can evaluate performance on the MNIST test dataset we created previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "epv3NVRmJ1gt",
    "outputId": "02dc7b3c-68a3-4d0c-d367-d6dbb6923250"
   },
   "outputs": [],
   "source": [
    "# Evaluate on test dataset.\n",
    "model.evaluate(test_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2pfaK4doXvcS"
   },
   "source": [
    "### Cumulative logits to probabilities\n",
    "\n",
    "Note that the output layer natively outputs cumulative logit values.  These can be  subsequently converted to probability estimates for each ordinal label utilizing the condor.ordinal_softmax() function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 221
    },
    "id": "5MAi4QxZyA2_",
    "outputId": "c856b16a-a015-4c65-a32c-b0d897120d91"
   },
   "outputs": [],
   "source": [
    "print(\"Predict on test dataset\")\n",
    "\n",
    "# Note that these are ordinal (cumulative) logits, not probabilities or regular logits.\n",
    "ordinal_logits = model.predict(test_dataset)\n",
    "\n",
    "# Convert from logits to label probabilities. This is initially a tensorflow tensor.\n",
    "tensor_probs = condor.ordinal_softmax(ordinal_logits)\n",
    "\n",
    "# Convert the tensor into a pandas dataframe.\n",
    "probs_df = pd.DataFrame(tensor_probs.numpy())\n",
    "\n",
    "probs_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can confirm that our probabilities sum to 1 as expected:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CuFGIJZvymSe",
    "outputId": "846f85cb-331d-4e9d-e82a-c3992ba7b1a8"
   },
   "outputs": [],
   "source": [
    "# Check that probabilities all sum to 1 - looks good!\n",
    "probs_df.sum(axis = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you have successfully generated CONDOR's predictions.  Depending on your use case, these may be sufficient for your purposes and if so you can stop here.  However, in the following sections we explore techniques for producing labels from the predicted probabilities.  These techniques will be required if your application requires a single class prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SZmVlStOItF8"
   },
   "source": [
    "### Label prediction\n",
    "\n",
    "Using the probabilities generated, we can produce point estimates of the labels for the MNIST images.  There are many valid techniques to produce point estimates from the probabilities.  Here we demonstrate two common techniques of calculating predicted labels.\n",
    "\n",
    "First we can simply select the label with the highest probability (i.e. we use the mode):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mA344vSf782T",
    "outputId": "442ae806-a019-49c8-80ce-6d37fbc40484"
   },
   "outputs": [],
   "source": [
    "# Probs to labels\n",
    "labels = probs_df.idxmax(axis = 1)\n",
    "labels.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use these labels to calculate the accuracy of our predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "G0Bbs1tb8uh8",
    "outputId": "c74f210e-4801-4b89-f6d9-1570edb30e0f"
   },
   "outputs": [],
   "source": [
    "np.mean(labels == mnist_labels_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "id": "xAUaY171_c22",
    "outputId": "bdfbd060-0e8c-47df-e0d9-2048cdbd0e1a"
   },
   "outputs": [],
   "source": [
    "# Compare to logit-based cumulative probs\n",
    "cum_probs = pd.DataFrame(ordinal_logits).apply(special.expit).cumprod(axis=1)\n",
    "cum_probs.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dQ67xh03JOii"
   },
   "source": [
    "Secondly we utilize the method of label prediction given by Equation 1 of the CONDOR paper (i.e. we use the median):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kNKZU84UBE7s",
    "outputId": "0702f534-b2bc-4142-ac19-d5baccffedbd"
   },
   "outputs": [],
   "source": [
    "labels2 = cum_probs.apply(lambda x: x > 0.5).sum(axis = 1)\n",
    "labels2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we calculate the accuracy of the labels using our test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5f-7BfbYBqSN",
    "outputId": "66e3a654-1a23-4583-c51a-95fa8b52549b"
   },
   "outputs": [],
   "source": [
    "np.mean(labels2 == mnist_labels_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often the two methods of label prediction agree, but not always:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AGlY8HecDFPO",
    "outputId": "d6569bb0-a815-4abc-8b04-da8e19356124"
   },
   "outputs": [],
   "source": [
    "np.mean(labels == labels2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OemBBdUmI6TI",
    "outputId": "d800914f-4a56-4e19-d39b-deb1d79ee7da"
   },
   "outputs": [],
   "source": [
    "print(\"Mean absolute label error version 1:\", np.mean(np.abs(labels - mnist_labels_test)))\n",
    "print(\"Mean absolute label error version 2:\", np.mean(np.abs(labels2 - mnist_labels_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you have fully implemented the CONDOR ordinal workflow, generated predicted probabilities and utilized two methods to produce point estimates of the labels. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2DpYEPQI6-gW"
   },
   "source": [
    "### Importance weights customization\n",
    "\n",
    "A quick example to show how the importance weights can be customized. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "STwPQdgHNne4",
    "outputId": "2e32e75d-8e98-4b7e-8046-2d57d67e906a"
   },
   "outputs": [],
   "source": [
    "model = create_model(num_classes = NUM_CLASSES)\n",
    "model.summary()\n",
    "\n",
    "# We have num_classes - 1 outputs (cumulative logits), so there are 9 elements\n",
    "# in the importance vector to customize.\n",
    "importance_weights = [1., 1., 0.5, 0.5, 0.5, 1., 1., 0.1, 0.1]\n",
    "loss_fn = condor.SparseCondorOrdinalCrossEntropy(importance_weights = importance_weights)\n",
    "\n",
    "model.compile(tf.keras.optimizers.Adam(learning_rate = learning_rate), loss = loss_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "73UOIams7TI_",
    "outputId": "881954e6-01bc-4f31-a1a8-459d07ad98a3"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "history = model.fit(dataset, epochs = num_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fJSm-gbwxTKt"
   },
   "source": [
    "## Amazon reviews and 5-star ratings\n",
    "\n",
    "Now we consider a wholly different problem - text-based Amazon product reviews with corresponding star ratings (via https://nijianmo.github.io/amazon/index.html#subsets).\n",
    "\n",
    "As well as introducing another dataset to which CONDOR can be successfully applied, this part of the tutorial will expand on some relevant topics that were not considered in the MNIST example.\n",
    "\n",
    "We start by downloading the necessary data:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VSdEFFwfoZL_"
   },
   "outputs": [],
   "source": [
    "!curl -o Prime_Pantry_5.json.gz http://deepyeti.ucsd.edu/jianmo/amazon/categoryFilesSmall/Prime_Pantry_5.json.gz "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we read the data from the downloaded file into a Pandas data frame and do some basic cleanup and preprocessing, extracting only the data that we need:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-vHiq67ioiUa",
    "outputId": "3e9352a8-c6f0-4acb-9d14-5aa3b73a5f6a"
   },
   "outputs": [],
   "source": [
    "data = []\n",
    "with gzip.open('Prime_Pantry_5.json.gz') as f:\n",
    "    for l in f:\n",
    "        data.append(json.loads(l.strip()))\n",
    "\n",
    "df = pd.DataFrame.from_dict(data)\n",
    "df = df[['overall', 'reviewText']]\n",
    "\n",
    "# There is a large amount of duplicate text in here, possibly due to paid/fraudulent reviews.\n",
    "df.drop_duplicates(\"reviewText\", inplace = True)\n",
    "\n",
    "# Some of the text is blank, which causes an obscure error about floating point conversion.\n",
    "df.dropna(inplace = True)\n",
    "\n",
    "print(len(df))\n",
    "print(df.head())\n",
    "\n",
    "outcome_col = \"overall\"\n",
    "text_col = \"reviewText\"\n",
    "\n",
    "# We subtract the minimum value from the outcomes so that they start at 0.\n",
    "df[outcome_col] = df[outcome_col].values - df[outcome_col].min()\n",
    "\n",
    "print(\"\\n\", df.overall.value_counts())\n",
    "\n",
    "# TODO: define automatically based on the number of unique values in the outcome variable.\n",
    "num_classes = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see (above) we have a data frame with product star ratings and corresponding text reviews.  You can also see the counts (number of entries) corresponding to each category of star rating.\n",
    "\n",
    "Now lets split the data into training and test sets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MrGlqgv7tv0A",
    "outputId": "bc47e862-ff48-4d0e-bcc4-e3e24951ad12"
   },
   "outputs": [],
   "source": [
    "# Train/Test split\n",
    "text_train, text_test, labels_train, labels_test = \\\n",
    "  train_test_split(df[text_col].values, df[outcome_col].values, test_size = 10000, random_state = 1)\n",
    "\n",
    "print(\"Training text shape:\", text_train.shape)\n",
    "print(\"Training labels shape:\", labels_train.shape)\n",
    "print(\"Testing text shape:\", text_test.shape)\n",
    "print(\"Testing labels shape:\", labels_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eF48EdMZuTxk"
   },
   "source": [
    "### Universal Sentence Encoder model (CONDOR applied with minimal code changes)\n",
    "\n",
    "The Universal Sentence Encoder encodes text into high dimensional vectors that can be used for text classification, semantic similarity, clustering and other natural language tasks.\n",
    "\n",
    "The model is trained and optimized for greater-than-word length text, such as sentences, phrases or short paragraphs. It is trained on a variety of data sources and a variety of tasks with the aim of dynamically accommodating a wide variety of natural language understanding tasks. The input is variable length English text and the output is a 512 dimensional vector. \n",
    "\n",
    "The following code shows how CONDOR can be applied to the Amazon review data for ordinal prediction, utilizing the existing Universal Sentence Encoder model, with minimal code changes.  CONDOR is designed to be easily added to existing models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eOqeIkoJuWcD",
    "outputId": "0ebfc6f3-8a36-4f7d-ad7e-4f712fb147ce"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "# This takes 20 - 30 seconds.\n",
    "\n",
    "# Clear our GPU memory to stay efficient.\n",
    "tf.keras.backend.clear_session()\n",
    "\n",
    "input_text = tf.keras.layers.Input(shape = [], dtype = tf.string, name = 'input_text')\n",
    "\n",
    "model_url = hub.load(\"https://tfhub.dev/google/universal-sentence-encoder-large/5\")\n",
    "\n",
    "base_model = hub.KerasLayer(model_url, input_shape = [],\n",
    "                            dtype = tf.string,\n",
    "                            trainable = False)\n",
    "                            \n",
    "embedded = base_model(input_text)\n",
    "\n",
    "x = tf.keras.layers.Dense(64, activation = 'relu')(embedded)\n",
    "x = tf.keras.layers.Dropout(0.1)(x)\n",
    "output =tf.keras.layers.Dense(num_classes-1)(x) \n",
    "\n",
    "model = tf.keras.Model(inputs = input_text, outputs = output)\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown above, we can load the model from its URL and make some minimal edits, including the addition of our CONDOR-required output layer.\n",
    "\n",
    "Next we compile the model, usind CONDOR's SparseCondorOrdinalCrossEntropy and the generate the same metrics as used previously with the MNIST data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HmMr5L_u_o64"
   },
   "outputs": [],
   "source": [
    "model.compile(loss = condor.SparseCondorOrdinalCrossEntropy(),\n",
    "              metrics = [condor.SparseOrdinalEarthMoversDistance(),\n",
    "                         condor.SparseOrdinalMeanAbsoluteError()],\n",
    "              optimizer = tf.keras.optimizers.Adam(learning_rate = 0.001))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can encode a test string and take a look at the first ten dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OwNLAJg33tc4",
    "outputId": "c96a180e-cc73-458f-bad5-0e1ed30040b2"
   },
   "outputs": [],
   "source": [
    "base_model(np.array([\"test_string\"])).numpy()[0, :10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will train the model using the test reviews as our training data and the corresponding star reviews as our labels.  As with the MNIST data example, CONDOR will perform rank-consistent ordinal prediction.  The advantage of ordinal prediction in a scenario such as predicting star reviews from text is clear - misclassifications close to the star value of the actual review will be preferable to misclassifications far from the true value.\n",
    "\n",
    "Note that the following code may take some time to run (up to several hours), depending on the specifics of your system:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "85MsPnRTxPEz",
    "outputId": "8d6a8187-7aaa-4148-83ca-2b582b23e972"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "history = model.fit(x = text_train,\n",
    "                    y = labels_train,\n",
    "                    epochs = 5,\n",
    "                    batch_size = 32, \n",
    "                    validation_split = 0.2,\n",
    "                    callbacks = [tf.keras.callbacks.EarlyStopping(patience = 2,\n",
    "                                                                  min_delta = 0.001,\n",
    "                                                                  restore_best_weights = True)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CXFi2qXPlehY"
   },
   "source": [
    "#### Evaluate the model\n",
    "\n",
    "Now we can evaluate model performance.  For comparison, CORAL achieves loss 0.7962, MAE 0.3195."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K3ocFVWnldzF"
   },
   "outputs": [],
   "source": [
    "model.evaluate(text_test, labels_test) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we can generate predictions on our test data using the model.  As previously with the MNIST data, the native outputs from the output later are cumulative logits which we convert to probabilities for each class/label using the condor.ordinal_softmax() method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "tqSBmDgXxkFb"
   },
   "outputs": [],
   "source": [
    "preds = model.predict(text_test)\n",
    "print(preds)\n",
    "\n",
    "probs = pd.DataFrame(condor.ordinal_softmax(preds).numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets have a look at some predicted probabilities versus the known labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KvD7ak27yA26"
   },
   "outputs": [],
   "source": [
    "print(probs.head(10))\n",
    "print(labels_test[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eNYR4nOjDJt_"
   },
   "source": [
    "#### Evaluate accuracy\n",
    "\n",
    "Lets evaluate the accuracy and mean absolute error of the model.  First we'll generate predictions using the label with highest probability (i.e. we use the mode, like we did with the MNIST data):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dZzpmNSGxmJv"
   },
   "outputs": [],
   "source": [
    "labels_v1 = probs.idxmax(axis = 1)\n",
    "print(\"Accuracy of label version 1:\", np.mean(labels_v1 == labels_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And as with the MNIST data we will again generate predictions using the method given by Equation 1 in the CONDOR paper (i.e. we used the median):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cum_probs = pd.DataFrame(preds).apply(special.expit).cumprod(axis=1)\n",
    "labels_v2 = cum_probs.apply(lambda x: x > 0.5).sum(axis = 1)\n",
    "print(\"Accuracy of label version 2:\", np.mean(labels_v2 == labels_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XYKpSMosDMuq"
   },
   "source": [
    "#### Evaluate mean absolute label error\n",
    "\n",
    "This is effectively an ordinal version of 1 - accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qJZNCzwfGqC0"
   },
   "outputs": [],
   "source": [
    "# These do not correspond with what we get from the model evaluation. Something must be off in one of these.\n",
    "print(\"Mean absolute label error version 1:\", np.mean(np.abs(labels_v1 - labels_test)))\n",
    "print(\"Mean absolute label error version 2:\", np.mean(np.abs(labels_v2 - labels_test)))\n",
    "\n",
    "print(\"Root mean squared label error version 1:\", np.sqrt(np.mean(np.square(labels_v1 - labels_test))))\n",
    "print(\"Root mean squared label error version 2:\", np.sqrt(np.mean(np.square(labels_v2 - labels_test))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ddSncBedI37-"
   },
   "outputs": [],
   "source": [
    "# Review how absolute error is calculated for ordinal labels:\n",
    "pd.DataFrame({\"true\": labels_test, \"pred_v2\": labels_v1, \"abs\": labels_v2 - labels_test}).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dUyJw-20B2AU"
   },
   "source": [
    "### Universal Sentence Encoder model (using pre-encoded labels for faster processing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GnCkqHv5CA_x"
   },
   "source": [
    "The \"Sparse\" versions of the CONDOR API are convenient and implementing them requires minimal changes to existing code. However there is a performance overhead compared to if we pre-encode the labels using CONDORs ordinal encoder method. This is because the sparse API is esssentially encoding on-the-fly inside the training loop rather than doing up-front.\n",
    "\n",
    "Furthermore, as we will see later, the labels do not always come encoded as 0,1,...,K-1. In these cases, using the CondorOrdinalEncoder will help transform labels into ordinal-ready values.\n",
    "\n",
    "In the code that follows we will implement up-front ordinal encoding of the labels using CONDOR's built-in functionality:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Rq1aA2ffB2Ai"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "enc = condor.CondorOrdinalEncoder(nclasses=num_classes)\n",
    "enc_labs_train = enc.fit_transform(labels_train)\n",
    "enc_labs_test = enc.transform(labels_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can compile the model.  Note that since we have pre-encoded the labels, we no longer use the 'Sparse' loss functions and metrics.  Rather we use corresponding versions that are designed for use with encoded labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ahRP7kkPB2Ai"
   },
   "outputs": [],
   "source": [
    "model.compile(loss = condor.CondorOrdinalCrossEntropy(),\n",
    "              metrics = [condor.OrdinalEarthMoversDistance(),\n",
    "                         condor.OrdinalMeanAbsoluteError()],\n",
    "              optimizer = tf.keras.optimizers.Adam(learning_rate = 0.001))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will train the model.  Note that we pass it the encoded labels this time around:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Fb2NU9kVB2Ai"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "history = model.fit(x = text_train,\n",
    "                    y = enc_labs_train,\n",
    "                    epochs = 5,\n",
    "                    batch_size = 32, \n",
    "                    validation_split = 0.2,\n",
    "                    callbacks = [tf.keras.callbacks.EarlyStopping(patience = 2,\n",
    "                                                                  min_delta = 0.001,\n",
    "                                                                  restore_best_weights = True)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally evaluate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OWBl7PuZB2Ai"
   },
   "outputs": [],
   "source": [
    "model.evaluate(text_test, enc_labs_test) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pre-encoding requires a little extra code, but it runs quickly and so the savings later will often be worth it.  Now you can caclulate accuracies etc like we did previously.\n",
    "\n",
    "You have now successfully implemented CONDOR on the Amazon review data using the Universal Sentence Encoder model.  Congratulations! You could stop here, or alterantively keep reading to learn more about the capabilities of CONDOR's ordinal encoder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xt9mAOqvE4Ep"
   },
   "source": [
    "## More examples of label encoding capabilities\n",
    "\n",
    "Here we further demonstrate some features of the ordinal encoder.\n",
    "\n",
    "First we pass a numpy array of classes to the ordinal encoder.  The encoder automatically determines how many classes there are and then orders them in the default sklearn OrdinalEncoder fashion (alphabetically in this case):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eGcYAoi0E_FI"
   },
   "outputs": [],
   "source": [
    "labels = np.array(['a','b','c','d','e'])\n",
    "enc_labs = condor.CondorOrdinalEncoder().fit_transform(labels)\n",
    "print(enc_labs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time we do the same, but using a basic list of labels in place of the numpy array from the previous example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9YuSAf1wFgAq"
   },
   "outputs": [],
   "source": [
    "labels = ['a','b','c','d','e']\n",
    "enc_labs = condor.CondorOrdinalEncoder().fit_transform(labels)\n",
    "\n",
    "print(enc_labs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we wish to specify that the order should be different from alphabetical. We do so by explicitly passing the category labels to the ordinal encoder, in order.  Note this would also allow \"missing\" categories to be included in proper order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FkzccbsjFwdv"
   },
   "outputs": [],
   "source": [
    "labels = ['low','med','high']\n",
    "enc = condor.CondorOrdinalEncoder(categories=[['low', 'med', 'high']])\n",
    "enc_labs = enc.fit_transform(labels)\n",
    "\n",
    "print(enc_labs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This handful of examples demonstrates all of the key behavior of the CONDOR ordinal encoder.  These and the MNIST and Amazon examples  above should provide you with all you need to get started implementing CONDOR in your models!\n",
    "\n",
    "### Good luck!"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "CONDOR TensorFlow demo.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}