docs/tutorials/decompositions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor Decompositions\n",
    "\n",
    "The philosophy of *tntorch* is simple: **one class for all formats**. [Different decompositions](main_formats.ipynb) (CP, Tucker, TT, hybrids) all use the same interface. \n",
    "\n",
    "*Note: sometimes the internal format will change automatically. For example, no recompression algorithm is known for the CP format, and running `round()` on a CP tensor will convert it to the TT format.*\n",
    "\n",
    "We will give a few examples of how to compress a full tensor into different tensor formats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([128, 128, 128])\n"
     ]
    }
   ],
   "source": [
    "import tntorch as tn\n",
    "import torch\n",
    "torch.set_default_dtype(torch.float64)\n",
    "import time\n",
    "\n",
    "import numpy as np\n",
    "X, Y, Z = np.meshgrid(range(128), range(128), range(128))\n",
    "full = torch.Tensor(np.sqrt(np.sqrt(X)*(Y+Z) + Y*Z**2)*(X + np.sin(Y)*np.cos(Z)))  # Some analytical 3D function\n",
    "print(full.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TT\n",
    "\n",
    "To compress as a low-rank tensor train (TT), use the `ranks_tt` argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D TT tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   3   3   1\n",
      "\n",
      "Compression ratio: 2097152/1920 = 1092.27\n",
      "Relative error: tensor(0.0005)\n",
      "RMSE: tensor(22.0745)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full, ranks_tt=3)  # You can also pass a list of ranks\n",
    "\n",
    "def metrics():\n",
    "    print(t)\n",
    "    print('Compression ratio: {}/{} = {:g}'.format(full.numel(), t.numcoef(), full.numel() / t.numcoef()))\n",
    "    print('Relative error:', tn.relative_error(full, t))\n",
    "    print('RMSE:', tn.rmse(full, t))\n",
    "    print('R^2:', tn.r_squared(full, t))\n",
    "    \n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The TT cores are available as `t.cores`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tucker\n",
    "\n",
    "Use the `ranks_tucker` argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D TT-Tucker tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      "  3   3   3\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   9   3   1\n",
      "\n",
      "Compression ratio: 2097152/1269 = 1652.6\n",
      "Relative error: tensor(0.0005)\n",
      "RMSE: tensor(22.0752)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full, ranks_tucker=3) \n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even though technically a TT-Tucker tensor, it has the [exact same expressive power](main_formats.ipynb) as a low-rank Tucker decomposition.\n",
    "\n",
    "The Tucker factors are `t.Us`. To retrieve the full Tucker core, use `tucker_core()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([3, 3, 3])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.tucker_core().shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CP\n",
    "\n",
    "Use the `ranks_cp` argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ALS -- initialization time = 0.045638084411621094\n",
      "iter: 0  | eps: 0.00098631 | total time:    0.0682\n",
      "iter: 1  | eps: 0.00092816 | total time:    0.0896 <- converged (tol=0.0001)\n",
      "3D CP tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      " <0> <1> <2>\n",
      " / \\ / \\ / \\\n",
      "3   3   3   3\n",
      "\n",
      "Compression ratio: 2097152/1152 = 1820.44\n",
      "Relative error: tensor(0.0009)\n",
      "RMSE: tensor(39.9936)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full, ranks_cp=3, verbose=True)  # CP is computed using alternating least squares (ALS)\n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CP factors are `t.cores` (they are all 2D tensors).\n",
    "\n",
    "## Hybrid Formats\n",
    "\n",
    "`ranks_tucker` can be combined with the other arguments to produce hybrid decompositions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D CP-Tucker tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      "  3   3   3\n",
      " <0> <1> <2>\n",
      " / \\ / \\ / \\\n",
      "3   3   3   3\n",
      "\n",
      "Compression ratio: 2097152/1179 = 1778.75\n",
      "Relative error: tensor(0.0035)\n",
      "RMSE: tensor(149.4028)\n",
      "R^2: tensor(1.0000)\n",
      "3D TT-Tucker tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      "  4   4   4\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   2   2   1\n",
      "\n",
      "Compression ratio: 2097152/1568 = 1337.47\n",
      "Relative error: tensor(0.0012)\n",
      "RMSE: tensor(51.8083)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full, ranks_cp=3, ranks_tucker=3)\n",
    "metrics()\n",
    "t = tn.Tensor(full, ranks_tt=2, ranks_tucker=4)\n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Error-bounded Decompositions\n",
    "\n",
    "If you instead pass the argument `eps`, a decomposition will be computed that will not exceed that relative error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D TT-Tucker tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      "  4   5   6\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   4   6   1\n",
      "\n",
      "Compression ratio: 2097152/2092 = 1002.46\n",
      "Relative error: tensor(8.3402e-06)\n",
      "RMSE: tensor(0.3594)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full, eps=1e-5)\n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That will always try to compress in both Tucker and TT senses, and therefore will always produce a TT-Tucker tensor. If you only want to compress, say, in the Tucker sense, you can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D TT-Tucker tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      "  5   4   7\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   28  7   1\n",
      "\n",
      "Compression ratio: 2097152/3021 = 694.191\n",
      "Relative error: tensor(4.0447e-06)\n",
      "RMSE: tensor(0.1743)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full)\n",
    "t.round_tucker(eps=1e-5)\n",
    "metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And conversely, for a TT-only compression:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3D TT tensor:\n",
      "\n",
      " 128 128 128\n",
      "  |   |   |\n",
      " (0) (1) (2)\n",
      " / \\ / \\ / \\\n",
      "1   4   6   1\n",
      "\n",
      "Compression ratio: 2097152/4352 = 481.882\n",
      "Relative error: tensor(8.3358e-06)\n",
      "RMSE: tensor(0.3592)\n",
      "R^2: tensor(1.0000)\n"
     ]
    }
   ],
   "source": [
    "t = tn.Tensor(full)\n",
    "t.round_tt(eps=1e-5)\n",
    "metrics()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}