1.2_taxonomies_djid.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7840c98c-1180-44fa-8be5-9e8039c0ace5",
   "metadata": {},
   "source": [
    "# Taxonomies - Dow Jones Intellingent Indentifiers (DJID)\n",
    "News content in Dow Jones is classificed across multiple taxonomies like `regions`, `industries`, `subjects` (aka topics), `companies`, `executives` among others.\n",
    "\n",
    "The current notebook shows how to interact with these taxonomies to convert codes to human-readable values or viceversa.\n",
    "\n",
    "In this notebook...\n",
    "* [Dependencies and Initialisation](#dependencies-and-initialisation)\n",
    "* [The Taxonomy Class](#the-taxonomy-class)\n",
    "* [Loading Taxonomies into Pandas DataFrames](#loading-taxonomies-into-pandas-dataframes)\n",
    "* [Export to a file](#export-to-a-file)\n",
    "* [Next Steps](#next-steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a607f00-fac3-44d0-b19e-3ecca4c528b6",
   "metadata": {},
   "source": [
    "## Dependencies and Initialisation\n",
    "Import statements and environment initialisation using the package `dotenv`. More details in the [Configuration notebook](0.2_configuration.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1d7f4a00-b889-4da7-9e3b-81d4ae4484d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "from factiva.news import Taxonomy\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()\n",
    "print('Done!')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "845f332a-6578-4a7d-bb71-aa1ffe05bdcf",
   "metadata": {},
   "source": [
    "## The Taxonomy Class\n",
    "\n",
    "The taxonomy class interacts with the API and thus, requires to authenticate by using the user_key parameter. By default it is taken from the environment variable `FACTIVA_USERKEY`.\n",
    "\n",
    "To get started, show the list of available taxonomies using the attribute `categories`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a0f2ca9f-adaa-4708-8fc1-f70769878e38",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['news_subjects',\n",
       " 'regions',\n",
       " 'hierarchyRegion',\n",
       " 'companies',\n",
       " 'hierarchyIndustry',\n",
       " 'industries',\n",
       " 'executives',\n",
       " 'hierarchySubject']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Taxonomy()\n",
    "t.categories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2759241a-8b61-4f70-be69-b1dbd0e49bf4",
   "metadata": {},
   "source": [
    "## Loading Taxonomies into Pandas DataFrames\n",
    "Simply assign the output of `get_category_codes(category:str)` to a variable.\n",
    "\n",
    "```Python\n",
    "hr = t.get_category_codes('industries')\n",
    "```\n",
    "\n",
    ">**Note**: The category `executives` returns a big dataset that may kill the Python kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0dbde025-c859-4d04-b835-5d0811f0f27e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(960, 1)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>code</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>i3dprn</th>\n",
       "      <td>3D/4D Printing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>i246</th>\n",
       "      <td>Abrasive Products</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>i4752105</th>\n",
       "      <td>Academic/Scientific/Trade Journals</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>i836</th>\n",
       "      <td>Accounting</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>iacc</th>\n",
       "      <td>Accounting/Consulting</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 description\n",
       "code                                        \n",
       "i3dprn                        3D/4D Printing\n",
       "i246                       Abrasive Products\n",
       "i4752105  Academic/Scientific/Trade Journals\n",
       "i836                              Accounting\n",
       "iacc                   Accounting/Consulting"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "taxonomy_name = 'industries'\n",
    "tc = t.get_category_codes(taxonomy_name)\n",
    "print(tc.shape)\n",
    "tc.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b41b80f",
   "metadata": {},
   "source": [
    "## Export to a file\n",
    "For convenience when working in other systems or loading a dictionary to a database table, the Pandas `DataFrame.to_*` options are a useful operation to export the loaded data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "1f9bf14e-231c-4a8a-a478-8474d2eddd40",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save to CSV\n",
    "tc.to_csv(f'./data/{taxonomy_name}.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "31c8a1ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save to JSON\n",
    "tc.to_json(f'./data/{taxonomy_name}.json', orient='table')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c65f57ae",
   "metadata": {},
   "source": [
    "# Next Steps\n",
    "\n",
    "* Understand how [companies](1.3_company_identifiers.ipynb) are identified and how those can be mapped to a predefined list.\n",
    "* Run an [explain](1.4_snapshot_explain.ipynb) to get a quick idea about how to query the archive.\n",
    "* Explore the available fields in the archive and how to apply conditions on them according to the [query reference](2.1_complex_large_queries.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}