GitGuardian-API-Automation-Examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6e0b3326",
   "metadata": {},
   "source": [
    "# Automate GitGuardian Worflows With APIs\n",
    "\n",
    "The real power of working with APIs is the ability to automate any workflow you can imagine...\n",
    "\n",
    "Note: this python code requires that [`requests`](https://pypi.org/project/requests/) be installed in the environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d865687",
   "metadata": {},
   "source": [
    "## Automation 1\n",
    "\n",
    "### Auto-assign the newest incident to a random member of the workspace.\n",
    "\n",
    "If you have just finished a historical scan or are working with a very large team that is constantly pushing code, then there is a good chance you will have discovered a lot of hardcoded credentials. These will show up in your GitGuardian dashboard as incidents. \n",
    "\n",
    "While you can manually assign these incidents to your team through the dashboard, one at a time, or multiple at once, with a little logic, you can assign them to a member of the team based on any criteria you choose. \n",
    "\n",
    "To demonstrate the concept, this example will simply assign the most recent triggered incident to a random member of the team.\n",
    "\n",
    "This reference automation: \n",
    "- Checks for unassigned, triggered incidents\n",
    "- Assigns the most recent incident to a random team member\n",
    "- Pre-builds the needed sharing link, just in case we need to involve the developer\n",
    "- Prints the updated incident to the screen for review\n",
    "\n",
    "*Note: You need to have Manager or Owner level access to be able to share incidents*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43534d06",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import authorization\n",
    "import json\n",
    "import random\n",
    "\n",
    "# Let's build our header with the bearer token to be used with each API call\n",
    "auth_headers = {'Authorization' : 'Token {}'.format(authorization.token) }\n",
    "\n",
    "# Get our current list of TRIGGERED incidents, listed with the most recent first.\n",
    "\n",
    "request_filters = {\n",
    "    \"ordering\":\"-date\", \n",
    "    \"status\":\"TRIGGERED\"\n",
    "}\n",
    "\n",
    "response = requests.get(\n",
    "    \"http://api.gitguardian.com/v1/incidents/secrets\", \n",
    "    request_filters, \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Store this JSON data in a var for ease of coding\n",
    "incident_data = response.json()\n",
    "\n",
    "# If there are no more incidents, we are done. \n",
    "if len(incident_data) <= 0:\n",
    "    print(\"There are no more incidents to assign.\")\n",
    "    exit(keep_kernel=True)\n",
    "else:\n",
    "    # Assuming there is one, let's get the first incident's id \n",
    "    first_incident_id = incident_data[0]['id']\n",
    "\n",
    "# To build our list of workspace members, we need a list to store the results    \n",
    "user_list = []\n",
    "# And a list of roles we want. We can only call the API for one role at a time.\n",
    "roles_to_retrieve = [\"manager\", \"owner\"]\n",
    "\n",
    "for role in roles_to_retrieve:\n",
    "    # Grab a random workspace member's id with `role` so we can assign them this incident\n",
    "    response = requests.get(\n",
    "        \"https://api.gitguardian.com/v1/members\", \n",
    "        {\"per_page\":\"100\", \"role\": role},\n",
    "        headers=auth_headers\n",
    "        )\n",
    "    # Build the list of user IDs. \n",
    "    for m in response.json():\n",
    "        user_list.append(m['id'])\n",
    "\n",
    "# print(user_list)\n",
    "\n",
    "# This if-else statement accounts for orgs of 1 member \n",
    "# print(len(user_list))\n",
    "if len(user_list) == 1:\n",
    "    assignee_member_id = user_list[0]\n",
    "else:\n",
    "    assignee_member_id = user_list[random.randint(0, (len(user_list)- 1))]\n",
    "\n",
    "\n",
    "## Assign the incident to the selected member of the org\n",
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/assign\".format(first_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"member_id\" : assignee_member_id}\n",
    "    )\n",
    "\n",
    "# Get the sharing URL and print it\n",
    "# It might already exist, for our incident, so let's check that first before we send another API call\n",
    "\n",
    "if incident_data[0]['share_url']:\n",
    "    print(\"You already shared this URL\")\n",
    "    print(\"Sharing URL is: \" + str(incident_data[0]['share_url']))\n",
    "else:\n",
    "    response = requests.post(\"https://api.gitguardian.com/v1/incidents/secrets/{}/share\".format(first_incident_id), headers=auth_headers, data = {\"auto_healing\" : \"true\"})\n",
    "    print(\"Sharing URL is: \" + str(response.json()['share_url'])) \n",
    "\n",
    "\n",
    "# And finally, let's show ALL the now updated details about our incident\n",
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}\".format(first_incident_id),\n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "719378a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ### This is an unassign call, for use when testing\n",
    "response = requests.post(\"https://api.gitguardian.com/v1/incidents/secrets/{}/unassign\".format(first_incident_id), headers=auth_headers, data = {\"member_id\" : assignee_member_id})\n",
    "print(json.dumps(response.json(), indent=2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d415326e",
   "metadata": {},
   "source": [
    "## Automation 2\n",
    "\n",
    "### Auto-assign incident to commit author if they are a member of the workspace.\n",
    "\n",
    "There might be times, especially on a smaller team, when the developer who triggered a new incident is also a member of the workspace who can best handle the resolution. \n",
    "\n",
    "While you can easily look this up manually through the dashboard, you can also leverage a little coding to do the lifting for you.\n",
    "\n",
    "This automation:\n",
    "- Checks for the most recent incident\n",
    "- Builds a list of workspace member emails\n",
    "- Checks to see if the commit author email is also in the member list\n",
    "    - If it is, it auto-assigns to that person\n",
    "    - Otherwise, it tells us that the author is not a member of the workspace\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41f5f48f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Uncomment the following 4 lines if you did not run the previous automation\n",
    "# import requests\n",
    "# import authorization\n",
    "# import json\n",
    "# auth_headers = {'Authorization' : 'Token {}'.format(authorization.token) }\n",
    "\n",
    "# Get our current list of TRIGGERED incidents, listed with the most recent first.\n",
    "request_filters = {\n",
    "    \"ordering\":\"-date\", \n",
    "    \"status\":\"TRIGGERED\"\n",
    "}\n",
    "response = requests.get(\n",
    "    \"http://api.gitguardian.com/v1/incidents/secrets\", \n",
    "    request_filters, \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Let's store this JSON data in a var for ease of coding\n",
    "incident_data = response.json()\n",
    "\n",
    "# If there are none, we are done. We can check the number of items with len()\n",
    "if len(incident_data) <= 0:\n",
    "    print(\"There are no more incidents to assign.\")\n",
    "    exit(keep_kernel=True)\n",
    "else:\n",
    "    # Assuming there is one, let's get the first incident's id \n",
    "    first_incident_id = incident_data[0]['id']\n",
    "\n",
    "# Build our list of workspace member's emails\n",
    "# note: if your list of users is larger than 100, not everyone will not be on this list.\n",
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/members\",\n",
    "    {\"per_page\":\"100\"},\n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Build the list of user IDs. \n",
    "user_email_list = []\n",
    "for m in response.json():\n",
    "    user_email_list.append(m['email'])\n",
    "\n",
    "# Get the email of the committer from the incident data\n",
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}\".format(first_incident_id), \n",
    "    headers=auth_headers\n",
    "    )\n",
    "comitter_email = response.json()['occurrences'][0]['author_info']\n",
    "\n",
    "# Let's check that the email address does exist and matches one of the emails in our workspace member list\n",
    "if comitter_email and any(comitter_email in email for email in user_email_list):\n",
    "    # If yes: assign the incident to the user via their email address\n",
    "    response = requests.post(\n",
    "        \"https://api.gitguardian.com/v1/incidents/secrets/{}/assign\".format(first_incident_id), \n",
    "        headers=auth_headers, \n",
    "        data = {\"email\" : comitter_email}\n",
    "        )\n",
    "    # print the updated incident information\n",
    "    print(\"This incident has been assigned to \"+str(comitter_email)+\" who also made the commit\")\n",
    "    print(json.dumps(response.json(), indent=4))\n",
    "        \n",
    "else:\n",
    "    print('The commit author is not a member of the workspace. No update to the incident has been made.')    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "975e4035",
   "metadata": {},
   "source": [
    "## Automation 3\n",
    "\n",
    "### Ignore an incident and mark it as a 'test credential' if all occurrences of a secret are in a /test/ directory and invalid.\n",
    "\n",
    "While it is _ALWAYS_ a good idea to manually verify that credentials in /test/ folders are really test credentials, if it is a credential you know is not valid, then you might choose to ignore them.\n",
    "\n",
    "This automation:\n",
    "- Checks for the most recent incident\n",
    "- Checks to see if: \n",
    "    - The incident is marked 'invalid'\n",
    "    - All occurrences were inside paths that contain '/test'\n",
    "        - If those 2 things are true - marks the incident as ignored with ignore_reason set to \"test_credential\"\n",
    "- Prints the updated incident for review\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb575d32",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Uncomment the following 4 lines if you did not run the previous automation\n",
    "import requests\n",
    "import authorization\n",
    "import json\n",
    "auth_headers = {'Authorization' : 'Token {}'.format(authorization.token) }\n",
    "\n",
    "# Get the current list of TRIGGERED incidents, listed with the most recent first.\n",
    "request_filters = {\n",
    "    \"ordering\":\"-date\", \n",
    "    \"status\":\"TRIGGERED\"\n",
    "}\n",
    "response = requests.get(\n",
    "    \"http://api.gitguardian.com/v1/incidents/secrets\", \n",
    "    request_filters, \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Let's store this JSON data in a var for ease of coding\n",
    "incident_data = response.json()\n",
    "\n",
    "# If there are none, we are done. We can check the number of items with len()\n",
    "if len(incident_data) == 0:\n",
    "    print(\"There are no more incidents to assign.\")\n",
    "    exit()\n",
    "else:\n",
    "    # Assuming there is one, let's get the first incident's id \n",
    "    first_incident_id = incident_data[0]['id']    \n",
    "    \n",
    "# Get the full incident data, including occurrence level details\n",
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}\".format(first_incident_id), \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Check if the secret is `invalid`\n",
    "# If it is, make a list of the filepaths for each occurrence\n",
    "\n",
    "filepath_list = []\n",
    "if response.json()['validity'] != \"invalid\":\n",
    "    print(\"The secrets are not marked as invalid!\")\n",
    "else:\n",
    "    for m in response.json()['occurrences']:\n",
    "        filepath_list.append(m['filepath'])\n",
    "    print(filepath_list)\n",
    "\n",
    "    # Check that the last step built a populated list and all the paths contain `/test/`\n",
    "    # and if that passes - let's mark the incident as ignored \n",
    "    if filepath_list and all('/test/' in filepath for filepath in filepath_list):\n",
    "        response = requests.post(\n",
    "            \"https://api.gitguardian.com/v1/incidents/secrets/{}/ignore\".format(first_incident_id), \n",
    "            headers=auth_headers, \n",
    "            data = {\"ignore_reason\" : \"test_credential\"}\n",
    "            )\n",
    "        print(json.dumps(response.json(), indent=4))\n",
    "    else:\n",
    "        print(\"Could not verify all occurrences were in /test/ folders\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b725c06b",
   "metadata": {},
   "source": [
    "## What is next??\n",
    "\n",
    "### You are only limited by your imagination and code\n",
    "\n",
    "I hope this notebook has been helpful in understanding what is possible with the GitGuardian API.  \n",
    "While this example used Python and is in a stand-alone Jupyter Notebook format, the API can be invoked with whatever language you choose to use and be embedded in any system where you can inject code. \n",
    "\n",
    "If you have other ideas for automations you want to see in this repository, I encourage you to make a pull request or open an issue. \n",
    "\n",
    "Always feel free to reach out to us at contact@gitguardian.com\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5c5c72c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}