API-Remediation-Workflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f8df52fc",
   "metadata": {},
   "source": [
    "# Secret Leakage Incident Remediation Using GitGuardian APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b524623",
   "metadata": {},
   "source": [
    "## Introduction And Format\n",
    "\n",
    "This Jupyter Notebook project is intended to introduce you to using the [GitGuardian API](https://docs.gitguardian.com/api-docs/introduction) from the perspective of remediating an incident.  \n",
    "\n",
    "While we cover many of the API calls available, it does not touch on all of them. For the full list, see the full [API reference documentation](https://api.gitguardian.com/docs).\n",
    "\n",
    "A note if you are new to [Jupyter Notebooks](https://jupyter.org/): Each code block is run on demand, but are all run within the same program, sharing the same memory. This means variables set in one block are callable from each following block. If you get output that says something is undefined, make sure you ran the block that defines it. \n",
    "\n",
    "### Format for API calls\n",
    "\n",
    "- The base URL for the latest version is api.gitguardian.com/v1 over HTTPS.\n",
    "- All data is sent and received as JSON.\n",
    "- All timestamps returned are ISO-8601 compliant, example: 2020-03-16T04:46:00+00:00 # for date-time\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c25dbc6",
   "metadata": {},
   "source": [
    "## 1. Authentication\n",
    "\n",
    "In order to use the GitGuardian API, you will need to create a [new personal access token](https://docs.gitguardian.com/api-docs/personal-access-tokens). See the documentation for more details. \n",
    "\n",
    "Steps to create and use a Personal Access Token for this notebook:\n",
    "- Go to your GitGuardian dashboard and click on API, then select the Person Access Tokens menu.\n",
    "- Create a new access token with the selected scope: \n",
    "    - scan \n",
    "    - incidnet:read\n",
    "    - incidnet:write\n",
    "    - incidnet:share\n",
    "    - members:read\n",
    "- Copy the new token to the clipboard. \n",
    "- *_Do not_* paste it into this document.\n",
    "- Create a new file in your project called \"authorization.py\" and add the following line and save the file:\n",
    "    - token = 'Paste-Your-GG-API-Token-Here'\n",
    "- Double check your `.gitignore` file to make sure this stays out of your source control. \n",
    "\n",
    "- ALTERNATIVELY - set up and use Hashicorp Vault or another such credentials manager. This is overall a better path, but let's keep things simple for this example repo.\n",
    "\n",
    "GitGuardian relies on each request to have a header that contains a bearer token. This is the first thing we will build with the code block below. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3ea10cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "## In this first section we authenticate\n",
    "## We will need these tools throughout the examples, so let's import them all now\n",
    "import requests\n",
    "import authorization\n",
    "import json\n",
    "\n",
    "# This line builds our header to include with each call to the API\n",
    "auth_headers = {'Authorization' : 'Token {}'.format(authorization.token) }\n",
    "\n",
    "# Before we attempt anything fancier, let's make sure the API is working and our authorization token worked. \n",
    "response = requests.get(\"https://api.gitguardian.com/v1/health\", headers=auth_headers)\n",
    "print(response)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c41005ed",
   "metadata": {},
   "source": [
    "If you see a `<Response [200]>` above, then you are good to go!\n",
    "If you see some other code, check to make sure your Access Token is valid or that your GitGuardian dashboard is available. See the full [GitGuardian API reference documentation](https://api.gitguardian.com/docs) for help with other response codes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3394f966",
   "metadata": {},
   "source": [
    "## List Incidents\n",
    "\n",
    "While you can always look up the list of incidents in the dashboard and apply many different filters, you can also programmatically call the platform to get a list of incidents.  \n",
    "\n",
    "While there are many ways to sort incidents, one convenient way to approach them is by status. \n",
    "\n",
    "There are [4 possible states for an incident](https://docs.gitguardian.com/internal-repositories-monitoring/remediate/prioritize-incidents#1-prioritize-with-the-incidents-table):\n",
    "    - Triggered\n",
    "    - Assigned\n",
    "    - Ignored \n",
    "    - Resolved\n",
    "\n",
    "Let's see if any triggered, yet-to-be-assigned incidents exist in your workspace.\n",
    "\n",
    "To filter for just the triggered incidents, you will add `{\"status\":\"TRIGGERED\"}` in the API request, after the URL. This is true of all filters you want to apply later on. Note: It is case-sensitive.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2edfbfd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(\n",
    "    \"http://api.gitguardian.com/v1/incidents/secrets\", \n",
    "    {\"status\" : \"TRIGGERED\"}, \n",
    "    headers=auth_headers\n",
    "    )\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "print(\"Total retrieved triggered incidents(not assigned): \"+ str(len(response.json())))\n",
    "print(\"By default, GitGuardian only returns 20 items per page of results.\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e60c083",
   "metadata": {},
   "source": [
    "If you have no triggered incidents, good job! That means you are on top of your secrets management game! \n",
    "\n",
    "But, for the purposes of this example application, we need a few to exist.  One safe way to do this is to reopen incidents that have invalid secrets and are not public.  You can do this from the GitGuardian dashboard. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf53598e",
   "metadata": {},
   "source": [
    "## Listing just the most recent incidents\n",
    "\n",
    "If you have a lot of incidents, then you are likely going to want to filter this list further. \n",
    "\n",
    "Here is an example of filtering for:\n",
    "- status=TRIGGERED - all incidents that have a status of TRIGGERED \n",
    "- ordering=-date - in an ordered list is shown descending from the most recent event\n",
    "- (optional) date_after=2023-03-01 - incidents that happened after a date YYYY-MM-DD. If you have a lot of incidents this is one way you can narrow your scope. \n",
    "\n",
    "Later in this notebook, we will want to reference the first listed triggered event, so let's save it in a variable when we run this code block.\n",
    "\n",
    "Here, instead of defining the filters inline with the URL, let's pre-define them for readability reasons\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e06af37",
   "metadata": {},
   "outputs": [],
   "source": [
    "request_filters = {\n",
    "#     \"date_after\":\"2023-03-01\",\n",
    "    \"ordering\":\"-date\", \n",
    "    \"status\":\"TRIGGERED\"\n",
    "}\n",
    "\n",
    "response = requests.get(\n",
    "    \"http://api.gitguardian.com/v1/incidents/secrets\", \n",
    "    request_filters, \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "# Let's save the ID of the first listed, newest triggered incident for later.\n",
    "my_incident_id = response.json()[0]['id']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "544c7103",
   "metadata": {},
   "source": [
    "## Assigning incidents\n",
    "\n",
    "Let's assign the most recent incident to someome. In this case, you.\n",
    "\n",
    "We will need to make our first POST command. To do that, we need to build a payload that consists of 2 things:\n",
    "\n",
    "1. incident_id - The id of the incident to retrieve\n",
    "\n",
    "And either one of the following:\n",
    "\n",
    "    2. email - email of the member to assign. This parameter is mutually exclusive with member_id.\n",
    "       or\n",
    "    3. member_id - id of the member to assign. This parameter is mutually exclusive with email.\n",
    "\n",
    "I trust you know your `email`. But how can we look up a workspace member's `email` or `member_id`? \n",
    "\n",
    "With the following API call!\n",
    "\n",
    "If you own the GitGuardian workspace you are authenticating to, then you will be listed first. The following code saves the ID of the first user listed in the variable `my_member_id`. You can reassign this variable later, so feel free to comment that line out below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad56c067",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/members\", \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "\n",
    "# Let's save the first listed member in a variable for later\n",
    "my_member_id = response.json()[0]['id']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c442d09d",
   "metadata": {},
   "source": [
    "### Making the assignment call\n",
    "\n",
    "Now you have the \n",
    "- `incident_id` \n",
    "- `member_id`, \n",
    "    - though we could use the `email` if we wanted to, \n",
    "\n",
    "Let's build a request to assign the first listed incident to a user using the member ID, which we found in the previous step.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "254f08fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you do not want to use the variables we assigned earlier\n",
    "# uncomment the appropriate line below and fill in with your pereferred IDs\n",
    "# my_incident_id = \"YOUR_ID\"\n",
    "# my_member_id = \"YOUR_ID\"\n",
    "\n",
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/assign\".format(my_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"member_id\" : my_member_id}\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "372046b4",
   "metadata": {},
   "source": [
    "## Unassign an incident\n",
    "\n",
    "You can also unassign incidents, which puts them back to 'triggered' status if they have not yet been resolved.\n",
    "\n",
    "The following block can be run as is since it uses the variables we set earlier. It will unassign the incident we just assigned above. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abd9a302",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/unassign\".format(my_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"member_id\" : my_member_id}\n",
    "    )\n",
    "\n",
    "# Print the updated incident data\n",
    "print(json.dumps(response.json(), indent=4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63632ed0",
   "metadata": {},
   "source": [
    "## Examining an incident\n",
    "\n",
    "Now that you have been assigned the incident let's take a closer look at the incident data. \n",
    "\n",
    "When you listed the incidents, you only revealed the top layer of available data. This is equivalent to the 'Incidents view' in the dashboard. Just as in the dashboard, you need to open the view of each specific incident to reveal more details, including occurrence information; you need to drill down into the `incidents/secrets` view with the API. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff8d5f98",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}\".format(my_incident_id), \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "759ae2d9",
   "metadata": {},
   "source": [
    "## Making a decision about remediating the incident\n",
    "\n",
    "Now you have to decide:\n",
    "\n",
    "1. Do you have enough information to resolve or ignore this incident?\n",
    "\n",
    "or\n",
    "\n",
    "2. Do you need to get more information from the committing author involved?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2261c030",
   "metadata": {},
   "source": [
    "### Scenario 1: You have enough information to resolve or ignore this incident\n",
    "\n",
    "Now that you know you can close this incident, the next decision is to `resolve` or to `ignore` the incident. \n",
    "\n",
    "#### Resolving incidents\n",
    "\n",
    "Let's first resolve it. \n",
    "\n",
    "In order to resolve an incident, we need 2 things: \n",
    "1. `incident_id` - we already set this above as my_incident_id.\n",
    "2. `secret_revoked` - this is a boolean we pass in as data; it is either true or false.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d71398e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/resolve\".format(my_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"secret_revoked\" : \"false\"}\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dc10e17",
   "metadata": {},
   "source": [
    "### Reopening incidents\n",
    "\n",
    "You can also re-open incidents, all you need is the `incident_id`.\n",
    "\n",
    "The following reopens the incident you just resolved. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c806e654",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/reopen\".format(my_incident_id), \n",
    "    headers=auth_headers\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f6e3a8c",
   "metadata": {},
   "source": [
    "#### Ignoring Incidents\n",
    "\n",
    "You can also choose to ignore incidents.\n",
    "\n",
    "Yet again, we need the `incident_id`\n",
    "\n",
    "But we also need an `ignore_reason`\n",
    "This has three possible states, expressed as strings\n",
    "- \"test_credential\"\n",
    "- \"false_positive\"\n",
    "- \"low_risk\"\n",
    "\n",
    "Let's ignore this one as `low_risk`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c0e9d22",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/ignore\".format(my_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"ignore_reason\" : \"low_risk\"}\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df7ab1dc",
   "metadata": {},
   "source": [
    "### Scenario 2. I need to get more information from the committing author involved\n",
    "\n",
    "You get more information from the developer involved by enabling sharing and sending the generated [sharing URL](https://docs.gitguardian.com/internal-repositories-monitoring/remediate/remediate-incidents#collaborating-with-developers) to the developer involved. \n",
    "*Remeber to treat these URLs as you would any other secret. The generated page will expose the original secret involved in the incident.*\n",
    "\n",
    "When requesting the share URL you have some options: \n",
    "- auto_healing - Default: false - Allow the developer to resolve or ignore through the share link\n",
    "- feedback_collection\t- Default: true - Allow the developer to submit their feedback through the share link\n",
    "- lifespan - Default: 0 - Lifespan, in hours, of the share link. If 0 or unset, a default value will be applied based on the workspace settings.\n",
    "\n",
    "For this example, leave `feedback_collection` and `lifespan` as their defaults and set `auto_healing` to `true`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49cf165c",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/incidents/secrets/{}/share\".format(my_incident_id), \n",
    "    headers=auth_headers, \n",
    "    data = {\"auto_healing\" : \"true\"}\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c553192",
   "metadata": {},
   "source": [
    "## Congratulations \n",
    "\n",
    "You have now used the API to :\n",
    "\n",
    "- List incidents\n",
    "- Find a workspace member's ID and emails\n",
    "- Assign an incidnet\n",
    "- Drill in see all the incident information, including the occurance data\n",
    "- Resolved the incident in 2 different ways\n",
    "\n",
    "I encourage you to experiment with available filters and leverage the full [GitGuardian API reference documentation](https://api.gitguardian.com/docs) to see what other options are available. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b2ce697",
   "metadata": {},
   "source": [
    "## Scanning for secrets with the API\n",
    "\n",
    "There is one more API call that I want to demonstrate: scanning for secrets.\n",
    "\n",
    "Most of the time, when you are scanning a repo, or any document, you are going to be better off using ggshield, the CLI. This is because it includes some logic to handle batching API calls better and deal with entire repositories at once. \n",
    "\n",
    "But in the interest of demonstration, I am including it here, scanning the first document that I told you to build when we started, `authorization.py`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72f11443",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "f = open(\"authorization.py\", \"r\", encoding=\"utf-8\")\n",
    "book = f.read()\n",
    "response = requests.post(\n",
    "    \"https://api.gitguardian.com/v1/scan\", \n",
    "    headers=auth_headers, \n",
    "    data = {\"filename\":\"authorization.py\", \"document\": book}\n",
    "    )\n",
    "\n",
    "print(json.dumps(response.json(), indent=4))\n",
    "\n",
    "f.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b536b330",
   "metadata": {},
   "source": [
    "Again, make sure you do not add this file to the git repository without accounting for it with .gitignore\n",
    "\n",
    "Also, in this repository, you will find another notebook with some sample automations using the API. Those are all built using what we covered in this notebook. \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}