pandas2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a44c3788-1b87-4f0b-aec7-c6d6f849fa26",
   "metadata": {},
   "source": [
    "# pandas2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d160743-5bc8-4085-a377-0dae8ab60314",
   "metadata": {},
   "source": [
    "Let's import pandas!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb4373f9-19c3-4ffa-a7a0-f8a923cafc92",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97e0843c-5b80-482c-b767-c207073f7028",
   "metadata": {},
   "source": [
    "#### If you are using Google Colab, you must run the next line of code. *If you are NOT using Google Colab, do NOT run the next line.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4efd26fc-e177-44da-9047-ea995943d06e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://raw.githubusercontent.com/aGitHasNoName/pandasBasics/main/forestfires.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae3537f5-ad45-4ec3-a78d-1f087ad2344a",
   "metadata": {},
   "source": [
    "<br><br>We're going to work with a new dataframe. This is a dataset from forest fires in NE Portugal. I have included the dataset as a csv file in today's materials, but the data is available publicly at this site: https://archive.ics.uci.edu/ml/datasets/Forest+Fires"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99e8a3d8-b6a6-433b-960c-e74335832a85",
   "metadata": {},
   "source": [
    "<br><br>Everyone can now run the next line of code to create a DataFrame from our csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2011bd39-be5f-4ac7-beab-6f137951e79b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"forestfires.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e81d576-c2aa-4b7c-9e2d-9f37389bc904",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a485479b-3eb4-4e2e-86bb-d2cdddb24b6b",
   "metadata": {},
   "source": [
    "Take a look at the new dataframe to get to know our data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27abf411-c252-4122-8018-11a09bef469f",
   "metadata": {},
   "source": [
    "How many rows and columns are in our data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b52eabc4-1876-4af6-a80c-dbba6e7785e2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8a26a640-5f0e-4786-b8f1-0159b644ab81",
   "metadata": {},
   "source": [
    "## <br><br><br>PART THREE: RENAMING COLUMNS, DROPPING ROWS AND COLUMNS, SORTING DATA, AND SAVING DATAFRAMES"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9455b792-35b5-44d9-9439-b80b394dc8fc",
   "metadata": {},
   "source": [
    "### <br>Renaming columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc4c34b4-4a3b-423e-878a-1ebc91e55c7e",
   "metadata": {},
   "source": [
    "Here's what our column names look like in the forest fire dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8107a48d-23ad-4469-96c8-c9b1a0e1b1ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e57d3dc4-8241-4be9-9023-3d62076a656d",
   "metadata": {},
   "source": [
    "Four of the columns end in \"\\_code\". Let's remove that part from the column names. We can use the `rename()` method. We need to pass the function a **dictionary** of the old name to be replaced as the key and the new name as the value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe94b5a2-4e3f-452a-b978-64a625395c68",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.rename(columns = {\"moisture_code\": \"moisture\", \n",
    "                     \"fuel_code\": \"fuel\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ecd9b2a-6136-4466-babb-73312f3ef0af",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3edc670-98af-4547-9e32-88b0fe7bf667",
   "metadata": {},
   "source": [
    "Uh-oh, the change didn't stick. We've encountered this before with strings, so we know the answer - reassign it to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62cd4555-5fc7-4f62-9fb6-61c36295db20",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.rename(columns = {\"moisture_code\": \"moisture\", \n",
    "                          \"fuel_code\": \"fuel\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bda3138b-584f-4006-9bce-42399e1f4751",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16eb5d84-6020-4266-b4f7-b2fbe1aa17b8",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e63ea6f1-93b4-4184-8279-651adf26c0c5",
   "metadata": {},
   "source": [
    "Write code to remove \"\\_code\" from the ends of the drought and initial_spread column names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "526c4262-0488-4527-bf01-d5ca75393406",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c11bb1e-9367-4568-afbc-80bf32196639",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a22b88d0-fa5f-4fab-9c77-1e82a788c0fe",
   "metadata": {},
   "source": [
    "### <br><br><br>Dropping rows and columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "454c2b6b-5bdc-4c3c-9169-d66b8a146475",
   "metadata": {},
   "source": [
    "Let's drop a single row from the DataFrame. How about row 2? You still have to assign `df` to a variable to make the change permanent:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b61d15dc-a17c-4770-b8ae-be2aca14ac3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88178e00-0160-484d-b413-7bb6e276a059",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af1c25e4-af49-41b5-87e8-1c93a7151fc1",
   "metadata": {},
   "source": [
    "<br>The index numbers did not reset when we dropped a row. 2 is missing!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6fc3b6d-08e4-4844-b69f-ccf60f10c010",
   "metadata": {},
   "source": [
    "We can reset the index and pretend like 2 was never there. The `reset_index()` function takes one keyword argument. If we don't pass the argument, `drop=True`, an extra column will get added to our DataFrame containing the old index numbers. Let's first reset the index without passing the argument, but we won't save that DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cec15ce-7f86-457c-9b20-8693c2d257d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c851da94-8756-404d-815f-9b25c3b901de",
   "metadata": {},
   "source": [
    "You can see that new column `index` contains the original index positions. Now let's save a new version of our DataFrame, with the indexes reset, but without that new column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64eed170-ea90-40e5-9017-c32c73b077ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "706ca53c-f75c-4fb2-af94-a074992b2efa",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00a34e80-97bd-4e47-808b-7ee2d7d38a5c",
   "metadata": {},
   "source": [
    "<br><br><br>The `drop()` function defaults to dropping rows. If we want to drop a column, we need to add one more argument. `axis=1` is used in pandas to refer to columns as opposed to rows (`axis=0`). The `axis` argument is used elsewhere in pandas, too. Let's drop the \"X\" column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f9fa114-21b7-4dce-9d86-41067dcf0944",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(\"X\", axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f140b38-5a8f-4ee4-88ea-3c5e77f14300",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4047294-1b5a-450c-b3b4-7d765031a2c8",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b181b50b-a1d8-44af-b60b-c2ee7e6a3ddd",
   "metadata": {},
   "source": [
    "Write code to view the last 5 rows of the DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f84ca95-0d09-458c-bc5b-2ce088116907",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "7f073f98-0ac3-4806-a563-fe9fd5b78073",
   "metadata": {},
   "source": [
    "Now write code to drop the very last row:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ecdb32f7-21ac-4114-8c38-f97d4a98c6f8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70255ad3-3185-470a-8602-4391f83fff5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcbc6186-e4b4-4658-83f7-e40aca964839",
   "metadata": {},
   "source": [
    "Write code to remove the \"Y\" column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5feb4c22-9dc6-4077-9aa6-773e62f88299",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f77b9457-a4d6-4aa2-b12f-6cf94277595b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33b8f215-7b5d-4cf3-a791-15645c00d26a",
   "metadata": {},
   "source": [
    "### <br><br><br>Sorting a DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0536e871-d413-4da5-a224-539a41349a51",
   "metadata": {},
   "source": [
    "There are two functions for sorting your DataFrame."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aa9d7b7-dcd2-472a-86a1-824022861707",
   "metadata": {},
   "source": [
    "If you want to sort by the index numbers, or if you want to sort by the column names (alphabetically), you use `sort_index`. It can take two arguments: the axis to sort by (row or column) and the order (ascending or not):"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44e11db2-60c9-4933-8eba-271972c77d87",
   "metadata": {},
   "source": [
    "The default arguments are to sort by row index with 0 at the top, which is how we've already been viewing the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "762345f6-8b7a-43ec-b0b7-dc443a2b516b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e310e557-efc5-4caf-bd8c-0ad0579d3b3a",
   "metadata": {},
   "source": [
    "Let's try more arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f061cf48-7edf-4c2e-b7be-51614091b78c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_index(ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52bafeba-a837-4a0e-b287-448ca4422622",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_index(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4bac656-ba2e-4258-9337-efb7e32d500b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_index(axis=1, ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b294a125-663b-4af9-b195-ce5d7e301884",
   "metadata": {},
   "source": [
    "<br><br><br>The second sort function, `sort_values()`, will sort the frame by the data in a column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "858a2967-f453-4b58-9f1b-4b704adceb0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_values(\"area_burned\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a0a7d97-c192-4194-8991-dea8617cfd80",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_values(\"day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e419243-6667-4862-9d57-c955c146d2d2",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80170267-0083-4131-9686-c2ee39f331dd",
   "metadata": {},
   "source": [
    "Write code to sort the DataFrame by the rain column, with the largest values at the top:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad1e5b7f-d696-43d7-be4c-fd9c5eab41f5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "bd065b40-884c-4b0b-b0a6-d2ccad291235",
   "metadata": {},
   "source": [
    "<br><br><br>You can also sort on multiple values by passing the `sort_values` function a list of column names instead of a single name. If we want to first sort by day, then by area burned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50d6dddf-9c37-4511-acfd-a06705b9dff8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sort_values([\"day\", \"area_burned\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17be944d-c7d8-459c-8b1b-3b3d6db0e70f",
   "metadata": {},
   "source": [
    "### <br><br><br>Saving your changed DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09651f16-096c-42e8-837a-63d72d7a5cab",
   "metadata": {},
   "source": [
    "We've made a lot of changes to the forest fire dataset. Let's save it as a new csv file. First, we can decide what we're going to call the new file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "501ecd8d-802a-46bc-a101-62e60cd16dcd",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_filename = \"fire_changed.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1be6ed49-9fea-4118-887a-aae610e4f070",
   "metadata": {},
   "source": [
    "Next, we can use the `to_csv()` method function to save the new file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0c28fbc-2ace-407d-84e4-5918092ba5db",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(new_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1795b3cb-5646-45b6-8324-3265dc261feb",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23287bf0-8bb4-4769-a6e0-7e670b5c604e",
   "metadata": {},
   "source": [
    "Remember what you learned from Part Two: Selecting Data. Create a new dataframe called df_rain that contains only the columns day, month, and rain: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dde0cf34-7429-49d0-ae39-e74dc7cd0c36",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "514a2113-ff40-41bb-bdf7-640161d0acd8",
   "metadata": {},
   "source": [
    "Run the next line of code to save a new filename for the rain dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57e91f70-f3f9-4500-8570-5fb90285de3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "rain_filename = \"forestFiresRain.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4388e937-5c75-4a85-afea-09e1ea0942c4",
   "metadata": {},
   "source": [
    "Write code to save the df_rain dataframe as a csv file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a0a9270-43f0-4a5f-85ca-c9af9711131c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "87f81cb4-10a3-486d-91cc-13ead798cb6a",
   "metadata": {},
   "source": [
    "## <br><br><br>PART FOUR: DATA AGGREGATION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "354d7ba4-b6d9-4946-863f-4c6c0e00975b",
   "metadata": {},
   "source": [
    "Data aggregation means taking many data points and reducing them to one number, whether it's a count, sum, mean, or other single statistic. Here are some DataFrame method functions:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92c9cea4-b108-426a-b45c-4e34aaaf9161",
   "metadata": {},
   "source": [
    "- [`.count()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.count.html)\n",
    "- [`.sum()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.sum.html)\n",
    "- [`.mean()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.mean.html)\n",
    "- [`.median()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.median.html)\n",
    "- [`.min()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.min.html)\n",
    "- [`.max()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.max.html)\n",
    "- [`.unique()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.unique.html)\n",
    "- [`.nunique()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.nunique.html)\n",
    "- [`.std()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.std.html)   #Standard error\n",
    "- [`.var()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.var.html)   #Variance\n",
    "- And more! https://pandas.pydata.org/docs/reference/series.html#computations-descriptive-stats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7512f421-b3ec-4907-91fd-133db3843f05",
   "metadata": {},
   "source": [
    "If you use a method function on the entire dataset, it will try its best to execute the method for all columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "506bf8c0-aa7d-46a4-b8ac-805e5abcac51",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1aff8bdf-0fce-4814-a4f3-c8b78ad3ec79",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06e8f8f8-abd9-44b2-b266-54dd50bf02c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f182ddbf-8165-4534-9bdb-05c5e221ce42",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd03b7de-3a91-4be9-8095-32fb2ebfc3da",
   "metadata": {},
   "source": [
    "<br>Not all functions will work on the entire DataFrame. Most of the time you are interested in only a subset of the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77cd48fc-6e9e-4c7a-9abd-09cbbc8a570b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"day\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f275f4e6-83c9-48e0-afd0-8b18afbdbc29",
   "metadata": {},
   "outputs": [],
   "source": [
    "list(df[\"day\"].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf96f512-021d-4bea-af0f-b7c021208ec0",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"month\"].nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a825e3ff-8ba2-4b0b-9cb1-a0037a1d5d8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"temp\"].var()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4eb489dd-5be6-4a1f-b9d0-04ee303a1dfa",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85c53f90-e776-47b0-8746-9759c737109d",
   "metadata": {},
   "source": [
    "Write code to find the mean humidity of the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7fc17e6-a05d-44a6-acb4-3f37886c2907",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ee6a45ac-3652-432d-aa9c-58e6aaeb5260",
   "metadata": {},
   "source": [
    "Write code to find the coldest temperature in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cbf6213-e015-4ae3-8a2d-a50a4b9500d3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dae13127-8e18-441a-8555-d2de21942526",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "094e8a28-eca5-4636-b4cf-d55fa47189c0",
   "metadata": {},
   "source": [
    "## <br><br>groupby"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac008ba8-c273-4186-b98b-0535c3a60c2c",
   "metadata": {},
   "source": [
    "Often, you will want to calculate the statistics for a particular subgroup of a data column."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90224728-5435-41ed-ac5a-a25789d9b977",
   "metadata": {},
   "source": [
    "For example, let's say we want to ask if more fires happen on certain days of the week. This code will tell you the count for every column in the DataFrame except the column that you are using to group your data (i.e. \"day\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60656711-6831-46ca-8d23-01eef5f89d1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"day\").count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba2699ef-74bb-4884-a7d3-8a0a2e480679",
   "metadata": {},
   "source": [
    "<br>We can make this easier to read by sorting the rows from lowest count to highest count. We can string multiple method functions onto our code, as long as they go in the correct logical order. We will pick any column to sort on, since they are all the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c0198a1-bb90-4478-886d-3ee199ab4bfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"day\").count().sort_values(\"X\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "705beddf-c156-4f7d-a977-91b4d01f5b1d",
   "metadata": {},
   "source": [
    "<br>It looks like weekends have more fires than weekdays."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e598c64a-fe48-4d1e-be96-9019ea4f5daa",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c332211-9a7c-49fe-9141-40215da1233d",
   "metadata": {},
   "source": [
    "Write code to find the mean for all the columns when grouped by month."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56f32a1e-5e73-4a5d-83e3-04e039640e67",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "19995e59-4205-44c1-84f9-7f8f6b41cc1c",
   "metadata": {},
   "source": [
    "<br><br><br><br>If you only want to see an aggregate total for one column in the DataFrame, you can add on the indexing technique we learned yesterday. With this code I will ask, What is the mean area burned on each day of the week?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34fb38d1-e20c-4c3b-8cf2-879978c31eb2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"day\")[[\"area_burned\"]].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45e4cb01-37c8-4df7-92e6-a20f333a4e32",
   "metadata": {},
   "source": [
    "<br>Again, we can also sort the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8704bd13-65db-4738-bd92-99a3308ac002",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"day\")[[\"area_burned\"]].mean().sort_values(\"area_burned\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c6d8bd1-82b5-4b45-818f-059121c6ef30",
   "metadata": {},
   "source": [
    "<br>So Saturday fires are the most destructive fires."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e788cde-12fa-42ff-ac69-e33ecf0f9005",
   "metadata": {},
   "source": [
    "<br>We can also add some other functions to our code, like round:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8949930-f8a2-4d6d-b3e4-c39a27520602",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"day\")[[\"area_burned\"]].mean().round(2).sort_values(\"area_burned\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1d7dd88-ca88-4792-b56e-3101ca17b6ca",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ffe388d-9ea7-4fb1-b164-a3c7cf6e68b1",
   "metadata": {},
   "source": [
    "Write code to count how many fires happened in each month (as a bonus, sort the results):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0498befa-df51-4384-b12f-89ba6678443a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "d4c3d928-c9eb-4f89-b242-1184f31be343",
   "metadata": {},
   "source": [
    "Write code to see the mean area burned for fires in each month (as a bonus, round and sort the results):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "735a6427-3a2b-46fb-814a-e25eb0ae6b58",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f634f961-e132-4969-b56d-9a7ea18cd45e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "648e16e3-1d26-4984-a724-db61f3783a94",
   "metadata": {},
   "source": [
    "## <br><br><br>PART FIVE: BASIC PLOTTING"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ea11ba1-3722-46f4-9a25-7f9b220c5742",
   "metadata": {},
   "source": [
    "Pandas allows you to make some basic plots without loading other packages. Plotting in pandas is good for exploring your data, and we'll focus on that today.\n",
    "<br><br>Pandas is not good for making good-looking, high-quality data visualizations. The Python library for that is called matplotlib. We will not be covering how to make data visualizations for publication, since this is not a matplotlib workshop.\n",
    "<br><br>Pandas' plotting capabilities are actually built on matplotlib, but in a much simpler format."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f4324f8-3119-441b-b98c-1be6e18d23a2",
   "metadata": {
    "tags": []
   },
   "source": [
    "### <br><br>Histograms"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a4fc7c-688f-455d-a258-b31dd597e811",
   "metadata": {},
   "source": [
    "One of the most common data exploration tasks you might do is to check the distributions of the columns in your dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ded9bb2d-8a27-4a36-b3a1-e0649f2e2651",
   "metadata": {},
   "source": [
    "We will use the `hist()` method function on the `temp` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "555464fd-90e2-47a8-b3a3-9fd65debcfa6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"temp\"].hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a818f4a9-fe92-46fb-bf02-b1890e0b9e3e",
   "metadata": {},
   "source": [
    "<br><br>Like I said, it's not pretty, but it tells us the story of our data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdc92213-b34c-4695-88cb-cf62f6d1f634",
   "metadata": {},
   "source": [
    "By default, `hist()` will divide the data into 10 bins. We can change that by passing a keyword argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56c99958-2254-43d6-9761-f34936b69c43",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"temp\"].hist(bins=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e767d2bf-7a9c-4732-a039-a9ccead148ee",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd89dcae-ce08-457e-bbf6-1b9b302a6601",
   "metadata": {},
   "source": [
    "Make a histogram of the `humidity` column. Specify that you want the data grouped into 15 bins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64c7c319-1706-49b5-b7df-15bee4f874c2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "340c8215-eace-46df-a971-4f21fe709b2f",
   "metadata": {},
   "source": [
    "<br><br><br>Another way to create a histogram in pandas is to do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00aa5519-8eb5-45bd-ac38-2a391caacc53",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist(column=\"temp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa15b86f-ee3b-49ff-aabd-8c0d79451116",
   "metadata": {},
   "source": [
    "<br>Both ways are doing the same thing. In one, we're making a histogram on a sample of our dataframe. In the second, we're calling `hist()` on the entire dataframe and then specifying the column with an argument."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb80c77a-0faa-437b-80db-eb026175048c",
   "metadata": {},
   "source": [
    "<br>If we don't specify a column, we will get histograms of all columns with numerical data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a95ab51-51c5-42a4-b69b-e53c3153d51a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2ff2372-aeee-4e76-84fa-a8a74557ca0a",
   "metadata": {},
   "source": [
    "<br>We can also ask for a list of columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "becdce2d-6f12-4ea8-b683-ca3e47c75176",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist(column=[\"temp\", \"humidity\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "125a5822-a1d7-4b79-a853-1cf178102b3f",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d6ca92c-54b6-45fb-8bf6-a1b544af93eb",
   "metadata": {},
   "source": [
    "In one line of code, create histograms for the `moisture_code` and `drought_code` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "965646e5-c458-4232-b221-a20199f84417",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "ac0c59fd-1852-45c1-b470-8e8c98d6d168",
   "metadata": {},
   "source": [
    "**<br><br><br>What other changes can we make to our histogram?**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c85dc43f-5eed-4402-9c71-d58508a64516",
   "metadata": {},
   "source": [
    "Let's look at the documentation. https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.hist.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f86bd1a-e985-4d42-a6ca-fae4bbad16f4",
   "metadata": {},
   "source": [
    "<br>We can get rid of the grid lines!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9b4df72-1afb-477d-995e-41e79d3be624",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist(column=\"temp\", grid=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3212ca45-18b4-4653-b8c4-cecc3cff721c",
   "metadata": {},
   "source": [
    "<br>We can also change the figure size. The `figsize` keyword argument takes a list containing two numbers: width in inches and height in inches."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ccda794-1719-44d5-b428-a4389cbcc829",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b90f4910-934d-48ba-962e-3be34421b37e",
   "metadata": {},
   "source": [
    "Remember this plot?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "990ff98f-48e2-4c60-b83e-821157b8ce8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c88d495-d653-40f1-833a-5c6ffed608bc",
   "metadata": {},
   "source": [
    "<br>Change the figure size in the next line of code until you can see the plots better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d3d456c-2cfa-4672-acf2-f22ff122af6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hist(grid=False, figsize=[2,2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a468baab-4db5-4059-9ff2-dd9be4355c05",
   "metadata": {},
   "source": [
    "## <br><br><br>Scatter plots to check for correlation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5291673f-b5cb-4dbf-991e-cb46346f7695",
   "metadata": {},
   "source": [
    "We can make a quick scatter plot to check for correlation between 2 columns. We have to use a slightly different format for our scatter plot function. We are going to do `df.plot.scatter()`. This function requires two arguments, the columns for your x and y axes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e156aa94-2419-44c8-a12f-efc70c5d6743",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot.scatter(x=\"temp\", y=\"humidity\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c800bf20-aa96-4993-a010-5da0e591f5b7",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bccf2886-c32d-4cff-bd3e-be52f279245b",
   "metadata": {},
   "source": [
    "Write code to create a scatter plot with the moisture_code column on the x-axis and the drought_code column on the y-axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c91c4a1-068b-48ba-9e30-6c813383ad61",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "e01406ba-81f4-448f-a713-0f24821809fa",
   "metadata": {},
   "source": [
    "Based on what we learned with `hist()`, can you add a grid to the scatter plot you just made and change the size so that it is a perfect square?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a79e81cc-f203-4ee2-ae7f-d815adff0851",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "37d257c1-0fba-4edf-a5ca-f002cc1a01b8",
   "metadata": {},
   "source": [
    "## <br><br><br>Slightly more complicated examples"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da86ac36-ba9c-45c4-9b9d-12d0c603e7d7",
   "metadata": {},
   "source": [
    "### <br>Removing Outliers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf4b1213-177e-4100-9840-c505088351f5",
   "metadata": {},
   "source": [
    "Let's look at the correlation between humidity and area_burned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19b001bd-2d81-4156-9ff3-3645b596cda0",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot.scatter(x=\"humidity\", y=\"area_burned\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e012b16-3d1d-4a67-8a01-4e423fa3fd4e",
   "metadata": {},
   "source": [
    "<br>We can see that a few outliers are clouding any relationship. We can remove them with a boolean, but we need to look at the plot above and decide where to make the cutoff. Let's try getting rid of only points above 400."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42ea18a5-4ba0-44b0-bfce-00044382322a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_no_outliers = df[df[\"area_burned\"] < 400]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a28e8e27-4b86-498e-bf15-02b8d1008dd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_no_outliers.plot.scatter(x=\"humidity\", y=\"area_burned\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69c44508-92b1-4c6c-b6b5-9fe0a154a0cd",
   "metadata": {},
   "source": [
    "<br><br>There are a large number of points with 0 for area_burned. Let's remove those, and only look at days where the fires spread."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "edf78f87-5f15-4ccc-a395-ca16c18880dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_no_outliers = df[(df[\"area_burned\"] > 0) & (df[\"area_burned\"] < 400)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8918fc00-3fba-4e28-9598-865a485a286c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_no_outliers.plot.scatter(x=\"humidity\", y=\"area_burned\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ce08e4-c584-4c5e-acbe-1b11fd8f67e7",
   "metadata": {},
   "source": [
    "### <br><br><br>Plotting with categorical data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8cdcc66-3f16-4d9f-a261-9eb3f8cfd867",
   "metadata": {},
   "source": [
    "Let's say we want to see the relationship between month and humidity. We can try a scatter plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a556cfcd-6347-4cc2-a489-82d73de36abc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot.scatter(x=\"month\", y=\"humidity\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3a2c3c5-fd85-473d-82dd-c5d8f3306605",
   "metadata": {},
   "source": [
    "<br>That doesn't work! Let's try another type of plot - a bar plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae535333-18b0-4c83-875e-2f9e31407eb4",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot.bar(x=\"month\", y=\"humidity\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06bd9942-37c8-4177-9c0b-f156a8b17518",
   "metadata": {},
   "source": [
    "<br>Also not what we're looking for. It's plotting each data point individually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2395c937-850f-4016-b867-989418e1cb46",
   "metadata": {},
   "source": [
    "What we really want to see is how the mean humidity changes each month. We can create a new DataFrame that only includes the data we need. We will group by month, select only the humidity column, and then find the mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1dfbe18e-1219-4eed-8f1e-9fac72bbb5cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "hum_mean = df.groupby(\"month\")[\"humidity\"].mean()\n",
    "hum_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc4c2300-d705-45e0-9e5b-e1208a27e3bc",
   "metadata": {},
   "source": [
    "<br>Now we have a nice series object that we can plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6c12c40-4f3c-4bc2-9a0c-08822680d957",
   "metadata": {},
   "outputs": [],
   "source": [
    "hum_mean.plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adf1c808-0b6d-4fd8-a2da-ea6f1b71dee5",
   "metadata": {},
   "source": [
    "<br>This is looking better, but we need to sort them by month, not alphabetically. We're going to worry about that in just a minute."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12d907f2-0f28-44a3-80d1-198e31df57cd",
   "metadata": {},
   "source": [
    "### <br><br>Exercise 5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57738428-6871-4532-81c5-d81655baaa55",
   "metadata": {},
   "source": [
    "Create a bar graph that shows mean temperature grouped by month. First you'll need to create a new series object with the means. Refer back to the humidity exercise directly above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06cd8ec2-0532-442b-bd9e-a844543a2d14",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6eea6f57-4ee7-4816-9fff-d3ceb8e29290",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "79d992a1-9827-4ee5-94f1-9f68f7acb7b3",
   "metadata": {},
   "source": [
    "<br><br><br>Let's deal with the sorting issue! We can create a new column in our DataFrame that contains a numerical value for the month. We will use a handy pandas function called `replace()`. It takes two arguments: a list of items to replace, and a list of the replacement values in the same order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "990ddb12-2223-4c8b-bd69-3314b7ad892b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"month_num\"] = df[\"month\"].replace([\"jan\", \"feb\", \"mar\", \"apr\", \n",
    "                          \"may\", \"jun\", \"jul\", \"aug\", \n",
    "                          \"sep\", \"oct\", \"nov\", \"dec\"], \n",
    "                         [1, 2, 3, 4, 5, 6, 7, 8, 9, \n",
    "                          10, 11, 12])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82b48313-aec3-485c-9d20-a9733f7699a5",
   "metadata": {},
   "source": [
    "<br>Now we will repeat the humidity plot we just did, but we will group by the new column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98b18090-2fb9-42ae-9573-8606fa837e33",
   "metadata": {},
   "outputs": [],
   "source": [
    "hum_mean = df.groupby(\"month_num\")[\"humidity\"].mean()\n",
    "hum_mean.plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e7f80e1-e51f-45b6-837f-64584f1cfd87",
   "metadata": {},
   "source": [
    "## <br><br><br>Other packages for exploring data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "619f5d0e-2e0c-4356-9a66-4e22140255ff",
   "metadata": {},
   "source": [
    "There are a few packages that have been created to help you visualize your data during the data exploration step without writing the code for yourself. They also have their limitations, however."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab6eee4b-9450-4b73-97a4-e0154654d31a",
   "metadata": {},
   "source": [
    "- pandas-profiling (gives in-depth summaries about your dataset) (I've been having trouble with the installation/dependencies lately)\n",
    "- sweetviz (designed as exploration prior to machine learning, so it requires you to split your data into train and test and select a target variable)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35bcc12c-1f6a-422a-9f0e-0e80a640e2c8",
   "metadata": {},
   "source": [
    "Both packages cover:\n",
    "- data type, unique values, missing values\n",
    "- Quantile statistics like minimum value, Q1, median, Q3, maximum, range, interquartile range\n",
    "- Descriptive statistics like mean, mode, standard deviation, sum, median absolute deviation, coefficient of variation, kurtosis, skewness\n",
    "- Histograms\n",
    "- Correlations of variables\n",
    "- Missing values matrix, count, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cab6242-4484-479a-afbc-7cb466b8592a",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}