notebooks/tut2_data_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <span style=\"color:#2c061f\"> Macro 318: Lecture 4 - 6 / Tutorial 2 </span>  \n",
    "\n",
    "<br>\n",
    "\n",
    "## <span style=\"color:#374045\"> Data, Stats and Math with Julia </span>\n",
    "\n",
    "\n",
    "#### <span style=\"color:#374045\"> Lecturer: </span> <span style=\"color:#d89216\"> <br> Dawie van Lill (dvanlill@sun.ac.za) </span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial we will start our discussion with how to work with data in Julia. \n",
    "\n",
    "We will then cover some basic statistics and in the last section move on to some fundamental ideas in mathematics (mostly related to calculus).  \n",
    "\n",
    "Please note that working with data in Julia is going to be different than working with data in Stata. \n",
    "\n",
    "I am just showing basic principles here so that you are aware of them. \n",
    "\n",
    "You do not need to memorise everything in this notebook. It is simply here as a good reference to have if you want to do some useful data work for macroeconomics. \n",
    "\n",
    "If you are more comfortable with Stata for working with data then you can continue on that path. I am simply offering an alternative. \n",
    "\n",
    "In the job market there are a few languages that are used for data analysis. \n",
    "\n",
    "The most popular ones are Stata, R, Python and Julia. \n",
    "\n",
    "At this stage Julia is not the most popular for data work, but it shares similarities with Python. \n",
    "\n",
    "So if you know Julia well, it will be easy to pick up Python. \n",
    "\n",
    "Julia is more popular for work related to numerical / scientific computation, which we will cover in some of the future tutorials. \n",
    "\n",
    "If you are interested in Python as an alternative to Julia you can always contact me and I can refer you to some resources. However, for most students it is more important to get the programming principles right without worrying too much about the language that they are using. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import Pkg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Pkg.add(\"CategoricalArrays\")\n",
    "Pkg.add(\"CSV\")\n",
    "Pkg.add(\"DataFrames\")\n",
    "Pkg.add(\"DataFramesMeta\")\n",
    "Pkg.add(\"Downloads\")\n",
    "Pkg.add(\"ForwardDiff\")\n",
    "Pkg.add(\"GLM\")\n",
    "Pkg.add(\"LinearAlgebra\")\n",
    "Pkg.add(\"Plots\")\n",
    "Pkg.add(\"Random\")\n",
    "Pkg.add(\"RDatasets\")\n",
    "Pkg.add(\"Roots\")\n",
    "Pkg.add(\"ShiftedArrays\")\n",
    "Pkg.add(\"SparseArrays\")\n",
    "Pkg.add(\"Statistics\")\n",
    "Pkg.add(\"Symbolics\")\n",
    "Pkg.add(\"Zygote\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using CategoricalArrays\n",
    "using CSV\n",
    "using DataFrames\n",
    "using DataFramesMeta\n",
    "using Downloads\n",
    "using ForwardDiff\n",
    "using GLM\n",
    "using LinearAlgebra\n",
    "using Plots\n",
    "using Random\n",
    "using RDatasets\n",
    "using Roots\n",
    "using ShiftedArrays\n",
    "using SparseArrays\n",
    "using Statistics\n",
    "using Symbolics\n",
    "using Zygote"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The primary package for working with data in Julia is `DataFrames.jl`.\n",
    "\n",
    " For a comprehensive tutorial series on this package I would recommend Bogumił Kamiński's [Introduction to DataFrames](https://github.com/bkamins/Julia-DataFrames-Tutorial)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DataFrames basics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we discuss basic principles from the DataFrames package. \n",
    "\n",
    "For the first topic we look at how to construct and access DataFrames. \n",
    "\n",
    "The fundamental object that we care about is the `DataFrame`. This is similar to a `dataframe` that you would find in R or in Pandas (Python).\n",
    "\n",
    "DataFrames are essentially matrices, with the rows being observations and the columns indicating the variables. \n",
    "\n",
    "Below is some code that sets the maximum number of rows and columns to be printed in the notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ENV[\"LINES\"] = 10; # set the max number of lines that will be printed\n",
    "ENV[\"COLUMNS\"] = 100; # set the max number of columns that will be printed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The easiest thing to construct is an empty DataFrame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DataFrame() # empty DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You could also construct a DataFrame with different keyword arguments. Notice the different types of the different columns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DataFrame(A = 2:5, B = randn(4), C = \"Hello\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the most common ways to use constructors is through arrays. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "commodities = [\"crude\", \"gas\", \"gold\", \"silver\"] # commodities\n",
    "last_price = [4.2, 11.3, 12.1, missing] # prices of the commodities (notice that the last value is missing)\n",
    "\n",
    "df = DataFrame(commod = commodities, price = last_price) # give names to columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice above that the `commod` column contains only string values. \n",
    "\n",
    "In the case of `price` we have decimal numbers with one missing number. \n",
    "\n",
    "The `DataFrames` package infers that these are floating point numbers.\n",
    "\n",
    "The question mark indicates the uncertainty about how to handle the missing value. \n",
    "\n",
    "One can also easily add a new row to an existing `DataFrame` using the `push!` function. \n",
    "\n",
    "This is equivalent to adding new observations (rows) to the variables. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_row = (commod = \"nickel\", price = 5.1)\n",
    "push!(df, new_row)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One could also use array comprehensions to generate values for the DataFrame,  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DataFrame([rand(3) for i in 1:3], [:x1, :x2, :x3]) # see how we named the columns with the symbol notation [:x1, :x2, :x3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create a DataFrame from a matrix, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = DataFrame(rand(3, 3), :auto) # automatically assign column names with :auto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Incidentally, you can convert the DataFrame into a matrix or array if you so wished, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next section we talk about accessing the element of a DataFrame as well as looking at some basic information about the DataFrame that we have on hand.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have our data set up in a DataFrame, we are often going to want to know some basic things about the contents. Let us construct a relatively large DataFrame. Most of the time we will be working with large datasets in economics, with thousands of rows and columns. You might be used to working with data in Excel, so things might feel foreign right now. However, I promise that once you start working with data in a programming language such as R, Julia or Python, your productivity will greatly increase. You only need to get over that initial apprehension on learning something new. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = DataFrame(rand(1:10, 1000, 10), :auto);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get some basic summary statistics on the data in the DataFrame using the `describe` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "describe(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to take a peak at the first few rows of the data we can use the `first` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first(y, 5) # first 5 rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are multiple ways to access particular columns of the DataFrame that we have created. The most obvious way is to to use `y.col` where `col` stands for the column name. This provides us the column in vector format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y.x2; # get a single column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another interesting way to access the column is the following, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[!, :x2]; # or y[!, 2] or y[:, :x2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can access several columns (the first two in this case) with the following command, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[:, [:x1, :x2]];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Getting rows is also quite easy (and similar to the way in which we access rows in arrays), "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[1, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the code above you can also easily change the values in the DataFrame. We could, for example, multiply each of the values in the second column by $2$ if we wanted. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first(y.x2, 2) # observe first two values in the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = y[!, :x2]; \n",
    "z *= 2; # multiply column by two\n",
    "\n",
    "first(z, 2) # observe newly mutated column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first(y.x2, 2) # important to note that this is unchanged"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us import some data and play around with it a bit. This is generally referred to as data wrangling. If you want to become a data scientist, then significant portion of your work is going to involve gathering and cleaning data. The analysis part only makes up a small percentage. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have created a dataset that is hosted on github at the following location -- https://github.com/DawievLill/Macro-318/blob/main/data/tut2_data.csv\n",
    "\n",
    "Now let us download this data with Julia. This is one possible way to do it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Downloads.download(\n",
    "    \"https://raw.githubusercontent.com/DawievLill/Macro-318/main/data/tut2_data.csv\", \n",
    "    \"tut2_data.csv\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now get the data into Julia!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sa_data = DataFrame(CSV.File(\"tut2_data.csv\", dateformat = \"yyyy/mm/dd\")) # specify the date format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we could have done the following, since we know the data is located in the `data` folder, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sa_data_1 = DataFrame(CSV.File(\"../data/tut2_data.csv\")); # if you don't understand this piece of code, you can simply move on. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we have succesfully imported data into Julia. \n",
    "\n",
    "What can we do with this data? \n",
    "\n",
    "We see that the data contains information on GDP, the repo rate, CPI and inflation. \n",
    "\n",
    "The first thing that we might want to do is visualise the data. \n",
    "\n",
    "This is always a good first step. \n",
    "\n",
    "After that we might want to look at some basic descriptive statistics, to get an idea of the properties of the data. \n",
    "\n",
    "However, before we do that, let us take a look at how to rename columns. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Renaming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two functions can be used to rename columns. \n",
    "\n",
    "The `names` function returns column names as a vector of strings, while the `propertynames` function returns a vector of symbols."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "names(sa_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "propertynames(sa_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the `rename!` function to change column names. \n",
    "\n",
    "This function can be used to rename all columns at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rename!(sa_data, [:date, :GDP, :interest, :CPI, :infl])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another option is to rename only some of the columns specified by their names,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rename!(sa_data, :GDP => :gdp, :interest => :repo, :CPI => :cpi, :infl => :inflation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For us to plot the data let us look at one the variables in the dataset. \n",
    "\n",
    "Let us consider GDP, which is the second column in the dataset. We can access GDP by calling the variable name, but we can also use the fact that it is located in the second column of the table. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdp_1 = sa_data.gdp;\n",
    "gdp_2 = sa_data[!, 2];\n",
    "gdp_3 = sa_data[!, :gdp];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check whether these variables give the same result as follows, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdp_1 == gdp_2 == gdp_3 # check that these give exactly the same result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us draw a basic plot of GDP and see if it aligns with your expectation of what GDP would look like in level terms. You can also compare this with US GDP in Chapter 2 of the Williamson textbook. I have used **nominal GDP** in this dataset. \n",
    "\n",
    "We will compare nominal with real GDP soon, with a simple calculation involving the inflation rate (although technically we should be using the GDP deflator to move from nominal to real terms).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_sa = sa_data[!, :date]\n",
    "plot(date_sa, gdp_1, legend = false, lw = 2, color = :blue, alpha = 0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the most popular transformations of GDP data is to take a natural logarithm. The reason for this is that differences between adjacent values in the GDP series represent growth rates once the series is \"logged\". We will get back to this point at a later stage. If we take a natural log of the series then the plot of GDP looks as follows, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "log_gdp = log.(gdp_1); # remember the dot syntax, since we are broadcasting the log over all the values of GDP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(date_sa, log_gdp, legend = false, lw = 2, color = :blue, alpha = 0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will see that the scale on the y-axis has changed after this transformation. The slope of this graph indicates the growth rate. A point that we will touch on soon. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also plot some of the other variables in our dataset, such as the inflation rate. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inflation_rate = sa_data.inflation\n",
    "plot(date_sa, inflation_rate, legend = false, lw = 2, color = :blue, alpha = 0.8)\n",
    "plot!([0], legend = false, lw = 1.5, seriestype = :hline, color = :black, ls = :dash, alpha = 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hodrick-Prescott filter (technical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following section is for the more technically inclined students. However, you can quickly read through this section to get a feeling for the main result. \n",
    "\n",
    "The Hodrick-Prescott filter is a tool that is used to remove the cyclical component of a time series. In other words it extracts the trend component of a time series.\n",
    "\n",
    "We will provide a brief formal description of what the filter is doing, and provide some code for how to operate it. \n",
    "\n",
    "It is not expected that you fully understand this process. \n",
    "\n",
    "This is a topic that you will encounter again in the Honours program. \n",
    "\n",
    "For now it is good enough to simply understand what the HP filter is doing. Please do not worry about the code. Those that are interested in the code can look at it, but it is a bit beyond what is expected of most of the students in this class, unless you have a background in programming and statistics. \n",
    "\n",
    "Formally, a time series $y_t$, such as GDP, is made up of a trend component $\\tau_t$, a cyclical component $c_t$ and an error component $\\varepsilon_t$ such that \n",
    "\n",
    "$$\n",
    "y_t = \\tau_t + c_t + \\varepsilon_{t}\n",
    "$$\n",
    "\n",
    "Given some value of $\\lambda$, there is a trend component that will solve the following minimisation problem, \n",
    "$$\n",
    "\\min _{\\tau }\\left(\\sum _{t=1}^{T}{(y_{t}-\\tau _{t})^{2}}+\\lambda \\sum _{t=2}^{T-1}{[(\\tau _{t+1}-\\tau _{t})-(\\tau _{t}-\\tau _{t-1})]^{2}}\\right)\n",
    "$$\n",
    "\n",
    "The first term $(y_{t}-\\tau _{t})^{2}$ represents the squared deviation of the trend from the series, which acts as a penalty on the cyclical component. The second term $[(\\tau _{t+1}-\\tau _{t})-(\\tau _{t}-\\tau _{t-1})]^{2}$, which is multiplied by $\\lambda$, represents the second difference of the squared trend component. This part penalises variations in the growth rate of the trend component. The higher the value of $\\lambda$, the higher this penalty.\n",
    "\n",
    "The value for $\\lambda$ is normally set to $1600$ for quarterly data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function hp_filter(y; w = 1600)\n",
    "\n",
    "    # y is original series to be smoothed\n",
    "    # w is smoothing parameter\n",
    "    # s is the output / filtered series\n",
    "\n",
    "    # ensure the correct shape (column vector)\n",
    "    if size(y, 1) < size(y, 2)\n",
    "        y = y'\n",
    "    end\n",
    "\n",
    "    t = size(y, 1)\n",
    "\n",
    "    a = 6 * w + 1\n",
    "    b = -4 * w\n",
    "    c = w\n",
    "    \n",
    "    d = [c b a]\n",
    "\n",
    "    d = ones(t, 1) * d\n",
    "\n",
    "    m = spdiagm(d[:,3]) + spdiagm(1 => d[1:t-1, 2]) + spdiagm(-1 => d[1:t-1,2])\n",
    "    m += spdiagm(2 => d[1:t-2, 1]) + spdiagm(-2 => d[1:t-2, 1])\n",
    "\n",
    "    m[1, 1] = 1 + w       \n",
    "    m[1, 2] = -2 * w\n",
    "    m[2, 1] = -2 * w      \n",
    "    m[2, 2] = 5 * w +1\n",
    "    m[t-1, t-1] = 5 * w + 1 \n",
    "    m[t-1, t] = -2 * w\n",
    "    m[t, t-1] = -2 * w    \n",
    "    m[t, t] = 1 + w\n",
    "\n",
    "    return s = m \\ y\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we see a plot with the trend in red and the original series in black. The trend is retrieved using the HP filter, with a value of $\\lambda = 1600$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hp_plot = plot(hp_filter(gdp_1), legend = false, lw = 2, alpha = 0.8, color = :red)\n",
    "plot!(hp_plot, gdp_1, lw = 2, alpha = 0.7, color = :black)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is the detrended business cycle, which we retrieve by subtracting the trend from the original series. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bus_cycle = gdp_1 - hp_filter(gdp_1);\n",
    "plot(date_sa, bus_cycle, lw = 2, alpha = 0.7, color = :black, legend = false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can see what happens if we use different values of $\\lambda$ for the trend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lambdas = [200, 1600, 10000]\n",
    "\n",
    "p1 = plot()\n",
    "\n",
    "for i in lambdas\n",
    "    plot!(p1, hp_filter(gdp_1, w = i), label = \"lambda = $i\", legend = :topleft, lw = 2, alpha = 0.8)\n",
    "end\n",
    "\n",
    "plot(p1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p2 = plot()\n",
    "\n",
    "for i in lambdas\n",
    "    plot!(p2, gdp_1 - hp_filter(gdp_1, w = i), label = \"lambda = $i\", legend = :bottomleft, lw = 2, alpha = 0.8)\n",
    "end\n",
    "\n",
    "plot(p2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Descriptive statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of the most widely used statistics are the mean (average) and standard deviation. They can be easily computed with the computer as follows,  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inflation_1 = sa_data[!, :inflation] # select the inflation column\n",
    "mean(inflation_1) # mean value of inflation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "std(inflation_1) # standard deviation of inflation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us plot a histogram for the data and also indicate where the mean is in this plot. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "histogram(inflation_1, legend = false, alpha = 0.5, bins = 20) # a histogram gives a general idea of what the distribution of values for inflation looks like.\n",
    "plot!([mean(inflation_1)],  seriestype = :vline, lw = 3, colour = :black, ls = :dash) # plots the mean value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, if we want descriptive statistics we can use the `describe()` function to give us some more information about the dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "describe(sa_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Growth rates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mathematics is so much easier when we get to use a computer. In this section I will introduce some of the basic mathematical theory that you need as a macroeconomist and then we will show you how that relates to programming."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Computing growth rates is quite important in macroeconomics. This is something that you will frequently encounter and it is important to know how to do this. The growth rate between subsequent two dates can be calculated as follows, \n",
    "\n",
    "$$\n",
    "\\left(\\frac{Y_{t} - Y_{t-1}}{Y_{t-1}}\\right) \\times 100 = \\left(\\frac{Y_{t}}{Y_{t-1}} - 1\\right) \\times 100 \n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are working with quarterly data and want to calculate the growth rate from one quarter to the same quarter next year then you should use, \n",
    "\n",
    "$$\n",
    "\\left(\\frac{Y_{t}}{Y_{t-4}} - 1\\right) \\times 100 \n",
    "$$\n",
    "\n",
    "In addition, we can calculate the monthly or quarterly growth rate at an annual rate, \n",
    "\n",
    "$$\n",
    "\\left(\\left[\\frac{Y_{t}}{Y_{t-1}}\\right]^{n} - 1\\right) \\times 100 \n",
    "$$\n",
    "where $n = 4$ represents quarterly growth and $n = 12$ gives monthly growth. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can calculate the average growth rate over $n$ years with the following formula, \n",
    "\n",
    "$$\n",
    "\\left(\\left[\\frac{Y_{t}}{Y_{t-n}}\\right]^{1/n} - 1\\right) \\times 100 \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples of growth rate calculations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us try calculating some growth rates with the data at hand. We will focus on GDP growth rates for this example, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdp_first = sa_data[1, :gdp]; # first value of the GDP series\n",
    "gdp_second = sa_data[2, :gdp]; # second value of the GDP series\n",
    "\n",
    "gdp_growth_1 = ((gdp_second - gdp_first)/gdp_first) * 100 # using the formula for growth between two periods. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is an alternative way to calculate an approximation to the growth rate from above. We can simply take the natural logarithm of the two values and subtract them from each other. In other words we have that, \n",
    "\n",
    "$$\n",
    "\\left(\\frac{Y_{t}}{Y_{t-1}} - 1\\right) \\times 100 \\approx \\log(Y_{t}) - \\log(Y_{t-1}) \\times 100\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(log(gdp_second) - log(gdp_first)) * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The values are not exactly the same. This is simply an approximation which is often used in practice. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As another example, let us calculate the quarterly growth rate of GDP and then plot the resulting values. This calculation is a bit more tricky since we are now going to apply it to the entire dataset. Let me explain the logic of what we are going to do here. \n",
    "\n",
    "First, we are going to create a new column that contains a lagged version of the original column for GDP. Why do we want to do this? The reason is that we want to be able to divide $Y_t$ by $Y_{t-4}$ from our formula above. However, we only have values for $Y_t$ in the $t$-th column and not $Y_{t-4}$. So for every row we need to create a corresponding lagged version of the original column. Before we do anything, let us take a look at the first few values of the dataset to see what we need to change. We only care about the first two columns, so we will only select those. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first(sa_data[!, 1:2], 8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sa_data.lagged_gdp = lag(sa_data[!, 2], 4);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the lagged GDP column has successfully been created and added to the DataFrame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "first(sa_data[!, :], 8) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can calculate the growth rate, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdp_growth_quarterly = (log.(sa_data.gdp) .- log.(sa_data.lagged_gdp)) .* 100; # log method\n",
    "gdp_growth_quarterly_1 = ((sa_data.gdp ./ sa_data.lagged_gdp) .- 1) .* 100; # formula method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(date_sa, gdp_growth_quarterly, legend = false, lw = 2, alpha = 0.7) # log method\n",
    "plot!(date_sa, gdp_growth_quarterly_1, legend = false, lw = 2, ls = :dash, color = :black) # formula method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the other formulas you can perform similar calculations. As an exercise you can attempt to use the other formulas to calculate annualised growth rates. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic math for macroeconomics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we will be discussing very basic mathematical concepts that relate to mathematical modeling. For this section we will be making heavy use of the notes on mathematics for economists by [Fan Wang](https://fanwangecon.github.io/Math4Econ/). Please go look at his website for more cool notes and give his repository a star. His code is mostly in Matlab, but he also has some Python and R code. You can easily translate Matlab to Julia, since the syntax of the languages are quite similar.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function is a **rule** that assigns to every element of $x \\in X$ a **single element** of the set $Y$. This is written as, \n",
    "\n",
    "$$\n",
    "f:X \\rightarrow Y\n",
    "$$\n",
    "\n",
    "The arrow indicates the mapping from the one set to another. When we write $y = f(x)$ we are mapping from the argument $x$ in the domain $X$ to a value in the co-domain $Y$. \n",
    "\n",
    "It is important to note that for a function we are assigning a single element from the set $X$ to the set $Y$.\n",
    "\n",
    "Let us illustrate this with some examples of functions and non-functions in Julia. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 0:π/100:2π\n",
    "y = sin.(x)\n",
    "\n",
    "plot(x, y, title = \"This is a function\", legend = false, lw = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function above is a portion of the $\\sin$ function over the interval from $0$ to $2\\pi$. However, the following graph that depicts a circle is not a function. It is a relation, but NOT a function. Can you see why this is the case? What is the defining feature of a function?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1; y = 1; r = 1\n",
    "θ = 0:π/50:2π\n",
    "\n",
    "x_unit = r .* cos.(θ) .+ x\n",
    "y_unit = r .* sin.(θ) .+ y\n",
    "\n",
    "plot(x_unit, y_unit, title = \"This is NOT a function\", legend = false, lw = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A linear function, which is also known as polynomial of degree 1 has slope $m$ and intercept $b$. Linear functions have constant slope. We will encounter the idea of slope again later when we talk about derivatives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = 0.5 # slope\n",
    "b = 1 # intercept\n",
    "\n",
    "ar_x = LinRange(-5, 10, 100)\n",
    "ar_y = ar_x .* m .+ b\n",
    "\n",
    "plot(ar_x, ar_y, legend = false, title = \"Linear function with slope $m and intercept $b\", lw = 2)\n",
    "\n",
    "vline!([0], ls = :dash, color = :black, alpha = 0.5, xticks = ([-2]))\n",
    "hline!([0], ls = :dash, color = :red, alpha = 0.5, yticks = ([1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In high school you probably determined the slope of this function using a the method of rise over run. In other words, the change in $y$ over the change in $x$. More \"formally\", you calculated $m = \\frac{\\Delta{y}}{\\Delta{x}}$. In this example, $\\Delta{y} = 1 - 0$ and $\\Delta{x} = 0 - (-2)$, so we have that $m = 1 / 2 = 0.5$. We will talk about this method of using a difference quotient again when we want to calculate the slope of a tangent line to a function in the section on derivatives. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Monomials and polynomials\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Functions that take the form $a \\cdot x^{k}$ are considered **monomials**. In this case $a$ is any real number and $k$ is a positive integer. The value of $k$ represent the degree of the monomial. Monomials can be added together to form **polynomials**. A general formulation for a polynomial of degree four would be, \n",
    "\n",
    "$$\n",
    "a + b\\cdot{x} + c \\cdot x^2 + d \\cdot x^3 + e \\cdot x^4\n",
    "$$\n",
    "\n",
    "where the coefficients $a, b, c, d, e$ could be positive or negative. In order to determine the degree of the polynomial, consider the monomials that constitute the polynomial. The monomial with the highest degree determines the degree of the whole polynomial. An explicit representation of a polynomial that you should all know quite well is something along the following lines, \n",
    "\n",
    "$$\n",
    "y = 2 + 5x^2\n",
    "$$\n",
    "\n",
    "Can you identify what the degree of this polynomial is? Let us plot some polynomials to get an idea of what they look like. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = -1:0.01:1\n",
    "y = 2 .+ 5 .* x .^ 2\n",
    "\n",
    "plot(x, y, legend = false, title = \"Polynomial of degree two\", lw = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us plot a polynomial of a degree three to see how this might differ. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = -3:0.01:3\n",
    "y = 2 .- 1 .* x .^ 2 .- 10 .* x .^ 3\n",
    "\n",
    "plot(x, y, legend = false, title = \"Polynomial of degree three\", lw = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Local and global maximum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function $f$ has a global maximum at $x^{*} \\in X$ if for all $x \\in X$, $f(x) \\leq f(x^{*})$. A function $f$ has a local maximum at $x^{*} \\in X$ if there exists and open interval $(a, b)$ such that $x^{*} \\in (a, b)$, and for all $x \\in (a, b), f(x) \\leq f(x^{x})$. Not all functions have a maximum. \n",
    "\n",
    "We are going to be dealing with functions such as utility functions, production functions and budget constraints in many of our economics models. \n",
    "\n",
    "As an example, in the case of a utility function the household is choosing the bundle of goods that provides the highest level of utility. \n",
    "\n",
    "Most production and utility functions will NOT have a local or global maximum. \n",
    "\n",
    "In the case of a log-utility function, higher consumption will give higher levels of utility, so no maximum exists here either. \n",
    "\n",
    "If we have a Cobb-Douglas production function, then output will increase with labour and capital. This means that the function is even increasing in its inputs and has not maximum. \n",
    "\n",
    "**NB**: It is only when we combine our preferences with a budget that we will be able to think about the optimal bindle choice given a fixed budget. The maximisation problem in this case will reveal some maximum. We will cover this example at length in the next tutorial. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quadratic utility"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is one nice utility function that has a maximum, the quadratic utility function. \n",
    "\n",
    "$$\n",
    "U(x) = x - \\alpha \\cdot x^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can quickly draw a plot of this utility function and by inspection determine where the maximum is going to be. The most efficient way to determine the maximum would be using a derivative and setting it equal to zero. However, we haven't introduced derivatives yet, so let us use this inefficient way to determine the maximum for now. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "npoints = 100\n",
    "a, b = (-10, 10)\n",
    "x = range(a, b, length = npoints)\n",
    "α = 0.2 \n",
    "\n",
    "U(x) = x .- α .* x .^ 2\n",
    "\n",
    "plot(x, U.(x), legend = false, lw = 2, title = \"Can you find the max?!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we can approximately guess from the graph where the highest value for this graph is going to be. It be somewhere in the interval between 10 and 15. However, this is not precise enough. We need some method to give us the exact answer, this is where derivatives will enter. Another approach is just to consider each of the utility values and then pick the maximum from the list. Luckily Julia has a function called `findmax()` that can do this for us. \n",
    "\n",
    "Don't worry too much about the code for now, this won't make too much sense at first. I will try and explain the basic idea, but if you don't get it don't worry. Try and come back to this code at a later stage once you are more familiar with programming in Julia. Then try and evaluate every line and see if you can make sense of what is happening here. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmax, ix = findmax(U.(x))\t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see from above that if we evaluate our utility function with the `findmax()` function a tuple is returned. The first value in the tuple is the function output at the maximum. In other words it is the value of utility where this function is maximised. It is the value on the y-axis. The second value in the tuple is the position in the list of $x$ values that we evaluated. If we go back in our code you will see that we created a range of $x$ values in the interval $-10$ to $10$. We created a grid of 100 points within the interval (generated 100 potential values for $x$) and inserted those $x$ values into the utility function at each of the given points. \n",
    "\n",
    "According to the `findmax()` function, the 63rd value in in that grid of $x$ values was the one that maximised the function. This is also referred to as the $\\argmax$ in mathematics. We can then find the $x$ value by looking at `x[ix]`, which is basically the same as `x[63]` in this case. Remember how to access elements in an array from the previous tutorial. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x[ix] == x[63]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The maximum here is $2.52525252 \\ldots$. If we chose a finer grid for $x$ we would be able to get a better approximation for the true value that maximises this function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x[ix] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the graph below we simply draw a scatter plot with the $x$ value being represented by the 63rd point in the $x$ vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scatter!([x[ix]], [fmax], color = :red, ms = 5)\n",
    "vline!([x[ix]], lw = 1.5, color = :black, ls = :dash, alpha = 0.5, xticks = ([x[ix]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Derivatives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear functions have a constant slope, as we have seen in one of the earlier examples. However, what about non-linear functions? What is the rate of change for a non-linear function as we move along its domain?\n",
    "\n",
    "Let $(x_0, f(x_0))$ be a point on the graph of $y = f(x)$. The derivative of $f$ at the point $x_0$ is the slope of the tangent line to the graph of $f$ at $(x_0, f(x_0))$. We can denote this derivative of a function $f$ at $x_0$ as\n",
    "\n",
    "$$\n",
    "f'(x_0) = \\frac{df(x_0)}{dx} = f_{x}(x_0)\n",
    "$$\n",
    "\n",
    "The notation above is interchangeable. The derivative is defined formally as, \n",
    "\n",
    "$$\n",
    "f'(x_0) = \\lim_{h \\rightarrow 0} \\frac{f(x_{0} + h) - f(x_0)}{h}\n",
    "$$\n",
    "\n",
    "In taking derivatives we normally revert to the derivative rules. The derivative rules are the following, \n",
    "\n",
    "1. Constant rule\n",
    "2. Power rule\n",
    "3. Chain rule\n",
    "4. Sum (difference) rule\n",
    "5. Product rule\n",
    "6. Quotient rule (we won't cover this, since you can use the product rule if needed)\n",
    "\n",
    "Then we can also speak of the exponential and log rule for derivatives. These are ones that will be frequently used. Let us have a brief discussion and examples of each of the rules. We will also accompany the solutions that are written out by hand with a solution generated by the computer to check that we are correct in our calculation. \n",
    "\n",
    "It is important to note that there are several ways in which you can take derivatives on the computer. The primary methods are, \n",
    "\n",
    "1. Symbolic differentiation\n",
    "2. Automatic differentiation\n",
    "3. Numerical differentiation\n",
    "\n",
    "We will not be going into detail on how these methods work. We will just use them in practice. We will show the symbolic and automatic differentiation approaches in this tutorial. For a good introduction to the approximation of derivatives using a computer you can look at these notes [here](https://mth229.github.io/derivatives.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Constant rule\n",
    "\n",
    "If $f(x) = k$ where $k$ is some constant then $f'(k) = 0$. As an application of the constant rule, let us determine the derivative of $f(x) = 8$. \n",
    "\n",
    "$$\n",
    "f'(8) = 0\n",
    "$$\n",
    "\n",
    "This is the easiest rule and can be computed in one line. Let us see what the computer does with this example. First we use the `Symbolics` package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@variables x\n",
    "D = Differential(x)\n",
    "\n",
    "y = 8\n",
    "D(y) # we can see that this now gives us the correct formulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(y)) # solution to the problem -- we see that this is the same as our answer. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we try the `Zygote` package. This uses automatic differentiation. Once again, we will not explain what this method entails. If you want to read more on the topic you are more than welcome. For those that are interested in machine learning and deep learning it will be compulsory to learn more about the concept of automatic differentiation at some point. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = 8\n",
    "\n",
    "f'(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that there is no answer here. This seems strange. Let us another approach with this rule. We can calculate the gradient at a certain point in another way.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Zygote.gradient(y -> 8, 8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the output is `nothing` here. This shows why we had no output in the previous code. When `nothing` is returned then there will be nothing to display. Another option for automatic differentiation is the `ForwardDiff` package. Let us quickly look at an example with this package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ForwardDiff.derivative.(f, 8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we get an answer of zero. Which is more in line with what one would expect. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Power rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For any positive integer $k$ the derivative of $f(x) = x^{k}$ at $x_0$ is, \n",
    "\n",
    "$$\n",
    "f'(x_0) = k \\cdot x_{0}^{k - 1}\n",
    "$$\n",
    "\n",
    "An example of this would be the following. Find the answer to \n",
    "\n",
    "$$\n",
    "f'(x) = x ^ 3\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What do you think the answer should be? Let us check with the computer to see what we get. First we try symbolic differentiation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = x ^ 3\n",
    "D(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Does this answer match with what you calculated by hand? Next we try automatic differentiation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = x ^ 3\n",
    "\n",
    "f'(x) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems that `Zygote` gives the same answer. Let us evaluate this function at a particular point with `ForwardDiff`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f'(5) == ForwardDiff.derivative(f, 5) # evaluated at the point x = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f'(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chain rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The chain rule is a bit more complicated then some of the other rules, but it is used frequently in economics. If we have a function $f(x) = p(q(x)) = (p \\circ q)(x)$ which is a composite of two differentiable functions $p(x)$ and $q(x)$ then the derivative at $x_0$ according to the chain rule is, \n",
    "\n",
    "$$\n",
    "f^{\\prime } (x_0 )=p^{\\prime } (q(x_0 ))\\cdot q^{\\prime } (x_0 )\n",
    "$$\n",
    "\n",
    "A good example of where this is applicable is the function \n",
    "\n",
    "$$\n",
    "f(x) = \\sqrt{(5x - 8)}\n",
    "$$\n",
    "\n",
    "In this case we have a composition of two functions, $p(x) = \\sqrt{x}$ and $q(x) = 5x - 8$. Here we need to take the derivatives separately and combine with our rule. We start with $p'(x)$. We haven't really talked about taking the derivative of a square root. So how do we proceed? Well, our square root can actually be written as $p(x)^{1/2}$. Now we can use our power rule from before to get $p'(x) = \\frac{x^{-1/2}}{2}$. Next we need to take the derivative of $q(x)$, which gives is $q'(x) = 5$ using the power rule (since $5 = 5 \\cdot x^{(1 - 1)}$). Now we combine all the components as per the chain rule, \n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "f'\\left( x \\right) & = p'\\left( {q\\left( x \\right)} \\right)\\,\\,q'\\left( x \\right)\\\\ \n",
    "&  = p'\\left( {5x - 8} \\right)\\,\\,q'\\left( x \\right)\\\\ \n",
    "&  = \\frac{1}{2}{\\left( {5x - 8} \\right)^{ - \\frac{1}{2}}}\\,\\left( 5 \\right)\\\\ \n",
    "&  = \\frac{1}{{2\\sqrt {5x - 8} }}\\,\\,\\left( 5 \\right)\\\\ \n",
    "&  = \\frac{5}{{2\\sqrt {5x - 8} }}\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us check whether the computer gives the same answers, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = sqrt(5x - 8)\n",
    "D(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = sqrt(5x - 8)\n",
    "\n",
    "f'(x) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sum (difference) rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given functions $p$ and $q$ that are differentiable at $x$ and with $f(x) = p(x) + q(x)$ we have that the derivative according to the sum rule is given by, \n",
    "\n",
    "$$\n",
    "f^{\\prime } (x)=p^{\\prime } (x)+q^{\\prime } (x)\n",
    "$$\n",
    "\n",
    "Let us consider an example with the application of this rule. Find the derivative of $f(x) = 2x^5 + 7$. In this case, $p(x) = 2x^5$ and $q(x) = 7$. \n",
    "\n",
    "$$\n",
    "\\begin{align*} \n",
    "f'(x)&=\\dfrac{d}{dx}\\left(2x^5+7\\right)\\\\\n",
    "&=\\dfrac{d}{dx}(2x^5)+\\dfrac{d}{dx}(7) & & \\text{Apply the sum rule.}\\\\\n",
    "&=2\\dfrac{d}{dx}(x^5)+\\dfrac{d}{dx}(7) & & \\text{Apply the constant multiple rule.}\\\\ \n",
    "&=2(5x^4)+0 & & \\text{Apply the power rule and the constant rule.}\\\\\n",
    "&=10x^4 & & Simplify. \n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, we check the answer on the computer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = 2x^5 + 7\n",
    "D(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = 2x^5 + 7\n",
    "\n",
    "f'(x) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are the most important rules that we will use the most. The following rules are also used, but less frequently. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Product rule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given functions $p$ and $q$ that are differentiable at $x$ and with $f(x)=p(x)\\cdot q(x)$ we have that the derivative is, \n",
    "\n",
    "$$\n",
    "f^{\\prime } (x)=p^{\\prime } (x)\\cdot q(x)+p(x)\\cdot q^{\\prime } (x)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the following example, find $f'(x) = (x^2+2)(3x^3−5x)$ by applying the product rule. \n",
    "\n",
    "If we set $p(x)=x2+2$ and $q(x)=3x3−5x$, then $p'(x)=2x$ and $q'(x)=9x2−5$ and therefore, \n",
    "\n",
    "$$\n",
    "f'(x)=p'(x)q(x)+q'(x)p(x)=(2x)(3x^3−5x)+(9x^2−5)(x^2+2)\n",
    "$$\n",
    "\n",
    "If we simplify this we have $f'(x) = 15x4+3x2−10$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check against the answer from computer, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = (x^2 + 2)*(3x^3 − 5x)\n",
    "D(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check against the automatic differentiation answer, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = (x^2 + 2)*(3x^3 − 5x)\n",
    "\n",
    "f'(x) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exponential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The exponential function is used a lot in economics. We will often have functions such as $f(x) = \\exp(a \\cdot x)$. The derivative in this case is given by, \n",
    "\n",
    "$$\n",
    "f'(x) = a \\cdot \\exp(a \\cdot x)\n",
    "$$\n",
    "\n",
    "In economics you will find the exponential function in the compounding of interest. The accumulated value of interest that is compounded continuously is given by \n",
    "\n",
    "$$\n",
    "A(t) = Pe^{rt}\n",
    "$$\n",
    "\n",
    "If you wanted to take a derivative of this function with respect to the variable $t$ then you would get the following answer, \n",
    "\n",
    "$$\n",
    "A'(t) = r \\cdot Pe^{rt}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "P = 10; r = 0.05 # choose arbitrary values for P and r\n",
    "\n",
    "f(x) = P * exp(r*x)\n",
    "\n",
    "f'(x) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logarithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we have the derivative of a log function, which is used almost everywhere in economics. If we have the function $f(x) = \\log(x)$ then the derivative is given by, \n",
    "\n",
    "$$\n",
    "f'(x) = 1 / x\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = log(x)\n",
    "\n",
    "f'(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Higher order derivatives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With these rules you should be able to tackle most problems that involve derivatives. These derivatives that we calculated were first order derivatives. We can actually take the derivative again with respect to the variable of interest and then we would have **second order derivatives**. The second order derivative is then a derivative of the first order derivative. \n",
    "\n",
    "While the first order derivative provides a rate of change (often a slope of a tangent line), the second order derivative gives us information on the rate of change of the rate of change. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Cobb-Douglas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can illustrate the usage of derivatives in economics with a basic example. Consider the Cobb-Douglas production function (which we will see again in the notebook on the Solow model). This is a simple explicit form for a production function that we often encounter in economics. The form of the function is as follows, \n",
    "\n",
    "$$\n",
    "F(K,L)=K^{\\alpha } \\cdot L^{\\beta}\n",
    "$$\n",
    "\n",
    "where $F$ is the production function and it is a function of the capital ($K$) and labour ($L$) inputs. In order to make this a univariate example, we fix the value of $K$, so that it isn't variable. The derivative with respect to labour gives us the marginal product of labour. The derivative is then given as, \n",
    "\n",
    "$$\n",
    "\\frac{{\\textrm{d}} Y(K,L)}{dL } = MPL(K,L)=(\\beta)\\cdot K^{\\alpha } \\cdot L^{\\beta - 1}\n",
    "$$\n",
    "\n",
    "The derivative that we have obtained above is simply another function and we can take additional derivatives. The second order derivative is given by, \n",
    "\n",
    "$$\n",
    "\\frac{{\\textrm{d}}^2 Y(K,L)}{dL^2 }=(\\beta )\\cdot (\\beta -1)\\cdot K^{\\alpha } \\cdot L^{\\beta -2}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate the derivatives with `Symbolics` as well, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@variables K L α β\n",
    "\n",
    "D = Differential(L)\n",
    "\n",
    "Y = K ^ (α) * L ^ (β)\n",
    "D(Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D(Y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "D2 = Differential(L) * Differential(L)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "D2(Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(D2(Y)) # second order derivative"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We seem to get the same answer as above if we do a quick check."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Curvature and second derivative"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our Cobb-Douglas example, we will graph the second derivative when $\\beta = 0.5$. The production function is concave. For any function that is twice continuously differentiable, the function is concave if and only if its second derivative is non-positive. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "α = 0.5; β = 0.5\n",
    "\n",
    "K = 1\n",
    "\n",
    "@variables L\n",
    "\n",
    "D = Differential(L)\n",
    "\n",
    "Y = K ^ (α) * L ^ (β)\n",
    "D(Y)\n",
    "\n",
    "diff_1 = expand_derivatives(D(Y));\n",
    "diff_2 = expand_derivatives(D2(Y));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(Y, 0.2, 3, label = \"f(x)\", lw = 2, alpha = 0.7)\n",
    "plot!(diff_1, 0.2, 3, label = \"First derivative\", lw = 2, ls = :dashdot, alpha = 0.7)\n",
    "plot!(diff_2, 0.2, 3, label = \"Second derivative\", legend = :bottomright, lw = 2, ls = :dash, alpha = 0.7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the first derivative is positive, which indicates that the slope of the function is always positive. In terms of the curvature though, the second derivative is always negative, which indicates that the slope of the function is positive but decreasing with an increase in labour. This means that the marginal return to labour is positive but decreasing with an increase in labour. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conditions for local maxima"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Previously we looked at the definitions for global and local maxima. In this section we describe the necessary conditions for optimality with respect to derivatives. In the univariate case (with only one variable), which we have mostly looked at thus far, the first and second order necessary condition for a local maxima is the following,  \n",
    "\n",
    "1. **FONC:** $f'(x^*) =0$\n",
    "2. **SONC** $f''(x^*) \\leq 0$ (and $f''(x^*) \\geq 0$ for local minima)\n",
    "2. (**SOSC** $f''(x^*) < 0$ (and $f''(x^*) > 0$ for local minima))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An easier way to determine the turning points for functions to use the first and second order conditions. Say that we want to determine the minima and maxima of the following function,\n",
    "\n",
    "$$\n",
    "f(x) = 200 - 30x + 8x^2 - 1/2x^3\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = 200 - 30 * x + 8 * x^2 - 0.5 * x^3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(f, 0, 12, lw = 2, legend = false)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It appears from the graph that there are going to be two turning points. We want to calculate the local maximum in the interval $[0, 12]$. First we take the first order condition to find the place where the slope is zero. Then we look at the second order condition to determine if it is a maximum or minimum. For this we are going to be using a root finding package in Julia, called `Roots`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "roots = find_zeros(f', (0, 12)) # we are essentially setting f'(x) = 0 with this package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot(f, 0, 12, lw = 2, legend = false)\n",
    "vline!([roots[1], roots[2]], lw = 2, alpha = 0.6, ls = :dash, color = :black, xticks = ([roots[1], roots[2]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to check if the second order derivatives evaluated at the turning points are positive or negative in order to determine if they are maxima or minima. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f''(roots[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This value is positive, so we don't believe that this is a maximum. We are looking for the maximum, and for that to be true we must have that the second order derivative is negative. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f''(roots[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we see that the value is negative, which indicates to us that this is a **maximum**. We can also look at original quadratic utility function to see whether that was a maximum. The code to find this maximum is the same as above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "α = 0.2 \n",
    "\n",
    "U(x) = x .- α .* x .^ 2\n",
    "\n",
    "utility_roots = find_zeros(U', (-10, 10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we get the correct answer, which differs marginally from the value we retrieved from the brute force grid method we used before. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Total revenue of the firm is given by $TR = 20Q - 2Q^2$ and the total cost function is $TC = Q^3 - 8Q^{2} + 20Q + 2$. Find the level of output that maximises total profit. First do this by hand and then try to solve the problem on the computer and see if you get the same answer. Verify that this level of output satisfies the condition that marginal revenue is equal to marginal cost. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions of multiple variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we talk about how to deal with functions $f: \\mathbb{R}^{n} \\rightarrow \\mathbb{R}$. This means functions that take values in $\\mathbb{R}^n$ and return a value along the real line. An example of this would be, \n",
    "\n",
    "$$\n",
    "f(x, y) = x^{2} + y^{2}\n",
    "$$\n",
    "\n",
    "This is a function that takes in two inputs and returns one output. We will mostly be working with functions that have two inputs. In other words, those functions for which $n = 2$, so that we are operating initially in $\\mathbb{R}^2$. Before we continue, I am going to be using two helper functions, that will allow us to easily some 3D plots. Do not worry too much about the code in the `minmax()` and `mmplotter()` functions, they are a bit more difficult to understand fully on the first reading. You are, however, equipped with all the tools to understand what is happening. This will simply require a careful reading of the first tutorial to figure out. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting in 3D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function minmax()\n",
    "\t\n",
    "\tv = collect(range(-2, stop = 2, length = 30))  # values\n",
    "\tmini = [x^2 + y^2 for x in v, y in v]\n",
    "\tmaxi = -mini   # max is just negative min\n",
    "\tsaddle = [x^2 + y^3 for x in v, y in v]\n",
    "\t\n",
    "\treturn Dict(:x => v,:min => mini, :max => maxi, :saddle => saddle)\n",
    "end;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function mmplotter(s::Symbol)\n",
    "\t\n",
    "    d = minmax()\n",
    "\n",
    "    surface(d[:x], d[:x], d[s], fillalpha = 0.7, legend = false, fillcolor =:viridis)\n",
    "end;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function mmcontour(s::Symbol)\n",
    "\t\n",
    "    d = minmax()\n",
    "\n",
    "    contour(d[:x], d[:x], d[s], fillalpha = 0.5, legend = false, fill = true, fillcolor = :viridis, color = :black)\n",
    "end;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can plot a set of points $(x, y, f(x, y))$ using a surface plot. The following provides the plot for the function $f(x, y) = x^{2} + y^{2}$ that we mentioned earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mmplotter(:max)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the surface plot, we can have a contour plot, which shows a top down view of the surface plot above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mmcontour(:max)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Partial derivatives\n",
    "\n",
    "For a function $f: \\mathbb{R}^{n} \\rightarrow \\mathbb{R}$ the concept of a derivative is extended to partial derivatives for $n$ variables. The partial of $x$ is defined by holding $y$ constant while a derivative in $x$ is taken. We saw an example of this with our calculation of marginal product of labour in a previous example.  \n",
    "\n",
    "The partial derivative of a function $f$ with respect to the variable $x$, can be written as $\\frac{\\partial{f}}{\\partial{x}}$, which is defined as,\n",
    "\n",
    "$$\n",
    "\\dfrac{∂f}{∂x}=f_x(x,y)=\\lim_{h→0}\\dfrac{f(x+h,y)−f(x,y)}{h}\n",
    "$$\n",
    "\n",
    "In this definition we see a new symbol, $\\partial$, which indicates a partial derivative. In this case we can also write the derivative with respect to $y$ so there are two partial derivatives for this function, since it is a function of two variables. \n",
    "\n",
    "The gradient of $f$, which is referred to as $\\nabla f$, is the vector valued function of partial derivatives $[\\partial f / \\partial x, \\partial f / \\partial y]$.\n",
    "\n",
    "#### Calculating partial derivatives by hand\n",
    "\n",
    "If we wanted to calculate the partial derivative for our some function we would have to do the following. Consider the following function, \n",
    "\n",
    "$$\n",
    "f(x,y)=x^2−3xy+2y^2−4x+5y−12\n",
    "$$\n",
    "\n",
    "The partial derivative of $f$ with respect to $x$ can be calculated using our definition above. First calculate $f(x + h, y)$,\n",
    "\n",
    "$$\n",
    "\\begin{align*} f(x+h,y) &=(x+h)^2−3(x+h)y+2y^2−4(x+h)+5y−12 \\\\ &=x^2+2xh+h^2−3xy−3hy+2y^2−4x−4h+5y−12. \\end{align*} \n",
    "$$\n",
    "\n",
    "Now we need to substitute this into our initial equation above and then simplify, \n",
    "\n",
    "$$\n",
    "\\begin{align*} \\dfrac{∂f}{∂x} &=\\lim_{h→0}\\dfrac{f(x+h,y)−f(x,y)}{h} \\\\ \n",
    "&=\\lim_{h→0}\\dfrac{(x^2+2xh+h^2−3xy−3hy+2y^2−4x−4h+5y−12)−(x^2−3xy+2y^2−4x+5y−12)}{h} \\\\ &=\\lim_{h→0}\\dfrac{x^2+2xh+h^2−3xy−3hy+2y^2−4x−4h+5y−12−x^2+3xy−2y^2+4x−5y+12}{h} \\\\ \n",
    "&=\\lim_{h→0}\\dfrac{2xh+h^2−3hy−4h}{h}\\\\ \n",
    "&=\\lim_{h→0}\\dfrac{h(2x+h−3y−4)}{h} \\\\ \n",
    "&=\\lim_{h→0}(2x+h−3y−4) \\\\ \n",
    "&=2x−3y−4. \\end{align*}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same thing can be done for the partial derivative for $f$ with respect to $y$. As an exercise, you might want to try and do this for $y$. The answer should be $-3x + 4y + 5$. \n",
    "\n",
    "Now, we won't always use this method to find the partial derivative. There are easier ways than using this definition above. For the partial derivative, we can think about it as taking a derivative of the one variable while holding the other constant. Exactly like we did for our example with the Cobb-Douglas production function.  \n",
    "\n",
    "So the easier way to take a partial derivative of $f$ with respect to $x$ would be to treat the variable $y$ like it was a constant and apply the usual single variable calculus rules. \n",
    "\n",
    "Let us consider the individual components of the function. \n",
    "\n",
    "In the case of $x^{2}$, we would have that the derivative is $2x$. \n",
    "\n",
    "For $-3xy$ we treat the $y$ like a constant, so the derivative is $-3y$.\n",
    "\n",
    "The next term, $2y^2$ falls away completely since it is constant. The same is true for $5y$ and $12$.\n",
    "\n",
    "Finally, we are left with $-4x$, with a derivative given by $-4$. \n",
    "\n",
    "If we then add these components together, we get our answer $2x - 3y - 4$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Differentiation via the computer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can compute the partials derivatives using the `Symbolics` package as follows, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@variables x y\n",
    "\n",
    "l = x^2 − 3x*y + 2y^2 − 4x + 5y − 12\n",
    "\n",
    "H = Differential(x);\n",
    "I = Differential(y);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "expand_derivatives(H(l)) # partial derivative of f(x, y) wrt to x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we can use automatic differentiation to calculate the partial derivatives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f(x) = x[1]^2 − 3x[1]*x[2] + 2x[2]^2 − 4x[2] + 5x[2] − 12\n",
    "\n",
    "∇f(x) = ForwardDiff.gradient(f, x)\n",
    "\n",
    "∂f_∂x(x, y) = ∇f([x, y])[1];\n",
    "∂f_∂y(x, y) = ∇f([x, y])[2];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this we can now find the gradient at any particular point, say for example we want the partial derivative of $f(x, y)$ with respect to $x$ at the point $(1, 2)$ we can do the following, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "∂f_∂x(1, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This has been a brief introduction to unconstrained optimisation. We will continue this discussion in our next tutorial. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.7.2",
   "language": "julia",
   "name": "julia-1.7"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}