4_team_data_scrape_luther.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Project Luther\n",
    "\n",
    "Kenny Leung - kenleung11@gmail.com\n",
    "\n",
    "Part 4/8 - Data scraping NBA team statistics\n",
    "\n",
    "This notebook documents the process of scraping advance statistics for all teams since 2003 from https://www.basketball-reference.com/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# import packages\n",
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "import pandas as pd\n",
    "import time\n",
    "import random\n",
    "import pickle\n",
    "from datetime import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Scraping\n",
    "\n",
    "I hypothesized that a rookie in the NBA will perform better/have a bigger contribution to a team with fewer \"good\" players (how good a player is can be estimated by a player's BPM). Here I am scraping season average advance statistics for all players on each team since 2003. First, I needed to obtain a list of url extensions for each team for each year. I started with getting the urls for all the teams in the 2017 season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = 'https://www.basketball-reference.com/leagues/NBA_2017.html'\n",
    "\n",
    "response = requests.get(url)\n",
    "page = response.text.replace('<!--', '').replace('-->', '')\n",
    "soup = BeautifulSoup(page, \"lxml\")\n",
    "\n",
    "url_list = []\n",
    "for x in soup.findAll(class_=\"table_outer_container\")[5].findAll('a'):\n",
    "    try:\n",
    "        url = x['href']\n",
    "        url_list.append(url)\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/teams/GSW/2017.html',\n",
       " '/teams/HOU/2017.html',\n",
       " '/teams/DEN/2017.html',\n",
       " '/teams/CLE/2017.html',\n",
       " '/teams/WAS/2017.html',\n",
       " '/teams/LAC/2017.html',\n",
       " '/teams/BOS/2017.html',\n",
       " '/teams/POR/2017.html',\n",
       " '/teams/PHO/2017.html',\n",
       " '/teams/TOR/2017.html',\n",
       " '/teams/OKC/2017.html',\n",
       " '/teams/BRK/2017.html',\n",
       " '/teams/MIN/2017.html',\n",
       " '/teams/SAS/2017.html',\n",
       " '/teams/IND/2017.html',\n",
       " '/teams/CHO/2017.html',\n",
       " '/teams/LAL/2017.html',\n",
       " '/teams/NOP/2017.html',\n",
       " '/teams/NYK/2017.html',\n",
       " '/teams/MIL/2017.html',\n",
       " '/teams/MIA/2017.html',\n",
       " '/teams/ATL/2017.html',\n",
       " '/teams/CHI/2017.html',\n",
       " '/teams/SAC/2017.html',\n",
       " '/teams/PHI/2017.html',\n",
       " '/teams/DET/2017.html',\n",
       " '/teams/ORL/2017.html',\n",
       " '/teams/UTA/2017.html',\n",
       " '/teams/MEM/2017.html',\n",
       " '/teams/DAL/2017.html']"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scrape for urls for all teams since 2003\n",
    "\n",
    "Now to get the url extensions for all teams since 2003. Noted that I cannot simply change the year in the url since there have been a few organizations that moved their team to another city, thus changing the url. For example, prior to 2008, the Oklahoma City Thunder were the Seattle Supersonics, and in 2012 the New Jersey Nets moved their team to Brooklyn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2003\n",
      "2004\n",
      "2005\n",
      "2006\n",
      "2007\n",
      "2008\n",
      "2009\n",
      "2010\n",
      "2011\n",
      "2012\n",
      "2013\n",
      "2014\n",
      "2015\n",
      "2016\n",
      "2017\n"
     ]
    }
   ],
   "source": [
    "url_template = 'https://www.basketball-reference.com/leagues/NBA_{year}.html'\n",
    "url_list = []\n",
    "\n",
    "for year in range(2003, 2018):\n",
    "    print(year)\n",
    "    url = url_template.format(year=year)  # get the url\n",
    "\n",
    "    response = requests.get(url)\n",
    "    page = response.text.replace('<!--', '').replace('-->', '')\n",
    "    soup = BeautifulSoup(page, \"lxml\")\n",
    "    \n",
    "    for x in soup.findAll(class_=\"table_outer_container\")[5].findAll('a'):\n",
    "        try:\n",
    "            url = x['href']\n",
    "            url_list.append(url)\n",
    "        except:\n",
    "            pass\n",
    "        \n",
    "    time.sleep(.5+2*random.random())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# pickle the list\n",
    "with open('team_year_url_list.pkl', 'wb') as fp:\n",
    "    pickle.dump(url_list, fp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with open(\"team_year_url_list.pkl\", 'rb') as picklefile: \n",
    "    url_list = pickle.load(picklefile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scraping the Advance Statistics for players on the 2016-2017 Boston Celtics\n",
    "\n",
    "Now that I have the url extensions for each team since 2003, I can loop through the list to scrape all the data I need. First, I wanted to make sure the code works by focusing on the Boston Celtics' 2016-2017 season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url = 'https://www.basketball-reference.com/teams/BOS/2017.html'\n",
    "\n",
    "response = requests.get(url)\n",
    "page = response.text.replace('<!--', '').replace('-->', '')\n",
    "soup = BeautifulSoup(page, \"lxml\")\n",
    "\n",
    "Year = int(soup.find_all('h1')[0].getText().split('\\n')[1][:4]) # get year\n",
    "Team = soup.find_all('h1')[0].getText().split('\\n')[2] # get team name\n",
    "\n",
    "column_headers = [th.getText() for th in soup.findAll(class_=\"overthrow table_container\")[7].findAll('th')][0:27]\n",
    "data_rows = soup.findAll(class_=\"overthrow table_container\")[7].findAll('tr')[1:]\n",
    "player_data = [[td.getText() for td in data_rows[i].findAll('td')] for i in range(len(data_rows))]\n",
    "\n",
    "for player in player_data:\n",
    "    player.insert(0, '') # insert an empty data column\n",
    "    \n",
    "for player in player_data:\n",
    "    player.insert(0, Year) # insert a column data for year\n",
    "    \n",
    "for player in player_data:\n",
    "    player.insert(0, Team) # insert a column data for team name\n",
    "    \n",
    "column_headers.insert(0,\"Team\") # insert team header\n",
    "column_headers.insert(0,\"Year\") # insert year header\n",
    "\n",
    "df = pd.DataFrame(player_data, columns=column_headers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scraping the Advance Statistics for each player on each team since 2003\n",
    "\n",
    "Now that the code works, I looped through the url_list I obtained above to get the advance stats for all players on each team for each season since 2003."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "url_template2 = \"https://www.basketball-reference.com{team_year}\"\n",
    "team_year_df = pd.DataFrame()\n",
    "\n",
    "for team_year in url_list:\n",
    "    print(team_year)\n",
    "    url = url_template2.format(team_year = team_year)\n",
    "    \n",
    "    response = requests.get(url)\n",
    "    page = response.text.replace('<!--', '').replace('-->', '')\n",
    "    soup = BeautifulSoup(page, \"lxml\")\n",
    "    \n",
    "    Year = int(soup.find_all('h1')[0].getText().split('\\n')[1][:4])\n",
    "    Team = soup.find_all('h1')[0].getText().split('\\n')[2]\n",
    "    \n",
    "    column_headers = [th.getText() for th in soup.findAll(class_=\"overthrow table_container\")[7].findAll('th')][0:27]\n",
    "    data_rows = soup.findAll(class_=\"overthrow table_container\")[7].findAll('tr')[1:]\n",
    "    player_data = [[td.getText() for td in data_rows[i].findAll('td')] for i in range(len(data_rows))]\n",
    "\n",
    "    for player in player_data:\n",
    "        player.insert(0, '')\n",
    "    \n",
    "    for player in player_data:\n",
    "        player.insert(0, Year)\n",
    "    \n",
    "    for player in player_data:\n",
    "        player.insert(0, Team)\n",
    "    \n",
    "    column_headers.insert(0,\"Year\")\n",
    "    column_headers.insert(0,\"Team\")\n",
    "    \n",
    "    df = pd.DataFrame(player_data, columns=column_headers)\n",
    "    team_year_df = team_year_df.append(df, ignore_index=True)\n",
    "\n",
    "    time.sleep(.5+2*random.random())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df.columns = ['Team', 'Year', 'null', 'Player', 'Age', 'G', 'MP', 'PER', 'TS%',\n",
    "       '3PAr', 'FTr', 'ORB%', 'DRB%', 'TRB%', 'AST%', 'STL%', 'BLK%', 'TOV%',\n",
    "       'USG%', 'null', 'OWS', 'DWS', 'WS', 'WS/48', 'null', 'OBPM', 'DBPM', 'BPM',\n",
    "       'VORP']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# dropped the null columns\n",
    "team_year_df = team_year_df.drop(['null'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# save to csv file\n",
    "team_year_df.to_csv('team_year_raw.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df = pd.read_csv('team_year_raw.csv',index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scraping the Per Game Statistics for each player on each team since 2003"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/teams/DAL/2003.html\n",
      "/teams/GSW/2003.html\n",
      "/teams/SAC/2003.html\n",
      "/teams/LAL/2003.html\n",
      "/teams/MIL/2003.html\n",
      "/teams/ORL/2003.html\n",
      "/teams/MIN/2003.html\n",
      "/teams/MEM/2003.html\n",
      "/teams/PHI/2003.html\n",
      "/teams/IND/2003.html\n",
      "/teams/NYK/2003.html\n",
      "/teams/SAS/2003.html\n",
      "/teams/PHO/2003.html\n",
      "/teams/NJN/2003.html\n",
      "/teams/POR/2003.html\n",
      "/teams/CHI/2003.html\n",
      "/teams/UTA/2003.html\n",
      "/teams/ATL/2003.html\n",
      "/teams/NOH/2003.html\n",
      "/teams/LAC/2003.html\n",
      "/teams/HOU/2003.html\n",
      "/teams/BOS/2003.html\n",
      "/teams/SEA/2003.html\n",
      "/teams/WAS/2003.html\n",
      "/teams/CLE/2003.html\n",
      "/teams/DET/2003.html\n",
      "/teams/TOR/2003.html\n",
      "/teams/MIA/2003.html\n",
      "/teams/DEN/2003.html\n",
      "/teams/DAL/2004.html\n",
      "/teams/SAC/2004.html\n",
      "/teams/LAL/2004.html\n",
      "/teams/MIL/2004.html\n",
      "/teams/DEN/2004.html\n",
      "/teams/SEA/2004.html\n",
      "/teams/MEM/2004.html\n",
      "/teams/BOS/2004.html\n",
      "/teams/LAC/2004.html\n",
      "/teams/MIN/2004.html\n",
      "/teams/PHO/2004.html\n",
      "/teams/ORL/2004.html\n",
      "/teams/GSW/2004.html\n",
      "/teams/CLE/2004.html\n",
      "/teams/ATL/2004.html\n",
      "/teams/NYK/2004.html\n",
      "/teams/NOH/2004.html\n",
      "/teams/WAS/2004.html\n",
      "/teams/SAS/2004.html\n",
      "/teams/IND/2004.html\n",
      "/teams/POR/2004.html\n",
      "/teams/MIA/2004.html\n",
      "/teams/NJN/2004.html\n",
      "/teams/DET/2004.html\n",
      "/teams/HOU/2004.html\n",
      "/teams/CHI/2004.html\n",
      "/teams/UTA/2004.html\n",
      "/teams/PHI/2004.html\n",
      "/teams/TOR/2004.html\n",
      "/teams/PHO/2005.html\n",
      "/teams/SAC/2005.html\n",
      "/teams/DAL/2005.html\n",
      "/teams/MIA/2005.html\n",
      "/teams/BOS/2005.html\n",
      "/teams/WAS/2005.html\n",
      "/teams/TOR/2005.html\n",
      "/teams/DEN/2005.html\n",
      "/teams/ORL/2005.html\n",
      "/teams/PHI/2005.html\n",
      "/teams/SEA/2005.html\n",
      "/teams/LAL/2005.html\n",
      "/teams/GSW/2005.html\n",
      "/teams/NYK/2005.html\n",
      "/teams/MIL/2005.html\n",
      "/teams/MIN/2005.html\n",
      "/teams/CLE/2005.html\n",
      "/teams/SAS/2005.html\n",
      "/teams/LAC/2005.html\n",
      "/teams/HOU/2005.html\n",
      "/teams/CHI/2005.html\n",
      "/teams/CHA/2005.html\n",
      "/teams/MEM/2005.html\n",
      "/teams/DET/2005.html\n",
      "/teams/IND/2005.html\n",
      "/teams/UTA/2005.html\n",
      "/teams/POR/2005.html\n",
      "/teams/ATL/2005.html\n",
      "/teams/NJN/2005.html\n",
      "/teams/NOH/2005.html\n",
      "/teams/PHO/2006.html\n",
      "/teams/SEA/2006.html\n",
      "/teams/WAS/2006.html\n",
      "/teams/TOR/2006.html\n",
      "/teams/DEN/2006.html\n",
      "/teams/MIA/2006.html\n",
      "/teams/LAL/2006.html\n",
      "/teams/PHI/2006.html\n",
      "/teams/DAL/2006.html\n",
      "/teams/SAC/2006.html\n",
      "/teams/GSW/2006.html\n",
      "/teams/BOS/2006.html\n",
      "/teams/MIL/2006.html\n",
      "/teams/CHI/2006.html\n",
      "/teams/CLE/2006.html\n",
      "/teams/ATL/2006.html\n",
      "/teams/LAC/2006.html\n",
      "/teams/CHA/2006.html\n",
      "/teams/DET/2006.html\n",
      "/teams/NYK/2006.html\n",
      "/teams/SAS/2006.html\n",
      "/teams/ORL/2006.html\n",
      "/teams/IND/2006.html\n",
      "/teams/NJN/2006.html\n",
      "/teams/NOK/2006.html\n",
      "/teams/UTA/2006.html\n",
      "/teams/MEM/2006.html\n",
      "/teams/MIN/2006.html\n",
      "/teams/HOU/2006.html\n",
      "/teams/POR/2006.html\n",
      "/teams/PHO/2007.html\n",
      "/teams/GSW/2007.html\n",
      "/teams/DEN/2007.html\n",
      "/teams/WAS/2007.html\n",
      "/teams/LAL/2007.html\n",
      "/teams/MEM/2007.html\n",
      "/teams/UTA/2007.html\n",
      "/teams/SAC/2007.html\n",
      "/teams/DAL/2007.html\n",
      "/teams/MIL/2007.html\n",
      "/teams/TOR/2007.html\n",
      "/teams/SEA/2007.html\n",
      "/teams/CHI/2007.html\n",
      "/teams/SAS/2007.html\n",
      "/teams/NJN/2007.html\n",
      "/teams/NYK/2007.html\n",
      "/teams/HOU/2007.html\n",
      "/teams/CHA/2007.html\n",
      "/teams/CLE/2007.html\n",
      "/teams/MIN/2007.html\n",
      "/teams/DET/2007.html\n",
      "/teams/BOS/2007.html\n",
      "/teams/LAC/2007.html\n",
      "/teams/IND/2007.html\n",
      "/teams/NOK/2007.html\n",
      "/teams/PHI/2007.html\n",
      "/teams/ORL/2007.html\n",
      "/teams/MIA/2007.html\n",
      "/teams/POR/2007.html\n",
      "/teams/ATL/2007.html\n",
      "/teams/GSW/2008.html\n",
      "/teams/DEN/2008.html\n",
      "/teams/PHO/2008.html\n",
      "/teams/LAL/2008.html\n",
      "/teams/UTA/2008.html\n",
      "/teams/ORL/2008.html\n",
      "/teams/IND/2008.html\n",
      "/teams/SAC/2008.html\n",
      "/teams/NOH/2008.html\n",
      "/teams/MEM/2008.html\n",
      "/teams/BOS/2008.html\n",
      "/teams/DAL/2008.html\n",
      "/teams/TOR/2008.html\n",
      "/teams/WAS/2008.html\n",
      "/teams/ATL/2008.html\n",
      "/teams/SEA/2008.html\n",
      "/teams/DET/2008.html\n",
      "/teams/CHI/2008.html\n",
      "/teams/CHA/2008.html\n",
      "/teams/MIL/2008.html\n",
      "/teams/NYK/2008.html\n",
      "/teams/HOU/2008.html\n",
      "/teams/PHI/2008.html\n",
      "/teams/CLE/2008.html\n",
      "/teams/NJN/2008.html\n",
      "/teams/MIN/2008.html\n",
      "/teams/SAS/2008.html\n",
      "/teams/POR/2008.html\n",
      "/teams/LAC/2008.html\n",
      "/teams/MIA/2008.html\n",
      "/teams/PHO/2009.html\n",
      "/teams/GSW/2009.html\n",
      "/teams/LAL/2009.html\n",
      "/teams/NYK/2009.html\n",
      "/teams/IND/2009.html\n",
      "/teams/DEN/2009.html\n",
      "/teams/UTA/2009.html\n",
      "/teams/CHI/2009.html\n",
      "/teams/DAL/2009.html\n",
      "/teams/ORL/2009.html\n",
      "/teams/BOS/2009.html\n",
      "/teams/SAC/2009.html\n",
      "/teams/CLE/2009.html\n",
      "/teams/POR/2009.html\n",
      "/teams/MIL/2009.html\n",
      "/teams/TOR/2009.html\n",
      "/teams/HOU/2009.html\n",
      "/teams/MIA/2009.html\n",
      "/teams/ATL/2009.html\n",
      "/teams/NJN/2009.html\n",
      "/teams/MIN/2009.html\n",
      "/teams/PHI/2009.html\n",
      "/teams/SAS/2009.html\n",
      "/teams/OKC/2009.html\n",
      "/teams/WAS/2009.html\n",
      "/teams/NOH/2009.html\n",
      "/teams/LAC/2009.html\n",
      "/teams/DET/2009.html\n",
      "/teams/MEM/2009.html\n",
      "/teams/CHA/2009.html\n",
      "/teams/PHO/2010.html\n",
      "/teams/GSW/2010.html\n",
      "/teams/DEN/2010.html\n",
      "/teams/UTA/2010.html\n",
      "/teams/TOR/2010.html\n",
      "/teams/ORL/2010.html\n",
      "/teams/MEM/2010.html\n",
      "/teams/HOU/2010.html\n",
      "/teams/CLE/2010.html\n",
      "/teams/NYK/2010.html\n",
      "/teams/DAL/2010.html\n",
      "/teams/LAL/2010.html\n",
      "/teams/ATL/2010.html\n",
      "/teams/OKC/2010.html\n",
      "/teams/SAS/2010.html\n",
      "/teams/IND/2010.html\n",
      "/teams/NOH/2010.html\n",
      "/teams/SAC/2010.html\n",
      "/teams/BOS/2010.html\n",
      "/teams/MIN/2010.html\n",
      "/teams/POR/2010.html\n",
      "/teams/PHI/2010.html\n",
      "/teams/MIL/2010.html\n",
      "/teams/CHI/2010.html\n",
      "/teams/MIA/2010.html\n",
      "/teams/WAS/2010.html\n",
      "/teams/LAC/2010.html\n",
      "/teams/CHA/2010.html\n",
      "/teams/DET/2010.html\n",
      "/teams/NJN/2010.html\n",
      "/teams/DEN/2011.html\n",
      "/teams/NYK/2011.html\n",
      "/teams/HOU/2011.html\n",
      "/teams/PHO/2011.html\n",
      "/teams/OKC/2011.html\n",
      "/teams/SAS/2011.html\n",
      "/teams/GSW/2011.html\n",
      "/teams/MIA/2011.html\n",
      "/teams/LAL/2011.html\n",
      "/teams/MIN/2011.html\n",
      "/teams/DAL/2011.html\n",
      "/teams/MEM/2011.html\n",
      "/teams/IND/2011.html\n",
      "/teams/UTA/2011.html\n",
      "/teams/SAC/2011.html\n",
      "/teams/ORL/2011.html\n",
      "/teams/TOR/2011.html\n",
      "/teams/PHI/2011.html\n",
      "/teams/LAC/2011.html\n",
      "/teams/CHI/2011.html\n",
      "/teams/WAS/2011.html\n",
      "/teams/DET/2011.html\n",
      "/teams/BOS/2011.html\n",
      "/teams/POR/2011.html\n",
      "/teams/CLE/2011.html\n",
      "/teams/ATL/2011.html\n",
      "/teams/NOH/2011.html\n",
      "/teams/NJN/2011.html\n",
      "/teams/CHA/2011.html\n",
      "/teams/MIL/2011.html\n",
      "/teams/DEN/2012.html\n",
      "/teams/SAS/2012.html\n",
      "/teams/OKC/2012.html\n",
      "/teams/UTA/2012.html\n",
      "/teams/MIL/2012.html\n",
      "/teams/SAC/2012.html\n",
      "/teams/MIA/2012.html\n",
      "/teams/PHO/2012.html\n",
      "/teams/HOU/2012.html\n",
      "/teams/MIN/2012.html\n",
      "/teams/NYK/2012.html\n",
      "/teams/GSW/2012.html\n",
      "/teams/IND/2012.html\n",
      "/teams/LAC/2012.html\n",
      "/teams/LAL/2012.html\n",
      "/teams/POR/2012.html\n",
      "/teams/ATL/2012.html\n",
      "/teams/CHI/2012.html\n",
      "/teams/DAL/2012.html\n",
      "/teams/MEM/2012.html\n",
      "/teams/ORL/2012.html\n",
      "/teams/WAS/2012.html\n",
      "/teams/PHI/2012.html\n",
      "/teams/NJN/2012.html\n",
      "/teams/CLE/2012.html\n",
      "/teams/BOS/2012.html\n",
      "/teams/DET/2012.html\n",
      "/teams/TOR/2012.html\n",
      "/teams/NOH/2012.html\n",
      "/teams/CHA/2012.html\n",
      "/teams/DEN/2013.html\n",
      "/teams/HOU/2013.html\n",
      "/teams/OKC/2013.html\n",
      "/teams/SAS/2013.html\n",
      "/teams/MIA/2013.html\n",
      "/teams/LAL/2013.html\n",
      "/teams/GSW/2013.html\n",
      "/teams/DAL/2013.html\n",
      "/teams/LAC/2013.html\n",
      "/teams/SAC/2013.html\n",
      "/teams/NYK/2013.html\n",
      "/teams/MIL/2013.html\n",
      "/teams/UTA/2013.html\n",
      "/teams/ATL/2013.html\n",
      "/teams/POR/2013.html\n",
      "/teams/TOR/2013.html\n",
      "/teams/BRK/2013.html\n",
      "/teams/CLE/2013.html\n",
      "/teams/MIN/2013.html\n",
      "/teams/BOS/2013.html\n",
      "/teams/PHO/2013.html\n",
      "/teams/DET/2013.html\n",
      "/teams/ORL/2013.html\n",
      "/teams/NOH/2013.html\n",
      "/teams/IND/2013.html\n",
      "/teams/CHA/2013.html\n",
      "/teams/MEM/2013.html\n",
      "/teams/WAS/2013.html\n",
      "/teams/CHI/2013.html\n",
      "/teams/PHI/2013.html\n",
      "/teams/LAC/2014.html\n",
      "/teams/HOU/2014.html\n",
      "/teams/MIN/2014.html\n",
      "/teams/POR/2014.html\n",
      "/teams/OKC/2014.html\n",
      "/teams/SAS/2014.html\n",
      "/teams/PHO/2014.html\n",
      "/teams/DAL/2014.html\n",
      "/teams/DEN/2014.html\n",
      "/teams/GSW/2014.html\n",
      "/teams/LAL/2014.html\n",
      "/teams/MIA/2014.html\n",
      "/teams/TOR/2014.html\n",
      "/teams/DET/2014.html\n",
      "/teams/ATL/2014.html\n",
      "/teams/WAS/2014.html\n",
      "/teams/SAC/2014.html\n",
      "/teams/NOP/2014.html\n",
      "/teams/PHI/2014.html\n",
      "/teams/NYK/2014.html\n",
      "/teams/BRK/2014.html\n",
      "/teams/CLE/2014.html\n",
      "/teams/CHA/2014.html\n",
      "/teams/IND/2014.html\n",
      "/teams/ORL/2014.html\n",
      "/teams/BOS/2014.html\n",
      "/teams/MEM/2014.html\n",
      "/teams/MIL/2014.html\n",
      "/teams/UTA/2014.html\n",
      "/teams/CHI/2014.html\n",
      "/teams/GSW/2015.html\n",
      "/teams/LAC/2015.html\n",
      "/teams/DAL/2015.html\n",
      "/teams/TOR/2015.html\n",
      "/teams/OKC/2015.html\n",
      "/teams/HOU/2015.html\n",
      "/teams/SAS/2015.html\n",
      "/teams/CLE/2015.html\n",
      "/teams/POR/2015.html\n",
      "/teams/ATL/2015.html\n",
      "/teams/PHO/2015.html\n",
      "/teams/DEN/2015.html\n",
      "/teams/BOS/2015.html\n",
      "/teams/SAC/2015.html\n",
      "/teams/CHI/2015.html\n",
      "/teams/NOP/2015.html\n",
      "/teams/WAS/2015.html\n",
      "/teams/DET/2015.html\n",
      "/teams/LAL/2015.html\n",
      "/teams/MEM/2015.html\n",
      "/teams/BRK/2015.html\n",
      "/teams/MIL/2015.html\n",
      "/teams/MIN/2015.html\n",
      "/teams/IND/2015.html\n",
      "/teams/ORL/2015.html\n",
      "/teams/UTA/2015.html\n",
      "/teams/MIA/2015.html\n",
      "/teams/CHO/2015.html\n",
      "/teams/PHI/2015.html\n",
      "/teams/NYK/2015.html\n",
      "/teams/GSW/2016.html\n",
      "/teams/OKC/2016.html\n",
      "/teams/SAC/2016.html\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/teams/HOU/2016.html\n",
      "/teams/BOS/2016.html\n",
      "/teams/POR/2016.html\n",
      "/teams/LAC/2016.html\n",
      "/teams/CLE/2016.html\n",
      "/teams/WAS/2016.html\n",
      "/teams/SAS/2016.html\n",
      "/teams/CHO/2016.html\n",
      "/teams/ATL/2016.html\n",
      "/teams/NOP/2016.html\n",
      "/teams/TOR/2016.html\n",
      "/teams/MIN/2016.html\n",
      "/teams/DAL/2016.html\n",
      "/teams/IND/2016.html\n",
      "/teams/ORL/2016.html\n",
      "/teams/DET/2016.html\n",
      "/teams/DEN/2016.html\n",
      "/teams/CHI/2016.html\n",
      "/teams/PHO/2016.html\n",
      "/teams/MIA/2016.html\n",
      "/teams/MEM/2016.html\n",
      "/teams/MIL/2016.html\n",
      "/teams/BRK/2016.html\n",
      "/teams/NYK/2016.html\n",
      "/teams/UTA/2016.html\n",
      "/teams/PHI/2016.html\n",
      "/teams/LAL/2016.html\n",
      "/teams/GSW/2017.html\n",
      "/teams/HOU/2017.html\n",
      "/teams/DEN/2017.html\n",
      "/teams/CLE/2017.html\n",
      "/teams/WAS/2017.html\n",
      "/teams/LAC/2017.html\n",
      "/teams/BOS/2017.html\n",
      "/teams/POR/2017.html\n",
      "/teams/PHO/2017.html\n",
      "/teams/TOR/2017.html\n",
      "/teams/OKC/2017.html\n",
      "/teams/BRK/2017.html\n",
      "/teams/MIN/2017.html\n",
      "/teams/SAS/2017.html\n",
      "/teams/IND/2017.html\n",
      "/teams/CHO/2017.html\n",
      "/teams/LAL/2017.html\n",
      "/teams/NOP/2017.html\n",
      "/teams/NYK/2017.html\n",
      "/teams/MIL/2017.html\n",
      "/teams/MIA/2017.html\n",
      "/teams/ATL/2017.html\n",
      "/teams/CHI/2017.html\n",
      "/teams/SAC/2017.html\n",
      "/teams/PHI/2017.html\n",
      "/teams/DET/2017.html\n",
      "/teams/ORL/2017.html\n",
      "/teams/UTA/2017.html\n",
      "/teams/MEM/2017.html\n",
      "/teams/DAL/2017.html\n"
     ]
    }
   ],
   "source": [
    "url_template2 = \"https://www.basketball-reference.com{team_year}\"\n",
    "team_year_df = pd.DataFrame()\n",
    "\n",
    "for team_year in url_list:\n",
    "    print(team_year)\n",
    "    url = url_template2.format(team_year = team_year)\n",
    "\n",
    "    response = requests.get(url)\n",
    "    page = response.text.replace('<!--', '').replace('-->', '')\n",
    "    soup = BeautifulSoup(page, \"lxml\")\n",
    "\n",
    "    Year = int(soup.find_all('h1')[0].getText().split('\\n')[1][:4])\n",
    "    Team = soup.find_all('h1')[0].getText().split('\\n')[2]\n",
    "\n",
    "    column_headers = [th.getText() for th in soup.findAll(class_=\"overthrow table_container\")[3].findAll('th')][1:28]\n",
    "    data_rows = soup.findAll(class_=\"overthrow table_container\")[3].findAll('tr')[1:]\n",
    "    player_data = [[td.getText() for td in data_rows[i].findAll('td')] for i in range(len(data_rows))]\n",
    "\n",
    "    for player in player_data:\n",
    "        player.insert(0, Year)\n",
    "\n",
    "    for player in player_data:\n",
    "        player.insert(0, Team)\n",
    "\n",
    "    column_headers.insert(0,\"Year\")\n",
    "    column_headers.insert(0,\"Team\")\n",
    "\n",
    "    df = pd.DataFrame(player_data, columns=column_headers)\n",
    "    team_year_df = team_year_df.append(df, ignore_index=True)\n",
    "\n",
    "    time.sleep(.5+2*random.random())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df.columns = ['Team', 'Year', 'Player', 'Age', 'G', 'GS', 'MP', 'FG', 'FGA', 'FG%', '3P',\n",
    "       '3PA', '3P%', '2P', '2PA', '2P%', 'eFG%', 'FT', 'FTA', 'FT%', 'ORB',\n",
    "       'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS/G']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# save to csv file\n",
    "team_year_df.to_csv('team_year_stats.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df = pd.read_csv('team_year_stats.csv',index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sorting and Grouping the data frame\n",
    "\n",
    "One of the features that I think will be an indicator on how well a rookie performs is the BPM stats of the players on the team in the previous year. For example, if a team contains many players with high BPM, it is less likely that a rookie will get an opportunity to contribute. Here, I wanted to group each team, each year, to find and sum the top 3, top 5, and top 7 player's BPMs. I also summed the BPM's of the entire team for each season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "teams = ['Atlanta Hawks', \n",
    "         'Boston Celtics', \n",
    "         'Brooklyn Nets', \n",
    "         'Charlotte Bobcats', \n",
    "         'Charlotte Hornets', \n",
    "         'Chicago Bulls', \n",
    "         'Cleveland Cavaliers', \n",
    "         'Dallas Mavericks', \n",
    "         'Denver Nuggets', \n",
    "         'Detroit Pistons', \n",
    "         'Golden State Warriors', \n",
    "         'Houston Rockets', \n",
    "         'Indiana Pacers', \n",
    "         'Los Angeles Clippers', \n",
    "         'Los Angeles Lakers', \n",
    "         'Memphis Grizzlies', \n",
    "         'Miami Heat', \n",
    "         'Milwaukee Bucks', \n",
    "         'Minnesota Timberwolves', \n",
    "         'New Jersey Nets', \n",
    "         'New Orleans Hornets', \n",
    "         'New Orleans Pelicans', \n",
    "         'New Orleans/Oklahoma City Hornets', \n",
    "         'New York Knicks', \n",
    "         'Oklahoma City Thunder', \n",
    "         'Orlando Magic', \n",
    "         'Philadelphia 76ers', \n",
    "         'Phoenix Suns', \n",
    "         'Portland Trail Blazers', \n",
    "         'Sacramento Kings', \n",
    "         'San Antonio Spurs', \n",
    "         'Seattle SuperSonics', \n",
    "         'Toronto Raptors', \n",
    "         'Utah Jazz', \n",
    "         'Washington Wizards']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "teams_abr = ['ATL', \n",
    "         'BOS', \n",
    "         'BRK', \n",
    "         'CHA', \n",
    "         'CHH', \n",
    "         'CHI', \n",
    "         'CLE', \n",
    "         'DAL', \n",
    "         'DEN', \n",
    "         'DET', \n",
    "         'GSW', \n",
    "         'HOU', \n",
    "         'IND', \n",
    "         'LAC', \n",
    "         'LAL', \n",
    "         'MEM', \n",
    "         'MIA', \n",
    "         'MIL', \n",
    "         'MIN', \n",
    "         'NJN', \n",
    "         'NOH', \n",
    "         'NOP', \n",
    "         'NOK', \n",
    "         'NYK', \n",
    "         'OKC', \n",
    "         'ORL', \n",
    "         'PHI', \n",
    "         'PHO', \n",
    "         'POR', \n",
    "         'SAC', \n",
    "         'SAS', \n",
    "         'SEA', \n",
    "         'TOR', \n",
    "         'UTA', \n",
    "         'WAS']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "team_dict = dict(zip(teams,teams_abr))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def team_abr(name):\n",
    "    '''\n",
    "    Change full team name to abbreviation to match draft dataframe\n",
    "    '''\n",
    "    return team_dict[name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df['Team'] = team_year_df['Team'].apply(team_abr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sorting dataframe based on BPM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "team_year_df;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of advance stats for top 3 players by BPM for each team \n",
    "top3 = team_year_df.sort_values('BPM', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(3) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'BPM': 'BPM_top3'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except bpm\n",
    "top3 = top3[['Team','Year','BPM_top3']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of advance stats for top 5 players by BPM for each team \n",
    "top5 = team_year_df.sort_values('BPM', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(5) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'BPM': 'BPM_top5'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except bpm\n",
    "top5 = top5[['Team','Year','BPM_top5']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# sum of advance stats for top 7 players by BPM for each team \n",
    "top7 = team_year_df.sort_values('BPM', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(7) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'BPM': 'BPM_top7'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except bpm\n",
    "top7 = top7[['Team','Year','BPM_top7']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of advance stats for each team\n",
    "top13 = team_year_df.sort_values('BPM', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(13) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'BPM': 'BPM_top13'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except bpm\n",
    "top13 = top13[['Team','Year','BPM_top13']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# merge dataframes\n",
    "BPMs = pd.merge(top3, top5, how='left', on=['Team', 'Year'])\n",
    "BPMs = pd.merge(BPMs, top7, how='left', on=['Team', 'Year'])\n",
    "BPMs = pd.merge(BPMs, top13, how='left', on=['Team', 'Year'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs.columns = ['Draft_Tm', 'Draft_Yr', 'BPM_top3', 'BPM_top5', 'BPM_top7', 'BPM_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['Yr_Tm'] = BPMs[\"Draft_Yr\"].map(str) + BPMs[\"Draft_Tm\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top3/top5'] = BPMs['BPM_top3']/BPMs['BPM_top5']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top3/top7'] = BPMs['BPM_top3']/BPMs['BPM_top7']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top3/top13'] = BPMs['BPM_top3']/BPMs['BPM_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top5/top7'] = BPMs['BPM_top5']/BPMs['BPM_top7']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top5/top13'] = BPMs['BPM_top5']/BPMs['BPM_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "BPMs['top7/top13'] = BPMs['BPM_top7']/BPMs['BPM_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# save to csv file\n",
    "BPMs.to_csv('team_bpm.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sorting dataframe based on PPG\n",
    "\n",
    "I also wanted to predict rookie year points per game (PPG) to compare whether the model predicts BPM or PPG better."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of stats for top 3 players by PPG for each team \n",
    "top3 = team_year_df.sort_values('PTS/G', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(3) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'PTS/G': 'PTS/G_top3'});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except ppg\n",
    "top3 = top3[['Team','Year','PTS/G_top3']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of stats for top 5 players by PPG for each team \n",
    "top5 = team_year_df.sort_values('PTS/G', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(5) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'PTS/G': 'PTS/G_top5'});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except ppg\n",
    "top5 = top5[['Team','Year','PTS/G_top5']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of stats for top 7 players by PPG for each team \n",
    "top7 = team_year_df.sort_values('PTS/G', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(7) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'PTS/G': 'PTS/G_top7'});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except ppg\n",
    "top7 = top7[['Team','Year','PTS/G_top7']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# sum of stats for top 13 players by PPG for each team \n",
    "top13 = team_year_df.sort_values('PTS/G', ascending=False) \\\n",
    ".groupby(['Team','Year']).head(13) \\\n",
    ".sort_values(['Year','Team']) \\\n",
    ".groupby(['Team','Year']).sum() \\\n",
    ".reset_index() \\\n",
    ".rename(columns={'PTS/G': 'PTS/G_top13'});"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# drop all other columsn except ppg\n",
    "top13 = top13[['Team','Year','PTS/G_top13']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# merge dataframes\n",
    "PPGs = pd.merge(top3, top5, how='left', on=['Team', 'Year'])\n",
    "PPGs = pd.merge(PPGs, top7, how='left', on=['Team', 'Year'])\n",
    "PPGs = pd.merge(PPGs, top13, how='left', on=['Team', 'Year'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs.columns = ['Draft_Tm', 'Draft_Yr', 'PPG_top3', 'PPG_top5', 'PPG_top7', 'PPG_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['Yr_Tm'] = PPGs[\"Draft_Yr\"].map(str) + PPGs[\"Draft_Tm\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top3/top5'] = PPGs['PPG_top3']/PPGs['PPG_top5']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top3/top7'] = PPGs['PPG_top3']/PPGs['PPG_top7']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top3/top13'] = PPGs['PPG_top3']/PPGs['PPG_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top5/top7'] = PPGs['PPG_top5']/PPGs['PPG_top7']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top5/top13'] = PPGs['PPG_top5']/PPGs['PPG_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "PPGs['top7/top13'] = PPGs['PPG_top7']/PPGs['PPG_top13']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# save to csv file\n",
    "PPGs.to_csv('team_ppg.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}