Merge pull request #6 from com-480-data-visualization/ms2

MS2

Author: gruvw

.gitignore

@@ -18,4 +18,11 @@
 .ionide
 
 .DS_Store
-data/*
+
+# Data files
+data/*
+data/**/*
+!data/**/
+!data/*.md
+!data/processed/*.md
+!data/**/.gitkeep

README.md

@@ -17,8 +17,8 @@ Project of Data Visualization (EPFL COM-480) - 2026
 
 ## Deliverables
 
-- [Milestone 1](./deliverables/MS1.md)
-- [Milestone 2](./deliverables/MS2.md)
-- [Milestone 3](./deliverables/MS3.md)
+- [Milestone 1](./deliverables/ms1/ms1.md)
+- [Milestone 2](./deliverables/ms2/ms2.pdf)
+- [Milestone 3](./deliverables/ms3/ms3.md)
 
 **Note**: Each deliverable comes with its associated GitHub release of the repository.

data/README.md

@@ -6,4 +6,11 @@ This folder contains the datasets that we will be using for our project. We have
 
 - nba_database/ : Contains player and team statistics for every NBA game from 1947 to the present. It is historical and comprehensive, updated daily. It provides a solid foundation for exploring basketball history, player performance, and team dynamics : https://www.kaggle.com/api/v1/datasets/download/eoinamoore/historical-nba-data-and-player-box-scores
 
-- nba_play_by_play_shot_data/ : A large-scale play-by-play and shot-detail dataset covering both NBA and WNBA games, collected from multiple public sources (e.g., official league APIs and stats sites). It provides every in-game event—from period starts, jump balls, fouls, turnovers, rebounds, and field-goal attempts through free throws—along with detailed shot metadata (shot location, distance, result, assisting player, etc.) : https://www.kaggle.com/api/v1/datasets/download/brains14482/nba-playbyplay-and-shotdetails-data-19962021
+- nba_play_by_play_shot_data/ : A large-scale play-by-play and shot-detail dataset covering both NBA and WNBA games, collected from multiple public sources (e.g., official league APIs and stats sites). It provides every in-game event—from period starts, jump balls, fouls, turnovers, rebounds, and field-goal attempts through free throws—along with detailed shot metadata (shot location, distance, result, assisting player, etc.) : https://www.kaggle.com/api/v1/datasets/download/brains14482/nba-playbyplay-and-shotdetails-data-19962021
+
+- nba_salary/ : Contains player/team salary data for the NBA from 1990 to 2026, scraped from HoopsHype (https://eu.hoopshype.com/salaries/players/ & https://eu.hoopshype.com/salaries/teams/) using the script in src (scrapping for each year, then merge into one csv, finally match to the personId with the player database for future use). It includes player names, seasons, and corresponding salaries : https://drive.google.com/drive/folders/1AI6Z8fIpP7RxhImdtOexrn6dgmB05izz?usp=share_link
+
+- nba_team_colors/ : Contains the main color palette for each NBA team (source: https://www.trucolor.net/portfolio/national-basketball-association-official-colors-franchise-records-1946-1947-through-present/) : https://drive.google.com/drive/folders/1-yfOkoxvzeTDWPNqtZtBHt1_TOxrQw8o?usp=sharing
+
+- processed/ : Contains the processed data files that are used for visualization. The files in this folder are generated from the raw datasets after cleaning, transforming, and aggregating the data to make it suitable for analysis and visualization : https://drive.google.com/drive/folders/1fipeswQbiNC3rnPsH2NQ-axi6x91Rk9n?usp=share_link
+

data/nba_salary/.gitkeep

@@ -0,0 +1,117 @@
+# Processed Data
+
+This folder contains the processed data files that are used for visualization.
+
+Can be found here: 
+
+## Description of files
+
+### player_seasons.csv
+
+Contains aggregated season-level statistics for players. Each row corresponds to a player's performance in a specific season, with the following columns:
+
+- `season`: The NBA season (e.g., 2020 for the 2020-2021 season)
+- `personId`: Unique identifier for the player (NBA player ID)
+- `firstName`, `lastName`: Player's first and last name
+- `gameType`: Regular Season or Playoffs
+- `teamScore`: Average number of points scored by the player's team in the season
+- `opponentScore`: Average number of points scored by the opponent teams in the season
+- `points`, `assists`, `rebounds`, `blocks`, `steals`, `turnovers`: Average per game general statistics for the player in the season
+- `pointsTotal`, `assistsTotal`, `reboundsTotal`, `blocksTotal`, `stealsTotal`, `turnoversTotal`: Sum throughout the season of the corresponding statistics for the player
+- `fieldGoalsMade`, `fieldGoalsAttempted`, `threePointersMade`, `threePointersAttempted`, `freeThrowsMade`, `freeThrowsAttempted`: Sum throughout the season of the corresponding shot statistics for the player
+- `plusMinusPoints`: Average plus/minus points per game for the player in the season (i.e., the average point differential when the player is on the court)
+- `foulsPersonal`: Average personal fouls per game for the player in the season
+- `numMinutes`: Average minutes played per game for the player in the season
+- `win`: Number of games won by the player's team in the season
+- `gamesPlayed`: Number of games played by the player in the season
+- `proportionThreePoint`: Proportion of three-point shots attempted out of total field goal attempts for the player in the season
+- `fieldGoalsPercentage`, `threePointersPercentage`, `freeThrowsPercentage`: Shooting percentages for the player in the season
+- `salary`: Average salary for the player in the season (if available, otherwise NaN)
+
+### player_games.csv
+
+TODO
+
+### shot_events.csv
+
+TODO
+
+### player_metadata.csv
+
+Contains metadata about players (from nba_api), with the following columns:
+
+- `personId`: Unique identifier for the player (NBA player ID)
+- `firstName`: Player's first name
+- `lastName`: Player's last name
+- `birthDate`: Player's birth date (in datetime format)
+- `height`: Player's height in centimeters
+- `weight`: Player's weight in pounds
+- `nbSeasons`: Number of seasons the player has played in the NBA
+- `jerseyNumber`: The jersey number the player wore in the NBA (e.g., 23)
+- `position`: The position the player played in the NBA (e.g., "Guard", "Forward", "Center")
+- `startYear`: The year the player started playing in the NBA
+- `endYear`: The year the player ended playing in the NBA
+- `draftYear`: The year the player was drafted into the NBA
+- `draftRound`: The round in which the player was drafted
+- `draftNumber`: The number of the pick in the draft
+
+### team_seasons.csv
+
+Contains aggregated season-level statistics for teams. Each row corresponds to a team's performance in a specific season, with the following columns:
+
+- `season`: The NBA season (e.g., 2020 for the 2020-2021 season)
+- `teamId`: Unique identifier for the team (NBA team ID)
+- `gameType`: Regular Season or Playoffs
+- `teamName`: Name of the team (e.g., "Lakers")
+- `teamCity`: City of the team (e.g., "Los Angeles")
+- `teamScore`: Average number of points scored by the team in the season
+- `opponentScore`: Average number of points scored by the opponent teams in the season
+- `assists`, `rebounds`, `blocks`, `steals`, `turnovers`: Average per game general statistics for the team in the season
+- `teamScoreTotal`, `opponentScoreTotal`, `assistsTotal`, `reboundsTotal`, `blocksTotal`, `stealsTotal`, `turnoversTotal`: Sum throughout the season of the corresponding statistics for the team
+- `fieldGoalsMade`, `fieldGoalsAttempted`, `threePointersMade`, `threePointersAttempted`, `freeThrowsMade`, `freeThrowsAttempted`: Average per game shot statistics for the team in the season
+- `fieldGoalsPercentage`, `threePointersPercentage`, `freeThrowsPercentage`: Shooting percentages for the team in the season
+- `proportionThreePoint`: Proportion of three-point shots attempted out of total field goal attempts for the team in the season
+- `plusMinusPoints`: Average plus/minus points per game for the team in the season (i.e., the average point differential when the team is on the court)
+- `foulsPersonal`: Average personal fouls per game for the team in the season
+- `numMinutes`: Average minutes played per game for the team in the season
+- `win`: Number of games won by the team in the season
+- `losses`: Number of games lost by the team in the season
+- `gamesPlayed`: Number of games played by the team in the season (should be 82 for regular season, but can be less for older seasons or playoffs)
+- `salary`: Average salary for the team in the season (if available, otherwise NaN)
+
+
+### team_games.csv
+
+Contains detailed game-level statistics for teams. Each row corresponds to a team's performance in a specific game, with the following columns:
+
+- `gameId`: Unique identifier for the game
+- `gameDateTimeEst`: Date and time of the game in Eastern Standard Time
+- `teamCity`: City of the team (e.g., "Los Angeles")
+- `teamName`: Name of the team (e.g., "Lakers")
+- `teamId`: Unique identifier for the team (NBA team ID)
+- `opponentTeamCity`: City of the opponent team (e.g., "Boston")
+- `opponentTeamName`: Name of the opponent team (e.g., "Celtics")
+- `opponentTeamId`: Unique identifier for the opponent team (NBA team ID)
+- `home`: Boolean indicating if the team was playing at home (0) or away (1)
+- `win`: Boolean indicating if the team won (1) or lost (0) the game
+- `season`: The NBA season (e.g., 2020 for the 2020-2021 season)
+- `gameType`: Regular Season or Playoffs
+- `teamScore`: Number of points scored by the team in the game
+- `opponentScore`: Number of points scored by the opponent team in the game
+- `numMinutes`: Duration of the game in minutes (should be 48 for regular season, but can be more for overtime games)
+- `assists`, `rebounds`, `reboundsDefensive`, `reboundsOffensive`, `blocks`, `steals`, `turnovers`: General statistics for the team in the game
+- `foulsPersonal`: Number of fouls committed by the team in the game
+- `fieldGoalsMade`, `fieldGoalsAttempted`, `threePointersMade`, `threePointersAttempted`, `freeThrowsMade`, `freeThrowsAttempted`: Shot statistics for the team in the game
+- `fieldGoalsPercentage`, `threePointersPercentage`, `freeThrowsPercentage`: Shooting percentages for the team in the game
+
+
+### team_metadata.csv
+
+Contains metadata about teams, with the following columns:
+
+- `teamId`: Unique identifier for the team (NBA team ID)
+- `teamAbbrev`: Abbreviation of the team name (e.g., "LAL" for Los Angeles Lakers)
+- `teamSlug`: Slug version of the team name (e.g., "los-angeles-lakers" should be used to get the team logo from the https://i.logocdn.com/nba/{year}/{teamSlug}.svg URL)
+- `Color1`, `Color2`, `Color3`, `Color4`, `Color5`: Colors associated with the team (in hexadecimal format, e.g., "#552583")
+- `seasonFounded`: The season in which the team was founded (e.g., 1947 for the first NBA season)
+- `seasonActiveTill`: The most recent season in which the team was active (e.g., 2100 for the active teams, or a past season for defunct teams)

data/nba_team_colors/.gitkeep

@@ -38,11 +38,50 @@ Please, fill the following sections about your project.
 
 **10% of the final grade**
 
+Two A4 pages describing the project goal.
+
+- Include sketches of the vizualiation you want to make in your final product.
+- List the tools that you will use for each visualization and which (past or future) lectures you will need.
+- Break down your goal into independent pieces to implement. Try to design a core visualization (minimal viable product) that will be required at the end.
+- Then list extra ideas (more creative or challenging) that will enhance the visualization but could be dropped without endangering the meaning of the project.
+
+Functional project prototype review.
+
+- You should have an initial website running with the basic skeleton of the visualization/widgets.
 
 ## Milestone 3 (29th May, 5pm)
 
 **80% of the final grade**
 
+For the final milestone, you need to create a **cool, interactive, and sufficiently complex D3.js (and other) data viz** on the dataset you chose.
+Tell a data story and explain it to the targeted audience.
+Create a **process book** that details how you got there, from the original idea to the final product.
+
+You need to deliver the following things:
+
+- 1. Github repository with a README
+    * Host the code and data on Github (if data is too big, link to a cloud storage) with your process book as a PDF file.
+    * Add a README file that explains the technical setup and intended usage.
+    * Code should be clean, manageable, and using the latest practices.
+- 2. Screencast
+    * Demonstrate what you can do with your viz in a fun, engaging and impactful manner.
+    * Talk about your main contributions rather than on technical details.
+    * **2 min video (not more).**
+- 3. Process book (max 8 pages)
+  * Describe the path you took to obtain the final result.
+  * Explain challenges that you faced and design decisions that you took.
+  * Reuse the sketches/plans that you made for the first milestone, expanding them and explaining the changes.
+  * Care about the visual/design of this report.
+  * **Peer assessment:** include a breakdown of the parts of the project completed by each team member.
+
+Grading Rubric:
+
+- Visualization: 35%
+- Technical Implementation: 15%
+- Screencast: 25%
+- Process book: 25%
+
+**Note:** Grades may vary across the team members, based on the process book and the peer assessment process. Please provide clear explanations.
 
 ## Late policy
 

data/processed/README.md

@@ -51,17 +51,17 @@ Even for people who aren't analytics nerds, it will be fun to know how the game
 We already explored our three main datasets to get a sense of their structure, coverage, and quality.
 Here's what we found.
 
-**Note**: You can check our full [data exploration notebook](../scripts/data_exploration/exploration.ipynb), but we also reported the main results below for your convenience.
+**Note**: You can check our full [data exploration notebook](../../scripts/data_exploration/exploration.ipynb), but we also reported the main results below for your convenience.
 
-- **Player statistics**: The dataset contains over 1.6 million rows, covering games from November 1946 all the way to the current 2025-26 season. Each row represents a single player's performance in a single game, with 35 columns including points, assists, rebounds, shooting percentages, and +/- impact. As a quick sanity check, looking at the 2015-16 season, Stephen Curry tops the scoring chart at 30.1 PPG, followed by James Harden and Kevin Durant — which matches historical records perfectly.</br><img src="../scripts/data_exploration/images/2015_players_table.png" width="600">
+- **Player statistics**: The dataset contains over 1.6 million rows, covering games from November 1946 all the way to the current 2025-26 season. Each row represents a single player's performance in a single game, with 35 columns including points, assists, rebounds, shooting percentages, and +/- impact. As a quick sanity check, looking at the 2015-16 season, Stephen Curry tops the scoring chart at 30.1 PPG, followed by James Harden and Kevin Durant — which matches historical records perfectly.</br><img src="../../scripts/data_exploration/images/2015_players_table.png" width="600">
 
-- **Team statistics**: Around 145'000 rows, also starting from 1946, with 48 columns per game entry. This includes all the usual box score stats plus richer game level data like fast break points, paint points, biggest lead, and lead changes. Looking at the 2015-16 season, the Warriors season stands out immediately with 73 wins, the all-time regular season record.</br><img src="../scripts/data_exploration/images/2015_teams_table.png" width="600">
+- **Team statistics**: Around 145'000 rows, also starting from 1946, with 48 columns per game entry. This includes all the usual box score stats plus richer game level data like fast break points, paint points, biggest lead, and lead changes. Looking at the 2015-16 season, the Warriors season stands out immediately with 73 wins, the all-time regular season record.</br><img src="../../scripts/data_exploration/images/2015_teams_table.png" width="600">
 
-- **Shot Details**: This dataset only starts from the 1996 season (when the NBA began tracking spatial shot data), but it's by far the richest one in terms of granularity. For the 2015-16 season alone there are ~208'000 shot attempts, each with court coordinates (`LOC_X`, `LOC_Y`), shot zone, distance, and whether it went in. Shot locations are centered on the basket, with x coordinates ranging from -250 to 250 (in tenths of a foot) and y from slightly negative (behind the backboard) up to the half-court line.</br><img src="../scripts/data_exploration/images/2015_barplot_shot_details.png" width="600">
+- **Shot Details**: This dataset only starts from the 1996 season (when the NBA began tracking spatial shot data), but it's by far the richest one in terms of granularity. For the 2015-16 season alone there are ~208'000 shot attempts, each with court coordinates (`LOC_X`, `LOC_Y`), shot zone, distance, and whether it went in. Shot locations are centered on the basket, with x coordinates ranging from -250 to 250 (in tenths of a foot) and y from slightly negative (behind the backboard) up to the half-court line.</br><img src="../../scripts/data_exploration/images/2015_barplot_shot_details.png" width="600">
 
-- **Awards**: The awards dataset has 3'465 entries across MVP, Defensive Player of the Year, Rookie of the Year, and more, going back to the 1956 season. It also includes vote share, so we can see not just who won but how dominant the win was (e.g. Nikola Jokić's 2021 MVP at 96.1% of votes). The All-Star selections go back to 1951 with 2'058 entries, and the End of Season Teams (All-NBA, All-Defensive, All-Rookie) cover 2'222 entries since 1947.</br><img src="../scripts/data_exploration/images/awards_table.png" width="600">
+- **Awards**: The awards dataset has 3'465 entries across MVP, Defensive Player of the Year, Rookie of the Year, and more, going back to the 1956 season. It also includes vote share, so we can see not just who won but how dominant the win was (e.g. Nikola Jokić's 2021 MVP at 96.1% of votes). The All-Star selections go back to 1951 with 2'058 entries, and the End of Season Teams (All-NBA, All-Defensive, All-Rookie) cover 2'222 entries since 1947.</br><img src="../../scripts/data_exploration/images/awards_table.png" width="600">
 
-- **Key plots**: The heatmaps below illustrate the coverage and density of the data nicely. The player heatmap shows career scoring totals by season for the top 50 all-time scorers, with Wilt Chamberlain peaking at 4'029 points in a single season, a staggering number. The team heatmap shows win totals by franchise across every season, making historical dominant runs (the 80s Celtics/Lakers, the 90s Bulls, the mid-2010s Warriors) immediately visible.</br><img src="../scripts/data_exploration/images/heatmap_players.png" width="600"></br><img src="../scripts/data_exploration/images/heatmap_teams.png" width="600">
+- **Key plots**: The heatmaps below illustrate the coverage and density of the data nicely. The player heatmap shows career scoring totals by season for the top 50 all-time scorers, with Wilt Chamberlain peaking at 4'029 points in a single season, a staggering number. The team heatmap shows win totals by franchise across every season, making historical dominant runs (the 80s Celtics/Lakers, the 90s Bulls, the mid-2010s Warriors) immediately visible.</br><img src="../../scripts/data_exploration/images/heatmap_players.png" width="600"></br><img src="../../scripts/data_exploration/images/heatmap_teams.png" width="600">
 
 Overall, the **data is in great shape** with very **high quality**.
 There are some minor typing differences we need to be aware of, but nothing that requires heavy cleaning.