session2.md

Session 2 Outline

Overview of Data Pipeline

• Anything you want to do repeatedly....try to automate as much as possible!

• Conceptually identify which components you can automate now vs which ones require enterprise infrastructure / resources

• Naming convention of files in the repo:

  • 1-, 2-, 3-, etc are all Jupyter notebooks
  • A-, B-, C-, etc are all RMarkdown docs

Data Pipeline docs

Notebook Exercise

Run through 1-read-in-data.ipynb.

Run through 2-demo-chart.ipynb.

Demo GitHub workflow to save any changes.

• Pull from master: git pull origin master
• Checkout new branch: git checkout -b clean-data
• Make changes locally
• Stage new changes: git add notebooks/1-read-in-data.ipynb
• Commit the change: git commit -m "Filtering data"
• Push to new branch: git push origin clean-data
• Create pull request
• Merge

References:

• Other Resources to complete data cleaning.
• GitHub Workflow for making a commit.

To Do

1. Make progress on 1-read-in-data.ipynb or create new notebook.
2. Make progress on 2-demo-chart.ipynb or create new notebook.
3. Make at least 1 more commit.