Merch-Store-DE-Task

This is an end-to-end data engineering task for merch store E-commerce platform which is aimed at creating batch processing pipeline for their internal data analytics team to automate the process of data extraction, transformation of data to meet business needs and loading the data to a unified data platform for the team to facilitate data analytics to answer business questios by the stakeholders.

Project Requirements And Dependencies

PostgreSQL database -- data source

Apache Airflow -- orchestration tool

Google Cloud Storage bucket -- data lake

Google Big Query -- data warehouse for analytics

Google Looker studio --- data visualization/ reporting tool

How To Make Use Of The Project

Link to medium article:

https://medium.com/@chibuokejuliet/end-to-end-data-engineering-project-using-apache-airflow-postgres-and-gcp-c385e5b306ef