Hello Data Engineer's

This is repo of me doing the DataTalks Data Engineering Zoomcamp

This Repo consists of all the Project, Weekly Notes & Answers

1^st Week

• This is about how to setup GCP with postgresql.
• Had a look at the yellow & green taxi data and extracted them
• Had some SQL refresher.

2^nd Week

• This week is about orchestration
• We used Mage for the orchestration. A note about - This is the First orchestrator I had used. I am not familar with airflow. But when looking mage It looks easier to build.
• Extracted the taxi data and did some tranformations and loaded into postgres database and also in google stoage bucket

3^rd Week

• What is data warehouse ?
• OLAP vs OLTP
• Had a experience in working with BigQuery
• Loading of Native tables from gcs and External table
• Partitioning & clustering of tables