For this project, I created the Medallion Architecture for data organisation in a lakehouse in Microsoft Fabric. I demonstrated how to level up your data quality from bronze (raw), to silver (validated), and finally to gold (enriched), all ready for super-efficient analytics. This was my in-person presentation for the Microsoft Fabric Nigeria Community and dbrownconsulting on 26th of October 2024.
- Created Lakehouse
- Data Ingestion.
- Silver Data Transformation.
- Gold Data Transformation.
- Data Modelling.
- Data Visualization in Power BI Service.
Datasource: Data was sourced from the popular adventure works data you can download here.
In Synapse Data Engineering, create three Lakehouses called Sales_Bronze, Sales_Silver and Sales_Gold.
.
- In the Sales_Bronze Lakehouse, Upload the Sample Adventureworks Dataset.
In the Sales_Silver Lakehouse, click on new Notebook to transform data for the Sales_Silver Lakehouse or you can import the Silver_Data_Transformation Notebook. Remember to copy/change the ABFS File Path of each table from the Sales_Bronze lakehouse before you Run ALL.
In the Sales_Gold Lakehouse, click on new Notebook to transform data for the Sales_Gold Lakehouse or you can import the Gold_Data_Transformation Notebook. Remember to copy/change the ABFS File Path of each table from the Sales_Bronze lakehouse before you Run ALL.
Now that we have the final data in the Sales_Gold Lakehouse, we need to model the semantic data before visualization in Power BI.
.
In Microsoft Fabric, you can auto-create Power BI reports for your analysis.