This repo includes data and codes used during the MSDS692, Data Acquisition class in Fall 2025.
Each folder indicates the day of the lecture (Ex. Day1, Day2, etc.) and includes corresponding example codes. Please note that I encourage you to work on Skeleton codes first, rather than just taking a look at example answer codes. Data are stored under the Data directory.
Detailed course schedules and slides are in the course Canvas page.
The following is the overall weekly learning plans.
- Week 1 Introduction to ETL, Virtual Environment, GCP Setup
- Week 2 Git Branch, File-based Acquisition and Cloud Storage, Streamlit Application
- Week 3 APIs for Programmatic Access
- Week 4 Creating APIs using FastAPI, GCP Cloud Runm, Containerization and Docker
- Week 5 Static Web Scraping, Dynamic Web Scraping
- Week 6 Dynamic Web Scraping, Retrieval from databases using SQLAlchemy
- Week 7 Chatbot Development using LLMs