🍄 2110446 Data Science Course at Chulalongkorn University (2022)

Welcome to the Data Science Course (2110446) at Chulalongkorn University! This repository contains the code, exercises, and resources to guide you through the fascinating world of data science, deep learning, and more. Each week, you'll dive into new concepts, reinforced with hands-on labs, to help you build a strong foundation in data science.

📚 Weekly Labs and Exercises

Week 1: Intro to Numpy, Pandas

1. Numpy: Introduction to numerical computing with Numpy.

2. Pandas: Basic data manipulation with Pandas.

3. Pandas with Youtube stat data: Analyzing YouTube video statistics with Pandas.

4. (Advanced) Pandas with Youtube stat data: Advanced analysis of YouTube video statistics.

5. Assignment (Pandas with Youtube stat data): Hands-on assignment analyzing YouTube video statistics.

Week 2: Data Preparation

1. EDA: Exploratory Data Analysis on loan data.

2. Impute Missing Value: Handling missing data in loan datasets.

3. Split Train/Test: Splitting datasets into training and testing sets.

4. Outliers with Log: Identifying and handling outliers using logarithmic transformation.

5. Outliers with Log (Titanic DataSet): Advanced outlier analysis using the Titanic dataset.

6. Assignment: Titanic dataset analysis assignment.

Week 3: Statistical Analysis

1. Basic Stat: Introduction to basic statistical analysis.

2. Intermediate Stat: Intermediate statistical techniques and their applications.

3. Assignment (Stat): Statistical analysis homework assignment.

Week 4: Regression

1. Assignment: Regression analysis assignment on bank marketing data.

Week 5: Traditional ML

1. Decision Trees: Implementing decision trees and random forests.

2. Linear Regression: Applying linear regression to predict outcomes.

3. Logistic Regression: Building and evaluating logistic regression models.

4. Neural Network: Introduction to basic neural networks for classification.

Week 6: Introduction to Deep Learning

1. Image Classification (Basic): Classify flowers using basic image classification techniques.
   

2. Image Classification (Advanced): Explore advanced flower classification with EfficientNet and pretrained weights.
   

3. Semantic Segmentation (UNET): Segment images of pets using the Oxford-IIIT pet dataset and UNET.
   

4. LSTM for Stock Price Prediction: Predict stock prices with Long Short-Term Memory (LSTM) networks.
   

5. SARIMAX for PM2.5 Forecasting: Forecast PM2.5 levels using the SARIMAX model.
   

   Assignment: Fashion MNIST Classification

   

Week 7: Data Extraction

1. Basic Web Scraping: Learn the basics of web scraping to extract data from web pages.
   

2. Wikipedia Data Extraction: Extract structured data from Wikipedia pages.
   

3. REST API Data Extraction: Interact with APIs to gather data programmatically.
   

4. Twitter Data Extraction: Collect data from Twitter using its API.
   

5. Web Automation with Selenium: Automate web browsing and data collection using Selenium.
   

   Assignment: Web Scraping Assignment

Week 8: Data Ingestion

All relevant code and scripts are available here.

1. Kafka Sample Producer: Learn how to produce data streams with Kafka.
   

2. Kafka Sample Consumer: Consume data streams from Kafka topics.
   

3. Kafka with AVRO: Handle data streams with AVRO serialization.

   • Producer: Open In GitHub
   • Consumer: Open In GitHub

4. Sensor Data Ingestion: Work with sensor data streams.

   • FileWriter Consumer: Open In GitHub
   • Counter Consumer: Open In GitHub

Download all source code here (week8_dataingestion.zip).

Week9: Spark

1. Basic Spark:
2. Spark SQL:
3. Spark ML:

Data Set:

1. Bank:
2. Star Wars:

Week10: Spark Streaming

1. Basic Spark Streaming:
2. Spark Streaming Window Operations:
3. Basic Structured Streaming:
4. Structured Streaming Window Operations:
5. Structured Streaming and Kafka:

Data Set:

Star Wars:

Week11: Airflow

All code is here: this link

Anyway, you can download all source codes for week11_airflow_and_fastapi through this link (week11_airflow_and_fastapi.zip).

** Updated python codes/notebooks will be posted here shortly before each lecture.

🛠 Environment Setup

The code in this repository is designed to run in Google Colab or a local Python environment. To get started locally, ensure you have Python 3.8+ installed and use the following steps to set up your environment:

git clone https://github.com/kaopanboonyuen/2110446_DataScience_2021s2.git
cd 2110446_DataScience_2021s2
pip install -r requirements.txt

📚 References

1. https://www.kaggle.com/code
2. https://www.tensorflow.org/tutorials
3. https://github.com/topics/machine-learning
4. https://archive.ics.uci.edu/ml/datasets.php
5. https://colab.research.google.com/notebooks/

🎓 License

This project is licensed under the MIT License. See the LICENSE file for more information.

🛡️ Disclaimer

This repository is for educational purposes only. All code and resources are provided as-is, without any guarantees or warranties.

For any questions or feedback, please contact me at Kao Panboonyuen.