Machine Learning Algorithms from Scratch 🚀

Introduction

This repository is a collection of machine learning algorithms that I implemented entirely from scratch using only Python. My goal was to deeply understand how these algorithms work under the hood, reinforce my math and programming skills, and gain hands-on experience by building them step by step without relying on libraries like scikit-learn.

What I Learned

• How core ML algorithms function at a mathematical and logical level
• The role of distance metrics, loss functions, activation functions, gradients, and optimization methods
• How to handle edge cases, tuning parameters, and validating results using synthetic datasets
• Debugging and visualizing results to improve model accuracy and intuition

Implemented Algorithms

Here are the 10 most common used machine learning algorithms I implemented from scratch:

1. K-Nearest Neighbors (KNN)

• Implemented using Euclidean distance
• Learned about instance-based learning and the curse of dimensionality

2. Linear Regression

• Implemented both with closed-form solution and gradient descent
• Understood loss functions and line fitting

3. Logistic Regression

• Used sigmoid function and binary cross-entropy loss
• Learned about classification and probabilistic interpretation

4. Decision Tree

• Built using recursive splitting with Gini Impurity
• Understood entropy, tree depth, and overfitting

5. Support Vector Machine (SVM)

• Implemented a linear SVM using gradient descent
• Learned about hinge loss and the concept of margin and hyperplanes

6. Perceptron

• A simple neural model for binary classification
• Reinforced understanding of linear classifiers and iterative learning

7. Naive Bayes Classifier

• Implemented the Naive Bayes classifier from scratch using NumPy
• Tests the classifier using a synthetic dataset from Scikit-learn

8. K-Means Clustering

• Unsupervised learning via iterative centroid updates
• Understood intra-cluster variance and convergence behavior

9. Principal Component Analysis (PCA)

• Implemented using eigen decomposition of the covariance matrix
• Learned about dimensionality reduction and variance maximization

10. Random Forest

• Implemented Random Forestand Deeply understand how decision trees work and how they are used in ensembles
• Learned how bootstrap sampling and feature randomness contribute to model robustness

Goals

• Gain an in-depth understanding of how each algorithm works
• Strengthen my mathematical intuition and Python skills
• Build a strong foundation for more complex ML/AI topics (like deep learning, LLMs, etc.)

Contribution

I welcome contributions, ideas, and improvements from the community. Whether you want to fix a bug, optimize performance, or add a new algorithm, feel free to open an issue or pull request.

---

Thanks for checking out my learning journey through core ML algorithms! 🚀