Pure Math Neural Network — 2-Layer MNIST MLP Classifier with Mini Batches, Momentum, L2 Regularization, and Evaluation Tools

This project started as a minimal two-layer neural network built from scratch in Python using only NumPy. Ultimately, I ended up improving on it with more advanced techniques (see features). As I improved on the model, I would stumble upon even better improvments and push myself to implement them without using any ML libraries such as PyTorch or TensorFlow. Feel free to fork and experiment, I found this to be a great project for ML fundementals.

🧠 Features

• 784 inputs -> 128 HL neurons -> 10 outputs
• Fully "manual" forward and backward propagation
• Mini-batch gradient descent with momentum
• L2 regularization
• Matplotlib accuracy/loss visualization and confusion matrix
• ReLU and softmax activations
• Explicit Cross Entropy Loss function

🚀 How to Run

1. Clone the Repository

git clone https://github.com/SawyerAlston/MNIST-NN-Pure-Math.git
cd MNIST-NN-Pure-Math

2. Install Dependencies

This project only uses Numpy (for complex math), Matplotlib (for plotting), and pandas (for data handling).

pip install -r requirements.txt

3. Modify Hyperparameters (optional)

You can change the following hyperparameters in config.py.

learning_rate = 0.01
epochs = 50
lambda_reg = 0.001
beta_mom = 0.85

4. Run Model

Running main.py will train, test, and evaluate results.

python main.py

📈 Model Performance

Stats:

Model will reach ~98% accuracy and ~0.08 loss after 50 epochs.

Iteration #: 0 | Accuracy: 0.8773827956989247 | Loss: 0.4424950074026731
Iteration #: 5 | Accuracy: 0.9697483870967742 | Loss: 0.10922517157056087
Iteration #: 10 | Accuracy: 0.9786 | Loss: 0.0831458624003045
Iteration #: 15 | Accuracy: 0.981666129032258 | Loss: 0.07446231924020297
Iteration #: 20 | Accuracy: 0.9835666666666667 | Loss: 0.07054676251877758
Iteration #: 25 | Accuracy: 0.9839489247311828 | Loss: 0.06814719627101269
Iteration #: 30 | Accuracy: 0.9845833333333334 | Loss: 0.06678000449212522
Iteration #: 35 | Accuracy: 0.9852833333333333 | Loss: 0.06518376537020491
Iteration #: 40 | Accuracy: 0.9852166666666666 | Loss: 0.06465588259409603
Iteration #: 45 | Accuracy: 0.9851166666666666 | Loss: 0.064073599008369
FINAL TEST RESULTS | Accuracy: 0.9788978897889788 | Loss: 0.07987769345952195

---

Evaluation Plots:

Accuracy:

Loss:

Confusion Matrix:

📦 Files

• config.py: configuration file for model
• data.py: downloads and sets up MNIST CSV data
• main.py: runs the model (train and test)
• model.py: core functions (init, forwardprop, backprop, etc.) of the model
• train_test.py: functions to train and test the model
• requirements.txt: required libraries
• Performance Plots/: plots of model performance

📝License

This project is licensed under the MIT License.