PAPER2CODE Repository

Welcome to the PAPER2CODE repository! This repository is dedicated to the implementation of state-of-the-art research papers in machine learning and deep learning from scratch. Our goal is to bridge the gap between theoretical research and practical implementation, providing clear, educational, and reproducible implementations of the models and methods described in these papers. We implement these models using TensorFlow and PyTorch, depending on the specific requirements of each project.

Repository Structure

The repository is organized into multiple subprojects, each focusing on a different research paper. Here’s an overview of the main directories:

• FC8_VGG/
• ResNet50/
• VisionTransformer/
• Unet/
• GAN/
• NLP-Evolution/

Overview

PAPER2CODE is an ever-growing repository that aims to continuously expand with new implementations of influential research papers in the field of machine learning and deep learning. As of now, the repository includes the following subprojects:

• FCN8_VGG: Implementation of the FCN-8 model based on Long et al. 2015.
• ResNet50: Implementation of the ResNet-50 architecture following He et al. 2016.
• Vision Transformer: Implementation of the Vision Transformer model as described in Dosovitskiy et al. 2020.
• Unet: Implementation of the U-Net model for biomedical image segmentation based on Ronneberger et al. 2015.
• GAN: Implementation of Generative Adversarial Networks based on Goodfellow et al. 2014.
• NLP-Evolution: A collection of implementations for various natural language processing models including Bigram, MLP, RNN, LSTM, GRU, and Transformer models. Each model is explained and implemented according to key papers:
  • Bigram: Predicts the next character based on the previous one.
  • MLP: Follows Bengio et al. 2003.
  • RNN: Based on Mikolov et al. 2010.
  • LSTM: Implements Long Short-Term Memory networks following Graves et al. 2014.
  • GRU: Based on Cho et al. 2014.
  • Transformer: Implements the Transformer model as described in Vaswani et al. 2017.
  • In-Run Data Shapley: Implements a unique data attribution method - In-run Data Shapley as described in Jiachen T. Wang et al. 2024

Getting Started

Prerequisites

To get started with the implementations in this repository, you need the following prerequisites:

• Python 3.7+
• TensorFlow
• PyTorch
• NumPy
• Matplotlib
• OpenCV
• Jupyter Notebook

Installation

1. Clone the repository:

   git clone https://github.com/silvaxxx1/PAPER2CODE.git
   cd PAPER2CODE

2. Install the required dependencies:

   pip install -r requirements.txt

Project-Specific Information

NLP-Evolution

The NLP-Evolution project explores various natural language processing models implemented in PyTorch. It includes implementations for:

• Bigram: A simple model predicting the next character based on the previous one.
• MLP: A multi-layer perceptron for sequence modeling.
• RNN: Recurrent neural network for capturing temporal dependencies.
• LSTM: Long Short-Term Memory networks for improved sequence modeling.
• GRU: Gated Recurrent Units as an alternative to LSTM.
• Transformer: A powerful model leveraging self-attention mechanisms.

For detailed usage and implementation, see the NLP-Evolution README.

Fun Experiment: Building Layers from Scratch

In the from_scratch_for_fun.py file, we implement MLP, LSTM, and GRU layers from scratch using NumPy. This exercise is intended to help gain a deeper intuition about these models and their components. It's not the primary focus of the repository but serves as an educational tool for understanding the underlying mechanisms of these models.

Contributing

If you'd like to contribute to this repository, please follow these steps:

1. Fork the repository.
2. Create a new branch (git checkout -b feature-branch).
3. Make your changes.
4. Commit your changes (git commit -am 'Add new feature').
5. Push to the branch (git push origin feature-branch).
6. Create a new Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.