Sign-SGD via Parameter-Free Optimization

This repository contains the code for experiments applying a range of optimization methods

Requirements for LLM experiments

To install requirements:

pip install -r requirements.txt

Training and Evaluation for LLM experiments

To train and evaluate the model in the paper, run this command:

./run_script.sh

VIT experiments

Code for experiments with Visual transformers can be found in the folder "vit"