Merge pull request #44 from OptimalFoundation/development

[Patch] Minor (embarassing) import issue + remove setup.py/cfg  + update deps

Author: bhavnicksm

.gitignore

@@ -13,4 +13,6 @@ src/nadir.egg-info/*
 
 nadir.egg-info/*
 
-tests/data/*
+tests/data/*
+
+build/*

README.md

@@ -8,21 +8,22 @@
 
 **Nadir** (pronounced _nay-di-ah_) is derived from the arabic word _nazir_, and means "the lowest point of a space". In optimisation problems, it is equivalent to the point of minimum. If you are a machine learning enthusiast, a data scientist or an AI practitioner, you know how important it is to use the best optimization algorithms to train your models. The purpose of this library is to help optimize machine learning models and enable them to reach the point of nadir in the appropriate context.
 
-PyTorch is a popular machine learning framework that provides a flexible and efficient way of building and training deep neural networks. This library, Nadir, is built on top of PyTorch to provide high-performing general-purpose optimisation algorithms.  
+**Nadir** follows the principles of Simplicity, Modularity and Composabilty. Read more in the [Core Philosophy](#core-philosophy) section. 
 
-# Table of Contents
+## Table of Contents
 
 - [Nadir](#nadir)
 - [Table of Contents](#table-of-contents)
 - [Installation](#installation)
 - [Simple Usage](#simple-usage)
+- [Core Philosphy](#core-philosophy)
 - [Supported Optimisers](#supported-optimisers)
 - [Acknowledgements](#acknowledgements)
 - [Citation](#citation)
 
 
 
-# Installation
+## Installation
 
 You can either choose to install from the PyPI index, in the following manner:
 
@@ -36,7 +37,7 @@ $ pip install git+https://github.com/Dawn-Of-Eve/nadir.git
 ```
 **Note:** Installing from source might lead to a breaking package. It is recommended that you install from PyPI itself.
 
-# Simple Usage
+## Simple Usage
 
 ```python
 import nadir as nd
@@ -52,7 +53,19 @@ optimizer = nd.SGD(model.parameters(), config)
 optimizer.step()
 ```
 
-# Supported Optimisers
+## Core Philosophy
+
+`Nadir` was built to provide a sense of uniformity and integration that might be lacking in the optimisation community, based on the simple idea that optimisers are not islands. They are usually inheriting characteristics from other optimisers and they provide inspiration to other optimisers. So why not make optimisers inheritable, composible and modular objects? 
+
+The core concepts that each optimiser in `Nadir` follows are:
+
+1. **Simplicity** is of key importance. We prefer readability and simplicity over performance. Experiment, test and verify what works and what does not with Nadir. Optimise and write custom fused kernels for your favorite optimisers after, for performance. 
+
+2. **Modularity** means that the each new optimiser should minimise on the extra new logic added by adding or editing only the parts that need editing. If you want to have a different momentum in Adam, you only change the function of Momentum after inheriting Adam. No need to write the entire code from scratch.
+
+3. **Composibility** implies that we can take things from one optimiser and add them to another without much effort. You can build a optimiser that is the mix of RAdam and NAdam with the properties of AdaBelief, if you so desire! That's what makes this library really powerful. 
+
+## Supported Optimisers
 
 | Optimiser 	| Paper 	                                                 |
 |:---------:	|:-----:	                                                 |
@@ -71,12 +84,12 @@ optimizer.step()
 |  **AdaBelief**| https://arxiv.org/pdf/2010.07468v5.pdf                     |
 |  **NAdam**    | http://cs229.stanford.edu/proj2015/054_report.pdf          |
 
-# Acknowledgements
+## Acknowledgements
 
 We would like to thank all the amazing contributors of this project who spent so much effort making this repositary awesome! :heart:
 
 
-# Citation
+## Citation
 
 You can use the _Cite this repository_ button provided by Github or use the following bibtex:
 

ROADMAP.md

@@ -1,5 +1,14 @@
+### 0.2.0
+- [ ] Readily usable micro-test for checking on optimizers
+- [ ] Make Config API optional
+- [ ] Add unit tests for the build
+
+
+
 ### 0.1.0
 
 - [x] Readily usable testing script on MNIST 
-- [ ] Implementation of SGD, Adam in PyPi Module of Nadir 
-- [ ] Have BaseOptimiser class, BaseMomentumOptimiser and BaseAdaptiveOptimiser class
+- [x] Implementation of SGD, Adam in PyPi Module of Nadir 
+- [x] Have BaseOptimiser class, BaseMomentumOptimiser and BaseAdaptiveOptimiser class
+
+

pyproject.toml

@@ -4,14 +4,13 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "nadir"
-version = "0.1.1"
 authors = [
   { name="Bhavnick Minhas", email="bhavnicksm@gmail.com" },
 ]
 maintainers = [
   { name = "Bhavnick Minhas", email="bhavnicksm@gmail.com"},
 ]
-description = "Nadir is a library of bleeding-edge DL optimisers built for speed and functionality in PyTorch for researchers"
+description = "Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻"
 readme = "README.md"
 requires-python = ">=3.7"
 classifiers = [
@@ -23,16 +22,23 @@ classifiers = [
 license = { text = "Apache 2.0"}
 dependencies = [
     "torch>=1.13.1",
+]
+dynamic = ["version"]
+
+[project.optional-dependencies]
+test = [
     "torchvision>=0.14.1",
     "tqdm",
     "wandb"
 ]
 
 [project.urls]
-"Homepage" = "https://github.com/Dawn-Of-Eve/nadir"
-"Bug Tracker" = "https://github.com/Dawn-Of-Eve/nadir/issues"
-
+"Homepage" = "https://github.com/OptimalFoundation/nadir"
+"Bug Tracker" = "https://github.com/OptimalFoundation/nadir/issues"
 
 [tools.setuptools]
 packages = ["nadir"]
-package-dir = {"" = "src"}
+package-dir = {"" = "src"}
+
+[tool.setuptools.dynamic]
+version = {  attr = "nadir.__version__" }

setup.cfg

@@ -17,9 +17,7 @@
 import torch
 import math
 
-from .base import BaseOptimizer
-from .base import BaseConfig
-
+from .adam import Adam, AdamConfig
 
 __all__ = ['RadamConfig', 'Radam']
 
@@ -32,7 +30,7 @@ class RadamConfig(AdamConfig):
   weight_decay : float = 0.
 
 class Radam(Adam):
-  def __init__ (self, params, config : LionConfig = LionConfig()):
+  def __init__ (self, params, config : AdamConfig = AdamConfig()):
     super().__init__(params, config)
     self.config = config