Implementation of the vanilla federated learning paper : Communication-Efficient Learning of Deep Networks from Decentralized Data.
Experiments are produced on MNIST, Fashion MNIST and CIFAR10 (both IID and non-IID). In case of non-IID, the data amongst the users can be split equally or unequally.
Since the purpose of these experiments are to illustrate the effectiveness of the federated learning paradigm, only simple models such as MLP and CNN are used.
Install all the packages from requirments.txt
- Download train and test datasets manually or they will be automatically downloaded from torchvision datasets.
- Experiments are run on Mnist, Fashion Mnist and Cifar.
- To use your own dataset: Move your dataset to data directory and write a wrapper on pytorch dataset class.
To run the experiement you must run the following commands.
- Firstly start the Server
python src/rest.py
- Secondly run the client:
python src/client.py
You can change the default values of other parameters to simulate different conditions. Refer to the options section.
The default values for various paramters parsed to the experiment are given in options.py. Details are given some of those parameters:
--dataset:Default: 'mnist'. Options: 'mnist', 'fmnist', 'cifar'--model:Default: 'mlp'. Options: 'mlp', 'cnn'--gpu:Default: None (runs on CPU). Can also be set to the specific gpu id.--epochs:Number of rounds of training.--lr:Learning rate set to 0.01 by default.--verbose:Detailed log outputs. Activated by default, set to 0 to deactivate.--seed:Random Seed. Default set to 1.--random:Randomize the dataset. Default is False--broken:0.05 of all the images will be skipped. Default is False--lr_type:Learningrate Options. Default is static learningrate.--weight:Decide the weight decay. Default is 1e-4
--iid:Distribution of data amongst users. Default set to IID. Set to 0 for non-IID.--num_users:Number of users. Default is 100.--frac:Fraction of users to be used for federated updates. Default is 0.1.--local_ep:Number of local training epochs in each user. Default is 10.--local_bs:Batch size of local updates in each user. Default is 10.--unequal:Used in non-iid setting. Option to split the data amongst users equally or unequally. Default set to 0 for equal splits. Set to 1 for unequal splits.