Highlight: We provide the implementation of balanced Extended Dirichlet strategy (see sim/data/partition.py) to enhance the original Extended Dirichlet strategy. This balanced version is useful when we want to assign an equal number of examples to each client.
Files
main_quadratic.py: The script for simple quadratic functions (that is, Groups 1-3, 6-8 in the paper).main_quadratic2.py: The script for quadratic functions with drastic Hessian change (that is, Groups 4-5, 9-10).save/QF/: The experimental results.plots/plots_QF.ipynb: The script for plotting the experimental results
Commands
The commands for main_quadratic.py.
nohup python main_quadratic.py -R 200 -K 10 -M 2 -P 2 --F1 0.5 1 --F2 0.5 -1 --lr 0 --seed 0 &
nohup python main_quadratic.py -R 200 -K 10 -M 2 -P 2 --F1 1 1 --F2 0 -1 --lr 0 --seed 0 &
nohup python main_quadratic.py -R 200 -K 10 -M 2 -P 2 --F1 0.75 1 --F2 0.25 -1 --lr 0 --seed 0 &
The commands for main_quadratic2.py.
nohup python main_quadratic2.py -R 200 -K 10 -M 2 -P 2 -L 0 --F1 0.5 1 --F2 0.5 -1 --lr 0 --seed 0 &
nohup python main_quadratic2.py -R 200 -K 10 -M 2 -P 2 -L 0 --F1 0.5 10 --F2 0.5 -10 --lr 0 --seed 0 &
Results
Files
main_logistic_regression.py: The script for logistic regression (numpy).sim/data/datasets_LR.py: The script for getting the datasets (for example, a9a, w8a).sim/data/partition_LR: The script for getting the partition maps of examples.save/LR/: The experimental results.plots/plots_LR2.ipynb: The script for plotting the experimental results.
Commands
nohup python main_logistic_regression.py -m LR -d a9a -R 50 -K 5 -M 1000 -P 10 --partition exdirb --alpha 1 100 --optim sgd --l2 0.0001 --lr 1 --global-lr 1.0 --batch-size 8 --seed 0 --eval-every 1 &
Results
The results for the setup
The results for the setup
The results for the setup
The results for the setup
Files
main_cwt.py: The script for training deep neural networks with SFL (specifically, Cyclic Weight Transfer).main_fedavg.py: The script for training deep neural networks with PFL (specifically, FedAvg).save/DNN/: The experimental results.plots/grid_search.ipynb: The script for grid search.plots/plots_DNN.ipynb: The script for plotting the experimental results.
Setups and Commands
nohup python {main_fedavg.py,main_cwt.py} -m wvgg9k4 -d {cifar10} -R 5000 -K 5 -M {500,1000} -P 10 --partition exdir --alpha {1,2,5} 100 --optim sgd --lr {0.00316, 0.01, 0.0316, 0.1, 0.316} --momentum 0 --weight-decay 0.0 --lr-scheduler exp --lr-decay 1.0 --batch-size 20 --seed 0 --eval-every 10 --clip {10,50} --log warning+log --device 0 &
The main differences in the experimental setups for Fashion-MNIST, CIFAR-10, and CINIC-10.
R=2000; M=500; seeds=0,1,2,3,4 # Fashion-MNIST
R=5000; M=500; seeds=0,1,2 # CIFAR-10
R=5000; M=1000; seeds=0,1,2 # CINIC-10
We use clip=10 for main_fedavg.py and clip=50 for main_cwt.py, respectively.
Results
Fashion-MNIST.
"FedAvg1.0_M500,10_K5_R2000,10_cnn_fashionmnist_exdir1,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R2000,10_cnn_fashionmnist_exdir1,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M500,10_K5_R2000,10_cnn_fashionmnist_exdir2,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R2000,10_cnn_fashionmnist_exdir2,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M500,10_K5_R2000,10_cnn_fashionmnist_exdir5,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R2000,10_cnn_fashionmnist_exdir5,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
CIFAR-10.
"FedAvg1.0_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir1,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir1,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir2,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir2,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir5,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M500,10_K5_R5000,10_wvgg9k4_cifar10_exdir5,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
CINIC-10.
"FedAvg1.0_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir1,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir1,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir2,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir2,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
"FedAvg1.0_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir5,100.0_sgd0.1,0.0,0.0_exp1.0_b20_seed0_clip10",
"CWT_M1000,10_K5_R5000,10_wvgg9k4_cinic10_exdir5,100.0_sgd0.01,0.0,0.0_exp1.0_b20_seed0_clip50",
