Momentum-added Stochastic Solver (MaSS): implemented in Tensorflow(Keras)
MaSS (Momentum-added Stochastic Solver) is an accelerated stochastic gradient method for training over-parametrized models. The code for the algorithm is in the file optimizers.py
Python: >= 3.5.2 Tensorflow: >= 1.8.0 Keras: 2.1.5
An example experiment: training a ResNet-32 using MaSS to classify the CIFAR-10 images. Run the code:
$ python3 train.py
or (specify CUDA device)
$ CUDA_VISIBLE_DEVICES=0 python3 train.py
Our experiment achieves on average 92.8% classification accuracy on the test set of CIFAR-10.