This framework called Perceptrovement implements all the algorithms from the bachelor thesis Improving the Perceptron-Algorithm by Tobias Krönke in 2011. Get your hands dirty and find out how the simple, yet famous perceptron-algorithm (Rosenblatt 1962) can help you solve your machine learning problem by incrementally learning a linear binary classification model. It was upgraded with new means to update the current model, voting and surviving model approaches, ideas to reduce the number of required iterations (epochs) over your data and different options to grasp the idea of SVMs' (soft) margins to increase robustness against noisy data and labels.
Included algorithms and functions
- Pegasos, Primal Estimated sub-GrAdient SOlver for SVM
Shai S. Shwartz, Yoram Singer und Nathan Srebro: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In: ICML ’07: Proceedings of the 24th international conference on Machine learning, pp. 807–814, 2007. ACM.
- MICRA, mistake-controlled rule algorithm and reduced epochs
Petroula Tsampouka und John Shawe-Taylor: Approximate maximum margin algorithms with rules controlled by the number of mistakes. In: Proceedings of the 24th international conference on Machine learning, ICML ’07,pp. 903–910, 2007. ACM.
- PA, passive-aggressive family of online binary classification algorithms
Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz und Yoram Singer: Online Passive-Aggressive Algorithms. Journal of Machine Learning Research, 7:551–585, 2006.
- PAUM, perceptron with uneven margins
Yaoyong Li, Hugo Zaragoza, Ralf Herbrich, John Shawe-Taylor und Jaz S. Kandola: The Perceptron Algorithm with Uneven Margins. In: Proceedings of the Nineteenth International Conference on Machine Learning, ICML ’02, pp. 379–386.
- support vector budgeting through removal and projection
Zhuang Wang, Koby Crammer und Slobodan Vucetic: Multi-Class Pegasos on a Budget. In: ICML, 2010.
- lamda-trick and alpha-bound
Roni Khardon und Gabriel Wachman: Noise Tolerant Variants of the Perceptron Algorithm. J. Mach. Learn. Res., 8:227–248, 2007.
- longest surviving and voting perceptrons
Yoav Freund und Robert E. Schapire: Large Margin Classification Using the Perceptron Algorithm. Machine Learning, 37(3):277–296, 1999.
- perceptrons in dual and primal (linear) form
Framework (Java 1.6 required, WEKA included): perceptrovement1106.zip
Thesis (German): Kroenke_Tobias.pdf
For Linux, Mac, Windows:
Extract the .zip file
Execute run.sh or run.bat (Windows) with Perceptrovement as your working directory
/usr/share/java/postgresql.jar may be removed from the class path
Quick usage instructions
Perceptrovement is made available as a classifier in the WEKA machine learning framework. With the explorer for example you can easily test it on appropriate data sets and easily adjust its options with the generic option editor. Mouse-over hints give you a quick idea of what does what. For more details please refer to the thesis.