Utilizing three publicly available and well respected datasets, Integral Regularization © was compared to standard neural network regularization techniques in the fields of finance, image processing, and education. For each dataset, Integral Regularization © out performed all seven unique combinations of the standard neural network regularization techniques: activity, dropout, and kernel.
Credit Card Fraud
The Credit Card Fraud Data Set is taken from real world European credit card fraud cases. The data set contains 284,807 credit card transactions, 30 features per sample, and highly imbalanced classes.
The data was standardized and fed into a fully connected dense layered neural network architecture. Cross entropy loss was used to classify transactions as: either "fraud" or "not fraud".
MNIST Handwritten Digits
The MNIST Handwritten Digits Data Set is comprised of 42,000 handwritten digits from zero through nine. Each sample is a 28 by 28 grey scale image.
The data was prepossessed by rescaling it into the range of minus three to three. Each image was augmented every epoch. The neural network architecture had initial convolution layers that fed into dense layers. Cross entropy loss was used to classify images into ten categories, one for each number.
U.S. Graduate Admissions
The U.S. Graduate Admissions Data Set is comprised of 500 sample points of real world, students data with seven features per sample. The features include GPA, test scores, and research experience.
The data was prepossessed with principal component analysis. The neural network architecture had seven dense fully connected layers. Mean squared error loss was used to fit the samples to their targets.