Abstract.

In this work, we research online augmentation – method of increasing the representativeness of the training set during the learning of artificial neural networks. We consider the most common transformations with preservation of labels, which will be useful in many practical tasks. Due the limitations of existing systems for online augmentation, among which the adding new transfor-mation functions and increased learning time, we introduce a new effective augmentation system without any impact on learning time. Experiments on the MNIST dataset with the developed sys-tem have shown that we outperform the current best result at SimpleNet(310K) architecture of neural network, reducing the error rate from 0.28% to 0.25%..

Keywords:

machine learning, artificial neural networks, online augmentation, augmentation on the fly, realtime augmentation.

PP. 150-157.

DOI: 10.14357/20790279180517

References

1. Polevoj D.V. Aktual’nye zadachi sozdaniya sistem massovogo vvoda s ispol’zovaniem opticheskogo raspoznavaniya dlya preobrazovaniya slozhno strukturirovannyh bumazhnyh dokumentov v gibridnyh informacionnyh sistemah // Conference on Systems analysis and information technologies SAIT-2011, V. 2, P. 192-195 (in Russian).
2. K. Bulatov. Smart IDReader: Document Recognition in Video Stream / K. Bulatov, V. V. Arlazarov, T. Chernov, O. Slavin and D. Nikolaev // 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, 2017, pp. 39-44. doi: 10.1109/ICDAR.2017.347.
3. The mnist database of handwritten digits [Electronical Resource]. URL: http://yann.lecun. com/exdb/mnist (date request 15.06.2018).
4. The CIFAR-10 dataset [Electronical Resource]. URL: https://www.cs.toronto.edu/~kriz/cifar.html (date request 15.06.2018).
5. Russakovsky O. ImageNet large scale visual recognition challenge / O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, L. Fei-Fei // International Journal of Computer Vision. – 2015. – Vol. 115, Issue 3. – P. 211-252.
6. Ronneberger O. U-Net: Convolutional Networks for Biomedical Image Segmentation / Ronneberger O., Fischer P., Brox T. // In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham.
7. Weiss G. The Effect of Class Distribution on Classifier Learning : An Empirical Study / Weiss G., Provost F. // Technical Report ML-TR-44, Department of Computer Science, Rutgers University. August 2, 2001.
8. Krizhevsky A. Learning multiple layers of features from tiny images. Master’s thesis. Department of Computer Science. University of Toronto. 2009. 60 p.
9. Zhang H. Mixup: Beyond Empirical Risk Minimization / Zhang H., Ciss M., Dauphin Y., Lopez D. // ArXiv e-prints [Electronical Resource] – ArXiv:1710.09412 – 2017 – URL: https://arxiv. org/abs/1710.09412 (date request 15.06.2018).
10. Zagoruyko S. Learning to Compare Image Patches via Convolutional Neural Networks / Zagoruyko S., Komodakis N. // The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015.
11. Fawzi A. Adaptive data augmentation for image classification / Fawzi A., Samulowitz H., Turaga D., Frossard P. // Image Processing (ICIP), 2016 IEEE International Conference on. 2016. С. 3688-3692.
12. Emelyanov S.O. Methods of training augmentation in the task of image classification / Emelyanov S.O., Ivanova A.A., Shvets E.A., Nikolaev D.P. // Sensornye sistemy [Sensory systems]. 2018. V. 32(3) (in Russian).
13. Arlazarov V.V. et al. Analysis of features of the use of fixed and mobile small-sized digital video camera for OCR // Information Technologies and Computer Systems, 3/2014. P. 71-81 (in Russian).
14. Chernyshova Y. Generation method of synthetic training data for mobile OCR system /Chernyshova Y. Gayer A. Sheshkus A. // Proc. SPIE 10696, Tenth International Conference on Machine Vision (ICMV 2017), 106962G (13 April 2018); doi: 10.1117/12.2310119.
15. Goodfellow I. Generative adversarial nets / I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio // Proceedings of the 27th International Conference on Neural Information Processing Systems. – 2014. – Vol. 2. – P. 2672-2680.
16. Perez L. The Effectiveness of Data Augmentation in Image Classification using Deep Learning / Perez L., Wang J. // ArXiv e-prints [Electronical Resource] – ArXiv:1712.04621 – 2017 – URL: https://arxiv.org/abs/1712.04621 (date request 01.06.2018).
17. Announcing NVIDIA DALI and NVIDIA nvJPEG [Electronical Resource]. URL: https://news. developer.nvidia.com/announcing-nvidia-daliand- nvidia-nvjpeg/ (date request 19.06.2018).
18. Bloice D. Augmentor: An Image Augmentation Library for Machine Learning / Bloice D., Stocker C., Holzinger A. // ArXiv e-prints [Electronical Resource] – ArXiv:1712.04621 – 2017 – URL: https://arxiv.org/abs/1712.04621 (date request 15.06.2018).
19. Hasanpour S.H. Let’s keep it simple, Using simple architectures to outperform deeper and more complex architectures / Hasanpour S. H., Rouhani M., Mohsen F., Sabokrou M. // ArXiv e-prints [Electronical Resource] – ArXiv:1608.06037 – 2016 – URL: https://arxiv.org/abs/1608.06037
(date request 25.05.2018).