Abstract.

A classic task of text classification was studied in many works, but current approaches mostly devoted to improvement of classification quality for what we call clean corpora, not containing typos. In this work we present results of modern classification models testing in the presence of noise for two languages – English and Russian.

Keywords:

neural networks; text classification; noise robustness.

PP. 174-182.

DOI: 10.14357/20790279180520

References

1. Armand Joulin, Edouard Grave, Piotr Bojanowski and Tomas Mikolov. 2016. Bag of Tricks for Efficient Text Classification. arXiv preprint arXiv: 1607.01759.
2. Valentin Malykh. 2018. Robust Word Vectors: Embeddings for Noisy Texts.
3. Yoon Kim, Yacine Jernite, David Sontag and Alexander M. Rush 2016. Character-Aware Neural Language Models. In AAAI, pages 2741–2749.
4. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 142–150.
5. Rubtsova Yuliya. 2014. Automatic Term Extraction for Sentiment Classification of Dynamically Updated Text Collections into Three Classes In Knowledge Engineering and the Semantic Web, pp140-149, Springer
6. Bochkarev, V. V., Shevlyakova, A. V., and Solovyev, V. D. 2015. The average word length dynamics as an indicator of cultural changes in society. Social Evolution & History, 14(2), 153-175.
7. Cucerzan, S. and Brill, E. 2004. Spelling correction as an iterative process that exploits the collective knowledge of web users. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.
8. Joulin, A., Grave, E., Bojanowski, P. and Mikolov, T. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759.
9. Howard, J. and Ruder, S. 2018. Fine-tuned Language Models for Text Classification. arXiv preprint arXiv:1801.06146.
10. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (pp. 6000-6010).
11. Xiang Zhang, Junbo Jake Zhao and Yann Le-Cun. 2017. Character-level Convolutional Networks for Text Classification. arXiv preprint arXiv: 1509.01626
12. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. arXiv preprint arXiv: 1408.5882
13. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches arXiv preprint arXiv: 1409.1259
14. Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate arXiv preprint arXiv:1409.0473
15. Bengio, Y., Simard, P. and Frasconi, P. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2), pp.157-166.
16. Tutubalina, Elena, and Nikolenko, Sergey. 2015. Inferring sentiment-based priors in topic models. In Mexican International Conference on Artificial Intelligence, pp. 92-104.
17. Niu, J., Yang, Y., Zhang, S., Sun, Z. and Zhang, W. 2018. Multi-task Character-Level Attentional Networks for Medical Concept Normalization. Neural Processing Letters, pp.1-18.
18. Vinciarelli A. Noisy text categorization, 2005. IEEE Transactions on Pattern Analysis and Machine Intelligence. Dec;27(12):1882-95.
19. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R., 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), pp.1929-1958.
20. Pineda, F.J. 1987. Generalization of back-propagation to recurrent neural networks. Physical review letters, 59(19), p.2229.
21. Glorot, X. and Bengio, Y. 2010, March. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 249-256).