Аннотация.

В работе представлен обзор общетематических (неспециализированных) порождающих диалоговых моделей, основанных на глубоком обучении. Рассмотрены основные проблемы построения диалоговых моделей, основанных на машинном обучении, и методы их решения. На русскоязычном корпусе проведено экспериментальное сравнение классической нейросетевой диалоговой модели «кодировщик-декодировщик» с ее модификацией, использующей механизм внимания.

Ключевые слова:

диалоговые системы, обработка естественного языка, генерация текста, нейронные сети, искусственный интеллект, глубокое обучение, модель «кодировщик-декодировщик».

Стр. 105-115.

 Полная версия статьи в формате pdf.

DOI: 10.14357/20790279190110

Литература

1. Ritter A., Cherry C., Dolan W.B. Data-driven response generation in social media //Proceedings of the conference on empirical methods in natural language processing. – Association for Computational Linguistics, 2011. – P. 583-593.
2. Sordoni A. et al. A Neural Network Approach to Context-Sensitive Generation of Conversational Responses //Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. – 2015. – P. 196-205.
3. Cho K., van Merrienboer B., Bahdanau D., Bengio Y. On the properties of neural machine translation: Encoder-decoder approaches. //Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, 2014. – P. 103-111.
4. Sutskever I., Vinyals O., Le Q.V. Sequence to sequence learning with neural networks //Advances in neural information processing systems. – 2014. – P. 3104-3112.
5. Schmidhuber Jürgen, Sepp Hochreiter. “Long short-term memory.” Neural Comput 9.8 (1997): 1735-1780.
6. Serban I.V. et al. Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation //AAAI. – 2017. – P. 3288-3294.
7. Graves A. Generating sequences with recurrent neural networks //arXiv preprint arXiv:1308.0850. – 2013.
8. Tiedemann J. News from OPUS-A collection of multilingual parallel corpora with tools and interfaces //Recent advances in natural language processing. – 2009. – P. 237-248.
9. Vinyals O., Le Q. A neural conversational model // ICML Deep Learning Workshop - 2015.
10. Serban I.V. et al. Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models //AAAI. – 2016. – P. 3776-3784.
11. Li J. et al. A Diversity-Promoting Objective Function for Neural Conversation Models // Proceedings of NAACL-HLT. – 2016. – P. 110-119.
12. Bahdanau D., Cho K., Bengio Y. Neural machine translation by jointly learning to align and translate // ICLR. – 2015.
13. Luong M.T., Pham H., Manning C.D. Effective approaches to attention-based neural machine translation //arXiv preprint arXiv:1508.04025. – 2015.
14. Shao L. et al. Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models //Proceedings of EMNLP. – 2017.
15. Li J. et al. A persona-based neural conversation model. //Proceedings of ACL. – 2016. – P. 994-1003
16. Al-Rfou R. et al. Conversational contextual cues: The case of personalization and history for response ranking //arXiv preprint arXiv:1606.00372. – 2016.
17. Asghar N. et al. Deep active learning for dialogue generation //Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (* SEM 2017). – 2017. – P. 78-83.
18. Li J. et al. Deep Reinforcement Learning for Dialogue Generation //Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. – 2016. – P. 1192-1202.
19. Papineni K. et al. BLEU: a method for automatic evaluation of machine translation //Proceedings of the 40th annual meeting on association for computational linguistics. – Association for Computational Linguistics, 2002. – P. 311-318.
20. Galley M. et al. deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets //Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). – 2015. – Vol. 2. – P. 445-450.
21. Banerjee S., Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments //Proceedings of the ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/ or summarization. – 2005. – P. 65-72.
22. Liu C.W. et al. How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation //Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. – 2016. – P. 2122-2132.
23. Li J. et al. Adversarial Learning for Neural Dialogue Generation //Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. – 2017. – P. 2157-2169.
24. Turing A. Computing intelligence and machinery //Mind. – 1950. – Vol. 59. – N. 2236.
25. Venkatesh A. et al. On Evaluating and Comparing Conversational Agents //arXiv preprint arXiv:1801.03625. – 2018.
26. Walker M.A. et al. PARADISE: A framework for evaluating spoken dialogue agents //Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics. – Association for Computational Linguistics, 1997. – P. 271-280.
27. Lowe R. et al. The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems //Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue. – 2015. – P. 285-294.
28. Ghazvininejad M. et al. A Knowledge-Grounded Neural Conversation Model //arXiv preprint arXiv:1702.01932. – 2017.
29. Kingma D.P., Ba J. Adam. A method for stochastic optimization //arXiv preprint arXiv:1412.6980. – 2014.