|
ИСТИНА |
Войти в систему Регистрация |
Интеллектуальная Система Тематического Исследования НАукометрических данных |
||
A Study on Manual and Automatic Evaluation for Text Style Transfer: The Case of Detoxificationстатья
- Авторы: Logacheva Varvara, Dementieva Daryna, Krotova Irina, Fenogenova Alena, Nikishina Irina, Shavrina Tatiana, Panchenko Alexander
- Сборник: Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval)
- Год издания: 2022
- Место издания: Association for Computational Linguistics Dublin, Ireland
- Первая страница: 90
- Последняя страница: 101
- DOI: 10.18653/v1/2022.humeval-1.8
- Аннотация: It is often difficult to reliably evaluate models which generate text. Among them, text style transfer is a particularly difficult to evaluate, because its success depends on a number of parameters. We conduct an evaluation of a large number of models on a detoxification task. We explore the relations between the manual and automatic metrics and find that there is only weak correlation between them, which is dependent on the type of model which generated text. Automatic metrics tend to be less reliable for better-performing models. However, our findings suggest that, ChrF and BertScore metrics can be used as a proxy for human evaluation of text detoxification to some extent.
- Добавил в систему: Шаврина Татьяна Олеговна