Zero-shot Cross-lingual Transfer of a Gloss Language Model for Semantic Change Detection

Rachinskiy, M.; Arefyev, N.

Авторы: Rachinskiy M., Arefyev N.
Сборник: Komp'juternaja Lingvistika i Intellektual'nye Tehnologii
Год издания: 2021
Издательство: РГГУ
Местоположение издательства: М.
Первая страница: 578
Последняя страница: 586
DOI: 10.28995/2075-7182-2021-20-578-586
Аннотация: Consulting word definitions from a dictionary is a familiar way for a human to find out which senses a particularword has. We hypothesize that a system that can select a proper definition for a particular word occurrence canalso naturally solve Semantic Change Detection (SCD) task. To verify our hypothesis, we followed an approachpreviously proposed for Word Sense Disambiguation (WSD) and trained a system that embeds word definitions andword occurrences into the same vector space. In this space, the embedding of the most appropriate definition hasthe largest dot product with a contextualized word embedding.The system is trained on an English WSD corpus. To make it work for the Russian language, we replaced BERTwith the multilingual XLMR language model and exploited its zeroshot crosslingual transferability. Despite notfinetuning the encoder model on any Russian data, this system achieves the second place in the competition, andlikely works for any of one hundred other languages XLMR was pretrained on, though the performance may vary.We then measure the impact of such WSD pretraining and show that this procedure is crucial for our results. Sinceour model was trained to choose a proper definition for a word, we propose an algorithm for the interpretation andvisualization of the semantic changes through time.By employing additional labeled data in Russian and training a simple regression model, that converts thedistances between output contextualized embeddings into more humanlike scores of sense similarity between wordoccurrences, we further improve our results and achieve the first place in the competition.
Добавил в систему: Арефьев Николай Викторович

	ИСТИНА	Войти в систему Регистрация
	Интеллектуальная Система Тематического Исследования НАукометрических данных
	Главная Поиск Статистика О проекте Помощь

ИСТИНА

Интеллектуальная Система Тематического Исследования НАукометрических данных

Zero-shot Cross-lingual Transfer of a Gloss Language Model for Semantic Change Detectionстатья

Прикрепленные файлы