A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Experts

Nitze, I.; Van der Sluijs, J.; Barth, S.; Bernhard, P.; Huang, L.; Kizyakov, A.; Lara, M.J.; Nesterova, N.; Runge, A.; Veremeeva, A.; Ward, J.M.; Witharana, C.; Xia, Z.; Liljedahl, A.K.

Авторы: Nitze Ingmar, Van der Sluijs Jurjen, Barth Sophia, Bernhard Philipp, Huang Lingcao, Kizyakov Alexander, Lara Mark J., Nesterova Nina, Runge Alexandra, Veremeeva Alexandra, Ward Jones Melissa, Witharana Chandi, Xia Zhuoxuan, Liljedahl Anna K.
Журнал: Permafrost and Periglacial Processes
Год издания: 2024
Издательство: John Wiley & Sons Inc.
Местоположение издательства: United States
Номер статьи: 2249
DOI: 10.1002/ppp.2249
Аннотация: Deep-learning (DL) models have become increasingly beneficial for the detection of retrogressive thaw slumps (RTS) in the permafrost domain. However, comparing accuracy metrics is challenging due to unstandardized labeling guidelines. To address this, we conducted an experiment with 12 international domain experts from a broad range of scientific backgrounds. Using 3 m PlanetScope multispectral imagery, they digitized RTS footprints in two sites. We evaluated label uncertainty by comparing manually outlined RTS labels using Intersection-over-Union (IoU) and F1 metrics. At the Canadian Peel Plateau site, we see good agreement, particularly in the active parts of RTS. Differences were observed in the interpretation of the debris tongue and the stable vegetated sections of RTS. At the Russian Bykovsky site, we observed a larger mismatch. Here, the same differences were documented, but several participants mistakenly identified non-RTS features. This emphasizes the importance of site-specific knowledge for reliable label creation. The experiment highlights the need for standardized labeling procedures and definition of their scientific purpose. The most similar expert labels outperformed the accuracy metrics reported in the literature, highlighting human labeling capabilities with proper training, site knowledge, and clear guidelines. These findings lay the groundwork for DL-based RTS monitoring in the pan-Arctic.
Добавил в систему: Кизяков Александр Иванович

	ИСТИНА	Войти в систему Регистрация
	Интеллектуальная Система Тематического Исследования НАукометрических данных
	Главная Поиск Статистика О проекте Помощь

ИСТИНА

Интеллектуальная Система Тематического Исследования НАукометрических данных

A Labeling Intercomparison of Retrogressive Thaw Slumps by a Diverse Group of Domain Expertsстатья

Прикрепленные файлы