Enhancing Human-in-the-Loop Learning for Binary Sentiment Word Classification (bibtex)

by B. Urcelay, C. J. Rozell and M. Bloch

Abstract:

While humans intuitively excel at classifying words according to their connotation, transcribing this innate skill into algorithms remains challenging. We present a human-guided methodology to learn binary word sentiment classifiers from fewer interactions with humans. We introduce a human perception model that relates the perceived sentiment of a word to the distance between the word and the unknown classifier. Our model informs the design of queries that capture more nuanced information than traditional queries solely requesting labels. Together with active learning strategies, our approach reduces human effort without sacrificing learning fidelity. We validate our method with theoretical analysis, providing sample complexity bounds. We also perform experiments with human data, demonstrating the effectiveness of our method in improving the accuracy of binary sentiment word classification.

Reference:

Enhancing Human-in-the-Loop Learning for Binary Sentiment Word ClassificationB. Urcelay, C. J. Rozell and M. Bloch. In IEEE Conference on Decision and Control (CDC), December 2024.

Bibtex Entry:

@inproceedings{urcelay.24,
	author = {Urcelay, B. and Rozell, C. J. and Bloch, M.},
	title = {Enhancing Human-in-the-Loop Learning for Binary Sentiment Word Classification},
	month = dec,
	year = {2024},
	booktitle = {IEEE Conference on Decision and Control (CDC)},
	address = {Milan, Italy},
	abstract = {While humans intuitively excel at classifying words according to their connotation, transcribing this innate skill into algorithms remains challenging. We present a
human-guided methodology to learn binary word sentiment classifiers from fewer interactions with humans. We introduce a human perception model that relates the perceived sentiment
of a word to the distance between the word and the unknown classifier. Our model informs the design of queries that capture more nuanced information than traditional queries solely
requesting labels. Together with active learning strategies, our approach reduces human effort without sacrificing learning fidelity. We validate our method with theoretical
analysis, providing sample complexity bounds. We also perform experiments with human data, demonstrating the effectiveness of our method in improving the accuracy of binary
sentiment word classification.}
}