Publication Date

Summer 7-13-2020

Abstract

Sentiment analysis methods extract the attitude of a text via systematic algorithms. To evaluate the validity of common sentiment analysis methods, we use polychoric correlation to compare computer-mediated methods and human-rated analogues. Our main topics of interest are the internal consistency of the raters' scores, the level of consensus among raters, and how well raters' scores correlate with those given by sentiment analysis methods for randomly collected Twitter data.

Our analysis found that there is good validity for methods that measure negative and positive sentiments in short texts, both in terms of inter-rater consistency and when comparing raters to computer-mediated sentiment analysis methods. The more complex sentiment pair anger and joy had lower levels of inter-rater consistency. Rater-computer consistency for anger and joy was the weakest among the methods tested, raising questions about the construct validity for measuring expressions of anger and joy in short texts.

Degree Name

Statistics

Level of Degree

Masters

Department Name

Mathematics & Statistics

First Committee Member (Chair)

Fletcher G. W. Christensen

Second Committee Member

Erik Barry Erhardt

Third Committee Member

Ronald Christensen

Language

English

Keywords

Sentiment, Sentiment Analysis, Polychoric, Polychoric Correlation

Document Type

Thesis

Download

Share

COinS