Can Valence Reflect Morality in Natural Language? A Preliminary Annotation Study

Present implementations of artificial intelligence (AI) ethics do not adequately take feelings, or affect, into account.

If AI should be aligned with human ethics, it seems reasonable to thoroughly investigate the possibility of AI behaviour that mirrors virtuous human ethical conduct, where feelings play a role in the actions, judgements or statements one makes.

Furthermore, while prominent theories of normative ethics are often discussed in terms of their differences and shortcomings, Virtue, Consequentialist, and Kantian Deontological ethics all share a common feature of considering human feeling to some degree while the popular descriptive ethics theory, Moral Foundations Theory, positions feelings as central to many of its foundations.

Therefore, in the present paper, a data set of moral valence is proposed, consisting of 500 annotations by six human participants for both action/judgement and consequence moral valence, ranging from -1 to 1 for text-presented scenarios from the Commonsense Norm Bank data set.

The resulting valence features share significant relationships with multi-class (immoral/discretionary/moral) and binary immoral/moral categories while additionally providing a noteworthy test set Matthew's correlation coefficient of 0.764 using regularised logistic regression for binary classification.

This provides early evidence of the usefulness of valence features for morality estimation of text, indicating that valenced consequences of responses for others can be considered toward more human morally-aligned AI.

In the interest of promoting further affective-moral computing research, this study's annotations will be made available for research on request.