Skip to main content

Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics.

McDougall, S., Reppa, I. and Taylor, J., 2022. Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics. Behavior Research Methods. (In Press)

Full text available as:

s13428-022-01973-7.pdf - Published Version
Available under License Creative Commons Attribution.

[img] PDF
Revised Manuscript Nudge and Bias in Subjective Ratings.pdf - Accepted Version
Restricted to Repository staff only
Available under License Creative Commons Attribution Non-commercial.


DOI: 10.3758/s13428-022-01973-7


Subjective ratings have been central to the evaluation of icon characteristics. The current study examined biases in ratings in relation to the context in which icons are presented. Context was manipulated between-participants, with some groups rating icon sets with limited variability, and others rating icon sets with wide variability. It was predicted that the context created by the icon set would influence participants’ ratings; when the range of icons was limited this would create bias given participants’ expectation that a full range of icon values was being presented. Six key icon characteristics were rated which were visual (visual complexity, appeal), affective (valence, feelings) and semantic (concreteness, semantic distance). Some icon characteristics were susceptible to rating bias while others were not. Where subjective judgements were being made of visual icon characteristics (appeal/complexity) and highly concrete icons which were very pictorial, there was clear evidence of substantial bias in ratings. The same susceptibility to bias was not evident when ratings relied solely on learned semantic associations or were associated with the emotional attributions made to icons. The dynamic nature of the ratings bias was demonstrated when the rating context was changed without participants’ knowledge. When participants rated further blocks of icons providing a different range of the to-be-rated characteristic, this resulted in rapid and dramatic changes in rating behaviour. These findings demonstrate the need for representative sampling of icon characteristics to avoid ratings bias. Practically, this is important when determining the usability of newly-designed icons sets in order to avoid over-valuing or under-valuing of key characteristics.

Item Type:Article
Uncontrolled Keywords:Ratings bias; Icon ratings; Icon design and usability; Concreteness; Complexity; Visual appeal; Affect
Group:Faculty of Science & Technology
ID Code:37497
Deposited By: Symplectic RT2
Deposited On:13 Sep 2022 12:10
Last Modified:17 Oct 2022 13:28


Downloads per month over past year

More statistics for this item...
Repository Staff Only -