How People Judge the Usability of a Desktop Graphic User Interface at Different Time Points: Is there Evidence for Memory Decay, Recall Bias or Temporal Bias?

Abstract

The System Usability Scale (SuS) survey is a widely respected tool for measuring usability. Generally, a SuS score is administered directly after a usability test to assess the usability and user experience of digital products. However, some researchers have used SuS as a survey as part of longitudinal ‘in the wild’ trials where SuS is often completed some period after the trial. The aim of this research was to determine if a participant’s memory of a product’s usability would change if a SuS survey was administered at different times after a test. Hence, we sought to understand if recalling the usability of a digital technology was affected by temporal bias or memory decay. This paper includes results and findings from two studies, study 1 involved evaluating a web application and study 2 involved evaluating a virtual learning environment. Collectively the two studies had 212 participants (n = 212). The findings conclude that there is no significant change of the user’s recollection of the usability of digital product as evidenced by an analysis of users who completed multiple SuS surveys over a short term period of 3 weeks or over an extended period of time of 6 months.