Users' performance in lab and non-lab environments through online usability testing: A case of evaluating the usability of digital academic libraries' websites

Abeer Alharbi, Pam Mayhew

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)
22 Downloads (Pure)


The factors related to the environment in which users operate may be of a vital importance when trying to understand how they experience a particular system. It is required that we find out how we can get to know those factors to investigate if they affect the users' performance in usability testing. An online usability study has emerged that can be attempted by a large, varied pool of users' anywhere with an Internet connection. Would the usage of an online usability study help to give comprehensive insight and an understanding of the whole user experience? That is especially interesting if the user operates remotely, as we are unaware of what the users might experience while performing the test (e.g., distractions and type of device used to attempt the test). Accordingly, a pilot study of ongoing research was conducted. An identical online usability-testing tool (Loop 11) through which to apply the online usability study was used in two environments: unrestricted (the user's natural environment), and totally restricted (a simulated lab environment). Ten subjects completed the test in the restricted environment and 20 completed it in the unrestricted environment. All of the subjects were asked to perform predefined search tasks on digital libraries' websites. Their performance was analyzed and compared against the two different environments. The results showed that online usability testing is a feasible method to gain comprehensive insight into how users attempt usability testing in a non-lab environment. The results of whether different environments affect test performance show no valuable differences in most of the study's measurements. The test subjects were frequently multitask while they performed the usability testing in an unrestricted environment, but they were highly distracted if they personally interrupted. The results encourage the researcher to conduct a formal version of this study to further examine the learned lessons from the pilot study.

Original languageEnglish
Title of host publicationProceedings of the 2015 Science and Information Conference, SAI 2015
PublisherThe Institute of Electrical and Electronics Engineers (IEEE)
Number of pages11
ISBN (Electronic)9781479985470
Publication statusPublished - 3 Sep 2015
EventScience and Information Conference, SAI 2015 - London, United Kingdom
Duration: 28 Jul 201530 Jul 2015


ConferenceScience and Information Conference, SAI 2015
Country/TerritoryUnited Kingdom


  • distraction
  • environment
  • online
  • remote
  • usability testing

Cite this