Leutner, Franziska; Codreanu, Sonia-Cristina; Brink, Suzanne and Bitsakis, Theodoros. 2023. Game based assessments of cognitive ability in recruitment: Validity, fairness and test-taking experience. Frontiers in Psychology, 13, 942662. ISSN 1664-1078 [Article]

Preview

Text fpsyg-13-942662.pdf - Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract or Description

Gamification and machine learning are emergent technologies in recruitment, promising to improve the user experience and fairness of assessments. We test this by validating a game based assessment of cognitive ability with a machine learning based scoring algorithm optimised for validity and fairness. We use applied data from 11,574 assessment completions. The assessment has convergent validity (r = 0.5) and test–retest reliability (r = 0.68). It maintains fairness in a separate sample of 3,107 job applicants, showing that fairness-optimised machine learning can improve outcome parity issues with cognitive ability tests in recruitment settings. We show that there are no significant gender differences in test taking anxiety resulting from the games, and that anxiety does not directly predict game performance, supporting the notion that game based assessments help with test taking anxiety. Interactions between anxiety, gender and performance are explored. Feedback from 4,778 job applicants reveals a Net Promoter score of 58, indicating more applicants support than dislike the assessment, and that games deliver a positive applicant experience in practise. Satisfaction with the format is high, but applicants raise face validity concerns over the abstract games. We encourage the use of gamification and machine learning to improve the fairness and user experience of psychometric tests.

Item Type:	Article
Identification Number (DOI):	https://doi.org/10.3389/fpsyg.2022.942662
Additional Information:	Data availability statement: The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Keywords:	game based assessment, cognitive ability, adverse impact, user experience, outcome parity, fairness, psychometric tests, recruitment
Departments, Centres and Research Units:	Institute of Management Studies
Dates:	Date Event 20 December 2022 Accepted 18 January 2023 Published
Item ID:	33141
Date Deposited:	14 Feb 2023 11:38
Last Modified:	14 Feb 2023 11:41
Peer Reviewed:	Yes, this version has been peer-reviewed.
URI:	https://research.gold.ac.uk/id/eprint/33141

View statistics for this item...

Edit Record (login required)