Computer Adaptive Testing in Higher Education : The Validity and Reliability of the approach

Abstract

This paper presents a summary of a six year study into the design, implementation and evaluation of a computer-adaptive test (CAT) for the assessment of Computer Science undergraduates in a UK university. In the first part of this project, a series of empirical studies were carried out in order to evaluate the contribution that the CAT approach could make to the assessment of Computer Science undergraduates. A brief summary of this research is presented in this paper. It was found in this research that the developed CAT was effective at tailoring the level of difficulty of the test to the ability of individual students. The two main groups of stakeholders, students and academic staff, both exhibited a positive attitude towards the CAT approach and the user interface. In the main part of this paper, the validity and reliability of the CAT approach is assessed. Two empirical studies were undertaken in order test the CAT’s validity and reliability and the results of these studies are presented here. Findings from this research are interpreted to show that in the context of assessment in Higher Education, the CAT developed in this research was valid and reliable. In the concluding section these findings are discussed in relation to other research in this area.