Vol.11, No.1, February 2022.                                                                                                                                                                           ISSN: 2217-8309

                                                                                                                                                                                                                        eISSN: 2217-8333

TEM Journal

TECHNOLOGY, EDUCATION, MANAGEMENT, INFORMATICS

Association for Information Communication Technology Education and Science

Implementing Item Response Theory (IRT) Method in Quiz Assessment System

Viska Mutiawani, Al Misky Athaya, Kurnia Saputra, Muhammad Subianto

© 2022 Viska Mutiawani, published by UIKTEN. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. (CC BY-NC-ND 4.0)

Citation Information: TEM Journal. Volume 11, Issue 1, Pages 210-218, ISSN 2217-8309, DOI: 10.18421/TEM111-26, February 2022.

Received: 17 October 2021.

Revised:   20 January 2022.
Accepted: 27 January 2022.
Published: 28 February 2022.

Abstract:

A quiz is one of the methods to test students’ abilities. Estimation of ability scores can be calculated by using the Classical Test Theory (CTT) and Item Response Theory (IRT) approaches. IRT is more sensitive to item characteristics that can estimate students’ ability. This study implements IRT in a quiz assessment system, compares the results of the CTT and IRT, then analyses the functionality and usability of the built system. The system was tested by 50 respondents using 30 question items with 5 answer choices. The collected data is automatically assessed by the system using CTT and IRT calculations. The results showed differences in ranking between CTT and IRT methods. Using the IRT method, there were 14 respondents who experienced an increase in rank, 19 respondents experienced a decrease in rank, and the remaining 17 respondents did not experience a change in ranking. The functionality of built system was tested with the Blackbox Testing method and it has passed all the test functions. The usability of the system was also tested by involving 23 respondents using the System Usability Scale (SUS) questionnaire and the SUS score showed that the system can be accepted by users.

Keywords – Item Response Theory, Classical Test Theory, quiz assessment system, Black box testing, System Usability Scale.

-----------------------------------------------------------------------------------------------------------

Full text PDF >  

-----------------------------------------------------------------------------------------------------------