Analysis and evaluation of question items: A solution to enhance the quality of multiple-choice test
Main Article Content
Abstract
Objective tests are among the most used assessment forms in educational institutions. However, designing multiple-choice tests of good quality is usually very difficult as it requires test designers to implement the testing, analysis and evaluation of question items for adjustment and improvement prior to use. This study presents how to analyze and evaluate multiple-choice questions based on Classical Test Theory. The data used in this study are the exam results performed by regular students majoring in Informatics Teacher education and Computer Science in their four basic Informatics exam papers in Dong Thap University, from the academic year of 2017-2018 to that of 2020-2021. Based on the parameters of the questions entirely calculated by Microsoft Excel software, the authors show how to classify good question items in the exam papers that can be included in the question banks for future use of testing and assessment activities, and at the same time how to identify unsatisfactory questions that should be revised for adjustment, improvement, or elimination.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Keywords
Difficulty index, Classical Test Theory, discrimination index, multiple - choice question
References
Airasian, P. W. (1994). Classroom assessment. New York: McGraw-Hill.
Al-Wardy, N. M. (2010). Assessment methods in undergraduate medical education. Sultan Qaboos University Medical Journal, 10(2), 203-209.
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education. California: Wadsworth Group.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Baker, F. (2001). The basics of Item Response Theory. ERIC: Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD.
Brennan, L. R. (2006). Educational measurement (4th ed.,). Washington DC: American Council on Education.
Bui, A. K., & Bui, P. N. (2018). Using IATA to analyze, evaluate and improve the quality of the multiple-choice questions in chapter power functions, exponential functions, and logarithmic functions. Can Tho University Journal of Science, 54(9C), 81-93.
Bui, N. Q. (2017). Evaluation of the quality of multiple-choice test bank for the module of Introduction to Anthropology by using the RASCH model and QUEST software. Science of Technology Development, 20(X3), 42-54.
Carroll, R. G. (1993). Evaluation of vignette-type examination items for testing medical physiology. Advances in Physiology Education, 264(6), 11-15.
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity, and reliability of multiple - choice questions for use in nursing research and education. Collegian, 12(1), 19-24.
Cox, K. R. (1976). How did you guess? Or what do multiple choice questions measure? Medical Journal of Australia, (1), 884–886.
Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391-418.
Davies, A. (1990). Principles of language testing. Oxford: Blackwell.
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for
assessments in medical education. Medical Education, 44 (1), 109-117.
Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38 (9), 1006-1012.
Duong, T. T. (2005). Testing and measurement academic achievement. Hanoi: Social Sciences Publishing House.
Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
Fowell, S. L., Southgate, L. J., & Bligh, J. G. (1999). Evaluating assessment: the missing link? Medical Education, 33(4), 276-281.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. New York: Routledge.
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.
Johnstone, A. H., & Ambusaidi, A. (2000). Fixed response: what are we testing?. Chemistry Education: Research and Practice in Europe, 1(3), 323-328.
Kheyami, D., Jaradat, A., Al-Shibani, T., & Ali, F. A. (2018). Item analysis of multiple-choice questions at the department of paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos University Medical Journal, 18(1), 68-74.
Lam, Q. T. (2011). Measurement in education - theory and application. Hanoi: Vietnam National University Publishing House.
Nguyen, B. H. T. (2008). Using Quest software to analyze objective test questions. Journal of Science and Technology, Da Nang University (2), 119-126.
Nguyen, P. H. (2017). Using GSP chart and ROC method to analyze and select mutiple choice items. Dong Thap University Journal of Science, 24(2), 11-17.
Nguyen, T. H. M., & Nguyen, D. T. (2006). Measurement assessment in the objective test: Question difficulty and examinees’ ability. Vietnam National University Journal of Science, (4), 34-47.
Nguyen, V. C., & Nguyen, Q. T. (2020). Applying ConQuest software with the two-parameter IRT model to evaluate the quality of multiple-choice test. HNUE Journal of Science, 65(7), 230-242.
Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64.
Odukoya, J. A., Adekeye, O., Igbinoba, A. O., & Afolabi, A. (2018). Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university. Quality & Quantity, 52(3), 983-997.
Patil, P. S., Dhobale, M. R., & Mudiraj, N. R. (2016). Item analysis of MCQs'-Myths and realities when applying them as an assessment tool for medical students. International Journal of Current Research and Review, 8(13), 12-16.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: multiple choice question examination versus assignment essay. Higher Education (35), 453-472.
Zubairi, A. M., & Kassim, N. A. (2006). Classical and Rasch analyses of dichotomously scored reading comprehension test items. Malaysian Journal of ELT Research, 2(1), 1-20.
Al-Wardy, N. M. (2010). Assessment methods in undergraduate medical education. Sultan Qaboos University Medical Journal, 10(2), 203-209.
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education. California: Wadsworth Group.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Baker, F. (2001). The basics of Item Response Theory. ERIC: Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD.
Brennan, L. R. (2006). Educational measurement (4th ed.,). Washington DC: American Council on Education.
Bui, A. K., & Bui, P. N. (2018). Using IATA to analyze, evaluate and improve the quality of the multiple-choice questions in chapter power functions, exponential functions, and logarithmic functions. Can Tho University Journal of Science, 54(9C), 81-93.
Bui, N. Q. (2017). Evaluation of the quality of multiple-choice test bank for the module of Introduction to Anthropology by using the RASCH model and QUEST software. Science of Technology Development, 20(X3), 42-54.
Carroll, R. G. (1993). Evaluation of vignette-type examination items for testing medical physiology. Advances in Physiology Education, 264(6), 11-15.
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity, and reliability of multiple - choice questions for use in nursing research and education. Collegian, 12(1), 19-24.
Cox, K. R. (1976). How did you guess? Or what do multiple choice questions measure? Medical Journal of Australia, (1), 884–886.
Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391-418.
Davies, A. (1990). Principles of language testing. Oxford: Blackwell.
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for
assessments in medical education. Medical Education, 44 (1), 109-117.
Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38 (9), 1006-1012.
Duong, T. T. (2005). Testing and measurement academic achievement. Hanoi: Social Sciences Publishing House.
Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
Fowell, S. L., Southgate, L. J., & Bligh, J. G. (1999). Evaluating assessment: the missing link? Medical Education, 33(4), 276-281.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. New York: Routledge.
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.
Johnstone, A. H., & Ambusaidi, A. (2000). Fixed response: what are we testing?. Chemistry Education: Research and Practice in Europe, 1(3), 323-328.
Kheyami, D., Jaradat, A., Al-Shibani, T., & Ali, F. A. (2018). Item analysis of multiple-choice questions at the department of paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos University Medical Journal, 18(1), 68-74.
Lam, Q. T. (2011). Measurement in education - theory and application. Hanoi: Vietnam National University Publishing House.
Nguyen, B. H. T. (2008). Using Quest software to analyze objective test questions. Journal of Science and Technology, Da Nang University (2), 119-126.
Nguyen, P. H. (2017). Using GSP chart and ROC method to analyze and select mutiple choice items. Dong Thap University Journal of Science, 24(2), 11-17.
Nguyen, T. H. M., & Nguyen, D. T. (2006). Measurement assessment in the objective test: Question difficulty and examinees’ ability. Vietnam National University Journal of Science, (4), 34-47.
Nguyen, V. C., & Nguyen, Q. T. (2020). Applying ConQuest software with the two-parameter IRT model to evaluate the quality of multiple-choice test. HNUE Journal of Science, 65(7), 230-242.
Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64.
Odukoya, J. A., Adekeye, O., Igbinoba, A. O., & Afolabi, A. (2018). Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university. Quality & Quantity, 52(3), 983-997.
Patil, P. S., Dhobale, M. R., & Mudiraj, N. R. (2016). Item analysis of MCQs'-Myths and realities when applying them as an assessment tool for medical students. International Journal of Current Research and Review, 8(13), 12-16.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: multiple choice question examination versus assignment essay. Higher Education (35), 453-472.
Zubairi, A. M., & Kassim, N. A. (2006). Classical and Rasch analyses of dichotomously scored reading comprehension test items. Malaysian Journal of ELT Research, 2(1), 1-20.
Most read articles by the same author(s)
- Van Canh Nguyen, Measuring student satisfaction of the quality of Dong Thap University library's services , Dong Thap University Journal of Science: Vol. 9 No. 4 (2020): Social Sciences and Humanities Issue (Vietnamese)
- Van Canh Nguyen, Van Tac Pham, Thi Bich Van Le, Measurement of students’ satisfaction in online courses at Dong Thap University , Dong Thap University Journal of Science: Vol. 12 No. 7 (2023): Social Sciences and Humanities Issue (English)
- Van Canh Nguyen, Assessing the students’ satisfaction level on training services at Dong Thap University , Dong Thap University Journal of Science: No. 40 (2019): Part A - Social Sciences and Humanities
- Van Canh Nguyen, Measuring the job requirements satisfaction level of teacher training majored graduates: A study based on employer feedbacks , Dong Thap University Journal of Science: Vol. 11 No. 6 (2022): Social Sciences and Humanities Issue (Vietnamese)
- Quoc Tuan Nguyen, Assessing the status of testing and evaluation activities for the in-service training of Dong Thap University through learners’ feedbacks , Dong Thap University Journal of Science: Vol. 10 No. 6 (2021): Social Sciences and Humanities Issue (Vietnamese)
- Thi Bich Van Le, Van Canh Nguyen, Children’s outdoor activity expressions at some kindergartens in Cao Lanh City, Dong Thap Province , Dong Thap University Journal of Science: No. 32 (2018): Part A - Social Sciences and Humanities
- Ho Cong Hau Nguyen, Van Tac Pham, Solutions to Training English essay writing skills for the English majors at Dong Thap University , Dong Thap University Journal of Science: No. 14 (2015): Part A - Social Sciences and Humanities
- Van Canh Nguyen, Applying the item response theory with two-parameter, three-parameter models in the evaluation of multiple choice tests , Dong Thap University Journal of Science: Vol. 10 No. 4 (2021): Social Sciences and Humanities Issue (Vietnamese)