Analysis and evaluation of question items: A solution to enhance the quality of multiple-choice test
Nội dung chính của bài viết
Tóm tắt
Objective tests are among the most used assessment forms in educational institutions. However, designing multiple-choice tests of good quality is usually very difficult as it requires test designers to implement the testing, analysis and evaluation of question items for adjustment and improvement prior to use. This study presents how to analyze and evaluate multiple-choice questions based on Classical Test Theory. The data used in this study are the exam results performed by regular students majoring in Informatics Teacher education and Computer Science in their four basic Informatics exam papers in Dong Thap University, from the academic year of 2017-2018 to that of 2020-2021. Based on the parameters of the questions entirely calculated by Microsoft Excel software, the authors show how to classify good question items in the exam papers that can be included in the question banks for future use of testing and assessment activities, and at the same time how to identify unsatisfactory questions that should be revised for adjustment, improvement, or elimination.
Chi tiết bài viết
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Từ khóa
Difficulty index, Classical Test Theory, discrimination index, multiple - choice question
Tài liệu tham khảo
Airasian, P. W. (1994). Classroom assessment. New York: McGraw-Hill.
Al-Wardy, N. M. (2010). Assessment methods in undergraduate medical education. Sultan Qaboos University Medical Journal, 10(2), 203-209.
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education. California: Wadsworth Group.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Baker, F. (2001). The basics of Item Response Theory. ERIC: Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD.
Brennan, L. R. (2006). Educational measurement (4th ed.,). Washington DC: American Council on Education.
Bui, A. K., & Bui, P. N. (2018). Using IATA to analyze, evaluate and improve the quality of the multiple-choice questions in chapter power functions, exponential functions, and logarithmic functions. Can Tho University Journal of Science, 54(9C), 81-93.
Bui, N. Q. (2017). Evaluation of the quality of multiple-choice test bank for the module of Introduction to Anthropology by using the RASCH model and QUEST software. Science of Technology Development, 20(X3), 42-54.
Carroll, R. G. (1993). Evaluation of vignette-type examination items for testing medical physiology. Advances in Physiology Education, 264(6), 11-15.
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity, and reliability of multiple - choice questions for use in nursing research and education. Collegian, 12(1), 19-24.
Cox, K. R. (1976). How did you guess? Or what do multiple choice questions measure? Medical Journal of Australia, (1), 884–886.
Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391-418.
Davies, A. (1990). Principles of language testing. Oxford: Blackwell.
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for
assessments in medical education. Medical Education, 44 (1), 109-117.
Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38 (9), 1006-1012.
Duong, T. T. (2005). Testing and measurement academic achievement. Hanoi: Social Sciences Publishing House.
Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
Fowell, S. L., Southgate, L. J., & Bligh, J. G. (1999). Evaluating assessment: the missing link? Medical Education, 33(4), 276-281.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. New York: Routledge.
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.
Johnstone, A. H., & Ambusaidi, A. (2000). Fixed response: what are we testing?. Chemistry Education: Research and Practice in Europe, 1(3), 323-328.
Kheyami, D., Jaradat, A., Al-Shibani, T., & Ali, F. A. (2018). Item analysis of multiple-choice questions at the department of paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos University Medical Journal, 18(1), 68-74.
Lam, Q. T. (2011). Measurement in education - theory and application. Hanoi: Vietnam National University Publishing House.
Nguyen, B. H. T. (2008). Using Quest software to analyze objective test questions. Journal of Science and Technology, Da Nang University (2), 119-126.
Nguyen, P. H. (2017). Using GSP chart and ROC method to analyze and select mutiple choice items. Dong Thap University Journal of Science, 24(2), 11-17.
Nguyen, T. H. M., & Nguyen, D. T. (2006). Measurement assessment in the objective test: Question difficulty and examinees’ ability. Vietnam National University Journal of Science, (4), 34-47.
Nguyen, V. C., & Nguyen, Q. T. (2020). Applying ConQuest software with the two-parameter IRT model to evaluate the quality of multiple-choice test. HNUE Journal of Science, 65(7), 230-242.
Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64.
Odukoya, J. A., Adekeye, O., Igbinoba, A. O., & Afolabi, A. (2018). Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university. Quality & Quantity, 52(3), 983-997.
Patil, P. S., Dhobale, M. R., & Mudiraj, N. R. (2016). Item analysis of MCQs'-Myths and realities when applying them as an assessment tool for medical students. International Journal of Current Research and Review, 8(13), 12-16.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: multiple choice question examination versus assignment essay. Higher Education (35), 453-472.
Zubairi, A. M., & Kassim, N. A. (2006). Classical and Rasch analyses of dichotomously scored reading comprehension test items. Malaysian Journal of ELT Research, 2(1), 1-20.
Al-Wardy, N. M. (2010). Assessment methods in undergraduate medical education. Sultan Qaboos University Medical Journal, 10(2), 203-209.
Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education. California: Wadsworth Group.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Baker, F. (2001). The basics of Item Response Theory. ERIC: Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD.
Brennan, L. R. (2006). Educational measurement (4th ed.,). Washington DC: American Council on Education.
Bui, A. K., & Bui, P. N. (2018). Using IATA to analyze, evaluate and improve the quality of the multiple-choice questions in chapter power functions, exponential functions, and logarithmic functions. Can Tho University Journal of Science, 54(9C), 81-93.
Bui, N. Q. (2017). Evaluation of the quality of multiple-choice test bank for the module of Introduction to Anthropology by using the RASCH model and QUEST software. Science of Technology Development, 20(X3), 42-54.
Carroll, R. G. (1993). Evaluation of vignette-type examination items for testing medical physiology. Advances in Physiology Education, 264(6), 11-15.
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity, and reliability of multiple - choice questions for use in nursing research and education. Collegian, 12(1), 19-24.
Cox, K. R. (1976). How did you guess? Or what do multiple choice questions measure? Medical Journal of Australia, (1), 884–886.
Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391-418.
Davies, A. (1990). Principles of language testing. Oxford: Blackwell.
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for
assessments in medical education. Medical Education, 44 (1), 109-117.
Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38 (9), 1006-1012.
Duong, T. T. (2005). Testing and measurement academic achievement. Hanoi: Social Sciences Publishing House.
Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
Fowell, S. L., Southgate, L. J., & Bligh, J. G. (1999). Evaluating assessment: the missing link? Medical Education, 33(4), 276-281.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. New York: Routledge.
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of Item Response Theory. Newbury Park: Sage.
Johnstone, A. H., & Ambusaidi, A. (2000). Fixed response: what are we testing?. Chemistry Education: Research and Practice in Europe, 1(3), 323-328.
Kheyami, D., Jaradat, A., Al-Shibani, T., & Ali, F. A. (2018). Item analysis of multiple-choice questions at the department of paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos University Medical Journal, 18(1), 68-74.
Lam, Q. T. (2011). Measurement in education - theory and application. Hanoi: Vietnam National University Publishing House.
Nguyen, B. H. T. (2008). Using Quest software to analyze objective test questions. Journal of Science and Technology, Da Nang University (2), 119-126.
Nguyen, P. H. (2017). Using GSP chart and ROC method to analyze and select mutiple choice items. Dong Thap University Journal of Science, 24(2), 11-17.
Nguyen, T. H. M., & Nguyen, D. T. (2006). Measurement assessment in the objective test: Question difficulty and examinees’ ability. Vietnam National University Journal of Science, (4), 34-47.
Nguyen, V. C., & Nguyen, Q. T. (2020). Applying ConQuest software with the two-parameter IRT model to evaluate the quality of multiple-choice test. HNUE Journal of Science, 65(7), 230-242.
Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64.
Odukoya, J. A., Adekeye, O., Igbinoba, A. O., & Afolabi, A. (2018). Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university. Quality & Quantity, 52(3), 983-997.
Patil, P. S., Dhobale, M. R., & Mudiraj, N. R. (2016). Item analysis of MCQs'-Myths and realities when applying them as an assessment tool for medical students. International Journal of Current Research and Review, 8(13), 12-16.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: multiple choice question examination versus assignment essay. Higher Education (35), 453-472.
Zubairi, A. M., & Kassim, N. A. (2006). Classical and Rasch analyses of dichotomously scored reading comprehension test items. Malaysian Journal of ELT Research, 2(1), 1-20.
Các bài báo được đọc nhiều nhất của cùng tác giả
- Nguyễn Văn Cảnh, Đo lường mức độ hài lòng của sinh viên về chất lượng dịch vụ của thư viện Trường Đại học Đồng Tháp , Tạp chí Khoa học Đại học Đồng Tháp: Tập 9 Số 4 (2020): Chuyên san Khoa học Xã hội và Nhân văn (Tiếng Việt)
- Nguyen Van Canh, Pham Van Tac, Le Thi Bich Van, Measurement of students’ satisfaction in online courses at Dong Thap University , Tạp chí Khoa học Đại học Đồng Tháp: Tập 12 Số 7 (2023): Chuyên san Khoa học Xã hội và Nhân văn (Tiếng Anh)
- Nguyễn Văn Cảnh, Đánh giá mức độ hài lòng của sinh viên về hoạt động phục vụ đào tạo tại Trường Đại học Đồng Tháp , Tạp chí Khoa học Đại học Đồng Tháp: Số 40 (2019): Phần A - Khoa học Xã hội và Nhân văn
- Nguyễn Văn Cảnh, Đo lường mức độ đáp ứng yêu cầu công việc của sinh viên tốt nghiệp các ngành đào tạo giáo viên: Một nghiên cứu dựa trên phản hồi của nhà sử dụng lao động , Tạp chí Khoa học Đại học Đồng Tháp: Tập 11 Số 6 (2022): Chuyên san Khoa học Xã hội và Nhân văn (Tiếng Việt)
- Nguyễn Quốc Tuấn, Đánh giá thực trạng hoạt động kiểm tra, đánh giá đối với các hệ đào tạo ngoài chính quy thông qua phản hồi của người học tại Trường Đại học Đồng Tháp , Tạp chí Khoa học Đại học Đồng Tháp: Tập 10 Số 6 (2021): Chuyên san Khoa học Xã hội và Nhân văn (Tiếng Việt)
- Lê Thị Bích Vân, Nguyễn Văn Cảnh, Một số biểu hiện về hoạt động vui chơi ngoài trời của trẻ mẫu giáo ở các trường mầm non thành phố Cao Lãnh, tỉnh Đồng Tháp , Tạp chí Khoa học Đại học Đồng Tháp: Số 32 (2018): Phần A - Khoa học Xã hội và Nhân văn
- Nguyễn Hồ Công Hậu, Phạm Văn Tặc, Biện pháp rèn kỹ năng viết luận của sinh viên chuyên tiếng Anh, Trường Đại học Đồng Tháp , Tạp chí Khoa học Đại học Đồng Tháp: Số 14 (2015): Phần A - Khoa học Xã hội và Nhân văn
- Nguyen Van Canh, Applying the item response theory with two-parameter, three-parameter models in the evaluation of multiple choice tests , Tạp chí Khoa học Đại học Đồng Tháp: Tập 10 Số 4 (2021): Chuyên san Khoa học Xã hội và Nhân văn (Tiếng Việt)