Effects of test item disclosure on medical licensing examination

Abstract

In 2012, the National Health Personnel Licensing Examination Board of Korea decided to publicly disclose all test items and answers to satisfy the test takers' right to know and enhance the transparency of tests administered by the government. This study investigated the effects of item disclosure on the medical licensing examination (MLE), examining test taker performance, psychometric characteristics, and factors affecting pass rates. This paper analyzed examinee performance data (n = 20,455) from 41 medical schools who took the MLE before (2009-2011) and after (2012-2014) the item disclosure policy (5548 total items). Changes in passing rates, performance of examinee, difficulty and reliability of the test, and factors affecting pass rate of the medical licensing examination before and after item disclosure were analyzed. In order to identify changes caused by item disclosure in the effects of student and school variables on the passing rate of MLE, Binary Logistic Hierarchical Linear Model was used. There was no significant change in pass rates before and after item disclosure. There was a modest increase in the proportion of test takers in the high-scoring group, following item disclosure. Degree completion status, gender, age of applicants and school mean were significant factors affecting pass rates, regardless of item disclosure. There was no difference between passing rates before and after item disclosure with respect to student- and school-level variables. Despite potential concerns for changes in test and examinee characteristics, empirical findings indicate that there was no significant difference caused by implementing item disclosure.