The cardiovascular in-training examination: Development, implementation, results, and future directions

Background The American College of Cardiology (ACC), in collaboration with the National Board of Medical Examiners (NBME), developed the first standardized in-training examination (ITE) for cardiovascular disease fellows-in-training (FITs). In addition to testing knowledge, this examination uses the newly developed ACC Curricular Milestones to provide specific, competency-based feedback to program directors and FITs. The ACC ITE has been administered more than 5,000 times since 2011. Objectives This analysis sought to report the initial experience with the ITE, including feasibility and reliability of test development and implementation, as well as the ability of this process to provide useful feedback in key content areas. Methods The annual ACC ITE has been available to cardiovascular disease fellowship programs in the United States since 2011. Questions for this Web-based, secure, multiple-choice examination were developed by a group of cardiovascular disease specialists and each question was analyzed by the NBME to ensure quality. Scores were equated and standardized to allow for comparability. Trainees and program directors were provided detailed feedback, including a list of the curricular competencies tested by those questions answered incorrectly. Results The ITE was administered 5,118 times. In 2013, the examination was taken by 1,969 fellows, representing 194 training programs. Among the 3 training years, there was consistency in the examination scores. Total test scores and scores within each of the content areas increased with each FIT year (there was a statistically significant difference in each cohort's average scale score across administration years). There was also significant improvement in examination scores across the fellowship years. Conclusions The ACC ITE is a powerful tool available to all training programs to assess medical knowledge. This examination also delivers robust and timely feedback addressing individual knowledge gaps, and thus, may serve as a basis for improving training curricula.