Establishing the validity and reliability evidence of preceptor assessment of student tool

Objective. To evaluate the validity and reliability evidence of the preceptor assessment of student tool (PAST) which was designed to assess doctor of pharmacy (PharmD) student rotation performance. Methods. Evaluation data were loaded into WINSTEPS software to conduct a Rasch rating scale analysis. Validity evidence was examined from construct and content validity perspectives, and reliability was assessed via student and item separation index and reliability coefficient. Data from 435 observations were included in the analysis. Results. All 19 items measured the same construct of interest and the five-point rating scale functioned appropriately and differentiated students’ ability. However, the item/person map indicated an absence of items at the end of the measurement continuum. Conclusion. Although adding items at the end of the measurement continuum may be beneficial, PAST showed good validity and reliability evidence when used to evaluate PharmD student rotations and is suitable to assess mastery learning.