Improving rating scales: Applying Rasch analysis to student pharmacists' attitudes towards herbal medications

Background: This study analyzed the psychometric properties of a cross-sectional survey of student pharmacists' attitudes regarding herbal medications using a polytomous Rasch model. Seven items assessed students' attitudes towards herbal medications using a 6-point agreement scale (0 to 5). Student responses were reviewed to identify outliers and aberrant response patterns, assess scale fit, and perform item analysis. Scale performance was assessed to determine if response categories were independent and equally represented. Items were reviewed for model fit and construct validity with significant item gaps identified using a z-test. Impact: A total of 324 students completed the survey. The initial 6-point scale was analyzed. Student responses with outfit mean-square (MNSQ) values >5 were removed, yet the scale did not have appropriate functionality. The categories were merged to form a 4-point scale with no improvement. Therefore, two different 3-point scale options were analyzed: response categories of 0, 1 to 3, and 4 to 5; and response categories of 0, 1 to 4, and 5. With ten students removed, both 3-point scales met the requirements for functionality and all items exhibited good fit with MNSQ values between 0.6 and 1.4, person-separation value of 1.29, and person-reliability value of 0.62. Recommendations: Both 3-point scales met the requirements for Rasch analysis. The most optimal scale was the 0, 1 to 4, and 5 option. Discussion: This study provides findings from the survey validity assessment alongside the survey results, which is useful for readers to have confidence in the quality of the study findings.