Development and validation of the Intervention Skills Profile–Skills: A brief measure of student social-emotional and academic enabling skills

Stephen P. Kilgus, Wes E. Bonifay, Katie Eklund, Nathaniel P. von der Embse, Casie Peet, Jared Izumi, Hyejin Shim, Lauren N. Meyer

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

The purpose of this study was to support the development and initial validation of the Intervention Selection Profile (ISP)–Skills, a brief 14-item teacher rating scale intended to inform the selection and delivery of instructional interventions at Tier 2. Teacher participants (n = 196) rated five students from their classroom across four measures (total student n = 877). These measures included the ISP-Skills and three criterion tools: Social Skills Improvement System (SSIS), Devereux Student Strengths Assessment (DESSA), and Academic Competence Evaluation Scales (ACES). Diagnostic classification modeling (DCM) suggested an expert-created Q-matrix, which specified relations between ISP-Skills items and hypothesized latent attributes, provided good fit to item data. DCM also indicated ISP-Skills items functioned as intended, with the magnitude of item ratings corresponding to the model-implied probability of attribute mastery. DCM was then used to generate skill profiles for each student, which included scores representing the probability of students mastering each of eight skills. Correlational analyses revealed large convergent relations between ISP-Skills probability scores and theoretically-aligned subscales from the criterion measures. Discriminant validity was not supported, as ISP-Skills scores were also highly related to all other criterion subscales. Receiver operating characteristic (ROC) curve analyses informed the selection of cut scores from each ISP-Skills scale. Review of classification accuracy statistics associated with these cut scores (e.g., sensitivity and specificity) suggested they reliably differentiated students with below average, average, and above average skills. Implications for practice and directions for future research are discussed, including those related to the examination of ISP-Skills treatment utility.

Original languageEnglish (US)
Pages (from-to)66-88
Number of pages23
JournalJournal of School Psychology
Volume83
DOIs
StatePublished - Dec 2020

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology

Cite this