The reliability of aggregated public opinion measures

Theory: Using generalizability theory, we discuss the O'Brien (1990) method of aggregate-level reliability assessment. We contrast this theoretical conception of reliability with the theory of individual-level internal consistency. Methods: The methodology used is the O'Brien (1990) aggregate-level reliability technique for the R:A design, where "R" denotes that survey respondents reside or are nested within aggregate units (A). The aggregate unit in this analysis is the state. One-way analysis of variance is used to estimate reliabilities. Results: Individual-level measures of reliability are inappropriate techniques when one is evaluating the reliability of aggregated measures of public opinion. Estimating aggregate-level reliability is methodologically simple and should be a common practice among the many researchers using aggregate-level variables created from individual-level data.