What bias? Changing standards, information effects, and human rights measurement

Human rights measurement efforts have been confronted with concerns of bias practically since quantitative measurement efforts began in the late 1970s and early 1980s. By and large, attacks have focused on biases to the source materials on which these measurement efforts rely—namely, the annual human rights reports produced by Amnesty International and the US Department of State. We take stock and seek to distinguish conceptually distinct types of bias that have been conflated in the past, yet plausibly affect human rights measures. In addition to revisiting reporting bias or organizational bias long identified in the literature, we also disentangle two types of bias that have been of concern more recently: bias attributable to changing standards and information effects. For each type of bias we identify, we provide an empirical implication as to the effect on human rights measures and, importantly, its spatial or temporal variation.