Hierarchical models that address measurement error are needed to evaluate the correlation between treatment effect and control group event rate

Objectives: To apply a hierarchical model (HM) that addresses measurement error in regression of the treatment effect on the control group event rate (CR). We compare HM to weighted linear regression (WLR) which is subject to measurement error and mathematical coupling. Study Design and Setting: We reviewed published HMs that address measurement error and implemented a Bayesian version in open-source code to facilitate adoption by meta-analysts. We compared WLR and HM across a very large convenience sample of meta-analyses published in the Cochrane Database of Systematic Reviews. Results: We applied both approaches (WLR and an HM that addresses measurement error) to 3193 meta-analyses that included 33,071 studies (average 10.28 studies per meta-analysis). A statistically significant slope suggesting an association between the treatment effect and CR was demonstrated with both approaches in 568 (17.19%) meta-analyses, with neither approach in 2036 (63.77%) meta-analyses, only with WLS in 229 (7.17%) and only with HM in 360 (11.28%) meta-analyses. The majority of slopes was negative (WLR 85%, HM 83%). In the majority of cases, HM had wider confidence intervals (72.53%) and slopes farther from the null (64.77%). Conclusion: Approximately 28% of meta-analyses demonstrate a significant association between the treatment effect and CR when HM is used to address measurement error, which can suggest frequent lack of portability of the relative effect across baseline risks. User-friendly open-source code is provided to meta-analysts interested in exploring this association.