Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations

Abstract

In comparative effectiveness research (CER), ensuring internal, construct, and external validity is crucial. Internal validity determines whether observed outcomes are causally linked to an intervention; construct validity assesses whether a study measures what it intends to; and external validity relates to generalizability in routine practice. Double-blind randomized trials optimize internal validity by minimizing bias and confounding, while construct validity is strengthened through pre-specified protocols and standardized data collection. However, controlled conditions limit external validity. Pragmatic RCTs improve generalizability but may compromise internal validity due to open-label designs. Observational CER studies-including observational studies following the target trial emulation framework-offer broader external validity and feasibility in less time and at lower cost. However, due to lack of random assignment, these studies are susceptible to measured and unmeasured confounding. Several techniques help mitigate these concerns, including a detailed pre-specified protocol, tools such as propensity score matching to balance measured confounders, falsification endpoint testing for assessing the presence of unmeasured confounders, and quasi-experimental designs (including instrumental variable analysis), which may be able to address both. Pre-specified sensitivity analyses and triangulation with complementary data sources further enhance robustness. Construct validity in observational CER depends on accurate patient profiling and validated computational phenotypes for identifying patients, exposures, and outcomes. Thoughtful study design and analytic rigor are essential for balancing these validity considerations. This brief review highlights these issues with examples from thrombosis research.