Assessing real-world evidence utilization in oncology health technology assessments: insights from external control-arm studies

Abstract

Background While health technology assessment (HTA) acceptance of contextual real-world data (RWD) studies describing burden of disease, disease natural history, or treatment pathways is relatively common, HTA practices for RWD studies addressing real-world clinical efficacy, such as those using external control arms (ECAs), are still evolving and less standardized. The aim of this study was to use data from HTA submissions and reports to understand common analytical methods and data considerations for submissions using RWD-based ECA. This evaluation used ECA studies as a basis for investigating the use of RWD to evaluate clinical efficacy. Methods Secondary data were compiled from selected oncology submissions to HTA agencies between January 2016 and December 2022 that incorporated RWD-based ECA data, using natural language-processing text-mining to identify and select relevant cases. Submissions were reviewed in six countries across Asia Pacific (Australia), Europe (France, Germany, UK), and North America (Canada and US). Submissions that were rated both positive and negative by HTA agencies were included, with HTA feedback organized into generalizability, confounding, data quality, and data analysis categories. Results Of 204 submissions identified, 100 cases were selected for the analysis of patterns highlighting sources of data for ECAs and RWD methodology best practices: Australia (n = 3), Canada (n = 34), France (n = 19), Germany (n = 15), UK (n = 26), and US (n = 3). A positive HTA recommendation was received by 69 of these 100 cases. Lung cancer was associated with the greatest number of cases/submissions. Retrospective cohort studies were the most common source of RWD, with inverse probability of treatment weighting/propensity score weight as the most common methodology used to generate real-world evidence. Most of the selected RWD-based ECA cases were from Canadian and UK HTA agencies. Positive comments focused on population adjustment, RWD viability, and alignment of data with standard of care (SoC) for that country/indication; negative comments focused on missing/limited data, lack of alignment with SoC, and potential risk of bias. Conclusion This study captured challenges in considering RWD-based ECAs for HTA submission and presents criteria for creating viable RWD studies using ECAs. Data source selection, patient population comparison, and transparent presentation of potential biases were important factors in enhancing the credibility and utility of RWD-based ECAs in HTA decision-making processes.