How to successfully use external comparison arm studies using real-world data –
Although randomized clinical trials (RCTs) are the gold standard for drug approval studies, the shift to precision medicine has increased the use of single-arm trials (SATs). SATs lack results in control patients. Therefore, to help contextualize study results, external comparison arms (ECAs) may be used, which compile data from external sources, such as patient registries and other medical records. However, methodological considerations should be undertaken to ensure the best conduct and minimize potential biases in ACE study designs.
Differences between RCTs and ECAs
By randomizing treatment and control groups, RCTs allow researchers to control for potential biases and the influence of unmeasured variables. Additionally, RCTs do not have to rely on two different data sources, which may have different operational definitions, evaluation methods, and measurement timelines. However, in some cases, RCTs are impractical and that is when SATs can be used.
When examining trends in precision medicine, which focus on specific biomarkers among patient populations in the same disease category, SATs allow researchers to evaluate new treatment options for patient cohorts smaller. To increase the validity of SAT results, researchers use real-world data (RWD) from patients with the same attributes to act as an external comparison arm. However, several factors should be considered before using ACE studies to ensure that the results are statistically valid.
Importance of sample size in ECAs
As with any research study, sample size plays a critical role in ensuring that study results can be used as valid evidence of treatment efficacy and safety. With respect to sample size for ACEs, there are several unique considerations including whether data for a treatment arm is already available, whether additional caution needs to be incorporated when estimating sample sizes necessary and how to incorporate the use of causal inference methods.
Populations and treatment conditions
Documentation of detailed descriptions of the ACE population is essential, including the mechanisms and conditions that led to patients being recorded in the data source. RWD sources typically only document comparison patients who actually received the treatment, not those who were to receive the treatment, which instead translates to a safety analysis population, not an intention-to-treat population. (ITT), and it is recommended to compare what is comparable also in terms of analysis populations.
A second consideration is to identify the target population for which the treatment should be standardized in terms of estimating the marginal effects of the treatment. This involves identifying estimates that will describe differences in treatment effects in different target populations, such as the mean treatment effect (ATE) for the entire patient population and the mean treatment effect on treated (ATT) or untreated (ATU), which focus on treated or untreated populations. Also, the mean treatment effect in the overlap population (ATO) is a possible estimate, focusing on the internal validity of the treatment comparison.
As with the study populations, the treatment conditions for the treatment and comparison groups should be clearly described. The description of the treatment group will be available via the clinical study protocol, and the conditions of the RWD comparison group should be similarly described (e.g., eligible doses, how the drug is administered, and frequencies of taking medication). Since there is a possibility of higher exposure time in the treatment group compared to the comparison group, due to the controlled setting of the SAT, researchers should assess whether implementing a minimum number of treatment cycles or exposure times for the two groups is useful to address baseline exposure differences in data sources, possibly as a sensitivity analysis.
Basic and Endpoint Considerations
Prior to data collection, baseline information should be established to ensure that the variable definitions of the two groups are as identical as possible. The index date, i.e. the date the trial officially begins, should be defined by the start of treatment, not the enrollment date, to create consistent definitions across data sources .
Inclusion and exclusion criteria should also be defined consistently across treatment and comparison groups as much as possible. For example, the definition of treatment lines in site-based RWD sources is based on the physician’s clinical judgment and may differ from physicians at other sites and the SAT algorithm. In order to create more consistency across all datasets, it is necessary to verify, and often reclassify, treatment lines and possibly other reference data.
The measurement of endpoints in RWD and SAT may differ, also, by means of different time points. For example, oncology studies with RWD endpoint progression (i.e. disease progression) may not be assessed using established classification rules, and some RWD sources may not even allow the application of such classification rules. Although assessments of RW progression are generally less stringent than SATs, a bias may be introduced, potentially in favor of the SAT drug. One consideration for validating RWD progression assessments is blinded central review, which involves the employment of reviewers who do not know which group the data is from.
Often propensity score (PS) models are used to analyze non-randomized data, but there are many other possible alternative methods, e.g. doubly robust, g-calculation, non-PS weighting, or non-matching methods. PS. Also, a combination of approaches is possible, for example performing a PS weighting or a g-calculation after an initial matching step.
Missing values and unmeasured covariates are central to the ECA design type and effective management of missing baseline covariate data is essential. Various sophisticated methods are possible and sensitivity analyzes must be applied to check the robustness of the results.
In the general case, RCTs are the optimal approach to clinical research and drug development. However, if an RCT is not feasible – for example, due to too small sample populations when studying ultra-rare diseases – SATs with ACEs that use real-world data can be a tool. powerful to help identify better treatments for patients.
In order to obtain statistically valid results, it will be critical to consider factors such as sample size, population and treatment conditions, baseline measurements, and endpoint comparisons when designing and of the analysis of a study. However, careful consideration will be key to successful use of ACEs, as each disease type and data source brings its own challenges in the research process.
About the Author
Dr. Gerd Rippin, Director of Biostatistics at IQVIA, received his BSc in Statistics in 1995 from the University of Dortmund, Germany, and his PhD in 1999 from the University of Mainz, Germany. He has spent most of his career within the CRO business, including as an entrepreneur and directly within the pharmaceutical industry. Dr. Rippin is a highly experienced biostatistician with over 20 years of expertise in applying statistical methods to clinical studies. His experience includes various indications and phases of medical research, and he has a particular interest in External Comparison Arm (ECA) studies and the application of complex real-world statistical methodology in general.