Bayesian adaptive trials

Data Science Methods

Bayesian adaptive trials are often used in comparative effectiveness research, and allow for modifying trial design features based on the data collected throughout the study. This design offers multiple advantages over conventional trials in which modifications are not generally allowed during enrollment and follow-up of patients. Below we outline the comparison between traditional versus adaptive trial designs, providing an overview of the adaptive trial implementation, and basic requirements for implementing it.

Shortcomings of conventional trial designs and where Bayesian adaptive trials excel

Randomized controlled trials are considered the current gold standard when it comes to the comparison of medical interventions. Despite their appeal, these studies present several limitations:

  1. Given the logistical complexity involved in conducting most trials, study completion can take a substantial amount of time, frequently stopping short of concluding since the total sample size is not attainable.

  2. Traditional trial designs – referred to as frequentist trials – require a statistical penalty every time an interim analysis is conducted to evaluate the intervention effectiveness. In other words, most researchers will avoid “looking under the hood” while the study is ongoing, keeping participating sites, patients, and the scientific community in the dark for years until its conclusion.

  3. For these same reasons, conventional trials usually avoid interim analyses to check which intervention might be performing better. Thus, the trial participants will keep receiving a treatment that might not be in their best interest, ultimately decreasing the potential benefit they could derive from participating in the study.

  4. Trial analyses are somewhat restricted. Specifically, trials are often designed to answer precise questions, with additional investigations being deemed exploratory and interpreted with caution.

  5. Traditional trials rely on the assumption that there is no previous knowledge about the intervention. Thus, it essentially disregards a lot of prior information about the proposed treatment.

  6. Upon completion of a traditional trial, most of the time, the research team across different institutions completely disbands. This dispersion implies the loss of all the lessons learned as a team, likely leading to the repetition of mistakes in further new trials.

Bayesian adaptive trials provide alternatives to the issues with their traditional, “frequentist cousins.” Namely:

  1. Compared to frequentist trials, it is possible to conduct a Bayesian adaptive one with 30% fewer patients, on average. This reduction in sample size can be translated into a substantial decrease in time to study completion, significant cost savings, and the ability to bring an intervention more quickly to clinical practice.

  2. Provided the appropriate planning of Bayesian adaptive trials – a process that involves a series of heavy-duty simulations – the final design can incorporate several interim analyses without any statistical penalties. This ability to look at which intervention might be “winning” while the trial is still ongoing, leads to many advantages. These advantages include the ability to update participating sites, patients, and other stakeholders about the study results, keeping them excited about the progress the study is making in terms of knowledge regarding the intervention. In other words, by not maintaining sites, patients, and others in the dark, the trial will avoid the traditional periods when the excitement regarding the study wears off while the trial completion is still far away.

  3. Bayesian analyses allow for the exploration of far more details than their frequentist counterparts. This ability to drill down into the data leads to more productive assessments, looking at clinical questions that are of interest to clinicians without being overly restricted by frequentist boundaries.

  4. Since interim analyses allow clinical researchers to have an initial feeling for which intervention might be the most effective, Bayesian adaptive trials can have their protocol changed, i.e., adapted, while the study is ongoing. While these changes are all planned during the initial design rather than conducted at random, possible adaptations could include the increase in the proportion of subjects receiving a given intervention. For example, imagine that intervention A was found to have been doing better than B during an interim analysis. However, the evidence is still not strong enough to declare it a winner. In a Bayesian adaptive trial, it is possible to change the allocation ratio to two individuals receiving intervention A for every individual receiving intervention B. This change in allocation ratio maximizes the average benefit to participants in the trial, thus providing an ethical advantage while not compromising its scientific integrity.

  5. Bayesian adaptive trials are particularly well-suited for sequential evaluations since they can use information from the first study to enhance the design of the next one. For example, imagine that in the first study, we compare two doses of a given intervention, dose A versus B. Once we complete this evaluation, if we now want to initiate a second study to test an intervention C, it would have to start from scratch. In a Bayesian adaptive trial, however, we can use the information from the first study as “prior information” while designing the second one. The use of prior information usually reduces sample size and improves trial effectiveness.

  6. Because it is possible to design Bayesian adaptive trials in sequence, several other logistical benefits will also ensue. For example, one advantage is the ability to keep the same high-performance sites as members of the network. This constancy removes the need for training all sites regarding standard protocols across trials, also giving you an advantage for knowing all Institutional Review Boards and their specific requirements. Ultimately, it provides an edge that is associated with maintaining a well-functioning team intact across projects.

Designing a Bayesian adaptive trial

We now provide the schematic representation of a Bayesian adaptive trial and the most common types of adaptations that are allowed in this design. Steps include: first, as a standard protocol for any trial, define the target population of the study, the intervention to be tested, and the outcomes to be assessed for the comparative effectiveness. In a Bayesian adaptive investigation, after these steps, we conduct extensive simulation planning for multiple scenarios where adaptations might occur. One of the adjustments can be dropping one of the intervention arms to focus on the more promising ones. For example, if an intervention arm with a lower dose of a given drug to be tested is found to be inferior to the standard-of-care control arm, then it is needless to expose the patients to this dosage, and it ultimately will be dropped from the scheduled design. Another adaptation can be the number of interim analyses that are needed to check the accumulating information regarding the current trial’s treatment effects to reach an optimal trial design. Sample size reassessment is another type of adaptation allowed in these studies, based on interim analyses of accumulating data. The sample size can be conditionally increased or decreased. Adaptations concerning drug dosage can lead to improving efficiency and quality in the estimates of the actual maximum tolerated dose (MTD) without jeopardizing the safety of the participants. Thus, a better understanding of the dose-response or dose-toxicity relationship can be achieved, along with easier identification of a safe and effective dose to use clinically. After the simulation stage and upon reaching a feasible trial design, we can implement it in an actual clinical scenario. This process will involve reanalyzing the data regularly, along with conducting the total number of interim analyses planned during the simulation without a statistical penalty. Thus, the final evaluations will allow for extensive drill-downs, and we will be able to focus on clinical decisions rather than p-values.

What you need to get started

The first step will be to define a research question, such as assessing the efficacy of a specific clinical intervention for some of the common conditions or diseases prevalent in the general population. Like any comparative effectiveness trial, the researchers will precisely define the target population and intervention characteristics, along with an assessment of relevant outcomes. While the intricacies of Bayesian adaptive trials are beyond the scope of this short article, here are the things you will need:

  1. Prospectively collect data from a large sample of individuals to provide answers to the intended clinical question. Of importance, take into consideration the amount of time during which you will be collecting this data.

  2. Make sure that the intervention assignment can be randomized without any bias, maintaining the integrity of a comparative effectiveness trial.

  3. Recruitment and retention of patients for the trial is one of the most challenging aspects. There can be multiple reasons behind these challenges, including lack of awareness, fear, distrust, or suspicions about the research, practical or personal obstacles, among others. Thus, as a first step towards tackling these challenges, you will have to make sure that the patients are aware that we will use their data collected during this study for further analyses.

  4. Another critical step to overcoming recruitment and retention challenges would be to convey “equipoise” to the patients. Equipoise here means that there is no advantage or disadvantage if the patients receive any of the trial treatments under investigation.

Like any conventional trial design, certain standard elements are essential for the effective implementation of a Bayesian adaptive trial. However, ultimately, adaptive trial designs can significantly reduce the use of resources and time as well as improve the likelihood of success of the study.