Introduction
Approximately 80% of clinical trials fall short of their recruitment goals (Clinical Trials Arena, 2018), risking the statistical power of the study, delaying timelines, and increasing costs. In the worst cases, enrollment issues can lead to an investigational product failing for reasons unrelated to its scientific merit. In less severe cases, the implementation of rescue services and overdue site closures inflate budgets, stretch team resources, and delay timelines. These challenges can be attributed to traditional planning and target-setting, which rarely reflect actual enrollment patterns, and study teams' execution, which is hamstrung by delayed data and limited signals of actual performance.
Lokavant is improving how enrollment targets are planned and updated, offering clinical operators opportunities to take action to reduce study delays and budget overruns.
Traditional approaches
Traditional approaches to enrollment forecasting are static and poorly utilize historical trial data. In most cases, teams calculate average enrollment rates based on public sources and limited internal data; these forecasts inform meaningful trial planning and trial monitoring decisions. This approach ignores within-trial fluctuations, making it difficult for teams to determine when and how to take actions to meet study timelines.
For example, once a site is activated it may have an artificially low recruitment rate as site staff learn how to operationalize the protocol (i.e., the "cold-start" or "burn-in" period). These simple global averages at the site, study, and country levels represent unattainably high goals, and decision-making may be negatively impacted as a result. Sites may be pushed too hard to reach these unattainable goals or they may become too complacent, blaming an arbritrary "burn-in" period. Furthermore, as novel issues arise in trials (e.g., stay at home orders that limit patient access to sites or competitive trials recruiting eligible patients), it is too difficult to estimate the severity of their impact over a longer enrollment period.
These inaccuracies are costly and there are currently no data-driven methods to determine how they will eventually impact overall enrollment. This prevents clinical operations teams from taking swift and decisive action to keep the study on track.
The Lokavant approach
In contrast to traditional methods, Lokavant uses a Bayesian approach1 to integrate historical data and within-trial data to create highly accurate trial enrollment projections.
The main benefits of Lokavant’s approach include:
-
incorporating historical data into enrollment projections, enabling accurate projections as the trial starts—when corrective and preventative actions (CAPAs) are most effective
-
creating projections through stochastic simulations2, which allow users to calculate the odds that they will hit their targets and track progress/key risk metrics more accurately
-
updating the model daily with observed trial enrollment data, increasing the precision of predictions throughout a trial
This approach yields a series of distributions, which present actionable risk-based metrics for both trial planning and trial execution. The outputs include:
-
Enrollment forecasts: 95% credible intervals for the forecast number of subjects enrolled over the duration of the trial
-
Enrollment probabilities: the probability that a trial will enroll a sufficient number of subjects before the given target completion date
-
Time probabilities: the probability that a trial will complete target enrollment within a given time range
Case study: How does this work in practice?
We present a retrospective analysis of one trial (Phase Ill, Hematology-Oncology) to show how the model estimates and continually updates relevant trial risks.
The study team projected enrollment using the industry standard approach by taking publicly available data and internal analysis to assume an average number of subjects enrolled per month based on site activations. This rate predicted that the enrollment target of 145 subjects would be met in 15 months (represented by the blue line in the figure below).
At day 60 Lokavant's model predicts that the study would miss its enrollment target. Using traditional methods, the study team identified and implemented a change order in month 15 of the study, extending the enrollment window by 23 months. Lokavant's approach identified the enrollment risk over 1 year before the formal mitigating action was initiated.
Later in the trial, on day 365, Lokavant's models continue to show that the study will only meet between 22% and 24% of the inital target patient accrual date. Lokavant’s simulations show that the trial will likely take somewhere between 36 to 47 months to finish enrolling; this proved to be highly accurate given the eventual total enrollment window of 43 months.
Enrollment challenges persisted even after the change order at month 15. The new rate was still inaccurate, and an additional change order was required to extend the recruitment window by an additional 8 months. Lokavant’s accurate enrollment predictions could have enabled the study team to adjust enrollment expectations 30 months prior to this second change order.
“Traditional approach” projections vs. actual enrollment
The Lokavant difference
Overview
Through the integration of historical data and within-trial learnings, Lokavant’s models display increased accuracy compared to traditional methods (see Table 1) enabling quicker and more data-driven action to be taken in trials.
By reducing the error in enrollment estimations, teams can focus on mitigation, site relationships and patient experience rather than identifying enrollment risks.
With a more accurate prediction of enrollment timelines, especially one that identifies a 36 month delay to complete patient accrual, study teams can take action, including re-training/re-evaluating existing sites, opening new sites, or extending the enrollment window immediately.
Enrollment slowdown
Even when accurate planning is in place, unexpected enrollment slowdowns can occur due to unforeseen issues. This trial exhibited a steep slowdown between days 720 and 840, a period where only 5 subjects were enrolled in the trial. Early identification of slowdown periods and insights on their impact to overall trial timelines are crucial for early and decisive intervention.
This enrollment slowdown was identified quickly by Lokavant’s models — the odds of successfully completing patient accrual drop from 60% to 35% by day 25 of the 120-day slowdown. Lokavant’s models can also quantify the effect of this slowdown on the trial, estimating a ~35-day delay, thereby quantifying the effect of the slowdown on the trial.4
Conclusion
Leveraging predictive analytics powered by historical and within-trial data, Lokavant's approach transforms trial enrollment planning from a flawed, obligatory process to a powerful tool for trial management.