Real-World Views On RWE: RECOVER Trial Seen As US Answer To UK’s RECOVERY

The real-world evidence effort centered on long-term outcomes from COVID-19 will be a model for the US, FDA Commissioner Robert Califf indicated during the recent International Society for Pharmacoepidemiology meeting.

Califf appeared to refer twice, though not explicitly by name, to the National Institutes of Health-run RECOVER long COVID trial that will include other federal agencies, such as the FDA, the Centers for Disease Control and Prevention and the Centers for Medicare and Medicaid Services during a “town hall” plenary session with leaders from other global health agencies. The RECOVER Initiative was announced at the end of July.

“Why do I have to call Israel to figure out what to do with the next vaccine dose?” Califf asked during the 25 August meeting, referring to US reliance on Israeli real-world data on responses to COVID-19 booster doses to new variants. “Stay tuned because there’s going to be a very significant effort across FDA, NIH, CDC and CMS to put together such a system.”

Califf alluded to the success of the UK’s RECOVERY trial, something he has often done in public forums, as the standard for a randomized real-world trial that sought answers to straightforward questions rather than attempting to collect as much data as possible on a wide range of marginally significant metrics.

“We’re going to try to give you a run for the money in the US to see if we can cobble together in our dissociated health system something that can compete,” Califf said of RECOVER.

The initiative comprises five trials, according to NIH, including testing Pfizer Inc.’s Paxlovid (nirmatrelvir and ritonavir) to see if it improves the symptoms of patients with long COVID.

FDA Chief Medical Officer Hilary Marston, Center for Biologics Evaluation and Research Director Peter Marks, CMS Chief Medical Officer Dora Hughes (succeeding the recently departed Lee Fleischer), and Nakela Cook, Patient Centered Outcomes Research Institute executive director, are named as ex-officio non-voting members on the trial executive committee.

The “town hall” served as a global regulatory forum on RWE with representatives from the European Medicines Agency, Medicines and Health Care products Agency, and Health Canada, in addition to agencies from Japan and Israel. Califf renewed his call for randomization of RWE trials and framed the broader concept of real-world evidence in “two dimensions,” data and methods. 

“The data can come from this whole spectrum of places ranging from a rarefied research clinic in a particular place that people have to go to in order to get data collected, to what happens in a healthcare system with electronic health records, to people’s homes where I had the chance to work for five years and at Alphabet,” Califf said in opening remarks.

“The other dimension is the method that we use, the research plan that we have to deal with those data and those methods can include anything from ‘we got a bunch of data, let’s analyze it’ to a very specific Phase I study at a research unit where every infusion is timed to the millisecond,” he continued. “Rather than have these divisions of real-world evidence versus randomization, I think that’s entirely wrong … We need to decide for the question that we have, what is the source of data and the research design that’s fit for that purpose.”

FDA issued a series of draft RWE guidances as part of its PDUFA VI commitments and finalized a December 2021 guidance on “considerations” for use of RWD/RWE to support regulatory decision-making at the end of August.

Health Canada Assistant Deputy Minister Pamela Aung-Thin noted that Health Canada has been evaluating the use of RWE in regulatory decision-making since 2017. “We’ve considered our RWE for decision-making as part of the totality of evidence in pre-market drug packages, and we’ve used RWE to help inform our signal detection pharmacovigilance and risk management activities in the post market.”

Califf showcased the value of his extensive experience as a trialist and with learning systems during his talk. He moved comfortably from digging into the weeds of complex trial design and analysis issues on COVID therapeutics to identifying big-picture uses of RWE in Alzheimer’s disease and obesity drugs. 

Califf highlighted the “time zero problem” of time-related bias that can lead to confounding results as a nagging problem for real-world trials.

“I think we're getting better and better at the methods of dealing with confounding, but if we're measuring the characteristics of people at a time that’s different from the inception point, I haven’t seen any solution to that problem that at least I would consider good,” he said. “I would say 99% of the observational studies on COVID that dealt with therapeutics, you might as well have said, ‘I don’t know, made it up or something’, because it could not deal with that issue.”

Then Califf addressed the use of RWD/RWE for broad common diseases as a postmarket, long-term follow up tool.

“I would argue that randomization with real-world data is going to be one of the most important tools going forward for common chronic disease where we need to understand effects of treatments over long periods of time, and also have heterogeneous, valid balances of risks and benefits in the population,” he said.

“I shouldn’t have to say more than Alzheimer’s drugs and obesity drugs for you to understand exactly what I’m talking about,” Califf added.

The FDA’s Oncology Center of Excellence views situations when there are feasibility challenges, ethical concerns, and a lack of clinical equipoise as the most appropriate for use of real-world data to determine efficacy in cancer, Associate Director of Pharmacoepidemiology Donna Rivera said.   

Rivera specified pediatrics, rare conditions, molecular subgroups, underrepresented populations and diseases with significant unmet need as those with the most potential for applying real-world data, highlighting the accelerated approval of Novartis AG’s alpelisib for a novel indication in PIK3CA-Related Overgrowth Spectrum (PROS) in April 2022. The approval was based on a single-arm clinical study in patients treated as part of an expanded access program for compassionate use enrolled across seven sites in five countries.

Data from the study was abstracted from medical charts and radiographic images at participating clinical sites. The primary endpoint was response at week 24 as determined by blinded independent central review.

Rivera went on to explain what made alpelisib an ideal candidate for use of RWE.

“To bring a bit of regulatory context to support this decision, the conduct of a single-arm study and use of data with blinded independent central review from expanded access programs was appropriate considering the rarity and high unmet medical need,” she said. “From a rationale perspective, there was mechanistic plausibility and early clinical data, making it challenging to conduct an RCT. It was a rare disease with a low prevalence of 14 people per million, with no currently approved available therapies in an area of high unmet medical need.”

“The plans were predefined for this, including predefined plans for study objectives, enrollment guidance and statistical analysis plans, as well as objective outcome validation again through blinded independent central review, which allowed in combination, the ability to provide substantial evidence of effectiveness and a positive risk benefit assessment,” Rivera added.

The quality of underlying real-world data continues to be a challenge for regulatory uses, Rivera said.

“There’s an increasing amount of real-world data,” she said. “However, there’s much less that results in real-world evidence, and I’ve been thinking a lot about why this is.”

“I think it’s a myriad of reasons, but ultimately, it’s the quality of the underlying data,” she added. “That’s been a limitation. The source quality of the data, the appropriateness of the design, and then how that pertains to the regulatory objectives.” Rivera also reiterated the necessity that the data be fit for purpose.

Other international regulators agreed.

“The challenge from the international perspective is we need some international standard about data quality or data reliability,” Yoshiaki Uyama, Japan Pharmaceuticals and Medical Devices Agency associate executive director, said. “The data itself, how we can assure that the data quality or the data management practices are appropriate and acceptable by the PMDA.”

Rivera highlighted the increasing importance of auditing source data.

“The auditing process has certainly become something increasingly more important as it pertains to real-world data because the information that’s collected may or may not have been collected in the same way” FDA traditionally thinks about, she said. 

Rivera said regulatory agencies must have access to the source data or certified copies and work “through the process early and often in conversation, for example, with the FDA to ensure that the data is going to be auditable that it’s going to meet regulatory requirements.” She added that with the alpelisib approval Novartis had images readily available to review.

“We have seen increases over time in real-world evidence coming in submissions,” Health Canada Senior Epidemiologist Melissa Kampman said. She noted several issues that lead to a rejection of RWE as a basis for evidence.

“Comparability issues are one of the major limitations, both in terms of the consideration when we’re either approving or rejecting,” Kampman said. “Missing data also tends to be a very large factor that does often lead to us not approving a drug based on the real-world evidence.”

Canadian and Israeli officials said oncology and rare diseases are the two areas where health agencies are seeing the most activity in the RWD/RWE space. 

“The greatest use is definitely in the oncology therapeutic area, and we see a lot of use for regulatory decisions, such as drug development, drug approvals, but also in the context of clinical decisions, such as comparing the effectiveness of different authorized treatments,” Michal Hirsch-Vexberg, Israel Ministry of Health head of pharmaceutical registration, said. “The oncology therapeutic area is definitely followed by the rare diseases therapeutic area.”

“Oncology and rare diseases are the two biggest areas for us as well,” Kampman said. “These are very much the most promising areas where we could use real-world data and real-world evidence in the future to help to fill the gaps.”

“What’s exciting now is we’re starting to see real-world data to really help us understand the impact and effectiveness of the decisions that we’re taking on a system-wide basis, but also on an individual patient-level basis, and helping us understand where measures are working, where they’re not, and potentially why they’re not and helping us to improve our regulatory position,” said Katherine Donegan, MHRA head of epidemiology.

“It really helped us explain to the public why certain decisions were being taken with the vaccination program in the UK,” Donegan said. “It might not have helped tackle misinformation that was already out there, but it certainly helped minimize the impact of these evolving data on the safety of the vaccines in terms of reducing uptake.” She added that the RECOVERY trial was one of the most impactful trials ever seen in the UK.

EMA Head of RWE Patrice Verpillat highlighted theDARWIN EURWE data network as a valuable resource and said the onus is now on regulators to become more comfortable with real-world data.

“Another important challenge is that the delivery of real-world evidence is not only the responsibility of the pharma industry … in terms of methods, that regulators have been quite slow adopters of new methodologies, because you need to see and wait how it works,” Verpillat said.