Skip to content

Applying RWE to Regulatory Applications

November 16, 2020

Industry experience in applying to support regulatory decisions has significantly expanded over the past year as the inclusion of RWE in regulatory submissions increased. Feedback stemming from regulator review of submissions leveraging RWD serves as a valuable opportunity to understand how is considered and reviewed in the regulatory context. This session was an open dialogue between the FDA and biopharma partners to share successes and address concerns in incorporating RWE in regulatory submissions.

Jump to transcript

Questions from the audience

As an industry, we're in agreement that an important prerequisite of using real-world data for regulatory decision-making is sufficiently high-quality data. As we continue to define what good data quality entails as a community, how is the agency assessing data quality for real-world data as you evaluate regulatory submissions that incorporate RWD?

Chana Weinstock: Thank you so much, great question. As this sort of submission becomes more and more common, there's been a lot of effort internally on our end to really standardize the way we approach these analyses and the advice we give about some of the data quality issues et cetera from our end. Towards that end, we now have an oncology center of excellence, real-world data, and real-world evidence program that's been established, and one of the areas of focus is regulatory review in terms of developing guidance that deals with data quality issues that we can communicate to external stakeholders. It also has really been involved in efforts to standardize our analysis methods and our advice that we're giving to companies and sponsors who approach us, and to develop shared resources internally for standardized characterization of real-world data sources and data quality, and to develop tools for regulatory review on our end. So, proposed methodology, standardize language, et cetera.

We have a lot of medical officers from the direct review divisions involved in this effort as well as statisticians, so this is a really important area that we're focusing on to try to speak to the data quality issues and the advice we're giving external stakeholders. We've also had many external collaborations with outside stakeholders such as Friends of Cancer Research, et cetera, to really get a sense of what the outside community is thinking about this, and to be able to communicate with pharma, et cetera, as to what our current thinking is on real-world evidence. So, thank you for the question. It's been a very important focus for us.

Dr. Patel, can you talk about the timeline of your analysis relative to your interactions with regulators? How and when did you engage them around your intended use of real-world data? Again, can you tell us about your interactions of regulators related to your intended use of real-world data?

Kiran Patel: Yes, for this specific program we started discussion much earlier during the process, and at the time of achieving the breakthrough designation, and that discussion continued. The challenge for us was more about having the access to the data since the implementation of the FGFR testing was not widely available at that time. But having a good communication with the agency allowed us to be able to provide this data even after the submission of the NDA. So, we got some grace period to allow for this data to be provided.

I'd say my suggestion would be to start those discussions very early as you start seeing the path towards registration since having that discussion with the agency much earlier would allow you to be able to get access to data that you need with the quality you need as well as the analysis that you need to conduct to be able to support the regulatory needs.

Thanks so much, Dr. Patel. And Dr. Weinstock, just as a follow-up, how and when should sponsors engage the agency in relation to their intended use of real-world data?

Chana Weinstock: Yes, thank you so much, and thank you, Dr. Patel. I would echo what you say about early interaction with us. Certainly, as soon as this is something you're considering, you're welcome to submit a meeting package or a question to the appropriate review division, and we have a lot of accumulated expertise in terms of statisticians, medical oncologists, et cetera, who are really thinking very hard and very deeply about this topic. So, as soon as you come up with a plan, you could certainly involve us and would love to be involved as early as possible because I think that's the best way to ensure success in terms of your regulatory approach.

Given the retrospective nature of the IBRANCE real-world data study, did you have a lot of missing data at the time of your analyses? And if so, how did this impact your interactions with health authorities? Again, did you have a lot of missing data and how did this impact your interactions with health authorities?

Albert L. Kraus: Yeah, it's a good question and missing data is a really hot topic here. We basically use three different types of real-world evidence. I would say in pharmacy data there's levels of the pharmacy system, and I understand there's levels that are a little more data surety around the information, and those are the levels we used. I forget the nomenclature on that, but in that, because it's based on medical claims and reimbursement and monies transferred and prescriptions provided, there's not a lot of different types of information, but there's also not a lot of missingness around a patient who got a script and paid for it through insurance, et cetera, et cetera.

So, that database, not as missing. On the safety side of the electronic health records as well as the efficacy side and the Flatiron data set, I would say there's a little more information missing there around some of the demographic elements. More broadly speaking, in our other efforts we've looked into, that has become a larger element. Whether it's ECOG status, other comorbidities known or unknown that may or may not be captured. In oncology, they don't often capture cardiac scenarios and risk. You can go back and do it in reverse like getting a Charlson cardiac status.

But that is an element you have to look at particularly with time to efforts. If you're looking at progression or survival, the prognosis and comorbidities of the populations and the matching is absolutely critical. We emphasize a little more tumor response, which is probably... It's certainly dependent on some of those factors, but a little bit less so. Our numbers were pretty small by the time we narrowed down to the populations we looked at in both endocrine-only as well as in palbociclib plus endocrine.

And so, there was some missing data. It wasn't viewed to be so strong as to raise into question the level of tumor responses seen because responses just don't happen to happen on their own and patients moving into therapy or moving in there with usually a disease, it's either not responding to prior therapy or a first-line occurrence. So, there was a fair bit of missing data. We didn't do any particular techniques to handle it in this case because the numbers were a bit small, but when we look at larger data sets, there are some different things you can do to back into looking at prognosis and matching of patients and matching of factors that might have been.

Chana went through this in detail in terms of saying that you get really worried about control arms in terms of are you really matching patients and the other risk factors. So, I'll stop there, but it's an important topic and you raise important questions.

Kadcyla's post-marketing evaluation using real-world data from US patients with breast cancer was used to fulfill regulatory needs in both the US and Europe. What have been your learnings on using real-world data from US patients for European regulatory decision-making?

Thibaut Sanglier: Hi, thank you for this question. So, basically, we did try to get some European data to answer the European health authorities, but in the end, it was such a challenge that US data was actually also appropriate because the question was around the underlying pharmacology and the pharmacological effect of the drug. It was one that they believed that US data would be appropriate in this situation.

Also, identifying European patient was a very large challenge for us because the countries are more fragmented and the population in each country is much smaller by definition. Wolfram?

Wolfram Hemmer: Yes, thank you. Yes, what I can add is indeed the numbers that were accessible in this with regard to patients with a low LVEF that were accessible through Flatiron were much higher than whatever we saw, which would be feasible from the registries, let alone European patients from the registries. I think that was the case we made here to replace basically the registries with the Flatiron study in this category three commitment. I think we could convince that this was the best way really to get hold of any such data, and I think it was accepted that they would be applicable to a good extent also with regard to European patients.

We were not very much specifically challenged on this aspect. Definitely, we were asked these questions not only from EU, but also from, for example, Health Canada, and were able to make the case that this would be applicable. That the data from the US would be applicable in the other areas.

This question is for all of our panelists. Based on what you've learned to date, what are some of the key opportunities you see for real-world data and regulatory decision-making in the next three to five years. Again, what are some key opportunities you see for real-world data and regulatory decision-making in the next three to five years? Keeping in mind we have less than five minutes left and we want to hear from everyone, maybe let's start with Dr. Sanglier, Dr. Hemmer, then Dr. Kraus, Dr. Patel, and lastly Dr. Weinstock. Dr. Sanglier, would you like to start?

Thibaut Sanglier: Yes, for sure. I think now we have more and more data becoming available and now we have more defined population, and also now we start to learn also much more in terms of how to use this data. We have seen a lot of the challenges, but we also have methods to limit those. So, learning the best-case scenarios and we have seen here that real-world data is actually extremely useful in these rare populations. So, probably that's where so far where real-world data is most relevant is the situation where we know that really clinical trials are not applicable. And when we have this better understanding of this smaller population really, I think that's where real-world data can play a key role really to complement the more traditional approach we have with the clinical trials data, randomized.

Wolfram Hemmer: Okay, try to be quick. This is Wolfram. Yeah, on top of the very interesting examples we heard today on the use of real-world data for decision-making I agree that what Thibaut had said that it will be really for rare diseases and circumstances where data are really sparse. So, I think filling data gaps will be definitely one of the options and it was part of our examples. But I'm also very curious to see in the next years the first examples of real-world data to be used to support at least the control arms in a regulatory setting.

Albert L. Kraus: Yeah, so a few areas, and I think we have a lot of work to see what gets more traction in certain areas, and we need a lot of case examples, but the one area I think it will obviously continue to be very important for safety signal monitoring as it has been in the past. But I think on an effectiveness side, I think it may well be especially important for extending randomized clinical trial data. FDA already has the option to approve outside of the study the inclusion/exclusion, and I think real-world evidence might allow more of that. In a way, I view IBRANCE, males with breast cancer that way.

But there are many places where we can widen the information source to get patients who would have been excluded in the trials or a wider catchment. It can be groups that never would have been studied. So, I think this could be quite productive. I also think it will continue to be used as a way to get external control arm information. Historically, we may have had series at MD Anderson, Mayo Clinic, wherever, the Farber. And now we're using electronic health records more in combination with series and other real-world data and registry data, and I think that will expand.

I think the concept of adding control arms, I mean, Chana mentioned that. I think it's important particularly in oncology where we have targeted therapies that target pretty strongly identified patient groups based on genomics or proteomics. If the treatment effect's extremely high, these trials might not even be doable, and we end up with a Glivec-like situation where it ends up being single-arm work and CML. Or other scenarios and we may be able to leverage real-world work to control more and more because the trials may fall apart. Anyone who finds out they're on placebo just drops.

So, I think more and more we're going to find examples there, which will maybe allow us to wade into using more control arm data stronger and stronger. I'll stop there. There's other things, but...

Kiran Patel: Yeah, hi. This is Kiran, so I may just extend to Dr. Kraus's statements. I really think that we're in a very different era in terms of drug development, where a lot of our drugs such as CAR-T's have demonstrated substantial clinical benefit. Connecting randomized studies perhaps may lead to loss of equipose very early on. So, some of the suggestions and proposals made, I really think that in the next few years will become real. There is still a lot of work to be done to be able to see how we can make this real. The electronic health records, I think as Dr. Weinstock mentioned, is still unstructured. I think there are ways to streamline that effort to really make much more consistent data out of that. We also know a lot of the clinical records does not reflect something simple such as ECOG performance status. So, there are ways and there are reasons to believe that those things has to align. The other thing also is this connecting the data sets using the pathology report to the clinical report to the radiology to imaging. There's still a lot of pieces that needs to be come together, but I really am hopeful that as we continue to put more and more efforts and commitments around doing that, that in the next two years we will be able to have much more structured data available to be able to meet the regulatory requirements.

Chana Weinstock: Thank you. I think there have been some really excellent ideas put forth by my co-panelists, and I'll echo that. I think as we get better at collecting the data and working through some of the methodologic challenges, I really think that some of the natural history questions and questions about targeted populations or molecularly-defined populations will be much easier to answer. I think in the next two years we'll be probably seeing a lot more of that sort of effort. I think the post-marketing safety piece is very important as well. In terms of the external control arm, I know I was the one who really focused on that. I think that's interesting. I don't know that it's ready for primetime in the near future, but we'll see. We'll see where things go in the next few years as we develop expertise in this area and as more approvals happen.


Jillian Motyl Rockland: Hi, everyone. Thanks for joining us today. My name is Jillian Rockland, and I lead the regulatory solutions team at Flatiron Health. I'm joined today by a few guest speakers: Dr. Chana Weinstock from FDA, Dr. Kiran Patel from Janssen, Dr. Albert Kraus from Pfizer, and Dr. Thibaut Sanglier from Roche. You'll hear from each of them shortly, and we'll also have Dr. Wolfram Hemmer from Roche joining us for the Q&A portion.

Please note that thanks to our many speakers here with us to share their perspectives today the webinar will run 15 minutes longer than scheduled until 2:15 PM Eastern. Some quick housekeeping items before we dive in. First, I'd like to draw your attention to the Q&A option available throughout the webinar. If you hover over your screen with your mouse, you'll see an option for Q&A towards the bottom of your screen within the black bar. At any point during today's presentation feel free to submit a question through this feature as everyone will be muted for the entire webinar. As a reminder, you'll only be able to see your own questions. We're excited to have some open dialogue at the end of the presentation, and we'll do our best to answer as many questions as we can at the end. But we'll follow up afterwards via email for any questions we don't get to and feel free to reach out to us after the webinar if you'd like to discuss any questions in more detail.

If during the webinar you notice your screen is zoomed in too far on a slide, you can click on view options at the top of the screen and click fit to window to zoom back out. Finally, if you have any technical questions or issues, please let us know via the Q&A tool, and we'll do our best to assist you. Lastly, before we jump in, please excuse any interruptions from our pets or loved ones as like most of you we're all working from home.

During our time today, we'll discuss how real-world data has been applied in regulatory settings. As an industry, we've learned and accomplished so much in the past couple of years in incorporating real-world data for regulatory decision-making. We're very excited today to hear from speakers from Janssen, Pfizer, and Roche on the opportunities, challenges, and learnings of using real-world data in regulatory submissions. And also, from the FDA, on the agency's current thinking on using real-world data in their decision-making process.

Before we get started, we'd like to learn a little bit about you, our audience. You should see a poll pop up on your screen momentarily. If you have two monitors, please check both monitors as it may appear on your other screen. Other attendees won't be able to see the response you choose. The question is for which of these applications has your organization considered or incorporated real-world data in regulatory submissions? You can select all that apply.

Real-world data has been used to characterize the natural history of disease or unmet medical need in a specific patient population of interest. Has been used as an external comparator. Has been used to provide information on a product's post-marketing safety and/or effectiveness. Or your organization hasn't considered using real-world data for the above applications yet. 10 more seconds and then we'll share the results.

Okay, let's end the poll and share the results. Great, so as you can see it looks like most of you have used or considered using real-world data to characterize the natural history of disease or unmet medical need. And while many of you have also used or considered using real-world data as an external comparator or to provide safety information or post-marketing information on effectiveness.

Now, we would like to turn the presentation over to our first speaker, Dr. Kiran Patel, Vice President of Clinical Development in the Solid Tumor Franchise at Janssen.

Kiran Patel: Great, thank you, Jillian. Hi, good morning and good afternoon everyone. I'm going to be presenting today the case stud of integrating clinical and genomics real-world data to support the new drug application for Erdafitinib, the first FGFR inhibitor approved in patients with second-line advanced or metastatic urothelial cancer with FGFR positive alterations. Next slide.

My disclosure, I work for Janssen, and I have vested interest and options working for the company. Next slide. So, I'll start with a brief introduction in reference to the high unmet medical need and the reason to generate the real-world evidence data. Patients with advanced bladder cancer have a poor outcome. The available therapies and approved therapies are all single-agent chemotherapy in second-line setting. And those chemotherapies have a very limited response rate and very poor survival.

The IO agents most recently approved such as PD-1 inhibitors have improved outcomes with an objective response rate of approximately 20% and survival extending beyond 10 months. However, not all patients benefit. There are published literature with limited data showing that the responses to these IO agents vary by the TCGA subtypes. In particular patients with luminal 1 subtype bladder cancer may be immunologically cold and may respond poorly to PD-1 inhibition. And therefore, the additional treatment options are needed for these patients with advanced bladder cancer. If you go to the next slide.

FGFR alterations are observed in approximately 20 to 25% of the patients with bladder cancer and also been known that these alterations are associated with non-T-cell-inflamed bladder tumors. It's found largely in these patients with luminal 1 bladder cancer where the IO agents have demonstrated limited to low response rate as shown in this table below. If you go to the next slide.

The phase two study of Erdafitinib in the FGFR+ advanced bladder cancer started almost simultaneously at the time of the development of other IO agents in bladder cancer. This study was a single-arm phase two study to demonstrate the activity of FGFR inhibitor Erdafitinib in patients with advanced bladder cancer. If you go to the next slide.

While the study was ongoing, the treatment paradigm was shifting where anti-PD-1 agents were also in development at the same time, and due to high response rate, led to accelerated approval of these agents. So, there were some patients enrolled in the phase two study that had received prior IO agents such as anti-PD-1, and they're shown here where 33 out of 210 patients enrolled in the study had received prior anti-PD-1.

In those patients with prior PD-1, the response to those PD-1 therapies was very limited as demonstrated here with 6.1% response rate. There was also data with another FGFR inhibitor in development at that time that showed that in 10 patients enrolled with the FGFR3 overexpression none of those patients have response and only one patient has stable disease. Next slide, please.

So, that led us to understanding the unmet medical need in light of the emerging change in treatment paradigm for these patients where PD-1 patients had received approval there were two critical questions for us to address in light of that. Number one is to really understand if this population with FGFR alterations, a subset of those patients with FGFR alterations have a different prognostic value compared to other patients who does not have this alteration.

And then number two critical question for us to address was what is the predictive value of this alteration in the outcome of treatment with available anti-PD-1 agents for which there were two things we did. One was to compare with FGFR-altered versus non-FGFR-altered bladder cancer tumors with the anti-PD-1 treatment. And number two was to do an indirect match comparison of FGFR-altered bladder cancer patients with the BLC2001 study patients enrolled with Erdafitinib versus the anti-PD-1 therapy. If you go to the next slide.

So, we used the Flatiron/Foundation Medicine real-world data. This was a combination of clinical and genomic data set linking the clinical information that includes full treatment histories and survival data from the Flatiron database. And using the genomic data set of FGFR alterations using the FoundationOne next-gen sequencing. So, in the data set of 25,000 plus patients, we identified 5,000 plus patients with advanced bladder cancer in the database of which 426 patients had the diagnosis of bladder cancer as well as had the Foundation Medicine genomic data set. Of which 300 or so patients have received treatment, and of those 300 patients, 124 patients have received anti-PD-1 treatment.

So, to answer the question, the first thing was to look at the demographics, and yes, please go to the next slide. We looked at the demographic and baseline characteristics of patients who had FGFR+ versus FGFR- disease. And you can see there was a very clear balance between two, but I also want to point out that the ECOG performance status, which is one of the key indicator to enroll patients in clinical study, approximately 80% of patients did not have the ECOG performance status. Go to the next slide.

Looking at the prognostic value, there was a trend in demonstrating shorter median survival in patients who had FGFR+ disease versus patients who did not. You can see from the numbers here the median overall survival was 7.2 months in the FGFR+ patients versus nine in the FGFR-. The hazard ratio was trending towards the direction to demonstrate benefit, although it was not statistically significant. Go to the next slide.

Looking at the predictive value of the FGFR with the use of anti-PD-1 therapy, patients who had FGFR+ disease had a median survival of 3.1 months, and in comparison to that, the patients with FGFR- had a median survival of 6.1 months. Again, trending positive with the hazard ratio of 1.33, but not statistically significant. This was maintained while doing the bivariate as well as multivariate Cox regression analysis adjusting for the baseline covariates. Go to the next slide, please.

And in looking at the indirect comparison using propensity score analysis where the treatment for overlapping population was adjusted for key variables such as line of therapy, age, sex, hemoglobin, smoking history as such, there was a clear difference in terms of the median survival in patients with FGFR+ disease. So, this is a clinical trial data set now doing an indirect comparison to that versus the data set from the Flatiron/Foundation Medicine showing 3.1 months survival in patients who have received anti-PD-1. So, if you go to the next slide, please.

In summary, we were able to demonstrate using the real-world evidence data to support the unmet medical need in patients with FGFR+ bladder cancer. Next slide, please. So, we'd like to do some comparison to this data set that I shared against the clinical literature and trials available. Of course, these are two different data sets. We know the clinical trials and literature data set that exist out there.

The Flatiron/Foundation Medicine data is contemporaneous. It has broad access to available therapies, and also, it's a comprehensive genomic profile. So, it's really combining two data sets together. That was the advantage. We were looking for specific population with FGFR alterations, and this data set was able to help support that there was no data out there in clinical trials or literature that can help us support that. If you go to the next slide.

However, there are downside to this and there are limitations to this, and some of the clinical limitations are, as you would expect, there is a selection bias with this type of data set. With the Flatiron data set, it was primarily at that time community-based clinics, and so the data set we received was based upon community settings or patients who are treated in the community setting versus clinical trial data set where all patients are treated at the academic institutions.

In addition to that, there are some missing elements as I mentioned ECOG performance status. There was also missing information on smoking history, hemoglobin, which are key confounding variables for patients with bladder cancer. In addition to that, there were different selection of comparison group, there were differential treatment misclassification, and also there were some statistical limitations especially with the small sample size of patients.

But there were a couple of key things that were also quite intriguing while doing this analysis. One was a survivorship bias whereby we have to exclude subjects who did not undergo molecular testing. Since we're combining two different data sets, that's something we had to apply to be able to make sure that we can bring more consistency in terms of the analysis. And due to the survivorship bias, we had to apply this delayed entry model using left truncation method, which I was not familiar with before we started the analysis. But I learned quite a bit during the assessment to be able to apply this to avoid the survivorship bias that I mentioned before. Go to the next slide.

In addition to those limitations, there are other considerations for real-world studies that could be applied for any studies, but also something that we learned during this process. As I mentioned, the adequate sample size is critical. Linkage to other data sources such as a claims database to capture complete information is also very critical because missing information could also lead to a very different outcome. And knowing that this is a real-world evidence data it's important to make sure that the missingness of the data is limited to some extent so that we can appropriately analyze and interpret the data.

Similarly, applying inclusion/exclusion criteria, especially looking at the control cohort. If the comparison is to a clinical study, it's important to apply the same criteria to minimize other variables. As well as minimizing the ascertainment bias especially with the exposure and outcomes that are listed here. I think it's important to really make sure that all efforts are made to make sure these important confounding factors are balanced before considering the analysis. If you go to the next slide.

And there are additional other considerations that were also learned during the process. Working with Flatiron as well as working with other vendors that allow sufficient time for data privacy issues that emerges from using this data set. So, it's good to manage that quite early on. Consider regulatory filing in contractual language. I always have come to realization that real-world evidence data could be categorized in different ways. The regulatory grade data set has to meet the regulatory standards, and so it's good to have that upfront agreement with the vendor in the contractual language.

The protocol and statistical plan must be in place prior to any analysis, so this is no different than conducting clinical study. The real-world data should also have a protocol and a clear understanding of what we plan to analyze. And then the rigor of data preparation for real-world evidence is equal to that of the submission of clinical trial data. So, again, applying those same principles in terms of the quality is critical for using real-world evidence. Next slide.

So, here is an overall conclusion leveraging real-world data. We were able to understand exactly what the issue was for this submission, which was that there are available therapies that has emerged as a standard of care. We use real-world data using Flatiron and Foundation Medicine to be able to combine genomics and clinical data set to be able to support this data set as a supportive submission for the new drug application for Erdafitinib. And that's my last slide. Thank you so much.

Jillian Motyl Rockland: Thank you, Dr. Patel. I'd now like to introduce our next speaker, Dr. Albert Kraus, Portfolio Lead and Global Regulatory Oncology at Pfizer.

Albert L. Kraus: Thank you, Jillian, and good morning everyone at least in the US. Good afternoon if you're coming from Europe or happy evening in Asia. Next slide. Yeah, I'm going to just spend a few minutes high-level overview of some work we did using real-world data and real-world evidence in the case of palbociclib in breast cancer. Next slide.

Just disclosures, I am a Pfizer employee. Obviously then own stock and options in Pfizer, and these views are my own and not necessarily those of the Pfizer corporation. Next slide. So, a little background on breast cancer in men. It's a rare disease, just over 2,600 new cases in 2019, about 500 deaths, but it's a dire disease. It's an incurable disease, dismal five-year survival rate of about 16% likely because men don't expect to get breast cancer, and so probably diagnosis is occurring a little later and the disease has progressed. But for knowledge of it being a disease that's not fully data found, but it's everybody's best guess.

Importantly in male patients with breast cancer, the tumors themselves express hormone receptors and have a biological similarity to the actual tumors in breast cancer in women. And so, that becomes an important element to this real-world assessment. Indeed because of that and some other factors, the US National Comprehensive Cancer Network, NCCN, guidelines indicate male patients with metastatic breast cancer should receive similar treatment as postmenopausal women with breast cancer. Next slide.

As well, just to indicate in the background, palbociclib, which is a CDK4 inhibitor, it's a cell cycle arrest kind of activity, is approved based on the back of multiple randomized trials with very substantial hazard ratios in big treatment in fact for the treatment of women who have hormone receptor-positive/HER2- breast cancer. And as well was incorporated into practice for men, which is why we could obtain some real-world data despite it not being approved in men. We hadn't done the trials including men. They were done in women. Due to the rarity of the disease in men, it's also not feasible to conduct a traditional large adequately powered randomized trial. Next.

Important, I think, thing about real-world evidence especially in these early days if you will in registration use is the fit of the evidence into the totality of evidence. In this case, the biology of breast cancer in men and women seems similar based on a lot of background cellular/molecular information. Clinical information in this class as well generally supports similar treatment effects and palbociclib itself as I mentioned has a large and consistent treatment effect. And I mentioned the other factors here. Next slide.

As well, I would say we did work before we went into this, for this as well as for other efforts in real-world evidence looking at the endpoint derivation in the real world because real-world assessment of both response and progression is different than in trials. In trials, we use RECIST. In the real world, docs are assessing tumor response on radiology, but not in the same manner as RECIST. And tumor progression itself again is assessed a little bit different in the clinic.

So, we went and did a little bit of work, and this is in women actually, but to look at the relation of progression outcome, response outcome in real-world versus in the trials, and we found a pretty substantial consistency, and this was presented at San Antonio in 2017. That formed a backbone of some confidence to look at real-world data. At least we felt the endpoints were reasonable based on this work. Next slide. The work in palbociclib in males itself was to evaluate real-world treatment patterns including clinical activity and safety to look at what we could learn in the benefit and risks of men with hormone receptor-positive/HER2- breast cancer. Next slide.

We used three real-world data sources. I like to call it in a way triangulating against the truth because the precision may be less in the real world. Although, two of these are more effective in the space than one safety. But we used, for looking at tumor response, electronic health records using the Flatiron health system. In the second source, we looked at pharmacy and medical claims databases using IQVIA data in large part to look at duration of therapy and prescription writing. And then the third safety database was using the Pfizer global ARGUS safety database to make sure we had a beat on safety in males, that we did have some male data from other development work. Next slide.

I won't get into all the elements here, but this needless to say, in a rare disease you start with a fair number of patients treated. We had predefined protocols as Kiran went through, and predefined assessment plans. And so, we have a number of detailed inclusion criteria to try to get the right patients. Then we have to make sure they were treated with palbociclib. There's some level of disease assessment that's adequate. A response assessment, progression assessment. We want to make sure they also received antihormonal agents in this case aromatase inhibitors or fulvestrant.

And then it narrows down to relatively small numbers in a rare disease in this case, and we'll get into this a little closer. But a little bigger numbers in prescription deliverables, smaller numbers in response assessment. Next slide. So, just to briefly give you a highlight on some of the data. This was duration of therapy. As you can see in the Kaplan-Meier group here, the duration of treatment and the blue line above is palbociclib plus letrozole in the frontline setting. It was longer than letrozole alone, about three times as long. Almost looking at that as a time to treatment failure kind of assessment. As well in the second line, which is not shown here or further than second line, palbociclib plus fulvestrant seemed to deliver a longer duration of therapy than fulvestrant alone. Next slide. When we look at the Flatiron Health response data, the , which was partial plus complete response as defined, not in a RECIST sort of manner, we had four responses out of 12 patients. When we looked at endocrine-only, we had one response out of eight patients. Knowing this was in a confined duration of time assessed so that the patient assessments would be done in a similar temporal environment.

We wanted to get a little more data on endocrine-only in males, and so we went to a different timeframe and picked up 16 patients with adequate data, et cetera, and two of those 16 have responded. So, suggesting a similar in that range of one out of eight for background therapy, whereas we had four out of 12 for the palbociclib. Next slide. Safety, bottom line here, just for the sake of time, we looked at some specific prespecified AEs in the Flatiron EHRs. We also scoured the Pfizer global safety database and we didn't see anything that looked different than the toxicity profile seen in women. Next slide.

FDA elements here. We did have multiple meetings with FDA, and I would encourage engagement and discussion with FDA on any proposals on several real-world designs, including these aspects that I'm presenting here on males. But there were other efforts and a lot of discussion input. There was a thorough review by FDA with a lot of questions and queries on all sorts of elements, data elements, interpretation elements, et cetera. And multiple site inspections for the work, both at Pfizer where we did the data crunching if you will, and stat analyses. But at Flatiron as well with the EHR and IQVIA with their aspects. Obviously, the inspections went well. No 483's issued. Next slide.

Some specific inclusions to this aspect is we felt real-world data sources supported that men with metastatic breast cancer appeared to drive some benefit from addition of palbociclib endocrine therapy. Safety profiles seemed consistent. So, we felt it was useful information. There are a number of limitations of these retrospective analyses in the small data set for sure. Kiran went through a number of them. I could go on about them too. There are limitations to anything we do, of course, but in these small data sets, even more in retrospective adds.

But we did submit this to FDA overall with the breadth of information that we previously obtained, and we were able to get an expanded indication including men into the label. And so, we think this is exciting new way, in this case, to slightly expand the label based on the strong evidence that's really this substantial evidence from the randomized trials in women. Next slide.

So, this just shows some changes in the USPI. We moved from being approved in women to adult patients, postmenopausal women, or men, talking about patients. We also had a section on a dosing schedule. I won't get into it here, but we got some fairly controversial information on use of LHRH agonist in men. You use those in women in certain instances, and it's recommended to be used in men, but there wasn't good compliance with use. And NCCN recommends they are used, and yet they're not in the real world that much. So, it was interesting. We put some languaging around there to hopefully help guide use a little better.

And then there is a statement around the safety of real-world evidence showing consistency with that seen in women. Next slide. I think this is the last one mostly. So, I think the key considerations too, we have to find fit for purpose examples of use of real-world evidence, and really tie it to the totality of all the evidence that we see on the drug. Non-clinical, clinical, and in this case, the fact that a randomized trial was not feasible also helped it. Next slide. And so, that's it. That's all I have. Thank you very much for your attention, and I'll turn it back to Jillian.

Jillian Motyl Rockland: Thank you for that informative presentation, Dr. Kraus. I will now turn it over to Dr. Thibaut Sanglier, Principal Data Scientist and Pharmacoepidemiologist, and Real-World Data Oncology at Roche.

Thibaut Sanglier: Thank you very much, Jillian. Good morning, good afternoon, good evening, everyone. So, I think we can move to the disclaimer page. I'm a full-time employee of Roche. Next slide, please. This presentation shares my view and not necessarily those of Hoffman-La Roche. Next slide, please.

For the next 10 minutes, I will share with you an example of how we used real-world data to address European commitment, and the fulfilling of this commitment led to an update of the core datasheet of Kadcyla and label updates in various countries. Throughout this the next 10 minutes, I will answer the three following questions: why real-world data? With a specific focus on why secondary data use was particularly relevant to us. What is the study about? The study I'm going to present, and what are the learnings we've drawn from this experience? Next slide, please.

In terms of clinical context and a bit of background, Kadcyla has different indications. One of them is in the metastatic setting where Kadcyla is indicated in patient previously treated with HER2 targeted therapies. In the clinical development program of Kadcyla, patient with a low left ventricular ejection fraction were excluded from the randomized clinical trial. So, patients with low LVEFs or LVEF lower than 50% were not included in the clinical trials. We had no cardiac safety data on the use of Kadcyla in patients with low LVEF after the approval.

Also, it's to be known that HER2 targeted therapies are also associated with an increased risk of developing left ventricular dysfunctions. So, two points that are important here. First, we're talking about the rare population. Low LVEF is quite rare in this population. And also, Roche committed to evaluate the cardiac safety outcome in patient initiating Kadcyla when they had a low left ventricular ejection fraction. Next slide, please. That is the timeline and regulatory context. Kadcyla was approved in the US in 2013, and in 2014 we had a category three commitment in the risk management plan of the company that aimed at addressing cardiac safety of the use of Kadcyla in this population. Next slide, please.

Initially, we proposed to evaluate the risk of interest in this population by using the metastatic breast cancer disease registries we were setting up at the time. When we made this proposal, we were very upfront with the health authorities, and we shared that we were under the impression that this population would be quite rare and that could be maybe a challenge in the recruitment of such a population. Next slide, please.

Actually, over the year when the registry started to recruit, we indeed started to face some challenges with respect to the recruitment, and that's why we started to inquire about using maybe different data that could help us to better document the risk of interest. That's why in 2016 we started to work with Flatiron, and we started to conduct a feasibility study in order to assess first the relevance of the data Flatiron had access to. But also, we wanted to assess the relevance of the technology that Flatiron could potentially use to answer our research question. Next slide, please.

Our intent was to use retrospective data, and from this retrospective data, identify a cohort of incident users. So, patients starting Kadcyla, and this user would have had to have an LVEF between 40 and 50%. Next slide, please. So, of course, we wanted to focus on baseline characteristic with a specific focus on cardiac condition and cardiac history. History of prior discontinuation to HER2 target therapy because of cardiac toxicity for instance.

Now, in terms of the outcome, we wanted to focus first on treatment-emergent adverse event, and we looked primarily at the incidence of an LVEF drop of more than 10% point from baseline, incidents of congestive heart failure, and other cardiac events. We also looked at symptoms and signs that were compatible with a clinical picture of CHF, of congestive heart failure, because we had some concern with maybe the risk of missing some outcome of interest. Next slide, please. Talking about the challenges and design consideration. We wanted to anticipate before designing the study, so the limitations here were potential limitation we considered before designing the study. First, we were aware that there was no background rate available because our population was really unknown at the time. We knew as well that there was a risk of protopathic bias meaning that patient who may start to experience early symptoms of the outcome could have been more likely to initiate Kadcyla.

And also, because we select a population conditional to the fact that they have some level of cardiac frailty, and we looked at cardiac outcome, we had to keep in mind that the outcomes we are measuring could be both actual new events, so incident cases, but also prevalent cases. And the rest of the slide talks more about limitation that are more general to the use of real-world data. So, in our case, in the context of safety, we wanted to define a risk period that was linked to the treatment duration, and treatment duration is not readily available in EHR, so we had to work with a range of sensitivity analyses on how to define this treatment duration.

We were also potentially in a situation of competing risk where the risk of death could compete with the risk of the cardiac events we wanted to measure. And also, then the other challenges that have also been raised in terms of misclassification, and the potential and the reporting of the outcome of interest, informative censoring, and missing data. Next slide, please.

So, what was really interesting for us at the time of the feasibility was the technology that the Flatiron team was suggesting to use. It was a technology... Next slide, please. The modular technology-enabled abstraction. For us, we knew the population was rare, and it was really interesting to work with Flatiron so they could query the structured data they have access to, to rapidly identify patient initiating Kadcyla for metastatic breast cancer. And then the team was able to use this modular technology-enabled abstraction to focus on the LVEF data of these Kadcyla patients. So, then it was possible to identify only the patient with the low LVEF, and then focus further abstraction on this patient. Next slide, please. On this slide we see the CONSORT chart of how we were able to identify our cohort of patient of interest. At the top of the chart, you see that we started with roughly 2,000 patients who started Kadcyla in the data set. Then the Flatiron used their modular abstraction to focus only on the LVEF data of these 2,000 patients. And then we had 135 patients who were potential candidates to be selected for analysis.

Now, applying other selection criteria we're able to identify 67 patient who had an LVEF in between 40 to 50% inclusive, and then we had a subset of those who were actually our primary cohort of patients with a left ventricular ejection fraction that was in between 40 to 49% and recorded within a two-month prior at the initiation of Kadcyla. One note here is that LVEF has many other lab values. It's sometimes reported as an absolute number, which is really helpful when you want to define a population according to a certain threshold. But also, sometimes it's reported as a range, so that's also why we had to work with the different types of cohorts.

On the next slide, we see just a snapshot of different results we had. Very briefly, on the top left corner, we see an estimation of the prevalence of different conditions we were interested in. What was really reassuring for us is that the prevalence of these conditions was actually quite in line with what we were expecting. On the right-hand side, you see the different outcomes and the different endpoint we use to look at the data and try to better quantify the cardiac risk in the safety population. So, we use different proportion, event rate, and the cumulative incidents at six months. Next slide, please.

In terms of the very high-level conclusions for this study. First, this study using Flatiron data, but also what we're seeing through the disease registry confirmed that the population of interest was actually quite rare. So, in very rare instances only Kadcyla was initiated in patients with low left ventricular ejection fraction. Most of the cases who presented, and one of the outcomes of interest during the follow-up actually presented with already a history of cardiac events or cardiac toxicity in the past before starting Kadcyla, and we did not observe a drastic increase of the cardiac risk in our study population.

For us, it speaks about the benefit-risk assessment that has to be done on a case-by-case basis in between the physician and the patients. And also, LVEF really has to be monitored throughout the course of HER2 treatments. Especially in this population. Next slide, please. Now, going back to the timelines and the outcome of this project. So, at the end of 2016. We were quite convinced that Flatiron was able to provide an added source of evidence that could be of relevance, and actually, this turned out to be the data we use in lieu of what was generated by the disease registries because the disease registry did not recruit enough patient at all. Next slide, please.

We proposed to the health authorities to go with Flatiron data to fulfill this proposal commitment and we registered the study on the ENCePP website before conducting the full analysis. We actually had the early interim analysis that was shared with the health authorities at the occasion of the PBRER. We had a publication of the result at the San Antonio Breast Cancer Symposium in 2019, and shortly after that, we received positive CHMP opinion in terms of the text we were submitting for the label.

This resulted in an update of the core datasheet for the compound and the update of the label of Kadcyla in Europe, in US, and Canada as well. The next slide. So, here is just an excerpt of the USPI with the study that is highlighted here in blue. Next slide, please. So, now going back to the beginning of this presentation. Why real-world data? First, it was about the research question that was really about routine clinical practice. And then why secondary data use? Because we intended initially to use prospective data, but in the end, it was so challenging to identify and recruit these patients that secondary data use was actually more fit for purpose approach.

This study was really about closing the gap on missing safety information in the population. It was not studied in clinical trials. And in terms of the learning for the team, and here it's really kudos to the regulatory team that did a fantastic job because it was possible for us to provide frequent update on the study status of both the registries and this study. It was also possible to share with health authorities and be very transparent with regard to the challenges we were experiencing.

Also, when these challenge were maybe too big, it was also possible to propose an alternate data source, so the Flatiron data. At the same time when we were making this proposal, it was possible for us to share the result of a feasibility. In the end, this study helped us to address European commitment that led to the update of the core datasheet of the product and change in the label of the product for Europe, US, and Canada, and the process is ongoing for other countries.

On the next slide, we have few references, and the next slide is an acknowledgment of the people who worked on this project over the years. Thank you very much.

Jillian Motyl Rockland: Thank you, Dr. Sanglier. Now, I'll introduce our last speaker before we get to the Q&A portion of the webinar. Dr. Chana Weinstock, team leader at CDER and Division of Oncology 1 at the FDA. Handing it over to you Dr. Weinstock.

Chana Weinstock: Thank you so much, Jillian. Good afternoon, my name is Chana Weinstock, and I'm a medical oncologist at FDA. Thanks so much to the previous speakers for their talk, and for outlining three very interesting case examples of regulatory submissions that included real-world evidence. Obviously, from FDA's end of things, this is a very big topic, and I certainly won't have time to do it justice in this brief talk, but I did want to touch on our current thinking on the topic. Instead of repeating the examples already discussed, I want to talk about some other directions that real-world evidence could take in a regulatory setting.

Thank you very much to the panelists and participants for joining today, and to Flatiron for inviting me to speak. Let's get real about real-world evidence and oncology drug approval. Next slide, please. So, I have no disclosures, and the opinions stated are my own. First, I'll be talking about the rationale behind the use of real-world evidence in a regulatory setting, some strengths and limitations, then I'll talk a bit about the use of external control arms in clinical trials. I'll touch on ongoing research in this area, and I'll briefly discuss COVID-19 and lessons to be learned from the pandemic experience. Next slide, please.

To begin with, why is it that we're so interested in the topic of real-world evidence at FDA? Well, to start with, it's because we have a congressional mandate from the 21st Century Cures Act that requires FDA to develop a framework and to issue guidance regarding use of real-world evidence in the regulatory setting. The PDUFA reauthorization builds on this requirement. So, there's a congressional mandate to think about and to develop a plan for how we deal with this topic. Next slide, please.

There are many potential advantages of real-world data, and to me, one of the most important is that there are many patients whose experiences are not captured in the context of a clinical trial. In addition to limitations of inclusion and exclusion criteria and geographic enrollment restrictions, there's the fact that a very small percentage of adult patients overall enroll on cancer clinical trials. So, real-world evidence has the potential to provide inclusivity by virtue of being more reflective and representative of the population who will use the drug after it's approved.

Additionally, real-world evidence has the potential to provide information on rare cancers. Next slide, please.

So, to back up and think about the rationale for inclusion of real-world data and approving drugs or biologic products or cancer indication. In my mind, I conceptualize this as a pyramidal structure in which there's clinical trial data generated by patients who've enrolled in clinical trials. A portion of those patients' experiences including efficacy and safety information can be captured and submitted to FDA for review and will subsequently be used in drug approval. Next slide.

However, if fewer than 5% of adult patients with cancer receive their care in the context of a clinical trial there's an enormous amount of potential data that exists regarding patients whose experiences may not be captured in the context of a clinical trial. Much of this real-world data exists in electronic form. It's been captured electronically, through as we've seen examples of electronic records, through insurance claims data, other billings data, et cetera. However, the question of whether this data is suitable to be used as evidence for regulatory approvals is very pertinent. Next slide, please.

So, let's talk about some of the potential challenges to use of real-world data. A lot of this has been covered already, so let's specifically use the example of electronic health records, EHR. On the one hand, almost every oncology patient treated today has EHR data recorded, which represents a relatively granular clinical picture of the patient experience. The challenges include the fact that data in pathology, radiology, and clinical notes are often unstructured. The way these data are captured through typing does not necessarily equal consistency and complete documentation. And very importantly clinical outcome measures for drug approvals may not be used or consistently recorded in practice. That makes using these data quite challenging in a regulatory context. Next slide, please.

As I said previously, the case studies and examples you heard prior to my talk discuss using real-world evidence to complement an approval done primarily based on clinical trial data. Highlighting some of the experiences that we had using that approach to the use of real-world evidence in recent years. What if we go a step further and talk about potentially using real-world evidence in the setting of an external control arm built into a trial itself. This was also an approach we've been asked about many times and had discussions about in various contexts.

Let's talk about the rationale for doing this sort of trial. In a conventional clinical trial, for example, the trial in the schema shown above, you might take 600 patients eligible for your trial, and randomize them to the experimental versus the control arm in a 1:1 ratio. Next slide, please. In a trial with an external control arm, you'd essentially only be conducting a single-arm trial with all trial patients on the experimental arm, and control data came from another setting on patients outside that clinical trial.

Your protocol and statistical analysis plan would prespecify the source of the external control data and obviously would specify how the analysis of this data would occur. This sort of trial design has never to date been used in a cancer drug approval, and that's because there are major concerns with this approach despite the advantages it might appear to offer in terms of patient numbers required for treatment. Next slide.

So, what are the potential problems with using an external source of data for a clinical trial like I've proposed? First, in terms of using real-world evidence as the source of the control arm data, one major problem is that patients with cancer treated on a clinical trial when matched and compared to those not treated on trials have actually been shown to live longer, just simply by virtue of being trial patients. This may implicitly bias the trials in favor of the experimental arm.

In general, randomization addresses the unquantifiable differences in a way that propensity score matching or other statistical methods cannot. There may be certain populations that are inherently less heterogeneous for whom external controls may be somewhat less problematic. For example, patients with molecularly defined tumor. However, I will again emphasize that no FDA approvals in oncology to date have relied on external control designs due in part to these issues. Next slide, please.

FDA has ongoing interest in this topic, and the Oncology Center of Excellence has published extensively related to real-world evidence. Here's one example of how we've contributed to the discussion, and that's our look at pragmatic trial endpoints. By looking back at the large collection of trial data we have at FDA from applications that have been submitted for review. By providing an aggregate look at that data, we can get a sense of how some endpoints that might be more easily captured in a real-world setting actually played out in a large collection of trial patients for whom data was carefully collected in the setting of the trial.

We retrospectively analyzed pragmatic clinical trials and points that lend themselves to the real-world setting. For example, time to treatment discontinuation. We looked at how these correlate with PFS and OS. These analyses were conducted in non-small cell lung cancer, renal cell carcinoma, and are ongoing, and hopefully, contribute to the discussion and understanding of these pragmatic trial endpoints. Next slide, please.

I'm also going to highlight Project Switch, which was an effort by my FDA colleagues to look at randomized clinical trials in several disease areas in which a later trial used a control arm that was the experimental treatment of an earlier trial. Or a control treatment for an earlier trial for the same patient population. The goals of this analysis was to compare the earlier trial arm as an external control to a later control experimental arm using statistical methods to control for bias.

The results of this analysis demonstrated that in some cases, external and concurrent controls were similar in effect size due to very specific biomarker selection and first-line treatment. This was not the case in more heterogeneous populations. Next slide, please.

So, just a parting thought. As you know, we're in the midst of a global pandemic, which has and will have many implications on all aspects of life as we know it, including on the conduct of clinical trials. Long before COVID, there was a call to make trials more patient-friendly by decentralizing clinical trials, and by bringing trial assessments to where patients live taking advantage of digital health technology. By necessity, we're deploying aspects of decentralized trial and exploring real-world data to respond to the COVID-19 pandemic.

So, what can we learn? It would be a shame to have this grand experiment pass us by and for trial conduct to continue unchanged. I challenge you all today to think of potential takeaways for clinical trial conduct. Next slide, please. Finally, FDA remains committed to Project Facilitate, which is our centralized effort to assist healthcare providers with requests to access investigational oncology products, and here's the contact information for this project. My final slide I'd like to acknowledge my FDA colleagues with interest in this topic and thank you so much for inviting me to speak today.

Jillian Motyl Rockland: Wonderful. Thank you so much all of our panelists.