The first patient in my morning cancer clinic asks me, with a mixture of hope and fear, "Is it working, Doc?" Her partner, who sits by her side, is happy that chemotherapy has helped her pain and is eager to hear that the radiology scans also show that the cancer has improved. These clinical and radiologic outcomes are critical to understanding whether a treatment is effective or needs to be changed. We all smile when we read the radiology report: the tumor is smaller! Each of us then silently wonders what will happen on the next scan.
In cancer clinical trials, we often know when treatment is beneficial by measuring the size of the tumor as it appears on radiographic scans. We then classify observed changes using an objective set of criteria and calculations, known as RECIST (Response Evaluation Criteria in Solid Tumors). But applying these specific radiographic measurements is not always feasible or practical in "real-world settings," where most cancer patients receive their care.
In routine practice, doctors often capture their impression of how a patient with cancer is doing using individually drafted clinician notes filed in the electronic health record (EHR). These notes may include a summary of results from radiology reports, lab tests or general information about the patient's health status. This is the information that oncologists use to make treatment decisions. Notably, the documentation does not necessarily contain the "source evidence" — such as scanned images or tumor size measurements — that are often available in a clinical trial database.
Progression of the cancer (i.e., worsening of disease) is one of the most common reasons to change treatment plans. Progression (whether identified by imaging, new clinical symptoms, worsening blood test results, etc.) demonstrates that the ongoing treatment no longer controls the cancer. At Flatiron, we are investigating ways to best identify real-world progression (rwP) events from the information contained in the patients' charts. A better understanding of how well and for how long a treatment may work (i.e., time from the start of a particular treatment until it stops working as indicated by a rwP event) is important for patients who are planning their lives and for their clinicians deciding on next steps.
Towards this end, we conducted three research studies using de-identified EHR data to evaluate outcomes in the real-world setting to assess if a patient's cancer was "unchanged" or "worse." Specifically, we aimed to understand and measure rwP – a critical event in a patient's cancer journey. So what did we do?
-
First, we needed to abstract the information about cancer progression from the patient's EHR. We also needed to confirm that the captured data was accurate and meaningful for the patient's care.
-
Then, we needed to repeat and scale this process for thousands of patient charts.
-
Finally, we needed to know if we could apply the rwP data to address questions and draw conclusions to gain new insights that could benefit future patients.
Abstract & Confirm
The first step, as described in our study published in Advances in Therapy , was to review 200 patient EHR charts to understand how changes in tumor size on imaging were documented. We found that imaging reports were very descriptive about whether the cancer improved or worsened. However, the RECIST system, which involves precise calculations of tumor size changes, was not used frequently. As a researcher and practicing medical oncologist, this makes sense to me: the overall picture of the patient as "better," "worse," or "the same" is generally much more important for decision-making compared to a specific number.
We then tried to apply the standard approach used in cancer clinical trials, RECIST criteria, to the tumor size information available in existing EHRs. The success of this approach rests on knowing the precise percentage difference in the size of each tumor, from the first scan until the current scan. As you can imagine, this can include a lot of tumors and scans! Since we did not have access to the original "raw" images to measure the tumors ourselves, we had to rely on what the radiologist documented in their report.
We found that the lack of tumor size details in EHR radiology reports prevented us from using the standard RECIST method as commonly used in clinical trials. This challenge was not surprising to me because documenting the precise diameter of each tumor can result in a long list! However, even without reliably being able to apply the RECIST method, we realized that, through the research study, we could access robust documentation in the EHR: the treating physician's notes.
We consulted with a diverse group of academic and FDA experts to develop an approach to capture when a patient's tumor was doing worse from information available in EHR notes. We started with "progression" because it is the cancer assessment most likely to lead to a change in treatment. We evaluated three ways to determine the most accurate and efficient method to curate real world progression (rwP):
-
Radiology-anchored approach: Based on the radiologist's evaluation of the patient's scan.
-
Clinician-anchored approach: Defined as the clinician's assessment of patient symptoms, clinical findings, and interpretation of relevant results and reports that are routinely summarized in EHR documents.
-
A combination of both approaches.
Given our reliance on the radiology RECIST criteria in clinical trials, we anticipated greater reliability with the radiology-anchored abstraction approach. However, in this pilot study we found that the clinician-anchored method was equally reliable at detecting when a patient's cancer progressed and was preferred by abstractors, largely because the approach was anchored in the complete context of the patient's clinical journey.
Crucially, the rwP events captured were followed by important events such as a change of treatment or death. We concluded that the clinician-anchored approach was feasible, reliable and meaningful. Next, we needed to apply the approach to an even bigger study of the tens of thousands of patients in one of our real-world datasets!
Repeat & Scale
Our second research study, published in JCO Clinical Cancer Informatics , was designed to determine if the clinician-anchored abstraction approach would successfully capture rwP from the de-identified EHR records of more than 30,000 patients with advanced non-small cell lung cancer (aNSCLC). Additionally, we evaluated real-world progression-free survival (the length of time a patient survived before the cancer worsened), a key progression-based endpoint to understand how well rwP translated to fundamental cancer treatment metrics. Based on this work, we now knew that the clinician-anchored approach to curating rwP from EHR records was feasible, reliable and scalable \.
Gain New Insights
After identifying a reliable, feasible, meaningful and scalable approach to assessing rwP as an endpoint, we then sought to derive clinically meaningful insights for a specific research question. Building on our prior collaborative research studies (see here and here ) — both published in The Oncologist with co-authors from the U.S. Food and Drug Administration — our third study, published in Cancer , expanded upon our description of early real-world utilization of immunotherapy treatment in patients with aNSCLC and their survival. We now could answer important questions such as "how long does immunotherapy keep the cancer from progressing in the real-world?"
After completing these three studies, we have begun to understand how cancer events such as progression can be captured from EHR data gathered during routine care. We identified a robust approach to determine when the patient's cancer worsens based on clinician's notes in the EHR, and evaluated how rwP correlated with overall survival. In sum, we are on a sound footing to perform meaningful assessments of real-world outcomes for large cohorts of patients with cancer. Ultimately, research about rwP in cancer will provide clinicians with more information to share with patients like mine who want to know what today's scan can tell them about their future.
