Flatiron Health Announces Nature Digital Medicine Publication of Machine Learning Development and Evaluation Framework Supporting Prospective Clinical Decisions Applied to Risk Prediction in Oncology
Flatiron Health announced the publication in Nature Digital Medicine of evidence from a machine learning (ML) approach that demonstrates the value of a ML-based clinical tool that can be used to support a clinician’s independent assessment of patient risk, using EHR-derived RWD, to avoid acute care events.
In partnership with the Huntsman Cancer Institute, the team’s research focused on translating the clinical challenge of surfacing patients who may be eligible for supplemental at-home care
into a technical problem. Flatiron engineers, designers, and data scientists worked with clinicians and administrators to understand their workflows, objectives and pain points. Collaboration between clinicians and engineers was essential to building a transparent and interpretable machine learning algorithm and defining how this could be embedded into existing clinical workflows to help make high-quality care easier and more efficient.
The proposed generalized framework provides guidance on crossing the chasm between research and practice, by positing six key steps that can enable the use of ML-based tools at the point of care effectively, responsibly and practically, including: Define the healthcare system quality improvement goal; Build or acquire ML-based clinical tool for prediction; Conduct retrospective evaluation of the tool; Conduct bias assessment; Conduct prospective evaluation; Embed and monitor the ML-based clinical tool in clinical workflow.
“It is critical to listen closely to the challenges described by frontline health workers and clinical administrators,” said Dr. Blythe Adamson, Principal Scientist at Flatiron Health. A key takeaway was that prospective validation of ML tool effectiveness was an important step in gaining the trust of care providers who incorporated it into their workflow. They achieved meaningful gains in efficiency of providing high-quality care for patients with cancer.”
A story motivating the development of this ML framework is featured in the Nature Health community.
- Ben-Israel D, Jacobs WB, Casha S, Lang S, Ryu WHA, de Lotbiniere-Bassett M, Cadotte DW. The impact of machine learning on patient care: A systematic review. Artif Intell Med. 2020 Mar;103:101785. doi: 10.1016/j.artmed.2019.101785. Epub 2019 Dec 31. PMID: 32143792.
- Coombs L, Orlando A, Wang X, Shaw P, Rich A, Lakhtakia, Titchener K, Adamson B, Miksad R, Mooney K. A machine learning framework supporting prospective clinical decisions applied to risk prediction in oncology. npj Digit. Med. 5, 117 (2022). https://doi.org/10.1038/s41746-022-00660-3
- Estevez M, Benedum CM, Jiang C, Cohen AB, Phadke S, Sarkar S, Bozkurt S. Considerations for the Use of Machine Learning Extracted Real-World Data to Support Evidence Generation: A Research-Centric Evaluation Framework. Cancers. 2022; 14(13):3063. https://doi.org/10.3390/cancers14133063
- Padula W et al. Machine Learning Methods in Health Economics and Outcomes Research—The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force. Value in Health, Volume 25, Issue 7, 1063 – 1080. Link
- Norgeot, B., Quer, G., Beaulieu-Jones, B.K. et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med 26, 1320–1324 (2020). https://doi.org/10.1038/s41591-020-1041-y
Flatiron Health is a healthtech company dedicated to helping cancer centers thrive and deliver better care for patients today and tomorrow. Through clinical and data science, we translate patient experiences into real-world evidence to improve treatment, inform policy, and advance research. Cancer is smart. Together, we can be smarter. Flatiron Health is an independent affiliate of the Roche Group. Flatiron.com @FlatironHealth