An initial set of studies went up this week at medRxiv, a new and somewhat controversial online outlet hosting preprinted clinical research reports—they haven’t yet been subject to peer review, much less journal editing—and the batch includes one dealing with AI.
The study, “Prospective and External Evaluation of a Machine Learning Model to Predict In-Hospital Mortality,” was conducted at Duke University.
The title tips the project’s objective. Senior author Mark Sendak, MD, MPP, and colleagues worked with EHR data from more than 75,000 hospitalizations to train, retrospectively validate and prospectively validate their model.
Noting that the ability to predict death before discharge at the time of admission could guide clinical and operational decision making—as well as possibly help improve outcomes—Sendak and team report that their model achieved an area under the precision recall curve (AUROC) of 0.90.
“Taken together, the findings in this study provide encouraging support that machine learning models to predict in-hospital mortality can be implemented on live EHR data with prospective performance matching performance seen in retrospective evaluations of highly curated research data sets,” the authors write. “The benefit-to-cost ratio of developing and deploying models in clinical settings will continue to increase as commonly available EHR data elements are more effectively utilized and opportunities to scale models externally are identified.”
Sendak et al. note that more research is needed to figure out how to integrate their machine-learning methodology into clinical workflows, scale it to sample size and calculate its potential impacts on outcomes.
MedRxiv is run jointly by Cold Spring Harbor Laboratory, Yale University and the BMJ. It prefaces each study it publishes with a caution: “This article is a preprint and has not been peer-reviewed. It reports new medical research that has yet to be evaluated and so should not be used to guide clinical practice.”
Earlier this month MedPage Today posted a video interview with two of the site’s co-founders, and a few commenters voiced concerns over posting clinical research before it’s been peer-reviewed.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.