If big data is to fulfill its potential for advancing the state of modern healthcare, developers of medical AI must be willing to show their work in detail, 25 or so researchers from around the world jointly assert in an article just published in Nature.
“[T]ransparency in the form of the actual computer code used to train a model and arrive at its final set of parameters is essential for research reproducibility,” the authors write, emphasizing the indispensability of the latter step in the scientific method.
The team took up the topic in response to the widely publicized 2020 Google Health study whose authors suggested that, in some scenarios, their AI system could be superior to experienced radiologists at finding cancers in mammograms.
The authors of the present Nature paper point out that no other research teams have been able to reproduce the findings.
They note part of the problem lies in Google Health’s reliance on licensed data sets out of the reach of most who could try to replicate Google Health’s results.
In a news release sent by Harvard T.H. Chan School of Public Health, John Quackenbush, PhD, a signatory of the Nature article, says the foundation of the scientific method is that “research results must be testable by others. Testability is even more important in clinical applications because we need a high level of confidence in our methods before they are used with patients.”
Quackenbush is chair of the biostatistics department at the Harvard school and a professor at the Dana-Farber Cancer Institute.
“In applications of artificial intelligence, this requires that the models, software code and data are available for independent validation,” he adds. “Transparency will accelerate research, advance patient care and build confidence among scientists and clinicians.”
The Nature paper is behind a paywall. The Harvard news release is posted here.
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.