Google AI did better than Six Radiologists

One of lung cancer’s most lethal attributes is its ability to trick radiologists. Some nodules appear threatening but turn out to be false positives. Others escape notice entirely, and then spiral without symptoms into metastatic disease.

On Monday, however, Google unveiled an artificial intelligence system that — in early testing — demonstrated a remarkable talent for seeing through lung cancer’s disguises.

study published in Nature Medicine reported that the algorithm, trained on 42,000 patient CT scans taken during a National Institutes of Health clinical trial, outperformed six radiologists in determining whether patients had cancer. It detected 5% more cancers and cut false positives — when cancer is suspected though a nodule is harmless — by 11% from reviewing a single scan. It performed on par with the radiologists when prior images of patients were also included in the evaluation.

The results underscore AI’s potential to improve lung cancer screening and help radiologists diagnose malignancies earlier and with greater accuracy — though the research did not show whether the Google system would help patients live longer. Lung cancer is by far the most common killer of Americans among cancers, resulting in about 160,000 deaths in 2018.

A large NIH study — the same one that supplied data for Google’s algorithm — established that screening of high-risk patients such as long-term smokers can reduce the risk of death by about 20%, but it also raised concerns about false positives leading to unnecessary testing that can harm patients. It reported that several deaths resulted from false positives that led patients to undergo invasive biopsies and other procedures. Ongoing concerns about the overall accuracy and benefit of screening have led to low rates of such testing. Meanwhile, about 50 percent of lung cancers are diagnosed after they have already spread, when the disease is harder to effectively treat.

Experts who were not involved in the study said Google’s system could make screening much more viable.

“These people have a technology that will improve the precision of screening tremendously,” said Dr. Otis Brawley, a professor of oncology and epidemiology at Johns Hopkins University and formerly executive vice president of the American Cancer Society. He said he is generally a skeptic of lung cancer screening, but emphasized that Google’s performance in reducing false positives was a significant step forward.

“It’s going to prevent more bad things from happening to people who are being screened,” said Brawley, adding that the system’s strong performance in that regard doesn’t necessarily mean it will save more lives from lung cancer.

Google’s system will require more rigorous testing — probably a randomized controlled trial — before it could be put into medical practice. The study was limited to patients who had already been treated, so it is impossible to tell whether the system, when used on new patients, will result in more effective care and better outcomes.

Google executives acknowledged that point in the study and an accompanying blog post, adding that they are working with clinical partners to further refine and validate the system. “To do a full assessment of this, you have to work with research organizations and run large-scale trials to understand how this technology will work at scale and on broad populations,” said Daniel Tse, a Google product manager who co-authored the study. He added that the company has had pre-submission discussions with the Food and Drug Administration to discuss approval criteria.

Google engineers who developed the AI system emphasized that it is not designed to replace radiologists, but to improve their ability to detect nodules and determine if they are dangerous. Existing computer-aided systems separate nodule detection and diagnosis into different tasks. Google’s system performs both functions, honing in on regions of interest within a scan and providing a risk score that indicates whether a given patient’s nodules are cancerous.

The system uses convolutional neural networks, a type of AI architecture, to learn the features of malignancy and point out the problematic areas by analyzing three-dimensional CT scans. That task is difficult and time-consuming for radiologists because they cannot examine three-dimensional scans the way a computer can; they must review hundreds of individual slices of the scan to hone in on problems. But the computer can review all the dimensions at once.

“We were able to train the [AI models] over the whole scan at a very high resolution,” said Shravya Shetty, a Google engineer and technical lead on the study. “Although radiologists can look at the slices, there are clear advantages to the model here.”

The authors reported that the system’s performance remained consistent when it was exposed to patients outside of the NIH dataset on which it was trained. The system reviewed scans of 1,700 patients from Northwestern Memorial HealthCare in Chicago, and produced similar results in classifying nodules and making diagnoses.

Experts said the Google software may be particularly useful for general radiologists, who often review patient lung scans in community hospitals in the U.S. Most lack the expertise of thoracic radiologists who specialize in lung disorders and are concentrated at large academic medical centers.

“To be able to make screening available to everyone, it can’t just be done by thoracic radiologists. It has to be done by all radiologists,” said Dr. Jorge Gomez, a medical oncologist at Mount Sinai Health System who serves as a national spokesman for the American Lung Association.

At Mount Sinai, he said, thoracic radiologists regularly participate in meetings to discuss patient care and point out the specific aspects of scans that indicate why nodules may or may not be malignant.

“That’s an incredible resource to have, and computers might be able to do that well,” he said. “This is a very important study that should prompt somebody with very deep pockets to do a randomized trial.”

Tse, the Google product manager, said the company is actively pursuing that work. “We are pushing things forward both internally within Google and with our partners,” he said, adding that he could not yet identify the time frame for the follow-up study.

Click here for the original article

Casey Ross

National Technology Correspondent

Casey covers the companies disrupting biopharma and health care.

Recent Posts

  • Researchers at the Technion–Israel Institute of Technology have developed a glue gun to put the human body back together when it has been seriously injured. The pins and stitches currently used to treat serious injuries come with drawbacks: They can be painful, they leave scars, they require high skill from the doctor, and they sometimes have to be removed after the tissues heal. Suture on the intestine, lungs or blood vessels often leak and therefore require a sealant. The medical glue that the researchers have developed is a “two in one,” said Prof. Boaz Mizrahi, head of the Biomaterials Laboratory of the Technion. It replaces both stitches and the sealant, and is good for both external and internal injuries, he said. All sorts of medical glues are already being used in dermatology, surgery, and other areas. Israeli startup Nanomedic Technologies Ltd., for example, has developed a medical device that it says can dress burns and other wounds with nano materials that mimic human tissue and peel off once the skin below is regenerated. Still, the glues currently in use to replace sutures and staples are limited by their mechanical properties and toxicity, the researchers said. Because they are very toxic, they can be utilized only on the surface of the skin. In addition, hardening of the glue may make the organ less flexible or the adhesion may not be sufficiently strong. With these limitations in mind, researchers have been on the hunt for a glue that is suitable for different tissues, nontoxic, and flexible after hardening. Such a glue would also need to decompose in the body after the tissue is fused together. Mizrahi worked together with doctoral student Alona Shagan and came up with what they say is a “very strong, nontoxic tissue adhesive that remains flexible even after solidification.” Their study...
  • Pinnacle Ventures has launched a pharmacogenomics programme to enable genetic testing to drive personalised prescribing decisions. The innovation arm of Pinnacle Midlands Health Network, a not-for-profit primary health care management company in New Zealand, is also working on embedding biomarker information into electronic health records and linking it to a clinical-decision support prescribing tool that can help prescribers by providing direct access to international pathways and guidelines. Pharmacogenetics involves prescriptions being tailored to a person’s genetic make-up, as people metabolise drugs in different ways, which can have a significant impact on a drug’s effectiveness. Ventures plans to do about 5,000 pharmacogenetic tests over the next 12 months, says chief executive John Macaskill-Smith. Some will be self-funded because individuals are struggling with their medications and others will be fully funded by Ventures, targeting specific groups within the Midlands population. Macaskill-Smith says it is a simple test that covers 65–70 per cent of medications frequently prescribed in New Zealand. “The New Zealand health system is under strain but using testing like this you could reduce the trial and error of prescribing and prevent adverse reactions to medications,” he said. Ethnicity plays a big part in how a person metabolises drugs, but the clinical trials that prescribing information are based on very rarely involve Māori or Pasifika test subjects. Macaskill-Smith said Ventures is partnering with key kiwi groups, Auckland University and Otago University medical schools and Callaghan Innovation to support research and develop a better understanding of how unique New Zealand populations respond to different medications. People who have a pharmacogenetic test can choose to consent to contributing their non-identifiable demographic information to researchers. Embedding the biomarker information into EHRs ensures a patient’s results are used for both current and future prescribing decisions, he said. Macaskill-Smith says a lot of direct-to-consumer online genetic-testing tools involve people...
  • Surgery students spend so much time on screens that they have lost the ability to perform simple tasks such as stitching and sewing up patients, a professor has warned. Roger Kneebone, a professor of surgical education at Imperial College, London, says the focus on academic knowledge has come at the expense of craftsmanship. “It is important and an increasingly urgent issue,” Kneebone told the BBC. “It is a concern of mine and my scientific colleagues that whereas in the past you could make the assumption that students would leave school able to do certain practical things – cutting things out, making things – that is no longer the case.” The professor, who teaches surgery to medical students, believes that this is down to an increase in technology which takes away the experience of handling materials and developing skills. ”An obvious example is of a surgeon needing some dexterity and skill in sewing or stitching,” he explained. ”A lot of things are reduced to swiping on a two-dimensional flat screen” Kneebone adds that a growing number of students are becoming “less competent and less confident” in using their hands, resulting in young professionals who “have very high exam grades but lack tactile general knowledge”. The professor will be speaking on Tuesday at the V&A Museum of Childhood in east London, at the launch of a report, published by the Edge Foundation, calling for more creativity in the curriculum. The report warns that entries to creative subjects have fallen by 20 per cent since 2010, including a 57 per cent fall in design and technology GCSE. Tristram Hunt, director of the Victoria and Albert Museum, who will be speaking alongside Professor Kneebone added: “Creativity is not just for artists. “Subjects like design and technology, music, art and drama are vitally important for...