IBM’s Watson Hasn't Beaten Cancer, But A.I. May Win

(Bloomberg Opinion) -- It’s a lot harder for a machine to match the best oncologists in cancer treatment than it was to beat human competitors on “Jeopardy.” It’s still a worthy goal. The problem is that so far the artificial-intelligence platform called Watson hasn’t matched the hype generated by IBM’s over-the-top advertising campaign.

Earlier this month, the medical website STAT reported that internal documents from International Business Machines Corp. revealed the computing system had recommended “unsafe and incorrect” cancer treatments. The system is being used in conjunction with human medical judgment, and there are no reports of patients being harmed. But in the documents obtained by STAT, doctors who had tried to use Watson to help them design treatment complained that the system wasn’t ready to practice medicine.

The machine learning revolution is coming to hospitals with or without Watson, especially because cancer is a data problem. Cancer cells are just a patient’s cells with coding errors. Each case of cancer is unique, and will respond best to different treatments depending on the patient’s own DNA and the errors in the cancer cells.

In medicine at least, Watson may not yet qualify as artificial intelligence, said Andrew Beam, a professor of biomedical informatics at Harvard. His field combines computer science, artificial intelligence and medicine. From what he understands of the Watson oncology program, the system is designed to encode and mimic the professional judgment of doctors from Memorial Sloan Kettering Cancer Center.

That could be extremely valuable. If it worked, patients even in the poorest or most remote areas could receive world-class care. The company promoted the system as a box that takes in patient information and spits out medical wisdom. But there are parts of medical intuition that can’t be computerized, Beam said. He said experts in computer science and artificial intelligence have long been skeptical of the Watson oncology program because the bold claims of its huge advertising campaign have not been accompanied by much evidence in peer-reviewed literature. (An IBM spokeswoman, Christine Douglass, said in response to recent news coverage of Watson’s cancer venture: “The opportunity for A.I. in health care is still nascent, but we are proud to be pioneers in this arena.”)

Some aspects of medical judgment are easier than others to program into a machine. Computers are great at recognizing images or even predicting the weather. These are relatively easy to assess: A computer can be “trained” with thousands of images and then tested to see if it can tell a cat from a dog from a tree. Or it can be trained and then tested with weather data from months past, to see how well it forecasts rain and snowstorms that we know happened.

Cancer treatment is more complicated, because humans are still figuring it out. Patients get multiple treatments that may result in remission periods, but whether there’s a remission and how long it lasts depends on a host of variables. “It’s something we struggle with a lot,” Beam said. “You want a ground-truth gold-standard correct answer for a given patient, and 99 percent of the time that doesn’t exist.”

Only a small fraction of cancer patients have their information recorded in a systematic way, said Isaac Kohane, a doctor and chairman of the biomedical informatics program at Harvard Medical School. That’s now starting to change. “Those of us in the AI community are extremely optimistic about how these techniques are going to revolutionize medicine,” he said. But with Watson, “it’s just unfortunate that the marketing arm got ahead of the capabilities.” 

Another expert, computational biologist Shirley Pepke, used machine learning to fight her own cancer, demonstrating what’s already possible but not available to ordinary patients. At 46, she was diagnosed with ovarian cancer, and after a round of chemotherapy failed to help, her doctors told her she would die. But she knew that the difference between her cancer and others that did respond to available drugs was encoded in the DNA of her tumor.

She was able to get her tumor DNA sequenced, and with the help of data scientist Greg Ver Steeg at the University of Southern California, compared it to a database with similar information on 400 other ovarian cancers. They used machine learning to find patterns — similarities between her tumor and others that responded better to chemotherapy.

The analysis turned up clues that she used to conclude that drugs called checkpoint inhibitors might offer one narrow thread of hope. At the time, they weren’t approved for ovarian cancer. She had to buy the drugs with her own money. They destroyed her thyroid and damaged her kidneys, she said, but she got a long, unexpected remission and was able to talk about her case three years later at a meeting on big data at Harvard Medical School.

Pattern recognition is where machines today excel, said Beam, and the FDA has recently approved several computerized systems for analyzing images. In one, he said, artificial intelligence demonstrated superhuman abilities to detect diabetic retinopathy — a complication of diabetes that can lead to blindness if not treated promptly. Computers can read all kinds of things from a picture of your retina, he said, including age, gender, blood pressure and whether you smoke.

Kohane said many companies are in the race to incorporate AI into medicine – pharma, data companies such as Google, and groups such as Flatiron Health, which is working to collect the kind of data needed to train AI systems. “It’s anybody’s guess who is going to be the market leader in this space,” he said. Artificial intelligence and big data are coming to doctors’ offices and hospitals. But it won’t necessarily look like the ads on TV.