Whenever Eric Karl Oermann has to tell a patient about a terrible prognosis, their first question is always: "how long do I have?" Oermann would like to offer a precise answer, to provide some certainty and help guide treatment. But although he's one of the country's foremost experts in medical artificial intelligence, Oermann is still dependent on a computer algorithm that's often wrong.
Doctors are notoriously terrible at guessing how long their patients will live.
Artificial intelligence, now often called deep learning or neural networks, has radically transformed language and image processing. It's allowed computers to play chess better than the world's grand masters and outwit the best Jeopardy players. But it still can't precisely tell a doctor how long a patient has left – or how to help that person live longer.
Someday, researchers predict, computers will be able to watch a video of a patient to determine their health status. Doctors will no longer have to spend hours inputting data into medical records. And computers will do a better job than specialists at identifying tiny tumors, impending crises, and, yes, figuring out how long the patient has to live. Oermann, a neurosurgeon at Mount Sinai, says all that technology will allow doctors to spend more time doing what they do best: talking with their patients. "I want to see more deep learning and computers in a clinical setting," he says, "so there can be more human interaction." But those days are still at least three to five years off, Oermann and other researchers say.
Doctors are notoriously terrible at guessing how long their patients will live, says Nigam Shah, an associate professor at Stanford University and assistant director of the school's Center for Biomedical Informatics Research. Doctors don't want to believe that their patient – whom they've come to like – will die. "Doctors over-estimate survival many-fold," Shah says. "How do you go into work, in say, oncology, and not be delusionally optimistic? You have to be."
But patients near the end of life will get better treatment – and even live longer – if they are overseen by hospice or palliative care, research shows. So, instead of relying on human bias to select those whose lives are nearing their end, Shah and his colleagues showed that they could use a deep learning algorithm based on medical records to flag incoming patients with a life expectancy of three months to a year. They use that data to indicate who might need palliative care. Then, the palliative care team can reach out to treating physicians proactively, instead of relying on their referrals or taking the time to read extensive medical charts.
But, although the system works well, Shah isn't yet sure if such indicators actually get the appropriate patients into palliative care. He's recently partnered with a palliative care doctor to run a gold-standard clinical trial to test whether patients who are flagged by this algorithm are indeed a better match for palliative care.
"What is effective from a health system perspective might not be effective from a treating physician's perspective and might not be effective from the patient's perspective," Shah notes. "I don't have a good way to guess everybody's reaction without actually studying it." Whether palliative care is appropriate, for instance, depends on more than just the patient's health status. "If the patient's not ready, the family's not ready and the doctor's not ready, then you're just banging your head against the wall," Shah says. "Given limited capacity, it's a waste of resources" to put that person in palliative care.
The algorithm isn't perfect, but "on balance, it leads to better decisions more often."
Alexander Smith and Sei Lee, both palliative care doctors, work together at the University of California, San Francisco, to develop predictions for patients who come to the hospital with a complicated prognosis or a history of decline. Their algorithm, they say, helps decide if this patient's problems – which might include diabetes, heart disease, a slow-growing cancer, and memory issues – make them eligible for hospice. The algorithm isn't perfect, they both agree, but "on balance, it leads to better decisions more often," Smith says.
Bethany Percha, an assistant professor at Mount Sinai, says that an algorithm may tell doctors that their patient is trending downward, but it doesn't do anything to change that trajectory. "Even if you can predict something, what can you do about it?" Algorithms may be able to offer treatment suggestions – but not what specific actions will alter a patient's future, says Percha, also the chief technology officer of Precise Health Enterprise, a product development group within Mount Sinai. And the algorithms remain challenging to develop. Electronic medical records may be great at her hospital, but if the patient dies at a different one, her system won't know. If she wants to be certain a patient has died, she has to merge social security records of death with her system's medical records – a time-consuming and cumbersome process.
An algorithm that learns from biased data will be biased, Shah says. Patients who are poor or African American historically have had worse health outcomes. If researchers train an algorithm on data that includes those biases, they get baked into the algorithms, which can then lead to a self-fulfilling prophesy. Smith and Lee say they've taken race out of their algorithms to avoid this bias.
Age is even trickier. There's no question that someone's risk of illness and death goes up with age. But an 85-year-old who breaks a hip running a marathon should probably be treated very differently than an 85-year-old who breaks a hip trying to get out of a chair in a dementia care unit. That's why the doctor can never be taken out of the equation, Shah says. Human judgment will always be required in medical care and an algorithm should never be followed blindly, he says.
Experts say that the flaws in artificial intelligence algorithms shouldn't prevent people from using them – carefully.
Researchers are also concerned that their algorithms will be used to ration care, or that insurance companies will use their data to justify a rate increase. If an algorithm predicts a patient is going to end up back in the hospital soon, "who's benefitting from knowing a patient is going to be readmitted? Probably the insurance company," Percha says.
Still, Percha and others say, the flaws in artificial intelligence algorithms shouldn't prevent people from using them – carefully. "These are new and exciting tools that have a lot of potential uses. We need to be conscious about how to use them going forward, but it doesn't mean we shouldn't go down this road," she says. "I think the potential benefits outweigh the risks, especially because we've barely scratched the surface of what big data can do right now."
Jessica Ware is obsessed with bugs.
My guest today is a leading researcher on insects, the president of the Entomological Society of America and a curator at the American Museum of Natural History. Learn more about her here.
You may not think that insects and human health go hand-in-hand, but as Jessica makes clear, they’re closely related. A lot of people care about their health, and the health of other creatures on the planet, and the health of the planet itself, but researchers like Jessica are studying another thing we should be focusing on even more: how these seemingly separate areas are deeply entwined. (This is the theme of an upcoming event hosted by Leaps.org and the Aspen Institute.)
Listen to the Episode
Entomologist Jessica Ware
D. Finnin / AMNH
Maybe it feels like a core human instinct to demonize bugs as gross. We seem to try to eradicate them in every way possible, whether that’s with poison, or getting out our blood thirst by stomping them whenever they creep and crawl into sight.
But where did our fear of bugs really come from? Jessica makes a compelling case that a lot of it is cultural, rather than in-born, and we should be following the lead of other cultures that have learned to live with and appreciate bugs.
The truth is that a healthy planet depends on insects. You may feel stung by that news if you hate bugs. Reality bites.
Jessica and I talk about whether learning to live with insects should include eating them and gene editing them so they don’t transmit viruses. She also tells me about her important research into using genomic tools to track bugs in the wild to figure out why and how we’ve lost 50 percent of the insect population since 1970 according to some estimates – bad news because the ecosystems that make up the planet heavily depend on insects. Jessica is leading the way to better understand what’s causing these declines in order to start reversing these trends to save the insects and to save ourselves.
The first thing Jeroen Perk saw after he partially regained his sight nearly a decade ago was the outline of his guide dog Pedro.
“There was a white floor, and the dog was black,” recalls Perk, a 43-year-old investigator for the Dutch customs service. “I was crying. It was a very nice moment.”
Perk was diagnosed with retinitis pigmentosa as a child and had been blind since early adulthood. He has been able to use the implant placed into his retina in 2013 to help identify street crossings, and even ski and pursue archery. A video posted by the company that designed and manufactured the device indicates he’s a good shot.
Less black-and-white has been the journey Perk and others have been on after they were implanted with the Argus II, a second-generation device created by a Los Angeles-based company called Second Sight Medical Devices.
The Argus II uses the implant and a video camera embedded in a special pair of glasses to provide limited vision to those with retinitis pigmentosa, a genetic disease that causes cells in the retina to deteriorate. The camera feeds information to the implant, which sends electrical impulses into the retina to recapitulate what the camera sees. The impulses appear in the Argus II as a 60-pixel grid of blacks, grays and whites in the user’s eye that can render rough outlines of objects and their motion.
Smartphone and computer manufacturers typically stop issuing software upgrades to their devices after two or three years, eventually rendering them bricks. But is the smartphone approach acceptable for a device that helps restore the most crucial sense a human being possesses?
Ross Doerr, a retired disability rights attorney in Maine who received an Argus II in 2019, describes the field of vision as the equivalent of an index card held at arm’s length. Perk often brings objects close to his face to decipher them. Moreover, users must swivel their heads to take in visual data; moving their eyeballs does not work.
Despite its limitations, the Argus II beats the alternative. Perk no longer relies on his guide dog. Doerr was uplifted when he was able to see the outlines of Christmas trees at a holiday show.
“The fairy godmother department sort of reaches out and taps you on the shoulder once in a while,” Doerr says of his implant, which came about purely by chance. A surgeon treating his cataracts was partnered with the son of another surgeon who was implanting the devices, and he was referred.
Doerr had no reason to believe the shower of fairy dust wouldn’t continue. Second Sight held out promises that the Argus II recipients’ vision would gradually improve through upgrades to much higher pixel densities. The ability to recognize individual faces was even touted as a possibility. In the winter of 2020, Doerr was preparing to travel across the U.S. to Second Sight’s headquarters to receive an upgrade. But then COVID-19 descended, and the trip was canceled.
The pandemic also hit Second Sight’s bottom line. Doerr found out about its tribulations only from one of the company’s vision therapists, who told him the entire department was being laid off. Second Sight cut nearly 80% of its workforce in March 2020 and announced it would wind down operations.
Ross Doerr has mostly stopped using his Argus II, the result of combination of fear of losing its assistance from wear and tear and disdain for the company that brought it to market.
Second Sight’s implosion left some 350 Argus recipients in the metaphorical dark about what to do if their implants failed. Skeleton staff seem to have rarely responded to queries from their customers, at least based on the experiences of Perk and Doerr. And some recipients have unfortunately returned to the actual dark as well, as reports have surfaced of Argus II failures due to aging or worn-down parts.
Product support for complex products is remarkably uneven. Although the iconic Ford Mustang ceased production in the late 1960s, its parts market is so robust that it’s theoretically possible to assemble a new vehicle from recently crafted components. Conversely, smartphone and computer manufacturers typically stop issuing software upgrades to their devices after two or three years, eventually rendering them bricks. Consumers have accepted both extremes.
But is the smartphone approach acceptable for a device that helps restore the most crucial sense a human being possesses?
Margaret McLean, a senior fellow at the Markkula Center for Applied Ethics at Santa Clara University in California, notes companies like Second Sight have a greater obligation for product support than other consumer product ventures.
“In this particular case, you have a great deal of risk that is involved in using this device, the implant, and the after care of this device,” she says. “You cannot, like with your car, decide that ‘I don’t like my Mustang anymore,’ and go out and buy a Corvette.”
And, whether the Argus II implant works or not, its physical presence can impact critical medical decisions. Doerr’s doctor wanted him to undergo an MRI to assist in diagnosing attacks of vertigo. But the physician was concerned his implant might interfere. With the latest available manufacturer advisories on his implant nearly a decade old, the procedure was held up. Doerr spent months importuning Second Sight through phone calls, emails and Facebook postings to learn if his implant was contraindicated with MRIs, which he never received. Although the cause of his vertigo was found without an MRI, Doerr was hardly assured.
“Put that into context for a minute. I get into a serious car accident. I end up in the emergency room, and I have a tag saying I have an implanted medical device,” he says. “You can’t do an MRI until you get the proper information from the company. Who’s going to answer the phone?”
Second Sight’s management did answer the call to revamp its business. It netted nearly $78 million through a private stock placement and an initial public offering last year. At the end of 2021, Second Sight had nearly $70 million in cash on hand, according to a recent filing with the Securities and Exchange Commission.
And while the Argus II is still touted at length on Second Sight’s home page, it appears little of its corporate coffers are earmarked toward its support. These days, the company is focused on obtaining federal approvals for Orion, a new implant that would go directly into the recipient’s brain and could be used to remedy blindness from a variety of causes. It obtained a $6.4 million grant from the National Institutes of Health in May 2021 to help develop Orion.
Presented with a list of written questions by email, Second Sight’s spokesperson, Dave Gentry of the investor relations firm Red Chip Companies, copied a subordinate with an abrupt message to “please handle.” That was the only response from a company representative. A call to Second Sight acting chief executive officer Scott Dunbar went unreturned.
Whether or not the Orion succeeds remains to be seen. The company’s SEC filings suggest a viable and FDA-approved device is years away, and that operational losses are expected for the “foreseeable future.” Second Sight reported zero revenue in 2020 or 2021.
Moreover, the experiences of the Argus II recipients could color the reception of future Second Sight products. Doerr notes that his insurer paid nearly $500,000 to implant his device and for training on how to use it.
“What’s the insurance industry going to say the next time this crops up?” Doerr asks, noting that the company’s reputation is “completely shot” with the recipients of its implants.
Perk, who made speeches to praise the Argus II and is still featured in a video on the Second Sight website, says he also no longer supports the company.
Jeroen Perk, an investigator for the Dutch customs service, cried for joy after partially regaining his sight, but he no longer trusts Second Sight, the company that provided his implant.
Nevertheless, Perk remains highly reliant on the technology. When he dropped an external component of his device in late 2020 and it broke, Perk briefly debated whether to remain blind or find a way to get his Argus II working again. Three months later, he was able to revive it by crowdsourcing parts, primarily from surgeons with spare components or other Argus II recipients who no longer use their devices. Perk now has several spare parts in reserve in case of future breakdowns.
Despite the frantic efforts to retain what little sight he has, Perk has no regrets about having the device implanted. And while he no longer trusts Second Sight, he is looking forward to possibly obtaining more advanced implants from companies in the Netherlands and Australia working on their own products.
Doerr suggests that biotech firms whose implants are distributed globally be bound to some sort of international treaty requiring them to service their products in perpetuity. Such treaties are still applied to the salvage rights for ships that sunk centuries ago, he notes.
“I think that in a global tech economy, that would be a good thing,” says McLean, the fellow at Santa Clara, “but I am not optimistic about it in the near term. Business incentives push toward return on share to stockholders, not to patients and other stakeholders. We likely need to rely on some combination of corporately responsibility…and [international] government regulation. It’s tough—the Paris Climate Accord implementation at a slow walk comes to mind.”
Unlike Perk, Doerr has mostly stopped using his Argus II, the result of combination of fear of losing its assistance from wear and tear and disdain for the company that brought it to market. At 70, Doerr says he does not have the time or energy to hold the company more accountable. And with Second Sight having gone through a considerable corporate reorganization, Doerr believes a lawsuit to compel it to better serve its Argus recipients would be nothing but an extremely costly longshot.
“It’s corporate America at its best,” he observes.