World’s First “Augmented Reality” Contact Lens Aims to Revolutionize Much More Than Medicine
Imagine a world without screens. Instead of endlessly staring at your computer or craning your neck down to scroll through social media feeds and emails, information simply appears in front of your eyes when you need it and disappears when you don't.
"The vision is super clear...I was reading the poem with my eyes closed."
No more rude interruptions during dinner, no more bumping into people on the street while trying to follow GPS directions — just the information you want, when you need it, projected directly onto your visual field.
While this screenless future sounds like science fiction, it may soon be a reality thanks to the new Silicon Valley startup Mojo Vision, creator of the world's first smart contact lens. With a 14,000 pixel-per-inch display with eye-tracking, image stabilization, and a custom wireless radio, the Mojo smart lens bills itself the "smallest and densest dynamic display ever made." Unlike current augmented reality wearables such as Google Glass or ThirdEye, which project images onto a glass screen, the Mojo smart lens can project images directly onto the retina.
A current prototype displayed at the Consumer Electronics Show earlier this year in Las Vegas includes a tiny screen positioned right above the most sensitive area of the pupil. "[The Mojo lens] is a contact lens that essentially has wireless power and data transmission for a small micro LED projector that is placed over the center of the eye," explains David Hobbs, Director of Product Management at Mojo Vision. "[It] displays critical heads-up information when you need it and fades into the background when you're ready to continue on with your day."
Eventually, Mojo Visions' technology could replace our beloved smart devices but the first generation of the Mojo smart lens will be used to help the 2.2 billion people globally who suffer from vision impairment.
"If you think of the eye as a camera [for the visually impaired], the sensors are not working properly," explains Dr. Ashley Tuan, Vice President of Medical Devices at Mojo Vision and fellow of the American Academy of Optometry. "For this population, our lens can process the image so the contrast can be enhanced, we can make the image larger, magnify it so that low-vision people can see it or we can make it smaller so they can check their environment." In January of this year, the FDA granted Breakthrough Device Designation to Mojo, allowing them to have early and frequent discussions with the FDA about technical, safety and efficacy topics before clinical trials can be done and certification granted.
For now, Dr. Tuan is one of the few people who has actually worn the Mojo lens. "I put the contact lens on my eye. It was very comfortable like any contact lenses I've worn before," she describes. "The vision is super clear and then when I put on the accessories, suddenly I see Yoda in front of me and I see my vital signs. And then I have my colleague that prepared a beautiful poem that I loved when I was young [and] I was reading the poem with my eyes closed."
At the moment, there are several electronic glasses on the market like Acesight and Nueyes Pro that provide similar solutions for those suffering from visual impairment, but they are large, cumbersome, and highly visible. Mojo lens would be a discreet, more comfortable alternative that offers users more freedom of movement and independence.
"In the case of augmented-reality contact lenses, there could be an opportunity to improve the lives of people with low vision," says Dr. Thomas Steinemann, spokesperson for the American Academy of Ophthalmology and professor of ophthalmology at MetroHealth Medical Center in Cleveland. "There are existing tools for people currently living with low vision—such as digital apps, magnifiers, etc.— but something wearable could provide more flexibility and significantly more aid in day-to-day tasks."
As one of the first examples of "invisible computing," the potential applications of Mojo lens in the medical field are endless.
According to Dr. Tuan, the visually impaired often suffer from depression due to their lack of mobility and 70 percent of them are underemployed. "We hope that they can use this device to gain their mobility so they can get that social aspect back in their lives and then, eventually, employment," she explains. "That is our first and most important goal."
But helping those with low visual capabilities is only Mojo lens' first possible medical application; augmented reality is already being used in medicine and is poised to revolutionize the field in the coming decades. For example, Accuvein, a device that uses lasers to provide real-time images of veins, is widely used by nurses and doctors to help with the insertion of needles for IVs and blood tests.
According to the National Center for Biotechnology Information, augmentation of reality has been used in surgery for many years with surgeons using devices such as Google Glass to overlay critical information about their patients into their visual field. Using software like the Holographic Navigation Platform by Scopsis, surgeons can see a mixed-reality overlay that can "show you complicated tumor boundaries, assist with implant placements and guide you along anatomical pathways," its developers say.
However, according to Dr. Tuan, augmented reality headsets have drawbacks in the surgical setting. "The advantage of [Mojo lens] is you don't need to worry about sweating or that the headset or glasses will slide down to your nose," she explains "Also, our lens is designed so that it will understand your intent, so when you don't want the image overlay it will disappear, it will not block your visual field, and when you need it, it will come back at the right time."
As one of the first examples of "invisible computing," the potential applications of Mojo lens in the medical field are endless. Possibilities include live translation of sign language for deaf people; helping those with autism to read emotions; and improving doctors' bedside manner by allowing them to fully engage with patients without relying on a computer.
"[By] monitoring those blood vessels we can [track] chronic disease progression: high blood pressure, diabetes, and Alzheimer's."
Furthermore, the lens could be used to monitor health issues. "We have image sensors in the lens right now that point to the world but we can have a camera pointing inside of your eye to your retina," says Dr. Tuan, "[By] monitoring those blood vessels we can [track] chronic disease progression: high blood pressure, diabetes, and Alzheimer's."
For the moment, the future medical applications of the Mojo lens are still theoretical, but the team is confident they can eventually become a reality after going through the proper regulatory review. The company is still in the process of design, prototype and testing of the lens, so they don't know exactly when it will be available for use, but they anticipate shipping the first available products in the next couple of years. Once it does go to market, it will be available by prescription only for those with visual impairments, but the team's goal is to bring it to broader consumer markets pending regulatory clearance.
"We see that right now there's a unique opportunity here for Mojo lens and invisible computing to help to shape what the next decade of technology development looks like," explains David Hobbs. "We can use [the Mojo lens] to better serve us as opposed to us serving technology better."
When I greeted Rodney Gorham, age 63, in an online chat session, he replied within seconds: “My pleasure.”
“Are you moving parts of your body as you type?” I asked.
This time, his response came about five minutes later: “I position the cursor with the eye tracking and select the same with moving my ankles.” Gorham, a former sales representative from Melbourne, Australia, living with amyotrophic lateral sclerosis, or ALS, a rare form of Lou Gehrig’s disease that impairs the brain’s nerve cells and the spinal cord, limiting the ability to move. ALS essentially “locks” a person inside their own body. Gorham is conversing with me by typing with his mind only–no fingers in between his brain and his computer.
The brain-computer interface enabling this feat is called the Stentrode. It's the brainchild of Synchron, a company backed by Amazon’s Jeff Bezos and Microsoft cofounder Bill Gates. After Gorham’s neurologist recommended that he try it, he became one of the first volunteers to have an 8mm stent, laced with small electrodes, implanted into his jugular vein and guided by a surgeon into a blood vessel near the part of his brain that controls movement.
After arriving at their destination, these tiny sensors can detect neural activity. They relay these messages through a small receiver implanted under the skin to a computer, which then translates the information into words. This minimally invasive surgery takes a day and is painless, according to Gorham. Recovery time is typically short, about two days.
When a paralyzed patient thinks about trying to move their arms or legs, the motor cortex will fire patterns that are specific to the patient’s thoughts.
When a paralyzed patient such as Gorham thinks about trying to move their arms or legs, the motor cortex will fire patterns that are specific to the patient’s thoughts. This pattern is detected by the Stentrode and relayed to a computer that learns to associate this pattern with the patient’s physical movements. The computer recognizes thoughts about kicking, making a fist and other movements as signals for clicking a mouse or pushing certain letters on a keyboard. An additional eye-tracking device controls the movement of the computer cursor.
The process works on a letter by letter basis. That’s why longer and more nuanced responses often involve some trial and error. “I have been using this for about two years, and I enjoy the sessions,” Gorham typed during our chat session. Zafar Faraz, field clinical engineer at Synchron, sat next to Gorham, providing help when required. Gorham had suffered without internet access, but now he looks forward to surfing the web and playing video games.
Gorham, age 63, has been enjoying Stentrode sessions for about two years.
The BCI revolution
In the summer of 2021, Synchron became the first company to receive the FDA’s Investigational Device Exemption, which allows research trials on the Stentrode in human patients. This past summer, the company, together with scientists from Icahn School of Medicine at Mount Sinai and the Neurology and Neurosurgery Department at Utrecht University, published a paper offering a framework for how to develop BCIs for patients with severe paralysis – those who can't use their upper limbs to type or use digital devices.
Three months ago, Synchron announced the enrollment of six patients in a study called COMMAND based in the U.S. The company will seek approval next year from the FDA to make the Stentrode available for sale commercially. Meanwhile, other companies are making progress in the field of BCIs. In August, Neuralink announced a $280 million financing round, the biggest fundraiser yet in the field. Last December, Synchron announced a $75 million financing round. “One thing I can promise you, in five years from now, we’re not going to be where we are today. We're going to be in a very different place,” says Elad I. Levy, professor of neurosurgery and radiology at State University of New York in Buffalo.
The risk of hacking exists, always. Cybercriminals, for example, might steal sensitive personal data for financial reasons, blackmailing, or to spread malware to other connected devices while extremist groups could potentially hack BCIs to manipulate individuals into supporting their causes or carrying out actions on their behalf.
“The prospect of bestowing individuals with paralysis a renewed avenue for communication and motor functionality is a step forward in neurotech,” says Hayley Nelson, a neuroscientist and founder of The Academy of Cognitive and Behavioral Neuroscience. “It is an exciting breakthrough in a world of devastating, scary diseases,” says Neil McArthur, a professor of philosophy and director of the Centre for Professional and Applied Ethics at the University of Manitoba. “To connect with the world when you are trapped inside your body is incredible.”
While the benefits for the paraplegic community are promising, the Stentrode’s long-term effectiveness and overall impact needs more research on safety. “Potential risks like inflammation, damage to neural tissue, or unexpected shifts in synaptic transmission due to the implant warrant thorough exploration,” Nelson says.
There are also concens about data privacy concerns and the policies of companies to safeguard information processed through BCIs. “Often, Big Tech is ahead of the regulators because the latter didn’t envisage such a turn of events...and companies take advantage of the lack of legal framework to push forward,” McArthur says. Hacking is another risk. Cybercriminals could steal sensitive personal data for financial reasons, blackmailing, or to spread malware to other connected devices. Extremist groups could potentially hack BCIs to manipulate individuals into supporting their causes or carrying out actions on their behalf.
“We have to protect patient identity, patient safety and patient integrity,” Levy says. “In the same way that we protect our phones or computers from hackers, we have to stay ahead with anti-hacking software.” Even so, Levy thinks the anticipated benefits for the quadriplegic community outweigh the potential risks. “We are on the precipice of an amazing technology. In the future, we would be able to connect patients to peripheral devices that enhance their quality of life.”
In the near future, the Stentrode could enable patients to use the Stentrode to activate their wheelchairs, iPods or voice modulators. Synchron's focus is on using its BCI to help patients with significant mobility restrictions—not to enhance the lives of healthy people without any illnesses. Levy says we are not prepared for the implications of endowing people with superpowers.
I wondered what Gorham thought about that. “Pardon my question, but do you feel like you have sort of transcended human nature, being the first in a big line of cybernetic people doing marvelous things with their mind only?” was my last question to Gorham.
A slight smile formed on his lips. In less than a minute, he typed: “I do a little.”
A new competition by the XPRIZE Foundation is offering $101 million to researchers who discover therapies that give a boost to people aged 65-80 so their bodies perform more like when they were middle-aged.
For today’s podcast episode, I talked with Dr. Peter Diamandis, XPRIZE’s founder and executive chairman. Under Peter’s leadership, XPRIZE has launched 27 previous competitions with over $300 million in prize purses. The latest contest aims to enhance healthspan, or the period of life when older people can play with their grandkids without any restriction, disability or disease. Such breakthroughs could help prevent chronic diseases that are closely linked to aging. These illnesses are costly to manage and threaten to overwhelm the healthcare system, as the number of Americans over age 65 is rising fast.
In this competition, called XPRIZE Healthspan, multiple awards are available, depending on what’s achieved, with support from the nonprofit Hevolution Foundation and Chip Wilson, the founder of Lululemon and nonprofit SOLVE FSHD. The biggest prize, $81 million, is for improvements in cognition, muscle and immunity by 20 years. An improvement of 15 years will net $71 million, and 10 years will net $61 million.
In our conversation for this episode, Peter talks about his plans for XPRIZE Healthspan and why exponential technologies make the current era - even with all of its challenges - the most exciting time in human history. We discuss the best mental outlook that supports a person in becoming truly innovative, as well as the downsides of too much risk aversion. We talk about how to overcome the negativity bias in ourselves and in mainstream media, how Peter has shifted his own mindset to become more positive over the years, how to inspire a culture of innovation, Peter’s personal recommendations for lifestyle strategies to live longer and healthier, the innovations we can expect in various fields by 2030, the future of education and the importance of democratizing tech and innovation.
In addition to Peter’s pioneering leadership of XPRIZE, he is also the Executive Founder of Singularity University. In 2014, he was named by Fortune as one of the “World’s 50 Greatest Leaders.” As an entrepreneur, he’s started over 25 companies in the areas of health-tech, space, venture capital and education. He’s Co-founder and Vice-Chairman of two public companies, Celularity and Vaxxinity, plus being Co-founder & Chairman of Fountain Life, a fully-integrated platform delivering predictive, preventative, personalized and data-driven health. He also serves as Co-founder of BOLD Capital Partners, a venture fund with a half-billion dollars under management being invested in exponential technologies and longevity companies. Peter is a New York Times Bestselling author of four books, noted during our conversation and in the show notes of this episode. He has degrees in molecular genetics and aerospace engineering from MIT and holds an M.D. from Harvard Medical School.
- Peter Diamandis bio
- New XPRIZE Healthspan
- Peter Diamandis books
- Longevity Insider newsletter – AI identifies the news
- Peter Diamandis Longevity Handbook
- Hevolution funding for longevity
XPRIZE Founder Peter Diamandis speaks with Mehmoud Khan, CEO of Hevolution Foundation, at the launch of XPRIZE Healthspan.