Giving robots self-awareness as they move through space - and maybe even providing them with gene-like methods for storing rules of behavior - could be important steps toward creating more intelligent machines.
Now roboticists are going a step further, training their creations to do even better: understand their own image in space and interact with humans like humans do. Today, there are already robot-teachers like KeeKo, robot-pets like Moffin, robot-babysitters like iPal, and robotic companions for the elderly like Pepper.
But even these reasonably intelligent creations still have huge limitations, some scientists think. “There are niche applications for the current generations of robots,” says professor Anthony Zador at Cold Spring Harbor Laboratory—but they are not “generalists” who can do varied tasks all on their own, as they mostly lack the abilities to improvise, make decisions based on a multitude of facts or emotions, and adjust to rapidly changing circumstances. “We don’t have general purpose robots that can interact with the world. We’re ages away from that.”
Robotic spatial self-awareness – the achievement by the team at Columbia – is an important step toward creating more intelligent machines. Hod Lipson, professor of mechanical engineering who runs the Columbia lab, says that future robots will need this ability to assist humans better. Knowing how you look and where in space your parts are, decreases the need for human oversight. It also helps the robot to detect and compensate for damage and keep up with its own wear-and-tear. And it allows robots to realize when something is wrong with them or their parts. “We want our robots to learn and continue to grow their minds and bodies on their own,” Chen says. That’s what Zador wants too—and on a much grander level. “I want a robot who can drive my car, take my dog for a walk and have a conversation with me.”
Columbia scientists have trained a robot to become aware of its own "body," so it can map the right path to touch a ball without running into an obstacle, in this case a square.
Jane Nisselson and Yinuo Qin/ Columbia Engineering
Today’s technological advances are making some of these leaps of progress possible. One of them is the so-called Deep Learning—a method that trains artificial intelligence systems to learn and use information similar to how humans do it. Described as a machine learning method based on neural network architectures with multiple layers of processing units, Deep Learning has been used to successfully teach machines to recognize images, understand speech and even write text.
Trained by Google, one of these language machine learning geniuses, BERT, can finish sentences. Another one called GPT3, designed by San Francisco-based company OpenAI, can write little stories. Yet, both of them still make funny mistakes in their linguistic exercises that even a child wouldn’t. According to a paper published by Stanford’s Center for Research on Foundational Models, BERT seems to not understand the word “not.” When asked to fill in the word after “A robin is a __” it correctly answers “bird.” But try inserting the word “not” into that sentence (“A robin is not a __”) and BERT still completes it the same way. Similarly, in one of its stories, GPT3 wrote that if you mix a spoonful of grape juice into your cranberry juice and drink the concoction, you die. It seems that robots, and artificial intelligence systems in general, are still missing some rudimentary facts of life that humans and animals grasp naturally and effortlessly.
How does one give robots a genome? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create.
It's not exactly the robots’ fault. Compared to humans, and all other organisms that have been around for thousands or millions of years, robots are very new. They are missing out on eons of evolutionary data-building. Animals and humans are born with the ability to do certain things because they are pre-wired in them. Flies know how to fly, fish knows how to swim, cats know how to meow, and babies know how to cry. Yet, flies don’t really learn to fly, fish doesn’t learn to swim, cats don’t learn to meow, and babies don’t learn to cry—they are born able to execute such behaviors because they’re preprogrammed to do so. All that happens thanks to the millions of years of evolutions wired into their respective genomes, which give rise to the brain’s neural networks responsible for these behaviors. Robots are the newbies, missing out on that trove of information, Zador argues.
A neuroscience professor who studies how brain circuitry generates various behaviors, Zador has a different approach to developing the robotic mind. Until their creators figure out a way to imbue the bots with that information, robots will remain quite limited in their abilities. Each model will only be able to do certain things it was programmed to do, but it will never go above and beyond its original code. So Zador argues that we have to start giving robots a genome.
How does one do that? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create. Genomes lay out rules for brain development. Specifically, the genome encodes blueprints for wiring up our nervous system—the details of which neurons are connected, the strength of those connections and other specs that will later hold the information learned throughout life. “Our genomes serve as blueprints for building our nervous system and these blueprints give rise to a human brain, which contains about 100 billion neurons,” Zador says.
If you think what a genome is, he explains, it is essentially a very compact and compressed form of information storage. Conceptually, genomes are similar to CliffsNotes and other study guides. When students read these short summaries, they know about what happened in a book, without actually reading that book. And that’s how we should be designing the next generation of robots if we ever want them to act like humans, Zador says. “We should give them a set of behavioral CliffsNotes, which they can then unwrap into brain-like structures.” Robots that have such brain-like structures will acquire a set of basic rules to generate basic behaviors and use them to learn more complex ones.
Currently Zador is in the process of developing algorithms that function like simple rules that generate such behaviors. “My algorithms would write these CliffsNotes, outlining how to solve a particular problem,” he explains. “And then, the neural networks will use these CliffsNotes to figure out which ones are useful and use them in their behaviors.” That’s how all living beings operate. They use the pre-programmed info from their genetics to adapt to their changing environments and learn what’s necessary to survive and thrive in these settings.
For example, a robot’s neural network could draw from CliffsNotes with “genetic” instructions for how to be aware of its own body or learn to adjust its movements. And other, different sets of CliffsNotes may imbue it with the basics of physical safety or the fundamentals of speech.
At the moment, Zador is working on algorithms that are trying to mimic neuronal blueprints for very simple organisms—such as earthworms, which have only 302 neurons and about 7000 synapses compared to the millions we have. That’s how evolution worked, too—expanding the brains from simple creatures to more complex to the Homo Sapiens. But if it took millions of years to arrive at modern humans, how long would it take scientists to forge a robot with human intelligence? That’s a billion-dollar question. Yet, Zador is optimistic. “My hypotheses is that if you can build simple organisms that can interact with the world, then the higher level functions will not be nearly as challenging as they currently are.”
In March, researchers published a review that lists which organic chemicals match up with certain diseases and biomarkers in the skin, saliva and urine. It’s an important step in creating a robot nose that can reliably detect diseases.
All diseases alter the organic compounds found in the body and their odors. Volatolomics is an emerging branch of chemistry that uses the smell of gases emitted by breath, urine, blood, stool, tears or sweat to diagnose disease. When someone is sick, the normal biochemical process is disrupted, and this alters the makeup of the gas, including a change in odor.
“These metabolites show a snapshot of what’s going on with the body,” says Cristina Davis, a biomedical engineer and associate vice chancellor of Interdisciplinary Research and Strategic Initiatives at the University of California, Davis. This opens the door to diagnosing conditions even before symptoms are present. It’s possible to detect a sweet, fruity smell in the breath of someone with diabetes, for example.
Hippocrates may have been the first to note that people with certain diseases give off an odor but dogs provided the proof of concept. Scientists have published countless studies in which dogs or other high-performing smellers like rodents have identified people with cancer, lung disease or other conditions by smell alone. The brain region that analyzes smells is proportionally about 40 times greater in dogs than in people. The noses of rodents are even more powerful.
Take prostate cancer, which is notoriously difficult to detect accurately with standard medical testing. After sniffing a tiny urine sample, trained dogs were able to pick out prostate cancer in study subjects more than 96 percent of the time, and earlier than a physician could in some cases.
But using dogs as bio-detectors is not practical. It is labor-intensive, complicated and expensive to train dogs to bark or lie down when they smell a certain VOC, explains Bruce Kimball, a chemical ecologist at the Monell Chemical Senses Center in Philadelphia. Kimball has trained ferrets to scratch a box when they smell a specific VOC so he knows. The lab animal must be taught to distinguish the VOC from background odors and trained anew for each disease scent.
In the lab of chemical ecologist Bruce Kimball, ferrets were trained to scratch a box when they identified avian flu in mallard ducks.
Glen J. Golden
There are some human super-smellers among us. In 2019, Joy Milne of Scotland proved she could unerringly identify people with Parkinson’s disease from a musky scent emitted from their skin. Clinical testing showed that she could distinguish the odor of Parkinson’s on a worn t-shirt before clinical symptoms even appeared.
Hossam Haick, a professor at Technion-Israel Institute of Technology, maintains that volatolomics is the future of medicine. Misdiagnosis and late detection are huge problems in health care, he says. “A precise and early diagnosis is the starting point of all clinical activities.” Further, this science has the potential to eliminate costly invasive testing or imaging studies and improve outcomes through earlier treatment.
The Nose Knows a Lot
“Volatolomics is not a fringe theory. There is science behind it,” Davis stresses. Every VOC has its own fingerprint, and a method called gas chromatography-mass spectrometry (GCMS) uses highly sensitive instruments to separate the molecules of these VOCs to determine their structures. But GCMS can’t discern the telltale patterns of particular diseases, and other technologies to analyze biomarkers have been limited.
We have technology that can see, hear and sense touch but scientists don’t have a handle yet on how smell works. The ability goes beyond picking out a single scent in someone’s breath or blood sample. It’s the totality of the smell—not the smell of a single chemical— which defines a disease. The dog’s brain is able to infer something when they smell a VOC that eludes human analysis so far.
Odor is a complex ecosystem and analyzing a VOC is compounded by other scents in the environment, says Kimball. A person’s diet and use of tobacco or alcohol also will affect the breath. Even fluctuations in humidity and temperature can contaminate a sample.
If successful, a sophisticated AI network can imitate how the dog brain recognizes patterns in smells. Early versions of robot noses have already been developed.
With today’s advances in data mining, AI and machine learning, scientists are trying to create mechanical devices that can draw on algorithms based on GCMS readings and data about diseases that dogs have sniffed out. If successful, a sophisticated AI network can imitate how the dog brain recognizes patterns in smells.
In March, Nano Research published a comprehensive review of volatolomics in health care authored by Haick and seven colleagues. The intent was to bridge gaps in the field for scientists trying to connect the biomarkers and sensor technology needed to develop a robot nose. This paper serves as a reference manual for the field that lists which VOCs are associated with what disease and the biomarkers in skin, saliva, breath, and urine.
Weiwei Wu, one of the co-authors and a professor at Xidian University in China, explains that creating a robotic nose requires the expertise of chemists, computer scientists, electrical engineers, material scientists, and clinicians. These researchers use different terms and methodologies and most have not collaborated before with the other disciplines. “The electrical engineers know the device but they don’t know as much about the biomarkers they need to detect,” Wu offers as an example.
This review is significant, Wu continues, because it can facilitate progress in the field by providing experts in all the disciplines with the basic knowledge needed to create an effective robot nose for diagnostic use. The paper also includes a systematic summary of the research methodology of volatolomics.
Once scientists build a stronger database of VOCs, they can program a device to identify critical patterns of specified diseases on a reliable basis. On a machine learning model, the algorithms automatically get better at diagnosing with each use. Wu envisions further tweaks in the next few years to make the devices smaller and consume less power.
A Whiff of the Future
Early versions of robot noses have already been developed. Some of them use chemical sensors to pick up smells in the breath or other body emission molecules. That data is sent through an electrical signal to a computer network for interpretation and possible linkage to a disease.
This electronic nose, or e-nose, has been successful in small pilot studies at labs around the world. At Ben-Gurion University in Israel, researchers detected breast cancer with electronic gas sensors with 95% accuracy, a higher sensitivity than mammograms. Other robot noses, called p-noses, use photons instead of electrical signals.
The mechanical noses being developed tap different methodologies and analytic techniques which makes it hard to compare them. Plus, the devices are intended for varying uses. One team, for example, is working on an e-nose that can be waved over a plate to screen for the presence of a particular allergen when you’re dining out.
A robot nose could be used as a real-time diagnostic tool in clinical practice. Kimball is working on one such tool that can distinguish between a viral and bacterial infection. This would enable physicians to determine whether an antibiotic prescription is appropriate without waiting for a lab result.
Davis is refining a hand-held device that identifies COVID-19 through a simple breath test. She sees the tool being used at crowded airports, sports stadiums and concert venues where PCR or rapid antigen testing is impractical. Background air samples are collected from the space so that those signals can be removed from the human breath measurement. “[The sensor tool] has the same accuracy as the rapid antigen test kits but exhaled breath is easier to collect,” she notes.
The NaNose, also known as the SniffPhone, uses tiny sensors boosted by AI to distinguish Alzheimer's, Crohn's disease, the early stages of several cancers, and other diseases with 84 to 98 percent accuracy.
Hossam Haick
Haick named his team’s robot nose, “NaNose,” since it is based on nanotechnology; the prototype is called the SniffPhone. Using tiny sensors boosted by AI, it can distinguish 23 diseases in human subjects with 84 to 98 percent accuracy. This includes early stages of several cancers, Alzheimer’s, tuberculosis and Crohn’s disease. His team has been raising the accuracy level by combining biomarker signals from both breath and skin, for example. The goal is to achieve 99.9 percent accuracy consistently so no other diagnostic tests would be needed before treating the patient. Plus, it will be affordable, he says.
Kimball predicts we’ll be seeing these diagnostic tools in the next decade. “The physician would narrow down what [the diagnosis] might be and then get the correct tool,” he says. Others are envisioning one device that can screen for multiple diseases by programming the software, which would be updated regularly with new findings.
Larger volatolomics studies must be conducted before these e-noses are ready for clinical use, however. Experts also need to learn how to establish normal reference ranges for e-nose readings to support clinicians using the tool.
“Taking successful prototypes from the lab to industry is the challenge,” says Haick, ticking off issues like reproducibility, mass production and regulation. But volatolomics researchers are unanimous in believing the future of health care is so close they can smell it.
A biotech in Cambridge, Mass., is targeting a rare disease called cystinosis with gene therapy. It's been effective for five patients in a clinical trial that's still underway.
Recently, AVROBIO compiled positive clinical data from this first and only gene therapy trial for the disease. The data show the potential of the therapy to genetically modify the patients’ own hematopoietic stem cells—a certain type of cell that’s capable of developing into all different types of blood cells—to express the functional protein they are deficient in. It stabilizes or reduces the impact of cystinosis on multiple tissues with a single dose.
Medical researchers have found that more than 80 different mutations to a gene called CTNS are responsible for causing cystinosis. The most common mutation results in a deficiency of the protein cystinosin. That protein functions as a transporter that regulates a lot metabolic processes in the cells.
“One of the first things we see in patients clinically is an accumulation of a particular amino acid called cystine, which grows toxic cystine crystals in the cells that cause serious complications,” explains Essra Rihda, chief medical officer for AVROBIO. “That happens in the cells across the tissues and organs of the body, so the disease affects many parts of the body.”
Jordan Janz, 23, meets Stephanie Cherqui, the principal investigator of his gene therapy trial, before the trial started in 2019.
Jordan Janz
According to Rihda, although cystinosis can occur in kids and adults, the most severe form of the disease affects infants and makes up about 95 percent of overall cases. Children typically appear healthy at birth, but around six to 18 months, they start to present for medical attention with failure to thrive.
Additionally, infants with cystinosis often urinate frequently, a sign of polyuria, and they are thirsty all the time, since the disease usually starts in the kidneys. Many develop chronic kidney disease that ultimately progresses to the point where the kidney no longer supports the body’s needs. At that stage, dialysis is required and then a transplant. From there the disease spreads to many other organs, including the eyes, muscles, heart, nervous system, etc.
“The gene for cystinosis is expressed in every single tissue we have, and the accumulation of this toxic buildup alters all of the organs of the patient, so little by little all of the organs start to fail,” says Stephanie Cherqui, principal investigator of Cherqui Lab, which is part of UC San Diego’s Department of Pediatrics.
Since the 1950s, a drug called cysteamine showed some therapeutic effect on cystinosis. It was approved by the FDA in 1994 to prevent damage that may be caused by the buildup of cystine crystals in organs. Prior to FDA approval, Cherqui says, children were dying of the disease before they were ten-years-old or after a kidney transplant. By taking oral cysteamine, they can live from 20 to 50 years longer. But it’s a challenging drug because it has to be taken every 6 or 12 hours, and there are serious gastric side effects such as nausea and diarrhea.
“With all of the complications they develop, the typical patient takes 40 to 60 pills a day around the clock,” Cherqui says. “They literally have a suitcase of medications they have to carry everywhere, and all of those medications don’t stop the progression of the disease, and they still die from it.”
Cherqui has been a proponent of gene therapy to treat children’s disorders since studying cystinosis while earning her doctorate in 2002. Today, her lab focuses on developing stem cell and gene therapy strategies for degenerative, hereditary disorders such as cystinosis that affect multiple systems of the body. “Because cystinosis expresses in every tissue in the body, I decided to use the blood-forming stem cells that we have in our bone marrow,” she explains. “These cells can migrate to anywhere in the body where the person has an injury from the disease.”
AVROBIO’s hematopoietic stem cell gene therapy approach collects stem cells from the patient’s bone marrow. They then genetically modify the stem cells to give the patient a copy of the healthy CTNS gene, which the person either doesn’t have or it’s defective.
The patient first undergoes apheresis, a medical procedure in which their blood is passed through an apparatus that separates out the diseased stem cells, and a process called conditioning is used to help eliminate the damaged cells so they can be replaced by the infusion of the patient’s genetically modified stem cells. Once they become engrafted into the patient’s bone marrow, they reproduce into a lot of daughter cells, and all of those daughter cells contain the CTNS gene. Those cells are able to express the healthy, functional, active protein throughout the body to correct the metabolic problem caused by cystinosis.
“What we’re seeing in the adult patients who have been dosed to date is the consistent and sustained engraftment of our genetically modified cells, 17 to 27 months post-gene therapy, so that’s very encouraging and positive,” says Rihda, the chief medical officer at AVROBIO.
When Janz was 11-years-old, his mother got him enrolled in the trial of a new form of cysteamine that would only need to be taken every 12 hours instead of every six. Two years later, she made sure he was the first person on the list for Cherqui’s current stem cell gene therapy trial.
AVROBIO researchers have also confirmed stabilization or improvement in motor coordination and visual perception in the trial participants, suggesting a potential impact on the neuropathology of the disease. Data from five dosed patients show strong safety and tolerability as well as reduced accumulation of cystine crystals in cells across multiple tissues in the first three patients. None of the five patients need to take oral cysteamine.
Janz’s mother, Barb Kulyk, whom he credits with always making him take his medications and keeping him hydrated, had been following Cherqui’s research since his early childhood. When Janz was 11-years-old, she got him enrolled in the trial of a new form of cysteamine that would only need to be taken every 12 hours instead of every six. When he was 17, the FDA approved that drug. Two years later, his mother made sure he was the first person on the list for Cherqui’s current stem cell gene therapy trial. He received his new stem cells on October 7th, 2019, went home in January 2020, and returned to working full time in February.
Jordan Janz, pictured here with his girlfriend, has a new lease on life, plus a new hair color.
Jordan Janz
He notes that his energy level is significantly better, and his mother has noticed much improvement in him and his daily functioning: He rarely vomits or gets nauseous in the morning, and he has more color in his face as well as his hair. Although he could finish his participation at any time, he recently decided to continue in the clinical trial.
Before the trial, Janz was taking 56 pills daily. He is completely off all of those medications and only takes pills to keep his kidneys working. Because of the damage caused by cystinosis over the course of his life, he’s down to about 20 percent kidney function and will eventually need a transplant.
“Some day, though, thanks to Dr. Cherqui’s team and AVROBIO’s work, when I get a new kidney, cystinosis won’t destroy it,” he concludes.