The media loves to tout scientific breakthroughs, and few are as toutable – and in turn, have been as touted – as CRISPR. This method of targeted DNA excision was discovered in bacteria, which use it as an adaptive immune system to combat reinfection with a previously encountered virus.
Shouldn't the editors at a Nature journal know better than to have published an incorrect paper in the first place?
It is cool on so many levels: not only is the basic function fascinating, reminding us that we still have more to discover about even simple organisms that we thought we knew so well, but the ability it grants us to remove and replace any DNA of interest has almost limitless applications in both the lab and the clinic. As if that didn't make it sexy enough, add in a bicoastal, male-female, very public and relatively ugly patent battle, and the CRISPR story is irresistible.
And then last summer, a bombshell dropped. The prestigious journal Nature Methods published a paper in which the authors claimed that CRISPR could cause many unintended mutations, rendering it unfit for clinical use. Havoc duly ensued; stocks in CRISPR-based companies plummeted. Thankfully, the authors of the offending paper were responsible, good scientists; they reassessed, then recanted. Their attention- and headline- grabbing results were wrong, and they admitted as much, leading Nature Methods to formally retract the paper this spring.
How did this happen? Shouldn't the editors at a Nature journal know better than to have published this in the first place?
Alas, high-profile scientific journals publish misleading and downright false results fairly regularly. Some errors are unavoidable – that's how the scientific method works. Hypotheses and conclusions will invariably be overturned as new data becomes available and new technologies are developed that allow for deeper and deeper studies. That's supposed to happen. But that's not what we're talking about here. Nor are we talking about obvious offenses like outright plagiarism. We're talking about mistakes that are avoidable, and that still have serious ramifications.
The cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed.
Two parties are responsible for a scientific publication, and thus two parties bear the blame when things go awry: the scientists who perform and submit the work, and the journals who publish it. Unfortunately, both are incentivized for speedy and flashy publications, and not necessarily for correct publications. It is hardly a surprise, then, that we end up with papers that are speedy and flashy – and not necessarily correct.
"Scientists don't lie and submit falsified data," said Andy Koff, a professor of Molecular Biology at Sloan Kettering Institute, the basic research arm of Memorial Sloan Kettering Cancer Center. Richard Harris, who wrote the book on scientific misconduct running the gamut from unconscious bias and ignorance to more malicious fraudulence, largely concurs (full disclosure: I reviewed the book here). "Scientists want to do good science and want to be recognized as such," he said. But even so, the cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed. In Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Millions, Harris describes how scientists must constantly publish in order to maintain their reputations and positions, to get grants and tenure and students. "They are disincentivized from doing that last extra experiment to prove their results," he said; it could prove too risky if it could cost them a publication.
Ivan Oransky and Adam Marcus founded Retraction Watch, a blog that tracks the retraction of scientific papers, in 2010. Oransky pointed out that blinded peer review – the pride and joy of the scientific publishing enterprise – is a large part of the problem. "Pre-publication peer review is still important, but we can't treat it like the only check on the system. Papers are being reviewed by non-experts, and reviewers are asked to review papers only tangentially related to their field. Moreover, most peer reviewers don't look at the underlying or raw data, even when it is available. How then can they tell if the analysis is flawed or the data is accurate?" he wondered.
Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Koff agreed that anonymous peer review is valuable, but severely flawed. "Blinded review forces a collective view of importance," he said. "If an article disagrees with the reviewer's worldview, the article gets rejected or forced to adhere to that worldview – even if that means pushing the data someplace it shouldn't necessarily go." We have lost the scientific principle behind review, he thinks, which was to critically analyze a paper. But instead of challenging fundamental assumptions within a paper, reviewers now tend to just ask for more and more supplementary data. And don't get him started on editors. "Editors are supposed to arbitrate between reviewers and writers and they have completely abdicated this responsibility, at every journal. They do not judge, and that's a real failing."
Harris laments the wasted time, effort, and resources that result when erroneous ideas take hold in a field, not to mention lives lost when drug discovery is predicated on basic science findings that end up being wrong. "When no one takes the time, care, and money to reproduce things, science isn't stopping – but it is slowing down," he noted. Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Scientists and publishers don't only cause the problem, though – they may also provide the solution. Both camps are increasingly recognizing and dealing with the crisis. The self-proclaimed "data thugs" Nick Brown and James Heathers use pretty basic arithmetic to reveal statistical errors in papers. The microbiologist Elisabeth Bik scans the scientific literature for problematic images "in her free time." The psychologist Brian Nosek founded the Center for Open Science, a non-profit organization dedicated to promoting openness, integrity, and reproducibility in scientific research. The Nature family of journals – yes, the one responsible for the latest CRISPR fiasco – has its authors complete a checklist to combat irreproducibility, à la Atul Gawande. And Nature Communications, among other journals, uses transparent peer review, in which authors can opt to have the reviews of their manuscript published anonymously alongside the completed paper. This practice "shows people how the paper evolved," said Koff "and keeps the reviewer and editor accountable. Did the reviewer identify the major problems with the paper? Because there are always major problems with a paper."
You are driving along the highway and see an electronic sign that reads: “3,238 traffic deaths this year.” Do you think this reminder of roadside mortality would change how you drive? According to a recent, peer-reviewed study in Science, seeing that sign would make you more likely to crash. That’s ironic, given that the sign’s creators assumed it would make you safer.
The study, led by a pair of economists at the University of Toronto and University of Minnesota, examined seven years of traffic accident data from 880 electric highway sign locations in Texas, which experienced 4,480 fatalities in 2021. For one week of each month, the Texas Department of Transportation posts the latest fatality messages on signs along select traffic corridors as part of a safety campaign. Their logic is simple: Tell people to drive with care by reminding them of the dangers on the road.
But when the researchers looked at the data, they found that the number of crashes increased by 1.52 percent within three miles of these signs when compared with the same locations during the same month in previous years when signs did not show fatality information. That impact is similar to raising the speed limit by four miles or decreasing the number of highway troopers by 10 percent.
The scientists calculated that these messages contributed to 2,600 additional crashes and 16 deaths annually. They also found a social cost, meaning the financial expense borne by society as a whole due to these crashes, of $377 million per year, in Texas alone.
The cause, they argue, is distracted driving. Much like incoming texts or phone calls, these “in-your-face” messages grab your attention and undermine your focus on the road. The signs are particularly distracting and dangerous because, in communicating that many people died doing exactly what you are doing, they cause anxiety. Supporting this hypothesis, the scientists discovered that crashes increase when the signs report higher numbers of deaths. Thus, later in the year, as that total mortality figure goes up, so do the percentage of crashes.
Boomerang effects happen when those with authority, in government or business, fail to pay attention to the science. These leaders rely on armchair psychology and gut intuitions on what should work, rather than measuring what does work.
That change over time is not simply a function of changing weather, the study’s authors observed. They also found that the increase in car crashes is greatest in more complex road segments, which require greater focus to navigate.
The overall findings represent what behavioral scientists like myself call a “boomerang effect,” meaning an intervention that produces consequences opposite to those intended. Unfortunately, these effects are all too common. Between 1998 and 2004, Congress funded the $1 billion National Youth Anti-Drug Media Campaign, which famously boomeranged. Using professional advertising and public relations firms, the campaign bombarded kids aged 9 to 18 with anti-drug messaging, focused on marijuana, on TV, radio, magazines, and websites. A 2008 study funded by the National Institutes of Health found that children and teens saw these ads two to three times per week. However, more exposure to this advertising increased the likelihood that youth used marijuana. Why? Surveys and interviews suggested that young people who saw the ads got the impression that many of their peers used marijuana. As a result, they became more likely to use the drug themselves.
Boomerang effects happen when those with authority, in government or business, fail to pay attention to the science. These leaders rely on armchair psychology and gut intuitions on what should work, rather than measuring what does work.
To be clear, message campaigns—whether on electronic signs or through advertisements—can have a substantial effect on behavior. Extensive research reveals that people can be influenced by “nudges,” which shape the environment to influence their behavior in a predictable manner. For example, a successful campaign to reduce car accidents involved sending smartphone notifications that helped drivers evaluate their performance after each trip. These messages informed drivers of their personal average and best performance, as measured by accelerometers and gyroscopes. The campaign, which ran over 21 months, significantly reduced accident frequency.
Nudges work best when rigorously tested with small-scale experiments that evaluate their impact. Because behavioral scientists are infrequently consulted in creating these policies, some studies suggest that only 62 percent have a statistically significant effect. Other research reveals that up to 15 percent of desired interventions may backfire.
In the case of roadside mortality signage, the data are damning. The new research based on the Texas signs aligns with several past studies. For instance, research has shown that increasing people’s anxiety causes them to drive worse. Another, a Virginia Tech study in a laboratory setting, found that showing drivers fatality messages increased what psychologists call “cognitive load,” or the amount of information your brain is processing, with emotionally-salient information being especially burdensome and preoccupying, thus causing more distraction.
Nonetheless, Texas, along with at least 28 other states, has pursued mortality messaging campaigns since 2012, without testing them effectively. Behavioral science is critical here: when road signs are tested by people without expertise in how minds work, the results are often counterproductive. For example, the Virginia Tech research looked at road signs that used humor, popular culture, sports, and other nontraditional themes with the goal of provoking an emotional response. When they measured how participants responded to these signs, they noticed greater cognitive activation and attention in the brain. Thus, the researchers decided, the signs worked. But a behavioral scientist would note that increased attention likely contributes to the signs’ failure. As the just-published study in Science makes clear, distracting, emotionally-loaded signs are dangerous to drivers.
But there is good news. First, in most cases, it’s very doable to run an effective small-scale study testing an intervention. States could set up a safety campaign with a few electric signs in a diversity of settings and evaluate the impact over three months on driver crashes after seeing the signs. Policymakers could ask researchers to track the data as they run ads for a few months in a variety of nationally representative markets for a few months and assess their effectiveness. They could also ask behavioral scientists whether their proposals are well designed, whether similar policies have been tried previously in other places, and how these policies have worked in practice.
Everyday citizens can write to and call their elected officials to ask them to make this kind of research a priority before embracing an untested safety campaign. More broadly, you can encourage them to avoid relying on armchair psychology and to test their intuitions before deploying initiatives that might place the public under threat.
I walked through the Dong Makkhai forest-products market, just outside of Vientiane, the laid-back capital of the Lao Peoples Democratic Republic or Lao PDR. Piled on rough display tables were varieties of six-legged wildlife–grasshoppers, small white crickets, house crickets, mole crickets, wasps, wasp eggs and larvae, dragonflies, and dung beetles. Some were roasted or fried, but in a few cases, still alive and scrabbling at the bottom of deep plastic bowls. I crunched on some fried crickets and larvae.
One stall offered Giant Asian hornets, both babies and adults. I suppressed my inner squirm and, in the interests of world food security and equity, accepted an offer of the soft, velvety larva; they were smooth on the tongue and of a pleasantly cool, buttery-custard consistency. Because the seller had already given me a free sample, I felt obliged to buy a chunk of the nest with larvae and some dead adults, which the seller mixed with kaffir lime leaves.
The year was 2016 and I was in Lao PDR because Veterinarians without Borders/Vétérinaires sans Frontières-Canada had initiated a project on small-scale cricket farming. The intent was to organize and encourage rural women to grow crickets as a source of supplementary protein and sell them at the market for cash. As a veterinary epidemiologist, I had been trained to exterminate disease spreading insects—Lyme disease-carrying ticks, kissing bugs that carry American Sleeping Sickness and mosquitoes carrying malaria, West Nile and Zika. Now, as part of a global wave promoting insects as a sustainable food source, I was being asked to view arthropods as micro-livestock, and devise management methods to keep them alive and healthy. It was a bit of a mind-bender.
The 21st century wave of entomophagy, or insect eating, first surged in the early 2010s, promoted by a research centre in Wageningen, a university in the Netherlands, conferences organized by the Food and Agriculture Organization of the United Nations, and enthusiastic endorsements by culinary adventurers and celebrities from Europeanized cultures. Headlines announced that two billion people around the world already ate insects, and that if everyone adopted entomophagy we could reduce greenhouse gases, mitigate climate change, and reign in profligate land and water use associated with industrial livestock production.
Furthermore, eating insects was better for human health than eating beef. If we were going to feed the estimated nine billion people with whom we will share the earth in 2050, we would need to make some radical changes in our agriculture and food systems. As one author proclaimed, entomophagy presented us with a last great chance to save the planet.
In 2010, in Kunming, a friend had served me deep-fried bamboo worms. I ate them to be polite. They tasted like French fries, but with heads.
The more recent data suggests that the number of people who eat insects in various forms, though sizeable, may be closer to several hundreds of millions. I knew that from several decades of international veterinary work. Sometimes, for me, insect eating has been simply a way of acknowledging cultural diversity. In 2010, in Kunming, a friend had served me deep-fried bamboo worms. I ate them to be polite. They tasted like French fries, but with heads. My friend said he preferred them chewier. I never thought about them much after that. I certainly had not thought about them as ingredients for human health.
Is consuming insects good for human health? Researchers over the past decade have begun to tease that apart. Some think it might not be useful to use the all-encompassing term insect at all; we don’t lump cows, pigs, chickens into one culinary category. Which insects are we talking about? What are they fed? Were they farmed or foraged? Which stages of the insects are we eating? Do we eat them directly or roasted and ground up?
The overall research indicates that, in general, the usual farmed insects (crickets, locusts, mealworms, soldier fly larvae) have high levels of protein and other important nutrients. If insects are foraged by small groups in Laos, they provide excellent food supplements. Large scale foraging in response to global markets can be incredibly destructive, but soldier fly larvae fed on food waste and used as a substitute for ground up anchovies for farmed fish (as Enterra Feed in Canada does) improves ecological sustainability.
Entomophagy alone might not save the planet, but it does give us an unprecedented opportunity to rethink how we produce and harvest protein.
The author enjoys insects from the Dong Makkhai forest-products market, just outside of Vientiane, the capital of the Lao Peoples Democratic Republic.
Between 1961 and 2018, world chicken production increased from 4 billion to 20 billion, pork from 200 million to over 100 billion pigs, human populations doubled from 3.5 billion to more than 7 billion, and life expectancy (on average) from 52 to 72 years. These dramatic increases in food production are the result of narrowly focused scientific studies, identifying specific nutrients, antibiotics, vaccines and genetics. What has been missing is any sort of peripheral vision: what are the unintended consequences of our narrowly defined success?
If we look more broadly, we can see that this narrowly defined success led to industrial farming, which caused wealth, health and labor inequities; polluted the environment; and created grounds for disease outbreaks. Recent generations of Europeanized people inherited the ideas of eating cows, pigs and chickens, along with their products, so we were focused only on growing them as efficiently as possible. With insects, we have an exciting chance to start from scratch. Because, for Europeanized people, insect eating is so strange, we are given the chance to reimagine our whole food system in consultation with local experts in Asia and Africa (many of them villagers), and to bring together the best of both locally adapted food production and global distribution.
For this to happen, we will need to change the dietary habits of the big meat eaters. How can we get accustomed to eating bugs? There’s no one answer, but there are a few ways. In many cases, insects are ground up and added as protein supplements to foods like crackers or bars. In certain restaurants, the chefs want you to get used to seeing the bugs as you eat them. At Le Feston Nu in Paris, the Arlo Guthrie look-alike bartender poured me a beer and brought out five small plates, each featuring a different insect in a nest of figs, sun-dried tomatoes, raisins, and chopped dried tropical fruits: buffalo worms, crickets, large grasshoppers (all just crunchy and no strong flavour, maybe a little nutty), small black ants (sour bite), and fat grubs with a beak, which I later identified as palm weevil larvae, tasting a bit like dried figs.
Some entomophagy advertising has used esthetically pleasing presentations in classy restaurants. In London, at the Archipelago restaurant, I dined on Summer Nights (pan fried chermoula crickets, quinoa, spinach and dried fruit), Love-Bug Salad (baby greens with an accompanying dish of zingy, crunchy mealworms fried in olive oil, chilis, lemon grass, and garlic), Bushman’s Cavi-Err (caramel mealworms, bilinis, coconut cream and vodka jelly), and Medieaval Hive (brown butter ice cream, honey and butter caramel sauce and a baby bee drone).
The Archipelago restaurant in London serves up a Love-Bug Salad: baby greens with an accompanying dish of zingy, crunchy mealworms fried in olive oil, chilis, lemon grass, and garlic.
Some chefs, like Tokyo-based Shoichi Uchiyama, try to entice people with sidewalk cooking lessons. Uchiyama's menu included hornet larvae, silkworm pupae, and silkworms. The silkworm pupae were white and pink and yellow. We snipped off the ends and the larvae dropped out. My friend Zen Kawabata roasted them in a small pan over a camp stove in the street to get the "chaff" off. We made tea from the feces of worms that had fed on cherry blossoms—the tea smelled of the blossoms. One of Uchiyama-san’s assistants made noodles from buckwheat dough that included powdered whole bees.
At a book reading in a Tokyo bookstore, someone handed me a copy of the Japanese celebrity scandal magazine Friday, opened to an article celebrating the “charms of insect eating.” In a photo, scantily-clad girls were drinking vodka and nibbling giant water bugs dubbed as toe-biters, along with pickled and fried locusts and butterfly larvae. If celebrities embraced bug-eating, others might follow. When asked to prepare an article on entomophagy for the high fashion Sorbet Magazine, I started by describing a clip of Nicole Kidman delicately snacking on insects.
Taking a page from the success story of MacDonald’s, we might consider targeting children and school lunches. Kids don’t lug around the same dietary baggage as the grownups, and they can carry forward new eating habits for the long term. When I offered roasted crickets to my grandchildren, they scarfed them down. I asked my five-year-old granddaughter what she thought: she preferred the mealworms to the crickets – they didn’t have legs that caught in her teeth.
Entomo Farms in Ontario, the province where I live, was described in 2015 by Canadian Business magazine as North America’s largest supplier of edible insects for human consumption. When visiting, I popped some of their roasted crickets into my mouth. They were crunchy, a little nutty. Nothing to get squeamish over. Perhaps the human consumption is indeed growing—my wife, at least, has joined me in my entomophagy adventures. When we celebrated our wedding anniversary at the Public Bar and Restaurant in Brisbane, Australia, the “Kang Kong Worms” and “Salmon, Manuka Honey, and Black Ants” seemed almost normal. Of course, the champagne helped.