The fate of novel ideas

Why did some good ideas, such as hand sanitizing in 19th-century hospitals or racial integration in the 20th century, take years to win widespread embrace? University of Utah postdoctoral researcher Wayne Johnson set out to identify the hurdles.

His research team’s program of five studies, which featured analyses of evaluations of films screened at Utah’s Sundance Film Festival and of products pitched on the television program Shark Tank, found that people are more likely to disagree about how valuable something is the less familiar it is. That is, the more unconventional a film or company, the wider the range of opinions it will garner.

“The problem is people see mixed reviews, interpret that as a bad sign, and then reduce their support or interest,” Johnson said. “People don’t realize that mixed reviews are to be expected when an idea is new.”

The greater an idea’s novelty, the greater disparity in responses it generates, according to Johnson’s study, published Friday, in the journal Nature Human Behaviour.

Missed opportunities

These conflicting evaluations result in missed opportunities. This is because the wide range of opinions is seen as a negative signal, rather than evidence that it’s “creative,” which Johnson defines as something both new and useful.

“The creativity research has been overwhelmingly focused on how you generate an idea. But from my perspective, lack of ideas often isn’t the right problem to solve,” Johnson said. “There are endless ideas out there. The difficulty is getting people to support and approve those ideas. In businesses, the bottleneck is very often the middle manager who rejects new ideas employees have. The idea is there, but it doesn’t get past a desk or a committee. That’s where creativity goes to die.”

Assessments of novel ideas vary because evaluators have fewer common reference points against which to evaluate them, making judgments more reliant on idiosyncratic knowledge and preferences, according to the findings.

The study was funded by Cornell University, where Johnson conducted the research as a graduate student with co-author Devon Proudfoot, an assistant professor of management.

Johnson got his own taste of the resistance experienced by novel ideas when, as a soldier in the U.S. Army, he led a unit that neutralized roadside bombs, or IEDs in military parlance, in Afghanistan in 2010.

More effective ways to avoid roadside bombs

At the time, the Army relied on tactics that were developed in Iraq, where U.S. soldiers used advanced machinery, including radars, to look for bombs without having to exit their armored vehicles. This strategy was more effective against Iraqi fighters who mostly relied on metal-bearing or military-grade ordnance hidden above ground, such as in trash or walls. But the U.S.’s high-tech gadgetry proved to have little value in Afghanistan, where fighters constructed homemade fertilizer-based explosives and buried them in that country’s dirt roads.

Johnson helped devise new tactics tailored to Afghanistan’s theater, but it required soldiers to abandon their equipment and exit their armored vehicles to eyeball the roads and ground for signs of concealed explosives.

“Some people told me, ‘Yeah, this is brilliant. Do it.’ And there’s some people who told me, ‘I don’t think you’ll survive two weeks if you do this,'” Johnson said. “Some were thinking with their Iraq experience, and others with fresh eyes for Afghanistan. They were assessing the idea from different frames of reference. It would be a terrible idea in Iraq, but great in Afghanistan. If I had just paid attention to the disagreement, I might have concluded it was too controversial a tactic to implement. The death and suffering of that missed opportunity would have been immense.”

The new tactics were immensely more effective and Johnson was quickly assigned to develop a training course to teach what his platoon had learned. However, it would take five more years for that course to be formally taught across the globe. Johnson describes this experience as having five weeks to generate the idea, five months to refine it, and five years to persuade others to implement it. This was a formative experience for him to realize that the most difficult part of innovation comes after the idea has been generated.

Years later as a business scholar, Johnson would explore how to more quickly integrate creative ideas into the marketplace. This experience taught him how a leader could easily miss a great creative idea because of conflicting feedback.

Although a sign of creativity, mixed reviews are seen in a negative light

“We found that people interpret [disagreement] in value judgments as a negative signal, reducing their support for novel ideas,” Johnson and Proudfoot’s paper concluded.

In one of the paper’s five studies, they examined 523 audience evaluations of films that premiered at Sundance between 2015 and 2022, comparing the amount of variability in films’ evaluations. (The 2024 festival opens next week in Park City.)

These evaluations reflect the opinions of film industry insiders, connoisseurs and trained critics. Conventional wisdom would predict their judgments would not be far apart. When it came to the more novel movies, however, their evaluations were far-ranging compared with those of more traditional films.

“The conclusion here is that the newer the idea is, the more even the experts are going to disagree about its value,” Johnson said. “And so we actually can’t count on experts to agree about whether something’s creative because newness makes them disagree about value.”

For another study, the researchers gathered 1,088 ventures pitched over 12 seasons of Shark Tank, a show where would-be entrepreneurs vie for acceptance from real-world investors who act as judges. Study participants rated a random subset of pitches on a seven-point novelty scale.

Then another set of participants rated the value of the 250 pitches rated most novel and the 250 pitches that were least novel.

Again, the ideas that were the most novel pitches saw wider ranges of evaluations.

Johnson argues decision makers should identify which reference points different evaluators are using. Then, they should focus on using evaluations with the most relevant, valid reference points. This strategy can create order from the confusion of many conflicting evaluations and make innovation opportunities clearer to see.

“People interpret disagreement as risk. The fact that disagreement increases with greater idea newness makes creative ideas seem less valuable,” Johnson said. “That throws sand in the gears of innovation.”

Instead of giving up or making negative conclusions when we see disagreement, he advises recognizing this could indicate creativity and prioritizing reviews that use the most relevant reference points.