The Importance of Behavioural Science & Market Research in New Product Development
The world of innovation appears to thrive on trendy celebrity collaborations, exciting new products, and limited-edition flavours or designs. It is dynamic, fun, and most importantly, a major business driver for brands. However, beneath the surface allure, innovation is far more complex than simply launching a new product and expecting instant success.
In the billion-dollar new product development industry, a staggering 95% of innovations fail, according to Professor Clayton Christensen. Just like other aspects of marketing, whether pricing, advertising, or positioning, successful innovation is rooted in strategy, science, and sometimes a little luck. Joining the elite 5% of successful innovations requires meticulous planning, market understanding, and the right approach.
Carlsberg understands these challenges firsthand. To educate aspiring marketers, they sought a time efficient, predictive, and easy to grasp way to illustrate the complexities of product development. Their goal was to equip students with the knowledge and tools to turn ideas into in-market successes.
Turning to System1’s Test Your Innovation platform, the Carlsberg team set out to put innovation theory into practice. They tested a series of new product concepts, designed by a class of students, to determine which would stand out on the shelf and where improvements could be made.
Like marketing theory, Test Your Innovation is built on the foundations of science and human behaviour, ensuring results closely reflect real-world market patterns. Predicting success is not as simple as asking consumers whether they would buy a product. As Daniel Kahneman’s research has shown, humans are inherently biased in rationalising decisions. We do not always do what we say or say what we do. The key to unlocking predictive insights lies in understanding emotional responses.
Unlike traditional testing methods, Test Your Innovation harnesses the predictive power of emotion. It draws on the work of Dr. Paul Ekman, allowing consumers to instinctively select one of his seven core universal human emotions, plus neutrality, to describe their reaction to a new product. This method prioritises intuition and System 1 decision-making, which behavioural science has shown to be the most reliable predictor of actual behaviour.
Using this approach, Carlsberg tested 10 hypothetical product ideas in both the UK and Switzerland with a nationally representative pool of consumers. Research by John Kearon, published in System1: Unlocking Profitable Growth, highlights a critical principle of successful innovation: Fluency. This concept refers to innovations that tap into familiarity, which triggers the emotion of Happiness. Kearon found that many innovations fail when they attempt to shock or surprise consumers too much. His research revealed a golden ratio for success: 80 percent Happiness to 20 percent Surprise.
Test Your Innovation enables marketers to quantify the balance between Happiness and Surprise, analyse consumer feedback, and refine ideas based on real emotional responses. With these insights, brands can confidently bring products to market, increasing their chances of success.
Of the 10 ideas tested, the findings reflected the real-world success rate of innovation. Only one idea achieved a 3 Star rating or higher, while 90% fell below this benchmark. Three concepts received a 2 Star rating, providing a strong foundation for further development. With consumer feedback, key brand attributes, and target appeal data, these ideas have the potential to be refined for greater success.
The most successful concept was a citrus flavoured twist on a functional beverage, generating 26% Happiness and 17% Surprise, achieving a balanced 60 to 40 ratio. The packaging, which was a spin on an existing brand in the market, was kept closely in line with its current look and feel. According to industry benchmarks, a 3 Star idea typically exhibits this mix. While the results suggest the concept leans slightly too much toward novelty to reach the 4- or 5-Star range, it still scored above average and shows strong market potential.
Consumer insights highlight opportunities for improvement. Expanding the flavour range could broaden appeal, while the brand name, unfamiliar to some, may require time to gain traction. Since this type of beverage (prebiotic soda) is still emerging in certain markets, a degree of novelty was expected. However, a well-executed marketing strategy focused on awareness and trust could boost familiarity; a celebrity partnership for example could further accelerate adoption by enhancing credibility and encouraging trial.
"This exercise aimed to highlight the challenges of new product development while emphasising the importance and benefits of market research. We wanted to show students that theory can only take you so far, and that to truly understand an idea's potential, you must seek feedback from the most important people, the consumers"
Thomas Amstutz CEO, Feldschlösschen
Select ideas with confidence by leveraging the power of emotion, the best predictor of future behavior.
Test up to 50 ideas in one test and benchmark against our database of more than 50,000 ideas. Customize the survey questions and your sample to capture insights from the general population and your target audience.
Test ideas such as new products, claims, taglines, packaging, concepts and more.
