Methodology

Behavioral science demonstrates the power of emotion to drive growth. Thus, to measure short- and long-term commercial effects like sales and market share gains, we leverage psychologist Paul Ekman’s seven universal human emotions. We also add neutrality as an option in our testing as it’s common for people to feel nothing about an ad or idea.

The approach differs from automated facial coding in that rather than relying on an algorithm to interpret how audiences feel, System1’s Test Your Ad and Test Your Innovation platforms ask people to indicate how they feel about an ad or idea. They select one of the seven basic emotions or neutral from a pictorial scale; an exercise that favors the fast processing that mimics how consumers make brand choices in the real world. In this way, across a large quantitative sample, it’s possible to establish how an ad or idea leaves people feeling – the blend of emotions it elicits.

Proven to predict growth

Because our Test Your Ad and Test Your Innovation metrics are driven by emotion, they are the most predictive of business outcomes. This predictiveness is backed by more than 250 validations for our Star Rating in relation to advertising, including in conjunction with the Institute of Practitioners in Advertising (IPA).

In a study conducted in 2009, System1 blind tested campaigns for which the IPA Effectiveness Databank held business effects data. Independent analysis by Peter Field on behalf of the IPA, using the IPA Effectiveness Databank, verified that campaigns generating a greater emotional response in System1’s testing achieved a greater number of very large business effects.

Marketers can Create with Confidence and secure buy-in from the C-suite knowing that the Star Rating is linked to commercial performance.

Combining media spend & Star Rating

Mark Ritson, brand consultant and founder of the Mini MBA, credits creative as the top growth factor that brands can control, second only to brand size. Consider two approaches to predicting growth – media weight (Extra Share of Voice), and media weight combined with creative quality (System1 Star Rating)

Your share of voice relative to your market share is a strong predictor of your brand’s growth the following year. Spend above your market size (Extra Share of Voice) and you will grow. Spend below your market size and you will lose market share.

In a study spanning over 4,000 ads, 32 categories, over £10bn in media spend, and over $0.5 trillion in revenue (Look out, Orlando Wood, IPA, 2021) when you add Star Ratings (creative quality) to spend data (media weight), a strong predictor of revenue growth becomes an exceptional one. In fact, the accuracy by which market share growth can be predicted nearly doubles.

Advertising quality acts like a multiplier for your spend. Aim for the high Star Ratings relative to your category and you will make your investment in advertising more profitable, giving you greater control over your long-term growth prospects.

Short-term impact

In addition to predicting marketing’s long-term impact, System1 also predicts the short-term impact of ads through emotional measurement. The Spike Rating is determined by the intensity of emotion that a consumer feels (irrespective of what that emotion is) and the strength of branding (learn more about Fluency below).

Our research with WARC, “How creativity boosts ROI,” shows that as creative quality increases, so does return on investment. This research was conducted by aligning Star Rating and Spike Rating results with WARC’s ROI database. The Star and Spike results were aggregated into three groups:

1-2=Low/Modest

3=Good

4-5=Strong/Exceptional

WARC’s ROI figures were averaged for each of these Star/Spike groups. In both cases, as creative quality increases, so does the return on investment.

Make your brand known

The short- and long-term effects of creative can’t be fully realized if consumers don’t know the work is from your brand. This is why System1’s Test Your Ad platform measures the strength of branding in ads. We ask consumers to identify which brand the ad was for. We then express the Fluency Rating as the % of people recognizing the brand by the end of the ad. 

The Fluency Rating is an insightful diagnostic measure, as it pinpoints how easily consumers can attribute creative to the brand, and whether adjustments may need to be made around branding and distinctive assets. Fluency also plays a role in the Spike Rating – brands can better tap into strong emotional responses if people can quickly and correctly identify the brand.

Taking these three metrics together, marketers have accurate predictions for growth and actionable improvements to maximize the impact of their marketing.