Skip to content
  • There are no suggestions because the search field is empty.

Penalty Analysis

See Which Attributes Are Costing You Overall Liking

Overview

Penalty Analysis answers one question: which attribute is pulling your Overall Liking down, and by how much?

It compares the respondents who thought an attribute was just right against the ones who wanted more of it or less. If the "too sweet" group rated your product two points lower overall, that gap is what the sweetness costs you. Across every attribute, that's how you decide what to fix first.

It runs on any project with an Overall Liking question and at least one Just-About-Right (JAR) attribute question. Those are designated for you — ask your Highlight contact if an attribute is missing.

Products is the label on a physical product study; on a concept study it reads Concepts, on a survey-only study Surveys.

Where to Find Penalty Analysis

Go to Insights, open the Analysis menu in the tab bar, and choose Penalty Analysis. Don't see it? Ask your account manager.

The tab is open while fielding runs, showing responses collected up to the moment the page loaded — reload for newer ones.

Reading the Table

You get one table per product picked in the Products selector, and three rows per attribute: the respondents who wanted more of it, the ones who thought it was just right, and the ones who wanted less. The row labels come from your own question, so they read as your respondents saw them — "Too Little", "Too Much".

Each row carries five columns:

  • n — how many respondents are in that group
  • Percent — that group's share of everyone who answered the attribute
  • Mean — that group's average Overall Liking score
  • Penalty — how much lower that group's liking is than the just-right group's. A positive number means they liked the product less
  • Weighted Penalty — the Penalty scaled by how many respondents it affected. This is the number that gets flagged

Groups smaller than 15% aren't given a penalty. Neither is the just-right row, at any size — it's the baseline the other two are measured against.

A Worked Example

Say Flavor is a JAR attribute on a study whose Overall Liking question uses a 9-point scale.

  • 65% said the flavour was just about right. Their average Overall Liking: 8.0
  • 20% said there was too much. Their average Overall Liking: 5.0

The Penalty is 8.0 − 5.0 = 3.0. That group liked the product three points less.

The Weighted Penalty is 3.0 × 0.20 = 0.60 — the three-point drop, scaled by the fifth of your audience it actually hit.

On a 9-point scale, 0.50 and above is Critical Risk, so that row turns red. At 14% rather than 20%, no penalty would have been calculated at all.

Yellow and Red Flags

Two flags colour the group's label cell: Moderate Risk in yellow and Critical Risk in red. A legend above the tables names them.

The thresholds come from your Overall Liking scale and adjust to its length, so a Critical Risk flag means the same severity however long your scale is. Flags compare across studies even when the raw numbers can't.

Each figure is where that band starts, so a Weighted Penalty of exactly 0.50 on a 9-point scale is Critical, not Moderate. The pair in force for your study is printed under the tables.

Penalty Analysis runs no significance testing. The flags come from the 15% rule and these bands, not from a statistical test.

Note: A Penalty can be negative, when the off-midpoint group liked the product more than the just-right group. Weighted Penalty ignores the sign, so such a row can still be coloured — check the sign before reading a colour as a problem.

What to Do With It

Work the red rows first, then the yellow, and treat an unflagged attribute as one this study gives you no reason to change.

The row tells you which way to move. A flag on "too little" means the attribute is under-delivered; a flag on "too much" means it's over-delivered. When both rows of one attribute are flagged, it's splitting your audience, and a single reformulation won't satisfy both groups.

Tip: Rank by Weighted Penalty, not Penalty. A three-point drop affecting 8% of respondents is a smaller commercial problem than a one-point drop affecting 60%.

Penalty Analysis tells you what an attribute costs you. It won't tell you who you'd lose by changing it — see Alienation Analysis for that.

Downloading Your Tables

Choose your settings first, then click Download CSV at the top right. The file matches what you've chosen, and adds a Risk column spelling out the flag as text.

Related

← Back to Insights Overview