Identify Attribute Impacts on Overall Liking
Overview
Penalty Analysis uses responses to "Just About Right" (JAR) questions about product attributes to help you understand how individual product attributes impact respondents' reaction to the Overall Liking question.
Accessing Penalty Analysis
Penalty Analysis is available for Projects that include an "Overall Liking" question and "Just About Right" (JAR) questions that ask respondents to rate attributes (ex. flavor, texture) on a scale like "Too Little," "Just About Right," or "Too Much". Once the project's surveys have closed, Penalty Analysis becomes available for relevant projects.
To access Penalty Analysis, navigate to the Insights section of a project, then click into the Analysis menu and select Penalty Analysis.
Understanding the Penalty Analysis Tables
Each tested product gets its own table, with rows for each JAR question and the attribute being measured (e.g., "JAR - Flavor"). Labels match your survey's wording for clarity. Each question breaks down into three rows:
- Low end: E.g., "Too Little" (bottom 2 box for a 5-point scale).
- Just About Right: Middle box.
- High end: E.g., "Too Much" (top 2 box for a 5-point scale).
The table includes four columns:
- Percent: Proportion of respondents in each group (e.g., a percentage for "Too Little").
- Mean: Average Overall Liking score for that group. Always shown for the "Just About Right" group; shown for top/bottom 2 box groups if they're 15% or more of responses.
- Penalty: Difference between the "Just About Right" mean and the top/bottom 2 box mean.
- Weighted Penalty: Penalty multiplied by the percent, showing the scaled impact.
Interpreting Risk Levels with Scale-Adjusted Thresholds
To ensure fair and consistent analysis across all your research, Penalty Analysis automatically detects the scale used for the "Overall Liking" question (e.g., 5, 7, or 9-point) and applies the appropriate risk thresholds. This means you can confidently compare results from different studies, even if they use different scales.
The analysis flags attributes with two risk levels, highlighted in yellow and red. In the user interface, you will see an indicator showing which threshold set is being applied for full transparency.
Here are the scale-adjusted thresholds we apply:
- 9-Point Scale:
- Moderate Risk: Weighted Penalty > 0.30
- Critical Risk: Weighted Penalty > 0.50
- 7-Point Scale:
- Moderate Risk: Weighted Penalty > 0.23
- Critical Risk: Weighted Penalty > 0.39
- 5-Point Scale:
- Moderate Risk: Weighted Penalty > 0.17
- Critical Risk: Weighted Penalty > 0.28
A high weighted penalty suggests an attribute is negatively impacting Overall Liking. By prioritizing attributes flagged with Critical Risk, you can focus your product development efforts on the areas that matter most to consumers.
Tip: Prioritize Critical Risk rows to address the most significant impacts on liking.
Example Calculation
Here's a simple example of how Penalty Analysis calculates the weighted penalty for a JAR question about "Flavor," assuming a 9-point Overall Liking scale was used.
Data:
- "Just About Right" (middle box): 65% of respondents, Mean Overall Liking = 8.0.
- "Too Much" (top 2 box): 20% of respondents, Mean Overall Liking = 5.0.
Calculation:
- Penalty: Difference between "Just About Right" mean and "Too Much" mean.
- 8.0 - 5.0 = 3.0
- Weighted Penalty: Penalty multiplied by the "Too Much" percentage (as a decimal).
- 3.0 × 0.20 = 0.60
- Risk Check: Compare to the 9-point scale thresholds.
- The Critical Risk threshold is >0.50.
- Since 0.60 is greater than 0.50, the attribute is flagged as Critical Risk (red).
This shows how a 3.0-point liking drop for 20% of respondents results in a critical issue to address.
Customizing Your Analysis
Product Groups
You can customize which products appear in your Penalty Analysis using Product Groups functionality:
- Add or Remove Products: Select specific products to include or exclude from the analysis using the product selector
- Combine Products with Product Groups: Mix individual products with pre-defined Product Groups for comparative analysis
- Work with Filters: Use Product Groups in combination with demographic or behavioral filters to analyze specific respondent subsets
This gives you the flexibility to focus your penalty analysis on the most relevant product combinations for your research objectives.
Filtering Options
Apply demographic, behavioral, or custom filters to narrow your analysis to specific respondent groups. Filters work in combination with Product Groups and Segments to provide highly targeted penalty analysis insights.
Exporting Your Analysis
Download CSV
Choose the settings you want to export, then click the "Download CSV" button at the top right of the Penalty Analysis page to export your complete analysis. The CSV export includes:
- All penalty calculations and weighted penalties for each product and attribute
- Risk level flags (Moderate Risk and Critical Risk indicators)
- Mean Overall Liking scores for each JAR response group
- Percentage breakdowns for "Too Little," "Just About Right," and "Too Much" responses
- Brief calculation explanation at the bottom
The exported file respects all your current page settings, including selected products, Product Groups, and filters, giving you a customized dataset ready for external analysis tools.
Applying Insights
Use Penalty Analysis to identify which attributes drag down overall liking the most. High weighted penalties, especially in red, indicate areas where tweaking the product (e.g., adjusting flavor or texture) could boost customer satisfaction.
← Back to Insights Overview