Alienation Analysis

Overview

Alienation Analysis in Insights helps you assess the risk of losing current users due to product changes by comparing Overall Liking (Liking + Purchase Intent) between products. Available under the Analysis tab once surveys are closed and all responses are collected, this tool is designed for Alienation Assessor research projects. The analysis provides a tiered risk classification — Low, Moderate, or High — so you can quickly gauge the severity of potential user loss. You can analyze your full audience or focus on specific segments for more targeted insights.

Accessing Alienation Analysis

Alienation Analysis becomes available after your surveys close and fielding ends, ensuring all responses are in. It's exclusive to Alienation Assessor research, which requires a specific recruit: all participants must review both the control (the existing or baseline product) and the test (the new or modified product), and they must be current users of the control.

To access the feature, navigate to Insights > Analysis > Alienation Analysis. In the Settings sidebar, you'll configure your analysis using the following controls:

Control: Select the baseline product (the current or original product that serves as the baseline for comparison)
Test: Select the product you're evaluating (the new or modified product to see how it might affect user preferences and purchase intent)
Filter: Optionally select a segment to focus the analysis on a specific audience subset

Filtering by Segment

The segment filter enables you to focus your alienation risk assessment on specific audience subsets. This is particularly useful when projects contain multiple research types or diverse user populations.

To apply a segment filter:

In the Settings sidebar, locate the Filter dropdown
Select a segment from the list to filter responses
The analysis automatically refreshes to show results for only that audience subset
The sample size (n) and all statistics update to reflect the segmented population
Clear the filter to return to analyzing the full population

Tip: Create and manage segments via Insights > Segments. You can also click Create Segments below the filter dropdown to navigate directly to segment management.

When no segment is selected, the analysis includes all eligible responses. When a segment is selected, only responses from users matching the segment criteria are included.

Defining Overall Liking

Overall Liking scores determine the categories:

High Overall Liking: Indicates liking and purchase intent (e.g., top 4 box for a 9-point scale, top 3 box for 7-point, top 2 box for 5-point).
Low Overall Liking: Indicates dislike and no purchase intent (e.g., bottom 5 box for 9-point, bottom 4 box for 7-point, bottom 3 box for 5-point).

These thresholds help identify which users are satisfied or dissatisfied with each product.

Understanding the Analysis

Alienation Analysis uses Overall Liking scores to compare user satisfaction between the control and test products. It categorizes respondents into four groups based on their overall liking scores:

Liked: Both control and test are above neutral.
Disliked: Both control and test are neutral or below.
Alienated: Control is above neutral, test is neutral or below.
Noise: Control is neutral or below, test is above neutral.

Only participants who answered surveys for both the control and test products are included in the calculations. The analysis compares the Alienated and Noise groups to determine a tiered risk level.

Interpreting the Table

After selecting the control and test, you'll see a table breaking down the four categories. The table shows counts and proportions for Alienated, Noise, Liked, and Disliked groups, along with a total row displaying the total population size for context.

The table helps you see how many users fall into the Alienated group (those who like the control but not the test) compared to the Noise group (which reflects natural variability). This comparison drives the risk assessment.

When a segment is applied, all values in the table reflect only the segmented population. This allows you to compare alienation risk across different audience groups by running separate analyses for each segment.

Tip: Hover over the Response Category column header for a detailed explanation of how each group is defined.

Interpreting the Risk Verdict

Alienation Analysis provides a color-coded risk verdict based on the relationship between the Alienated and Noise percentages. The verdict uses a three-tiered classification:

Low Risk (green): Minimal potential for losing loyal customers, making the change relatively safe to consider.
Moderate Risk (amber): Some potential for losing loyal customers, suggesting further analysis may be warranted.
High Risk (red): Significant potential for losing loyal customers, suggesting the change may require further refinement or reconsideration.

How Risk Is Determined

The risk level depends on two factors: how the alienation percentage compares to noise, and the magnitude of alienation.

When alienation is higher than noise (more users are alienated than benefiting):

Low Risk: Less than 10% of users are alienated
Moderate Risk: 10% to 19% of users are alienated
High Risk: 20% or more of users are alienated

When alienation is equal to or lower than noise (natural variability accounts for the alienation):

Low Risk: Noise is at or below 10%, indicating stable preferences with minimal risk
Moderate Risk: Noise exceeds 10%, suggesting general preference instability that warrants further analysis

When alienation does not exceed noise, the maximum risk level is Moderate Risk, since the alienation observed can be attributed to natural variability rather than a genuine loss of loyalty.

Understanding the Verdict Description

The verdict includes a contextual description that varies based on whether alienation or noise is the dominant factor:

When alienation exceeds noise: The description reports the alienation percentage and explains what it means for customer retention. For example, "14% of [Control] loyalists are unlikely to purchase [Test]. This indicates some potential for losing loyal customers, suggesting further analysis may be warranted."
When noise is equal to or higher: The description highlights both percentages to show the cross-preference pattern. For example, "8% of [Control] loyalists are unlikely to purchase [Test], but 10% of [Test] loyalists show the same reluctance toward [Control]. Both levels are low, indicating stable preferences with minimal risk."

When a segment is selected, the verdict is calculated based only on the segmented population. This allows you to compare risk across different audience groups.

Applying the Insights

Use Alienation Analysis to prioritize decisions based on the risk tier:

Low Risk: The product change is relatively safe. Proceed with confidence while monitoring post-launch metrics.
Moderate Risk: The product change shows some potential for user loss. Consider conducting additional research — such as Penalty Analysis — to identify which specific attributes might be driving alienation before proceeding.
High Risk: The product change poses significant risk of losing loyal customers. Consider refining the product or testing alternative formulations before moving forward.

Example use cases for segment filtering:

Assess alienation risk specifically among "Early Adopters" or "Power Users"
Evaluate product changes separately for different demographic segments
Focus analysis on users who match specific behavioral criteria
Compare alienation across segments to identify which user groups are most impacted

← Back to Insights Overview