Designing Audience & Sample Size

Designing the right audience and sample size is one of the most important parts of product research. Audience definition and sample size are not independent decisions; they are tightly linked to your research objective. When either is misaligned, even technically sound data can lead to misleading conclusions.

This article is designed as practical guidance for designing sound research when testing with Highlight. You can leverage Highlight to better understand your target audience, optimize your product, make a go/no product decision, gather ratings and reviews, and more. This guide helps you decide for your specific use case who to talk to, how many people you need in your test, what tradeoffs you’re making, and when results are “good enough.”

Step 1: Start With the Decision You’re Trying to Make

Instead of asking:

“How many respondents do we need?”

Ask:

“What decision will this research support?”

Your research objective should directly determine:

Who to include: how broad or narrow the audience should be
What level of precision is needed: directional vs. validation
How results will be interpreted and used

Common Research Objectives

Stage	Objective	Methodology	Typical Precision Needed
Early	Concept testing	Quantitative	Lower; directional
Early	Product learning	Qualitative	Lower; diagnostic
Nearing launch	Product validation	Quantitative	Higher, go/no go decision
Nearing launch, or post	Competitive benchmarking	Quantitative	Higher, stable benchmarking
Nearing launch, or post	Claims substantiation	Quantitative	Higher, messaging validation
Post-launch	Line extension learning with users	Qualitative	Moderate; specific & opinion-rich

Step 2: Choose the Right Audience

Below are the most common CPG research audiences — from broadest to narrowest. In most cases, broader is better unless there’s a clear reason to narrow.

Audience Type	Who They Are	Best Used For	When Not to Use!
Gen pop	General adult population, regardless of category usage	Quick pulse surveys White-space exploration Category entry potential Claim comprehension testing	Product performance testing Situations requiring real usage context
Category purchasers	People who purchased/used the category in a defined timeframe (e.g., deodorant buyers past 6 months)	Need states & occasions Competitive benchmarking Early-stage concept screening Prototype testing Pre-launch validation	Understanding current user behaviors Highly technical product evaluation of a new versus old formula, such as Alienation Testing
Product concept acceptors	Category purchases who are willing to use a product like the one being tested	Need states & occasions Competitive benchmarking Early-stage concept screening Prototype testing Pre-launch validation	Understanding current user behaviors Highly technical product evaluation of a new versus old formula, such as Alienation Testing
Brand-aware consumers	People familiar with the brand (regardless of usage)	Message testing Line extension	Category-expanding innovation that requires new/prospective users
Current product users/loyalists	People who use or have used a specific product type/format in the past	Alienation testing Line extension Usage and satisfaction	Category-expanding innovation that requires new/prospective users

Common Mistakes to Avoid

Using brand loyalists to estimate total demand or interest in a new product or concept
Assuming bigger samples solve poor targeting or audience definition
Over-screening early-stage innovation and narrowing your audience too early

Understanding Incidence Rate

The narrower your audience, the lower your incidence rate — which increases cost and timing risk. Incidence rate (or IR) is the percentage of people who qualify for your study out of everyone you ask. It tells you how common your target audience is within the broader population you’re sampling from. If you send survey to 1,000 people and 300 qualify by falling into your audience definition, your Incidence Rate = 30%. That means 3 out of every 10 people you contact are eligible.

A high incidence audience would be adults 18–65 who purchased shampoo in past 6 months, representing 60%+ of the total US adult population. A low incidence audience would be women 25–40 who bought sulfate-free shampoo in the past month, only buy salon brands, and have color-treated hair.

Lower IR = higher recruitment cost + longer timelines! You can use the Highlight platform to estimate your incidence rate and feasibility of your test by filling out the audience details of your project.

Step 3: Determine Sample Size (aka “base size” or “n size”)

Now that you know who you want to test with, you can determine how many testers you need.

Think about:

Is this high-risk or early learning?
Do I need direction or validation?
Am I comparing product to product, or audience to audience?
For physical product tests, how many samples or physical products can I produce for testing?
Are there any subgroups I’m interested in, such as looking at those aware of my brand vs those unaware?

Practical Guidelines

Physical Product Tests or IHUTs
Sample Size	Best For
50 - <100n per product	Early product learning, directional reads Overall impressions Single audience, no deep audience segmentation
100- <200n per product	Product optimization tests Light subgroup analysis Balanced precision and feasibility
200n+	Validation Reliable subgroup comparisons Benchmarking

Digital Tests or Those with survey only, no product sent
Sample Size	Best For
100-200n	Early concept or audience learning, directional reads Overall impressions Single audience, no deep audience segmentation
200- <500n	Concept or audience optimization tests Light subgroup analysis Balanced precision and feasibility
500n+	Concept or audience validation Reliable subgroup comparisons Benchmarking

Minimum Sample Sizes

The sample size ranges above reflect Highlight’s recommendations for physical and digital research. That said, you may increase or decrease your sample size depending on your study objectives, budget, timeline, and risk tolerance. Below are the minimum sample sizes we recommend to maintain various levels of statistical rigor.

Highlight Minimum Recommendations

Confidence intervals reflect the long-run success rate of the method. For example at 95% Confidence, if we were to test our product with many subsets of our total population, about 95% of those intervals would contain the true population parameter. This reflects how confident we are in the method over repeated tests, and its ability to yield an accurate result when working from a sample of consumers rather than an entire population.

80 or 85% Confidence Interval (Directional Reads)
- 50n per study minimum
- 25n per subgroup
90% Confidence Interval (Moderate Precision)
- 100n per study minimum
- 50n per subgroup
95% Confidence Interval (High Precision)
- 200n per study minimum
- 75n per subgroup

Think of this as a sliding scale:

Higher confidence → Larger base sizes → Greater precision
Lower confidence → Smaller base sizes → Directional guidance

The key is aligning your base size with the decision risk. Not every study requires 95% confidence — but high-stakes decisions often do.

Applied Examples: Audience & Sample Design in Practice

The scenarios below illustrate how objective, audience, and sample size choices work together in real-world research — including the risks and tradeoffs involved.

Scenario	Primary Objective	Audience Definition	Sample Size	Risk Level	Why this risk level	Key Tradeoff
Prototype IHUT Prototypes Head to Head	Identify top 2 performers to advance	Product concept acceptors (no brand requirement)	75-100n per product	Low-moderate	No external benchmark; comparative ranking vs. validation	Gain clarity on relative strengths Cannot infer market readiness
Validation HUT Reformulation vs. Category Leader	Determine parity or superiority vs. category leader in real-world use	Regular users of product; exclude strong category-leader loyalists	150-200n per product	High	Strong benchmark anchor Small differences matter Real-world variance increases noise	Gain realism & credibility Lose tolerance for weakness
Digital Concept Test Packcept Evaluation	Understand perceived value & purchase intent of 5 packcepts	Category purchasers	100n per concept	Low	Hypothetical evaluation No real product experience required	Gain speed & scalability Lose behavioral confirmation

Final Takeaways!
All research is constrained by budget, timelines, incidence, recruitment feasibility, and operational realities. Running good research is not about eliminating constraints — it’s about making intentional decisions within them.

You may need to reduce sample size because you can’t produce enough units for testing. That’s okay! You can compensate with richer qualitative inputs — open ends, photos, videos — to increase depth of understanding even if breadth is limited. Tradeoffs don’t weaken research when they’re deliberate. They strengthen it!

Before launching your study on Highlight, ask yourself:

What decision will this confidently support?
Where might results skew — and in what direction?
What conclusions should not be drawn from this data?

When you can answer those clearly, you’re ready.

Use this guide as your blueprint to test with confidence. And when you’re ready — launch on Highlight and turn smart design into decisive momentum.