Effect size is a measure of the magnitude of an observed effect: the practical scale of a difference, association, or change detected in data. Unlike statistical significance alone, it helps assess whether a result may matter for a business, marketing, or research decision.
In market research, effect size helps distinguish a formally detected difference from one that may actually influence customer behaviour, brand perception, sales, or product choice.
What is effect size?
Effect size is a statistic that describes the strength of a phenomenon observed in a study. It can represent the scale of a difference between groups, the strength of an association between variables, the extent of change over time, or a model’s ability to explain variation in an outcome.
In effect size in statistics, it does not answer whether a result could have occurred by chance under the assumptions of a given statistical test. That is the role of p-values and significance tests. Effect size instead describes the estimated magnitude of a difference or association; whether that magnitude is practically important depends on the research and decision-making context.
For example, a study may find a difference in stated purchase intention between audiences exposed to two advertising messages. A significance test may indicate that the difference is unlikely under the null hypothesis, but effect size helps assess whether the advantage of one message is marginal, moderate, or large enough to justify changing the campaign.
The choice of measure depends on the type of data and the research question. Common measures include:
- Cohen’s d – for assessing the standardised difference in means between two groups, for example, the average concept rating among customers in two segments;
- Pearson’s r – for describing the direction and strength of an association between two quantitative variables, for example, customer satisfaction and intention to repurchase;
- odds ratio – for comparing the odds of a particular behaviour between groups, for example, purchase after exposure to a campaign versus no such exposure;
- eta squared or partial eta squared – for assessing the proportion of variability associated with a factor in analysis of variance, with partial eta squared accounting for other effects in the model;
- R squared – for assessing the proportion of variation in the outcome explained by a regression model.
Effect size can be reported in the original units of measurement, for example as a difference in average ratings, or in standardised form. Standardised measures facilitate comparisons across questions, groups, and studies, but require careful interpretation in the context of a specific market category.
Using effect size in practice
Effect size is especially useful when research findings are intended to support a choice between alternative actions. It is used by market researchers, data analysts, marketing teams, product managers, and people responsible for offer development and customer experience.
In quantitative research, effect size supports the assessment of whether differences between segments are worth using operationally. If two segments evaluate price, packaging, or brand messaging differently, statistical significance alone does not determine whether separate communication is justified. Effect size helps determine whether the difference is sufficiently large to support campaign personalisation, product modification, or a change in sales priorities.
Typical applications include:
- comparing the performance of advertising creatives, landing pages, or promotional offers in A/B tests;
- assessing the impact of a campaign on brand awareness, purchase consideration, or other brand-tracking metrics;
- analysing differences between current, lapsed, and prospective customers;
- measuring associations between service quality, satisfaction, loyalty, and recommendation;
- assessing the impact of price changes, product features, or subscription models on purchasing decisions;
- comparing results before and after implementing a new solution in customer-experience research.
In B2B projects, effect size can indicate whether a difference in supplier evaluation between purchasing decision-makers and solution users is practically meaningful. In B2C research, it can help assess whether a new packaging line meaningfully changes product perception or purchase intention.
In mixed-methods research, quantitative effect-size results can be considered alongside qualitative material. In-depth interviews, focus groups, or analysis of open-ended responses can help explain why an observed effect occurred, which mechanisms may underlie it, and for which audiences it matters most.
Effect size and related methods
Effect size is part of statistical interpretation, not a standalone data-collection method. It is most often used alongside hypothesis tests, confidence intervals, regression analysis, analysis of variance, and experiments.
It is important to distinguish effect size from statistical significance. Statistical significance depends, among other things, on sample size and data variability. With a large sample, even a very small difference may be statistically significant. Conversely, an effect with potentially important business implications may not reach statistical significance in a small or highly variable sample. Therefore, sound interpretation should consider both elements.
Effect size should also be distinguished from business significance. A large statistical effect does not always mean high economic value, just as a relatively small effect may matter in a category with high sales volume, high margins, or strategic importance. Assessment should consider market context, the cost of implementing a change, the size of the target group, and the potential impact on key metrics.
In practice, effect size is often presented with confidence intervals. This combination helps assess not only the estimated magnitude of an effect but also the precision of that estimate. A wide interval indicates greater uncertainty, which is particularly important when analysing niche segments, small customer groups, or rare purchasing behaviours.
How to interpret effect size in research?
The question of how to interpret effect size in research has no single universal answer, because appropriate interpretation depends on the measure used, the study design, data quality, and the decision the analysis is intended to support. Common interpretation thresholds can provide a reference point, but they should not replace knowledge of the category being studied.
When interpreting effect size, it is useful to follow several steps. First, determine exactly what the statistic describes: a difference in means, the strength of an association, change over time, or a model’s explanatory or predictive performance. Then relate the result to the natural scale of the phenomenon, previous measurements, results for competing alternatives, and the business objective of the project.
The following questions can be helpful:
- Is the observed difference noticeable to a customer or user?
- Does the magnitude of the effect justify the cost of implementing the recommended change?
- Does the effect occur across the whole population, or only in a particular segment?
- Is the result stable after accounting for other variables, such as price, customer tenure, purchase channel, or company profile?
- How precise is the estimate, and do the data support limiting the risk of overinterpretation?
It is also worth remembering that effect size does not automatically demonstrate a causal relationship. A causal conclusion is justified primarily when the study design allows control of confounding factors, for example in an experiment, a randomised A/B test, or an appropriately designed quasi-experimental study.
Reliable reporting of effect size increases the usefulness of market research because it shifts attention from the question of whether a difference exists to the question of its real magnitude. As a result, marketing, product, and sales decisions can be based not only on the formal result of a test but also on an assessment of its practical value.