The decision about how many respondents to invite to a study is more often driven by budget pressures than methodology – and this is precisely when the risk arises that the findings will look impressive in a presentation but prove useless for business decision-making. Sample size is not a matter of intuition or a round number in the brief, but the result of the project’s objective, population structure, and acceptable margin of error. The text below explains what the number of respondents truly depends on and how to avoid the trap of “a sample that is too small for an analysis that is too broad.”<\/p>\n
Sample size determines two things at once: the precision of estimates (that is, how narrow the interval containing the true population value is) and the ability to conduct subgroup analyses – comparisons across segments, regions, and age groups. A sample that is sufficient for reporting an overall result will very often prove too small when the client wants to see the distribution of responses across subgroups. This is the most common point at which a research project breaks down analytically, even though it has formally been delivered in line with the brief.<\/p>\n
The second dimension is sample representativeness. Representativeness does not result from size alone – a sample of several thousand people recruited exclusively through a single online channel may still be highly biased. Representativeness is determined by the sampling method: whether every member of the population had a known, non-zero probability of being included in the sample, whether the demographic structure reflects that of the population, and whether appropriate analytical weights were applied. This is why the question “how many respondents” must always be asked alongside the question “which respondents, and from where.”<\/p>\n
The third factor is the nature of the phenomenon being studied. The rarer the behavior or attitude, the larger the sample needed to capture it. If the proportion of interest in the population is only a few percent, a sample of several hundred people will produce a confidence interval so wide that the result loses its value for decision-making. Conversely, for widespread phenomena, where responses are distributed close to fifty-fifty, the same sample size may be sufficient in terms of the maximum error for estimating a proportion. Sample size should therefore always follow from the research question, not the other way around.<\/p>\n
The conventional approach to sample size is based on four parameters: the expected confidence level (most commonly set at ninety-five percent), the acceptable margin of sampling error (usually around a few percentage points), the assumed response distribution, and the population size. The lower the acceptable margin of sampling error and the higher the confidence level, the larger the required sample – although this relationship is not linear: halving the error requires approximately quadrupling the sample size.<\/p>\n
However, the formula itself is only a starting point. In project practice, what matters is which analyses will be conducted using the data. If the report is to include comparisons across five customer segments, each of them must have a sample size that supports meaningful inference – which means that the total sample grows several times beyond the minimum calculated for the overall population. The same applies to multivariate analyses: regressions, factor analyses, and segmentations each have their own requirements regarding the minimum number of observations per variable.<\/p>\n
Sampling is the second pillar of the decision. The most commonly used approaches are:<\/p>\n
The choice of method depends on the availability of a sampling frame, the nature of the population, and the type of conclusions to be drawn. In B2B research, where the population may be small and highly heterogeneous, random sampling is often difficult to implement – and in such cases, a well-designed purposive sample may provide greater insight than a formally correct but unfeasible random sample.<\/p>\n
As Hume’s Institute experts point out, “more respondents” does not always mean “better data” – what matters is who is asked, not only how many people are asked. A sample of 2,000 people recruited from a single consumer panel can produce a more biased picture than a carefully designed sample of several hundred respondents recruited from different sources, with structure controls and post-stratification weights. Sample size is a necessary condition for reliability, but never a sufficient one.<\/p>\n
In mixed-methods projects, the quantitative sample size is often determined in parallel with the plan for the qualitative component. In-depth interviews or focus groups can identify variables worth controlling for in the quantitative study, which in turn affects the structure of the strata and the final sample size. This linkage reduces the risk that the sample will be selected for a hypothesis that proves inadequate to the actual market structure.<\/p>\n
The first and most common mistake is confusing sample size with sample representativeness. Size answers the question “how precisely,” while structure answers the question “how accurately.” A sample of 10,000 people from a single recruitment source is not representative of the general population, regardless of how impressive it looks in a report headline. Conversely, a sample of several hundred respondents selected in line with the population structure and appropriately weighted can provide estimates with comparable diagnostic value.<\/p>\n
The second mistake is ignoring the effects of clustering and weighting. When a sample is drawn in clusters (for example, by facilities, municipalities, or companies), the effective sample size is lower than the nominal size – because respondents within the same group are more similar to one another than would result from pure random sampling. Failing to account for this adjustment leads to underestimated confidence intervals and overestimated certainty in the conclusions.<\/p>\n
The third mistake concerns subgroup analyses. Clients often request “cross-sections” for segments whose nominal size is only a few dozen people. Inference from such small subsamples is subject to a very wide margin of sampling error – wide enough that differences between segments often become statistically indistinguishable from noise. This is a trap that even experienced analysts fall into when pressure for a “rich report” outweighs methodological discipline.<\/p>\n
The fourth mistake is overinterpreting non-representative samples. Social media polls, website surveys, or quick questions in a newsletter have their value – but it is exploratory rather than confirmatory value. Treating them as the basis for firm conclusions about the entire population is a methodological error, regardless of how many respondents took part.<\/p>\n
The fifth mistake, characteristic of B2B projects, is forcing random sampling where the population is so small and specific that a purposive or expert approach is more appropriate. Sample representativeness in the classical statistical sense then ceases to be an adequate criterion – it is replaced by the validity of selecting key informants and informational saturation.<\/p>\n
There is no universal answer – sample size depends on the study objective, population structure, planned subgroup analyses, and the acceptable margin of sampling error. In general population studies focused on a single metric, samples numbering in the hundreds are often used, but when the report is to include segment comparisons, the required sample size increases severalfold. It is worth defining the analytical questions first and calculating the sample size only afterward.<\/p>\n
A confidence level is a parameter of the interval estimation procedure: at a confidence level of ninety-five percent, in a long series of repetitions of a study conducted using the same method, approximately that proportion of confidence intervals would contain the true population value. A higher confidence level provides more conservative inference but requires a larger sample or a wider error interval.<\/p>\n
Purposive sampling is used where the population is small, hard to reach, or highly heterogeneous in qualitative terms – typically in B2B research, expert research, qualitative research, and case studies. Its advantage is the precise alignment of respondents with the research question, at the cost of being unable to make formal statistical generalizations. In mixed-methods projects, both approaches are often combined to achieve both depth and measurability.<\/p>\n