Blank cells may appear in the results dataset, a respondent may drop out of the survey, or skip a question about income, purchase frequency, or brand rating. Data imputation can make it possible to use the research material without prematurely discarding observations, but an incorrectly selected method can produce a result that only appears precise.
The key issue is not simply filling in a missing value, but determining what the missing data represent, where they came from, and whether their occurrence changes the picture of the study population. Only then can it be decided whether deleting the case, analyzing available responses, or imputing missing data is the safer option.
When does data imputation help, and when can it distort the result?
Data imputation is helpful when missing responses are limited, can be explained, and can be reliably estimated based on other information collected in the study. It does not involve guessing a respondent’s answer. It is a statistical procedure in which a blank value is replaced with a calculated or randomly drawn value in accordance with an adopted model that uses relationships present in the data.
In a survey project, missing data may result from different causes. A respondent may have accidentally skipped a question in an online survey, failed to remember specific information, considered the question too personal, or discontinued the interview before it was completed. These situations are not methodologically equivalent. If someone did not provide the number of purchases because they did not remember this information, using other questions describing their behavior may be considered. However, if they refused to answer a question about income, the missing value may be related to the income level itself or to attitudes toward disclosing data. A simple mean imputation may then conceal an important difference between respondents.
Three main missing data mechanisms are described in survey data analysis. The first is missing completely at random, where the probability of missingness depends neither on observed data nor on the missing value itself. The second is missing at random, where, after accounting for information available in the dataset, the probability of missingness does not depend on the unobserved value. For example, it may be related to age, the survey administration channel, or the answer to another question. The third mechanism occurs when the probability of missingness depends on the unobserved value even after available data have been taken into account. In practice, this is the most difficult situation because the data do not directly allow the hidden response to be reconstructed.
Data imputation is generally more justified for data missing at random than for missing data that depend on unobserved values. If some respondents did not answer a question about satisfaction, but their answers regarding service, price, product use, and willingness to recommend are known, an imputation model can use these relationships. However, if dissatisfied individuals were more likely to abandon the survey just before the satisfaction question, filling in data based on the remaining respondents may overstate the final rating.
It is worth distinguishing between a missing response for an individual variable and complete interview nonresponse. A single blank field can sometimes be filled in because other responses are available for that person. When a respondent did not take part in the study or an interview was rejected due to low quality, there are usually no responses on which to base imputation. Auxiliary data, for example from the sampling frame or the study administration system, may nevertheless be helpful. This phenomenon requires separate monitoring, including for non-response bias.
The most important principle is that imputation does not fix a weak research design or eliminate uncertainty. It can reduce losses resulting from missing responses, but only when the missingness mechanism has been identified and the method matches the type of variable and the purpose of the analysis.
How should a data imputation method be selected for the type of missingness and the research design?
The selection of a method should begin before running a statistical model. A missing data audit is needed, covering their extent, placement in the questionnaire, differences between respondent segments, and the relationship between blank responses and other variables. In an online survey, it is worth checking whether missing responses are concentrated around questions that require cognitive effort, are sensitive, or are displayed after a long series of questions. In telephone interviews, the way a question is asked or an interviewer’s decision to code an answer as “don’t know” may be important.
As Hume’s Institute experts point out, “before selecting an imputation method, it is necessary to determine why data are missing – accidental and intentional missingness require different approaches.” This principle protects against automatically applying a single technique to the entire dataset. A model may be computationally correct while still being inappropriate for the way missing responses arose.
The most commonly used solutions include simple methods and model-based methods. Simple methods can be useful at the working stage, but require caution in analyses reported to the client. Model-based methods may better preserve relationships between variables, but applying them requires data preparation and verification of assumptions. Single model-based imputation may still underestimate variability and fail to fully account for uncertainty associated with missing data.
Before selecting a solution, it is worth distinguishing between the following approaches:
- Deleting cases with missing data – involves analyzing only complete interviews or complete observations for a given model. It is transparent, but may substantially reduce the analytical material and lead to bias, especially when missing data are related to the variables being analyzed.
- Mean, median, or mode imputation – replaces a missing value with a typical value for the entire sample or a selected group. It may be useful as a supporting method for technical variables, but reduces variation in responses and underestimates variance.
- Hot deck imputation – a missing response is replaced with the response of a respondent with similar characteristics. The method is intuitive in survey research, provided that similarity criteria are well defined.
- Regression imputation – a value is predicted based on other variables, such as service ratings, frequency of product use, and respondent profile. It requires checking whether the model accurately describes the relationship present in the data.
- Multiple imputation – creates several versions of the dataset with different possible values for missing data and then combines the analysis results. It makes it possible to account for uncertainty related to completing missing data and is therefore generally safer than assigning a single value when the model assumptions are met.
A practical example: in a customer satisfaction study, some individuals skip a question about rating a mobile app, but it is known whether they use the app, how often they log in to their account, and how they rate other contact channels. First, individuals who do not use the app should be separated from those eligible to answer but leaving the question blank. The first group does not constitute missing data in the usual sense because the question does not apply to them. The second group requires an analysis of the missingness mechanism. Only then can data imputation be considered, using variables that describe service use.
Another case concerns a question about monthly spending. If a respondent selects a range category rather than providing an exact amount, this should not automatically be treated as a blank response. It is partial information that can be analyzed as a category or included in a model designed for interval-censored data. Converting it into a single amount without justification reduces data quality because it introduces an appearance of precision.
Validation is necessary after imputation has been performed. Variable distributions are compared before and after completion, relationships between key questions are checked, and it is assessed whether imputed values create unnatural clusters. In the case of multiple imputation, information should also be retained about the procedure, the number of datasets created, the variables used in the model, and the rules for combining results. Documentation is part of the research toolkit because it allows for a subsequent assessment of how results based on observed data differ from results that account for imputed missing data.
What most often harms data quality when imputing missing data?
The most common error is treating all blank fields as the same type of problem. A questionnaire may include “don’t know” responses, “not applicable” responses, refusals, technical skips, and discontinued interviews. Each of these codes has a different meaning. “Not applicable” often results from survey logic and should not be imputed. “Don’t know” may describe a genuine lack of respondent knowledge, which can itself be an important study finding. A refusal, in turn, may indicate that the question is sensitive.
Another pitfall is mean imputation for opinion variables and rating scales. When an answer is missing for a question rated on a scale, entering the mean causes artificial middle responses to be added to the dataset. Result dispersion decreases, and differences between groups may be smoothed out. In a brand study, this may create the impression of greater agreement in opinions than actually exists among respondents.
Data imputation without considering the structure of the questionnaire is also problematic. For example, a question about rating the complaint handling process should be asked only of individuals who have filed a complaint. Filling in responses for people without such experience would be a definitional error rather than a technical one. Before analyzing missing data, filters, conditional questions, and survey routing must be checked.
It is also worth avoiding a situation in which the same dataset is used to build an imputation model and then to draw overly categorical conclusions without indicating the extent of uncertainty. Imputed values are estimates, not recovered observations. Reporting should be transparent about which variables were completed, which method was used, and whether results change materially after applying an alternative procedure.
Another error is confusing imputation with weighting. Data weighting corrects the share of specific groups in the sample, whereas missing data imputation fills in missing values in responses. Both procedures may be used in a single project, but they address different sources of error. Their order and interaction should be planned by the analyst rather than determined automatically by a data analysis program.
It can also be risky to use imputation for a variable that is the main study outcome without conducting a sensitivity analysis. If a key indicator is based on a question with a large number of refusals, it is advisable to compare the result for observed data, results after different imputation methods, and conservative scenarios. The aim is not to select the most favorable result, but to assess whether the conclusion remains stable despite missing responses.
Data quality does not automatically improve after blank cells are filled. Sometimes it is more reliable to leave the value missing, limit the analysis to an appropriate subgroup, or present the result with a clear caveat. Imputation is a tool for addressing a specific problem, not a mandatory stage of every survey data analysis.
How should missing data be reviewed before analyzing results?
The review should begin before tables and models are created. This makes it possible to separate technical errors from genuine missing responses and determine which variables require particular caution. A good practice is to prepare a missingness map for the entire dataset and for key analytical segments.
The following sequence of work is useful in a research project:
- Check response codes and distinguish refusals, “don’t know” responses, out-of-range responses, and missingness resulting from a filter.
- Assess which questions contain missing data and whether it is concentrated at a particular stage of the survey.
- Compare respondents with and without missing data in terms of available characteristics, such as profile, method of participation in the study, and answers to related questions.
- Determine whether deleting cases changes the composition of the analyzed group or limits the ability to conduct the planned analyses.
- Select a data imputation method appropriate for the type of variable, the missingness mechanism, and the analytical purpose.
- Validate the result and document the procedure in the analytical documentation.
Such a review also makes it possible to identify fieldwork issues. If missing responses occur more often in one data collection channel or with a specific version of the questionnaire, the cause may lie in study implementation rather than respondent behavior. In that case, data imputation should not replace an assessment of the data collection process.
The purpose of the analysis should also be considered. For a simple description of response distributions, an analysis of available cases with a clear indication of the percentage base may be sufficient, provided that the missingness mechanism does not materially distort the distribution being described. Modeling relationships between multiple variables more often requires a procedure that reduces the loss of observations and accounts for uncertainty. The same variable may therefore be handled differently in a descriptive table and in a statistical model, provided that the decision is justified and transparent.
In research practice, the most useful principle is proportionality: the more important a variable is to the study objective and the more missing responses may change the interpretation of the result, the more thorough the assessment should be. The goal is not to impute everything, but to understand what each missing value represents and how it affects the final analysis.
Frequently asked questions
What is data imputation?
Data imputation is a procedure for replacing a missing value with an estimate derived from other information available in the dataset. It may take a simple form, such as assigning the median, or use a statistical model and multiple auxiliary variables. Its purpose is to limit the effects of missing responses, not to pretend that the missingness never occurred.
When can cases with missing data be deleted?
Deleting cases is acceptable when missing data are limited and are not related to respondent characteristics or the outcome being analyzed. Before making such a decision, the removed individuals should be compared with those retained in the analysis. If they differ in important variables, deletion may introduce greater bias than cautious data imputation.
What imputation methods are used in survey research?
Survey data analysis uses, among other methods, median or mode imputation, hot deck imputation, regression models, and multiple imputation. The choice depends on the variable type, the missing data mechanism, and the intended use of the result. Categorical questions require methods that preserve the category structure, while scales and numerical variables additionally require control of the distribution and relationships with other responses.
Ask about statistical analysis of data from your study
Hume’s Institute helps diagnose missing data, select imputation procedures, and document their impact on analysis results. Contact makes it possible to determine the scope of work appropriate to the questionnaire design and the available research material.