Data gaps in quantitative data analysis - what are they and how to deal with them?

Basic types of data deficiencies

The most common division of data deficiencies found in the literature relates to whether or not they are the result of chance - whether they are random or not, and whether other factors influence their occurrence. Identifying the mechanisms of their occurrence and indicating which type of deficiencies the analyst is dealing with is important in order to choose appropriate methods to deal with them.

We divide data deficiencies into:

1. MCAR (Missing completely at random): missing data are completely random and do not depend on any other values in the data set. This means that the probability of missing data is the same for all observations, regardless of the values of both observed and unobserved data.
   Let us look at Table 1, in which we have several variables: age, education and income. Missing data in the ‘Age’ column could be an example of MCAR, assuming that omitting this value is completely random and does not depend on other variables in the dataset. An example would be when a respondent accidentally omits an answer to an age question in a survey.
2. MAR (Missing at random): missing information is random, but its occurrence may depend on other observed data in the set, but not on the value of the missing data.
   Incomplete information in the income column may be an example of MAR-type missing values if we assume that the probability of skipping an income question increases with the age of the respondents. However, the probability of missing data itself does not depend directly on unknown income values.
3. MNAR (Missing not at Random): Missing data are not random and their probability of occurrence depends on the value of the missing data in themselves.
   Incomplete information in the ‘Education’ column could be an example of MNAR if we assume that the lower the education, the more likely it is that education data will not be provided. In this case, the missing value is directly related to the value that should be given, introducing a systematic error in the missing data.

 

Table 1. The table presents sample data with data gaps that can be regarded as MCAR, MAR and MNAR gaps.

No	Age	Education	Income
1	25	NA	4 000
2	NA	NA	4 500
3	35	Secondary	NA
4	45	Secondary	5 500
5	55	Secondary	NA

* NA (Not Available)

Recognising the type of data gaps in the dataset being analysed is key to choosing the right method to handle them. Each requires a different strategy in order for the analysis to be reliable and the conclusions to be correct and qualitative.

Methods of dealing with data gaps

The ways of dealing with data gaps range from very simple approaches to complex imputation methods. Each type mentioned has its own advantages and limitations. One basic approach is to delete or omit data where there are gaps. In contrast, we can count data imputation among the more complex methods that are often used. Both statistical models and machine learning algorithms require careful selection and validation. This is needed to ensure that the imputation process does not introduce additional errors or biases. It is also crucial to understand the mechanism that led to the missing data, as different mechanisms may require different coping strategies.

Popular methods for dealing with missing data:

1. Deleting and skipping data: when deleting observations with missing data, they are removed from the analysed dataset. In contrast, with data skipping, observations with missing data remain in the dataset but are not considered in the analysis, allowing the analyst to retain the option of using them in further research. This method may lead to a loss of information, and the statistical model created on such data may be inaccurate. The approach is only valid if the gaps are of the MCAR type. If the missing data are not completely random, it is not safe to remove observations with missing values or substitute missing data one at a time. In such situations, multiple imputation should be used. In an MCAR situation with a small number of missing data, excluding units of observations from the analysis will not lead to a significant burden on parameter estimation and can be used.
2. Single imputation: missing data can be replaced by values calculated from existing data. Missing values are replaced by a constant value, e.g. the mean, median or dominant for a given variable. They can also be predicted by linear regression or other regression techniques from the available data. It is worth bearing in mind, however, that substituting data from the mean or median may result in a skewed distribution and unrepresentative data, especially if the data are not missing at random.

3. Multiple imputation: for each missing value, several possible substitute values are generated, thus creating multiple complete data sets. Each is then analysed separately, and the results of these analyses are combined to produce final estimates and inferences that take into account the uncertainty associated with the missing data. The advantage of multiple imputation is that it allows the estimation of not only the value of missing data, but also the uncertainty associated with these estimates. This approach makes the results of the analyses reliable and less prone to potential errors due to the arbitrary selection of a single value for missing data. Multiple imputation is particularly useful in studies where missing data are unavoidable and their omission or inappropriate handling could lead to erroneous conclusions. This method takes into account the interdependencies between variables in the dataset, allowing for accurate estimation of missing values.

 

PS IMAGO PRO also offers a fully automatic multiple imputation mode that selects the most appropriate imputation method based on the characteristics of the data, while leaving the user free to customise the model.

This procedure generates several possible values for missing data and creates several complete sets. For each of these, the user receives a summary of the results and one mixed result (from all the created sets).

The Multiple Imputation procedure, is useful when the missing data are not completely random and makes it possible to obtain:

• the most accurate substitution of missing data through multiple imputation,
• automation of the imputation process,
• visualisation and diagnostics of missing data patterns to understand the patterns of missing data.

The use of PS IMAGO PRO in dealing with missing data significantly increases the accuracy of analyses in the presence of incomplete information, automating the imputation process and preserving key statistical assumptions. Missing value analysis and Multiple imputation reduce data preparation time and facilitate the assessment of imputation quality through detailed reports. This approach not only facilitates accurate estimations, but also helps in the clear interpretation and effective communication of research results.