Causes and effects of measurement errors in educational attainment

From 2002 – 2016 Statistics Norway conducted the fieldwork in the Norwegian edition of The European Social Survey (ESS). ESS is a cross-national survey program that has been conducted every two years since 2002, each round covering around 25 European countries. The main aims are to describe stability and change in social structure, social conditions, and attitudes in Europe. The samples are representative of all persons aged 15 and above resident within private households in each country, regardless of their nationality, citizenship or language. Individuals are selected by strict random probability methods at every stage. All the interviews are conducted in face-to-face mode.

The final product of the ESS is a public use file that everybody can download. Due to the Statistics Act of Norway the policy of Statistics Norway prohibits to give information from administrative registers without some terms. The policy is that public use files only contains survey information from the questionnaire (except for gender, age and residence/region). In the Norwegian edition of the European Social Survey (N-ESS) the respondents are asked about their educational attainment instead of merging it from a register. For quality control or research purposes inside Statistics Norway it is in accordance with the policy to merge administrative information to any given survey. As part of the internal process quality control of the data collection of the N-ESS all the elements of the sample were merged to relevant administrative registers. Hence N-ESS offer a unique possibility to study data quality with respect to educational attainment. We have two independent sources of information for all respondents and can study the effects of non- sampling errors in surveys.

In this document we examine the agreement between level of education as measured both in surveys and from register information and to examine whether these two measurements of education would give similar outcomes in regression analyses of a set of dependent variables. The analysis was based on Rounds 5–8 of the Norwegian part of the ESS (N-ESS) combined with register data on education from Statistics Norway. We examined the differences between the two measurements of education both with crosstabulation and with a multinomial regression analysis of a threefold classification of agreement. Finally, seven dependent variables that varied by source, number of levels, in continuous and categorical versions, and a set of control variables were regressed on education. The agreement rate for the total sample aged 15–104 was 62.4%, or 66.3% after excluding cases with missing values, and the rate increased with the aggregation of education. The multinomial analysis showed that agreement varied by age, mostly due to the low agreement rate for young respondents, who also showed the highest propensity to report a higher education than that shown in the register measurement. The validation analysis indicated that the differences in the effects of education were small, and even in a few comparisons with significant outcomes, these differences would not lead to different substantial conclusions. In conclusion, our main expectations were confirmed: the agreement between the survey and the register measurements was relatively high, and the differences were mainly located in the adjacent categories. The agreement was highest for the middle-aged respondents and lowest for the youngest respondents. The tendency to report a higher level of education than that in the register measurement was negatively related to age, with an opposite relationship for reporting a lower level of education than that in the register measurement. Finally, the validation analysis indicated that the measurements based on the survey and the register measurements in most instances may be used interchangeably. However, in projects targeting young people, a survey measurement of education is recommended because of the time lag of the register information.

 

Authors: Øyvin Kleven and Kristen Ringdal