Survey fatigue reduces financial literacy scores by 5-15 percent
A Bank of Canada staff working paper finds that survey fatigue significantly reduces financial literacy scores. When financial literacy questions are placed at the end of a survey, respondents are 5-15 percent more likely to answer 'Don't know' instead of providing correct responses.
Question placement impacts accuracy
A Bank of Canada (BOC) staff working paper investigates how survey fatigue affects financial literacy measurements.
Using a randomized A/B experiment embedded in the November 2020 Cash Alternative Survey, researchers placed the 'Big Three' financial literacy questions either at the beginning or near the end of a longer survey.
The study reveals that when these questions appear at the end, respondents are significantly more prone to selecting 'Don't know' answers.
This shift primarily comes at the expense of correct responses, leading to a measurable drop in financial literacy scores by 5 to 15 percent.
The findings highlight that survey design, specifically question placement, can causally influence reported financial literacy levels.
Fatigue disproportionately affects vulnerable groups
The research details that the impact of survey fatigue is not uniform across all demographics.
The increase in 'Don't know' responses is particularly pronounced among women, younger adults, and individuals with lower educational attainment.
University-educated participants show less susceptibility.
Notably, the study finds that the rate of incorrect answers remains stable across experimental groups, suggesting that incorrect responses might offer a more robust benchmark for cross-survey comparisons.
The methodology involved non-probability quota sampling, with subsequent weighting and benchmarking against a probability survey to address potential biases.
Refining financial literacy measurement
This study provides a crucial methodological insight for financial literacy surveys, highlighting the need to account for survey design.
Its findings suggest that raw financial literacy scores may underestimate actual knowledge, particularly for certain demographics.
This calls for a re-evaluation of how cross-country comparisons and trend analyses are conducted.