Bias in Online Health Surveys: Identifying and Overcoming Challenges
DOI:
https://doi.org/10.15503/emet2024.20.28Keywords:
online surveys, bias, online research, data validity, bias mitigationAbstract
Aim. Online surveys have become a vital tool in health research due to their cost- effectiveness, speed, and global reach. However, their methodological limitations – par- ticularly various forms of bias – may affect the validity and generalisability of findings. This review aims to provide an overview of the most common types of bias in online health surveys and to discuss strategies for their mitigation.
Methods. A narrative literature review was conducted to identify and categorise the primary sources of bias associated with online survey research in health-related contexts. The review focused on three main areas: survey design and presentation bias, cognitive and psychological response biases, and selection bias. Studies addressing these issues were examined to extract examples and recommended mitigation approaches.
Results. The review identified several key sources of bias. Poor survey design, exces- sive length, or misleading visual formatting – was associated with reduced data quality and increased dropout rates. Response biases, such as social desirability, recall errors, or avoidance of sensitive questions, were found to skew data, particularly in self- reported health measures. Selection bias emerged as a major concern, as access to and familiarity with digital technologies significantly influences who participates in online surveys. Indi- viduals with poorer health or limited digital literacy are often underrepresented.
Conclusions. Despite their numerous advantages, online health surveys are vulner- able to various forms of bias that can compromise data validity. Researchers must care- fully consider survey design, sample recruitment, and respondent characteristics. Apply- ing strategies such as pre-testing, use of clear language, adaptive sampling techniques, and bias correction methods can enhance the quality of online survey data. When ethical standards are upheld and methodological rigour is applied, online surveys can remain a powerful and reliable tool in public health research.
Downloads
References
Bethlehem, J. (2010). Selection bias in web surveys. International Statistical Review, 78(2). 161–188. doi: 10.1111/j.1751-5823.2010.00112.x.
Cantuaria, M. L., & Blanes-Vidal, V. (2019). Self-reported data in environmental health studies: Mail vs. web-based surveys. BMC Medical Research Methodology, 19, 238. doi 10.1186/s12874-019-0882-x. Centers for Disease Control and Prevention. (2013). The BRFSS data user guide. Centers for Disease Control and Prevention. Retrieved from https://www.cdc.gov/brfss/data_documentation/
index.htm
Choi, B. C., & Pak, A. W. (2005). A catalog of biases in questionnaires. Preventing chronic disease, 2(1), A13. Evenson, K. R., Goto, M. M., & Furberg, R. D. (2015). Systematic review of the validity and reliability
of consumer-wearable activity trackers. International Journal of Behavioral Nutrition and Physi-
cal Activity, 12, 159. doi: 10.1186/s12966-015-0314-1.
Eysenbach, G. (2004). Improving the quality of web surveys: The Checklist for Reporting Results
of Internet E-Surveys (CHERRIES). Journal of Medical Internet Research, 6(3), e34. doi: 10.2196/
jmir.6.3.e34.
Eysenbach, G., & Wyatt, J. (2002). Using the Internet for surveys and health research. Journal of
Medical Internet Research, 4(2), E13. doi: 10.2196/jmir.4.2.e13.
Geldsetzer, P. (2020). Use of rapid online surveys to assess people’s perceptions during infectious
disease outbreaks: A cross-sectional survey on COVID-19. Journal of Medical Internet Research,
(4), e18790. doi: 10.2196/18790.
Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-
analysis. Public Opinion Quarterly, 72(2). 167-189. doi: 10.1093/poq/nfn011.
Hlatshwako, T. G., Shah, S. J., Kosana, P., Adebayo, E., Hendriks, J., Larsson, E. C., Hensel, D. J., Erausquin, J. T., Marks, M., Michielsen, K., Saltis, H., Francis, J. M., Wouters, E., & Tucker, J. D. (2021). Online health survey research during COVID-19. The Lancet Digital Health, 3(2), e76-e77.
doi: 10.1016/S2589- 7500(21)00002-9.
THEORY
Jensen, R. E., Snyder, C. F., Abernethy, A. P., Basch, E., Potosky, A. L., Roberts, A. C., Loeffler, D. R., & Reeve, B. B. (2014). Review of electronic patient-reported outcomes systems used in cancer clinical care. Journal of oncology practice, 10(4), e215–e222. doi: 10.1200/JOP.2013.001067.
Jorm, A. F. (2000). Mental health literacy: Public knowledge and beliefs about mental disorders. The British Journal of Psychiatry, 177(5), 396-401. doi.org/10.1192/bjp.177.5.396.
Jylhä, M. (2009). What is self-rated health and why does it predict mortality? Towards a unified con- ceptual model. Social Science & Medicine, 69(3), 307-316. doi: 10.1016/j.socscimed.2009.05.013. Kevin, B. W. (2005). Researching internet-based populations: Advantages and disadvantages of
online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10(3). doi.org/10.1111/j.1083-6101.2005. tb00259.x.
Kreuter, F., Presser, S., & Tourangeau, R. (2008). Social desirability bias in CATI, IVR, and web sur- veys: The effects of mode and question sensitivity. Public Opinion Quarterly, 72(5), 847-865. doi: 10.1093/poq/nfn063.
McCambridge, J., Kalaitzaki, E., White, I. R., Khadjesari, Z., Murray, E., Linke, S., Thompson, S. G., Godfrey, C., & Wallace, P. (2011). Impact of length or relevance of questionnaires on attrition in online trials: Randomized controlled trial. Journal of Medical Internet Research, 13(4), e96. doi: 10.2196/jmir.1733.
Singh, S., & Sagar, R. (2021). A critical look at online survey or questionnaire-based research studies during COVID-19. Asian Journal of Psychiatry, 65, 102850. doi: 10.1016/j.ajp.2021.102850.
Singer, E., & Ye, C. (2012). The use and effects of incentives in surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 112–141. doi: 10.1177/0002716212458082.
Smyth, J. D., Dillman, D. A., Christian, L. M., & Stern, M. J. (2006). Effects of using visual design prin- ciples to group response options in web surveys. International Journal of Internet Science, 1(1). 6-16. Schnell, R., Noack, M., & Torregroza, S. (2017). Differences in general health of internet users and non-users and implications for the use of web surveys. Survey Research Methods, 11(2), 105–123.
doi: 10.18148/srm/2017.v11i2.6803.
Toepoel, V., & Lugtig, P. (2015). Online surveys are mixed-device surveys: Issues associated with the
use of different (mobile) devices in web surveys. 2190-4936. 2015. doi: 10.12758/mda.2015.009. Tourangeau, R. (2000). Remembering what happened: Memory errors and survey reports. In A. A. Stone, J. S. Turkkan, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, & V. S. Cain (Eds.), The science of
self-report: Implications for research and practice (29-48). Lawrence Erlbaum Associates Publishers. Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5). 859-
doi: 10.1037/0033-2909.133.5.859.
van Gelder, M. M., Bretveld, R. W., & Roeleveld, N. (2010). Web-based questionnaires: The future
in epidemiology? American Journal of Epidemiology, 172(11). 1292-1298. doi: 10.1093/aje/kwq291. Willis, G. B., & Artino, A. R., Jr (2013). What Do Our Respondents Think We’re Asking? Using Cog- nitive Interviewing to Improve Medical Education Surveys. Journal of graduate medical education,
(3). 353–356. doi: 10.4300/JGME-D-13-00154.1.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Karolina Szałata, Monika Czekalska, Luiza Łabuzińska, Martyna Bukowiec, Weronika Worosz, Laura Magdalena Sikorska, Gabriela Szpila, Angela Ćwil
This work is licensed under a Creative Commons Attribution 4.0 International License.
