By Yan Wang, Ph.D.
Self-report is the most commonly used method to collect data in behavioral research, but its reliability and validity have always been hotly debated. Obtaining reliable and valid data on behavioral questions especially sensitive ones (e.g., risky sex, condom use) represents a longstanding challenge for public health research.
Researchers have attempted a number of methods—the indirect questioning technique, the randomized response technique, the nominative technique, and the bogus pipeline procedure—to better assess sensitive questions in research. Application of these methods is limited, probably due to the complex procedures and ethical concerns. Furthermore, most of these methods attempt to circumvent the sensitive nature of a survey question rather than directly tackle it to ensure reliability and validity.
Self report is most commonly used data collection method in behavioral research, but its reliability and validity are problematic
Construal level theory (CLT), the theory linking psychological distance to level of thinking (i.e., abstract versus concrete), provides the potential to improve self-report survey method.
The truthfulness in answering a question is, to a great extent, conditioned by the respondent’s perception of intrusiveness, threat of disclosure, or social desirability of the particular question. Construals are mental constructions of the universe at different psychological distances with self using here and now as the reference point. When people talk or think about other people, they usually use a high construal level of thinking, which is more abstract and less context-dependent. Sensitivity of a question is presumed to decline with the increase of psychological distance. This desensitization mechanism provides the potential to improve self-report data quality.
Construal level theory (CLT) has potential to improve self-report survey method
Social distance can function as a manipulation of construal level. When the target of the same question changes from self to socially distant others, the interpretation of a survey question and the composition of the response will automatically switch to higher construal levels. Therefore, the response to the same question for socially distant others would be more reliable because the response will be:
- A composite according to the established attitude or belief of the respondent,
- More abstract, and
- Less likely to be affected by contextual factors.
Reliability and validity of self-report may increase as social distance increases
The goal of a survey is to elicit information that is not only highly reliable but also accurate and valid regarding participants themselves. Based on the construal level theory, an individual’s assessment of the attitudes and beliefs of others toward a sensitive question also contains a certain amount of information about his or her own attitudes and beliefs (used as the reference for construal), although the amount of information may decline with social distance. The combination of self-assessment with the assessments of socially distant others may provide more valid data than using only self-assessment.
Construal level theory based multi-target survey method shows significant improvements in reliability and predictive validity
We tested the CLT-based survey method among 11,32 young adults in China to investigate their sexual openness, actual risky sexual behaviors, and history of sexually transmitted diseases. In addition to a conventional self-report on five questions tapping into the participant’s attitude toward five types of sexual behaviors (e.g., premarital sex, extramarital sex, and commercial sex) on a five-point Likert scale, we also asked the participants to estimate how many of rural residents/urban residents/foreigners would hold favorable attitude toward these behaviors on a five-point scale. We used a series of latent variable modeling strategies to extract and aggregate the information based on participants’ assessments of self and other three social groups. The latent score derived using the CLT-based method showed much higher reliability and better predictive validity than the conventional self-report.
Funding Acknowledgement: This study was supported by NIH grant (R01 MH086322, PI: Xinguang Chen).
Reference
Chen, X., Wang, Y., Li, F., Gong, J., Yan, Y. (2015). Development and evaluation of the Brief Sexual Openness Scale: A construal level theory based approach. PLoS ONE, 10(8): e0136683. doi:10.1371/journal.pone.0136683
About the Author
Yan Wang, Ph.D., is a research assistant professor of epidemiology at University of Florida. She has training and expertise in both psychology and epidemiology. Dr. Wang received her Ph.D. in child and family studies from Syracuse University in 2013. With an interdisciplinary perspective, her research focuses on using advanced methodology to investigate the complex etiological processes involved in risk behaviors, especially alcohol and tobacco use and sexual risk behaviors among youth and vulnerable populations such as rural residents and persons living with HIV/AIDS. She has worked on a number of NIH-funded projects, including those on mental health and risk behaviors among urban, rural-to-urban migrants, and rural residents in China; alcohol use among persons living with HIV/AIDS in Florida; and advanced quantum modeling on sexual risk behaviors. More information on her research can be found here: http://epidemiology.phhp.ufl.edu/yan-wang-phd/
