Executive Summary
Introduction: Why ask questions on sexual orientation?
Health, economic, and social surveys have always had to adapt to changing demands and changing times. In recent years, public policy debates have heightened the need for high quality scientific data on the sexual orientation of adults and young people in the United States. Discussions of civil rights, program evaluation, public health, and the delivery of human services must rely on sound facts and analyses that come from survey research, but often those facts are not available in the context of gay-related policy issues because lesbian, gay, and bisexual (LGB) people cannot be identified in surveys without specific questions pertaining to sexual orientation.
Fortunately, several private and some publicly funded surveys in the United States have begun to ask questions that allow identification of dimensions of sexual orientation, which has allowed researchers to identify the important role of sexual orientation as a predictor of health, social, and economic outcomes. Indeed, the failure to account for sexual orientation effects may lead to inaccurate scientific conclusions about targeting health interventions or identifying health risk factors, for example.
Asking questions on sexual orientation is not only necessary for scientific, practical, and policy purposes, but the recent research documented in this report demonstrates that it is also possible to include such questions on surveys without sacrificing data integrity or respondent retention. This report addresses the questions that arise once researchers have decided to include sexual orientation questions, including what to ask, where to ask it, and how to analyze the data, all in the context of a diverse population. The report outlines some “best practices” for actually putting the decision to ask sexual orientation questions into practice.
The report presents the findings from a multi-year effort of an expert panel of scholars from several disciplines in the health and social sciences, including economics, sociology, psychology, epidemiology, public health, and political science. Thanks to a generous grant from the Ford Foundation, we have had the opportunity to conduct original methodological research, analyze newly available sources of data, discuss issues with administrators and researchers in statistical agencies, and meet to cull all of those experiences into this document.
What to ask
Questions on existing large-scale surveys have varied widely, and we have learned a great deal from the different survey experiences about the types of questions that have worked and how to avoid problems. Conceptually, sexual orientation has three major dimensions, and below we present the recommended item for each dimension that draws on our research and experiences with using these items:
- Self-identification: how one identifies one’s sexual orientation (gay, lesbian, bisexual, or heterosexual)
Recommended Item: Do you consider yourself to be:
a) Heterosexual or straight;
b) Gay or lesbian; or
c) Bisexual? - Sexual behavior: the sex of sex partners (i.e. individuals of the same sex, different sex, or both sexes).
Recommended Item: In the past (time period e.g. year) who have you had sex with?
a) Men only,
b) Women only,
c) Both men and women,
d) I have not had sex - Sexual attraction: the sex or gender of individuals that someone feels attracted to.
Recommended Item:
People are different in their sexual attraction to other people. Which
best describes your feelings? Are you:
a) Only attracted to females?
b) Mostly attracted to females?
c) Equally attracted to females and males?
d) Mostly attracted to males?
e) Only attracted to males?
f) Not sure?
We also recommend that sexual orientation be asked separately from marital status and cohabitation in surveys. However, for all surveys – including those that do not directly ask about sexual orientation – we recommend that the marital status and cohabitation questions include response options that take into account the diversity of families and the changing legal circumstances of sexual minority individuals and households. At a minimum, we recommend that all marital status questions allow a response option for “living with a partner” and, ideally, that a complete household sex roster for adults and children be available for researchers to maximize the usefulness of this information.
How and where to ask
The next issue concerns making decisions about how to conduct the survey—the mode—and where to place the questions. The researcher’s concern is often that respondents either will not answer sensitive questions like sexual orientation or will answer with an inaccurate response. The choice of an appropriate mode of data collection will mitigate these problems. In particular, enhancing the privacy of the survey environment appears to encourage respondents to answer sensitive questions, including those related to sexual orientation, and to report accurately. Careful placement, mode adaptations, and interviewer training may improve the quality of sexual orientation data that is collected by a given survey by providing for a level of privacy that is sufficient to encourage accurate responses.
Survey mode: When possible, we recommend placing sexual orientation-related questions on self-administered portions of a survey. This method could involve inclusion of a subset of questions on a paper-and-pencil self-administered questionnaire (as done in the General Social Survey) or inclusion on a self-administered computer-assisted interview (as done in the National Survey of Family Growth).
Interviewer training: We recommend that training be provided to interviewers who will ask questions about a dimension of sexual orientation. Training should explain the reason(s) the question has been added and should review any clarifying language to be provided about response categories and privacy protection of responses.
Placement of questions: We recommend that the placement of questions on sexual orientation be tailored to meet the needs and goals of each survey. For general public health surveys that include questions about IV related behaviors, sexual behaviors, or reproductive behaviors, we recommend including questions about recent (e.g. within the past year or within the past five years) and lifetime same-sex sexual behavior with other questions related to sexual behavior or at the end of the relevant module. For other surveys (except school-based classroom surveys of adolescents which are discussed below), we recommend including a direct sexual orientation identity question at the end of the standard “Demographics” section. For surveys with a strong reason to include multiple measures of sexual orientation (e.g. mental health), we recommend including a battery of questions about same-sex sexual attraction, same-sex sexual behavior, and sexual orientation identity. For surveys that include a self-administered component for “sensitive” topics, we recommend including sexual orientation identity, attraction, and behavior questions in that section.
Age considerations
What to ask: Sexual orientation questions have been asked on large-scale school-based surveys of adolescents around the world since the mid-1980s. Because physical sexual maturity, sexual orientation, and sexual relationships most commonly develop during the adolescent years, all of the sexual orientation questions have limitations that should be considered. Because many adolescents are not sexually experienced, questions that focus on the gender of sexual partners will likely misclassify the majority of adolescents with respect to sexual orientation. Attraction is generally a better measure for adolescent populations, except in studies specifically focused on sexual health and sexual risks, although a significant proportion of younger adolescents may not have experienced sexual attractions yet.
Survey mode: Adolescence is also the time when sexual orientation-based harassment is the most prevalent, so the stigma associated with specific identity labels may reduce response rates or increase false responses, unless care is taken to ensure privacy and anonymity during survey administration.
Placement of questions: Care should be taken that any sexual orientation question not be placed next to sexual abuse questions. Doing so may yield higher non-response rates. Careful consideration must also be given to survey mode in regard to placement, since many paper-and-pencil surveys of young people place their demographics questions at the beginning of the survey. Placement of sexual orientation questions in the demographics section will mean that most students are still on the same page when the sexual orientation question is viewed, possibly making it more difficult to ensure privacy or anonymity.
Racial/ethnic and culture considerations
The context of racial/ethnic diversity leads to additional methodological considerations related to the cross-cultural equivalence of sexual orientation measures. These considerations are relevant for studying not only racial/ethnic diversity within sexual minority groups but also sexual diversity within racial/ethnic minority groups.
What to ask: Sexual orientation survey items should be culturally appropriate, relevant, acceptable, and compatible with the respondent’s understanding of the construct that the question is intended to measure. However, differences in how sexuality is understood in different racial/ethnic populations underscore the difficulties in generalizing sexual orientation as a social construct and raise questions about cultural equivalence. Also, more research is needed to better understand how attraction and behavior are mapped onto sexual orientation identities. Therefore, when possible we recommend that surveys assess multiple dimensions of sexuality, such as measures of sexual behavior, sexual attraction, and self-identity. When measuring sexual diversity within racial/ethnic minority groups, researchers might also consider including additional response options for sexual orientation identity terms, such as two-spirit, same gender loving, homosexual, down low, or queer, that may turn out to be more relevant for non-white populations.
Sampling: Given differences in population sizes and constructions of sexuality across race/ethnicity and culture, we cannot employ the same set of assumptions about LGB people of color as we do with white LGB communities when determining sampling strategies or devising sampling frames. In some studies, the likelihood of being recruited and the willingness to participate in studies may vary by race/ethnicity. In sampling strategies to study sexual minorities, the careful choice of screening questions when developing sexual minority samples is crucial, and questions that capture a broad range of individuals with same-sex behavior, as well as LGB identities, may result in a more racially and ethnically diverse sample. Choices of sampling frames and sampling methods should also be attentive to methods that will reduce the likelihood of under-representation of some racial/ethnic groups.
Data analysis issues: An additional consideration when analyzing data on racial/ethnic minority LGB concerns the need to understand factors that mediate the choice of identity categories, particularly discrimination and acculturation.
Collecting data on transgender status and gender nonconformity
This report primarily addresses survey questions on sexual orientation, which includes sexual identity, sexual behavior, and sexual attraction. However, the social and political community for LGB people also includes transgender people who would remain invisible on surveys even with a question on sexual orientation. As an umbrella term, transgender refers to people whose gender expression defies social expectation. More narrowly, the term transgender describes a smaller group of people who experience incongruence between birth sex and gender identity.
The health and well-being of transgender people may be among the poorest in the United States. Our lack of knowledge about how to identify transgender respondents on general population surveys hinders efforts to improve the health and socioeconomic status of this marginalized community. While this report cannot make specific recommendations on transgender-related measurement given our current level of methodological knowledge, we describe various measurement approaches and related issues and considerations in an effort to further research on transgender health and welfare.
Analysis of sexual orientation data
Responsible analyses of sexual orientation data must be cognizant of several important factors that have been observed to potentially distort or misrepresent important nuances. Most importantly, a growing body of research analyzing associations between sexual orientation and a wide variety of social and health outcomes suggests that the best research requires several important practices:
- Careful consideration of differences among non-heterosexual responses: Several studies provide evidence that sexual minority respondents can be quite different in their demographic composition and social, health, or economic outcomes. Whenever the size of the sub-groups allows, bisexuals should be separated from lesbian and gay respondents and men and women should be considered separately. In some cases, those who do not respond to sexual orientation questions, select other, or select “I don’t know” should not be considered LGB.
- A thorough understanding of how survey methodologies can affect reliability and validity of responses: Survey mode, skip patterns, and survey goals will influence reporting of minority sexual orientations.
- Separate subgroup analyses of outcomes: Evidence also suggests substantial differences in characteristics of sexual minorities across a variety of demographic sub-groupings. Researchers should always be aware that attributes attributed to the LGB community are largely associated primarily with white LGB individuals since they represent the largest racial/ethnic grouping within the population.
- Taking contextual issues into account: Researchers should try to provide appropriate context for their research to aid in the interpretation of results.
- A clear understanding of time frames: Researchers should be cautious when analyzing data collected over relatively long periods of time. Social norms may have changed over the period of time data was collected and the willingness of individuals to report same-sex experiences or LGB identities has increased over time. When relying on questions about past sexual behavior, researchers should be cognizant of the issues associated with asking questions about long reference periods. For example, more variation in sexual behavior and attraction may arise from longer time frames and responses to such time frames require longer recall.
- Recognition of potential sources of measurement error: Researchers should consider the possibility of false positives since errors made by those in a large population (e.g., heterosexuals) potentially misclassify individuals into a very small population (e.g., sexual minorities).