Science & Technology

12 Recommendations To Protect the Integrity of Survey Research

Survey research plays a vital role in informing policy decisions, political campaigns, and understanding societal needs. However, recent concerns about refusal rates, inaccurate forecasts, contradictory findings, and declining trust in institutions have raised questions about the reliability of surveys. In response, an article published in PNAS Nexus brings together 20 experts from various sectors who propose 12 recommendations aimed at enhancing the accuracy and credibility of surveys amidst evolving technology and societal challenges.

Science requires data, and survey research is one important means of gathering it. Surveys provide a scientific way of acquiring information that is used to inform policy decisions, guide political campaigns, clarify the needs of stakeholders, enhance customer service, help society understand itself, and improve the quality of life in the United States.

In recent years, concerns have been raised about growing rates of refusal to participate in surveys, as well as about inaccurate forecasts in high-profile elections, polls with contradictory findings, the declining trust in government and media institutions that fund such research, and skepticism fueled by political polarization. “Although polling is not irredeemably broken,” the authors of a new article write, “changes in technology and society create challenges that, if not addressed well, can threaten the quality of election polls and other important surveys on topics such as the economy.”

In this article, published today (March 28, 2023) in the journal PNAS Nexus, 20 experts from diverse fields – including academia, science, government, nonprofits, and the private sector – offer a dozen recommendations to improve the accuracy and trustworthiness of surveys.

The authors’ recommendations aim to better align the practices of survey research with three scientific norms: transparency, clarity, and correcting the record. Among the recommendations are that surveyors and survey researchers should:

“If survey panelists’ responses have been potentially biased by responses to earlier surveys or questionnaires, researchers and readers need to know that,” said co-author Kathleen Hall Jamieson, director of the Annenberg Public Policy Center of the University of Pennsylvania.

The authors also recommend the creation of a publicly available, professionally curated archive of identified technical problems in polling tools and their remedies. And they recommend that professional organizations and universities develop and disseminate a guide to survey research for use in high school courses as well as a brief “users guide” designed for laypeople and journalists who do not have training in statistics or survey research.

“Around the world, survey research produces vital information,” said co-author Arthur Lupia, a distinguished professor at the University of Michigan. “These new recommendations have the potential to increase the quality of future surveys for generations to come.”

See the full list of recommendations online.

The recommendations emerged from a virtual two-day retreat in November 2021 convened by Marcia McNutt, president of the National Academy of Sciences, and cohosted by The Annenberg Foundation Trust at Sunnylands and the Annenberg Public Policy Center (APPC). The sessions were coordinated by Lupia and Jamieson, who also is the Sunnylands program director. Among the participants were the authors of this paper, who include past and current editors of major academic journals, past presidents of the American Political Science Association and the American Association for Public Opinion Research (AAPOR), a past director of the U.S. Census Bureau, scholars who have led some of the nation’s largest university-based election surveys, and individuals responsible for the creation and maintenance of major government and nonprofit survey datasets.

Read more about this retreat and other NAS-Sunnylands-APPC retreats on APPC’s website.

The authors of the PNAS Nexus article are:

“Protecting the integrity of survey research” was published online as an open-access article in PNAS Nexus on March 28, 2023.

Reference: “Protecting the integrity of survey research” by Kathleen Hall Jamieson, Arthur Lupia, Ashley Amaya, Henry E Brady, René Bautista, Joshua D Clinton, Jill A Dever, David Dutwin, Daniel L Goroff, D Sunshine Hillygus, Courtney Kennedy, Gary Langer, John S Lapinski, Michael Link, Tasha Philpot, Ken Prewitt, Doug Rivers, Lynn Vavreck, David C Wilson and Marcia K McNutt, 28 March 2023, PNAS Nexus.
DOI: 10.1093/pnasnexus/pgad049

Founded in 1993, the Annenberg Public Policy Center aims to enlighten the public and policymakers on the significance of communication in fostering comprehension of political, scientific, and health matters across local, state, and federal levels.

Back to top button