November 7, 2017

Transatlantic Dialogues: In Europe and North America, Publics More Supportive Than Experts of Direct Democracy

Methodology

About Pew Research Center’s survey of foreign policy experts

The survey of foreign policy experts was conducted among invitees of the German Marshall Fund’s annual Brussels Forum and alumni of the GMF’s fellowship and educational programs, including: the Marshall Memorial Fellowship, the Transatlantic Inclusion Leaders Network, the Manfred Wörner Seminar and the American Political Science Association Congressional Fellowship. The survey was self-administered online via SurveyMonkey from Jan. 18 to March 16, 2017.

GMF sent requests for survey participation via email to a total of 3,109 individuals. These requests were on official letterhead and signed by Pew Research Center President Michael Dimock and German Marshall Fund President Karen Donfried. To encourage participation, all individuals were sent survey reminders every two weeks during February and every week in March up until the conference, for a total of six reminders throughout the fieldwork period. A total of 585 experts completed the survey; for purposes of analysis, responses were restricted to only respondents from North America and Europe, for a final total of 547 respondents. Surveys were considered valid responses when a respondent completed at least half of the closed-ended, substantive questions. The overall response rate for the survey was 19%.

The survey is weighted according to three demographic variables: (1) gender, (2) employment sector and (3) nationality. The weighting information was provided by GMF, based on the organization’s membership lists. Results are similar with or without weights. The overall design effect of the survey is 1.14, yielding a margin of error of ±4.5 percentage points for the North American and European sample (N=547) and an effective sample size of 481.

The open-ended survey responses were coded by two Pew Research Center researchers. The unit of analysis was each participant’s full response to the question “And over the next 12 months, in your opinion, what will be the biggest challenge for relations between Europe and the U.S.?” The question was posed in an open-ended format and respondents could write as much or as little as they wished. The researchers worked together using early responses to generate and revise the list of coding categories. They then separately coded the responses and achieved high levels of intercoder reliability (Krippendorff’s alpha = 0.89).

About Pew Research Center’s Spring 2017 Global Attitudes Survey

Results for the general public come from Pew Research Center’s Spring 2017 Global Attitudes Survey. The medians cited in the text are from the 12 transatlantic countries included in the Spring 2017 Global Attitudes Survey: the United States, Canada, France, Germany, Greece, Hungary, Italy, the Netherlands, Poland, Spain, Sweden and the United Kingdom. The surveys for these 12 countries were conducted by telephone and face-to-face interviews from Feb. 16 to April 17, 2017, among 12,462 adults under the direction of Princeton Survey Research Associates International and Kantar Public UK. The results are based on national samples. More details about our international survey methodology and country-specific sample designs are available here.