Identify at least one strength and one weakness for the open-ended questions. Identify at least one strength and one weakness for the closed-ended questions. For each weakness, p

2/7/23, 7:05 PM Survey Research: Spr23 PSYC 005 #38039 RESEARCH METHODS IN PSYCHOLOGY-Online

https://canvas.pasadena.edu/courses/1131870/pages/survey-research?module_item_id=15132919 1/3

Survey Research

The following lecture on Survey Research was written by Heidi Iseman, a graduate student in Clinical Psychology at Cal State University, Fullerton. She is currently working on her master's thesis. She and her thesis advisor are using survey research to conduct the research.

Overview of Survey Research What is a survey and why would you use one when conducting psychological research? Survey research can be either qualitative or quantitative and involves asking respondents to provide self-report responses to questions about their thoughts, feelings, and behavior. Researchers who use survey research prefer to have random samples that are as large as possible so their findings represent the population as much as possible. Surveys are a great research method to use if you need a large number of participants.

Survey research started around the beginning of the 20 century when English and American “social surveys” were used to learn more about social issues, such as poverty. Surveys are widely used by researchers within the government, as well as among academics in many fields, such as psychology, sociology, political science, market research, and public health.

Why is survey research useful? In addition to its important uses in many different fields, survey research has provided prevalence information about mental disorders in the United States. One such survey is The National Comorbidity Survey: http://www.hcp.med.harvard.edu/ncs (https://www.google.com/url? q=http://www.hcp.med.harvard.edu/ncs&sa=D&usg=AFQjCNHGgRvKIdEBbW5iqCRTVDJMgBDeaA) . This large-scale mental health survey has gathered information about the prevalence of mental disorders and how they are associated with important outcome variables. This type of study helps inform policymakers, as well as mental health providers.

Constructing Surveys When a respondent answers a survey question, there is a psychological process that he or she goes through. One cognitive model of responding to a survey question involves first interpreting the question, then retrieving information, forming a judgment, formatting a response, and lastly, editing the response (Sudman, Bradburn, & Schwarz, 1996). The chapter provides an example of this process using a survey question about alcohol use. First, respondents need to be able to accurately interpret the question. This is why it is important for researchers to use language that is free from jargon and easy to understand. Next, the respondent would need to retrieve information from his or her memory regarding how much he or she drinks. Different factors might influence how well the participant is able to retrieve information, such as cognitive ability and the presence of a substance use disorder. Then, the respondent would use the information retrieved to arrive at a judgment of how much he or she actually drinks. It is important to remember that this is a judgment and therefore is subjective. The survey researcher has to trust that the respondent is able to accurately judge their drinking behavior. After the participant is makes a judgment, he or

th

2/7/23, 7:05 PM Survey Research: Spr23 PSYC 005 #38039 RESEARCH METHODS IN PSYCHOLOGY-Online

https://canvas.pasadena.edu/courses/1131870/pages/survey-research?module_item_id=15132919 2/3

she would then format their responses to fit the answer choices made available to them. Lastly, the respondent may edit their response based on the options available.

One important factor to consider when interpreting survey responses is context effects. Context effects refer to how the context in which the question is presented, rather than the actual content, influence responses. An example of a context effect is an item-order effect, where the order in which the question is presented influences how participants respond. In other words, responses are affected by earlier questions. Respondents may be psychologically primed or have certain schema activated when one question is presented before another. Participants can also be influenced by the order in which response options are presented, which is why counterbalancing is an important technique. Counterbalancing involves randomizing the response options so the findings are not unduly influenced by order effects.

There are different types of items that can be included in a survey. One type is an open-ended item, where respondents are asked a question where they can provide their own unique answer. One example provided in the text is “What is the most important thing to teach children to prepare them for life?” This question is open-ended because the respondent has the freedom to provide his or her own response. These types of items are useful when researchers are conducting qualitative research or are at the early stages of research project development.

In contrast, closed-ended items involve presenting a question and a set of predetermined responses for the participant to choose from. These types of items are useful for quantitative research and are used when researchers are aware of what types of responses they should be getting. Closed-ended items lend themselves well to quantitative research because they do not need to be re-coded, as might an open-ended item. These types of items can be easily converted onto a spreadsheet and statistically analyzed. When researchers use closed-ended items that are quantitative in nature, a rating scale is often used. A rating scale provides an ordered set of responses that the participant can choose from. One of the most popular types of rating scales is a Likert Scale. The original Likert Scale was developed in the 1930s by a researcher named Rensis Likert. A Likert Scale involves presenting the respondent with a question where he or she can either agree or disagree and then measuring their response along a 5-point scale ranging from Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, andStrongly Disagree.

Effective survey items follow the BRUSO acronym. They are brief, relevant, unambiguous, and objective. Questionnaire items should not be overly long, irrelevant to the research study, difficult to interpret, or subjective. It is important to carefully edit and screen the items included in the survey to ensure that the responses obtained are valid and reliable. The chapter provides some examples of poor and effective items.

After the researcher has chosen the items to be included in the survey, he or she must format the survey. Typically, a survey begins with an introduction where the researcher describes the purpose of the study, acknowledges academic affiliations or sponsorships, and recognizes and thanks the participant for taking part in the study. The researcher must obtain informed consent from the participant and should provide information, such as how long the survey will take to complete, the types of questions that will be asked, as well informing the participant that he or she can withdraw from the study at any time.

Conducting Surveys Once you have created your survey, the next step is to determine how you are going to conduct it. First, you will have to find a sample of individuals who will take your survey. There are two main forms of sampling. Probability sampling refers to when the researcher knows the

2/7/23, 7:05 PM Survey Research: Spr23 PSYC 005 #38039 RESEARCH METHODS IN PSYCHOLOGY-Online

https://canvas.pasadena.edu/courses/1131870/pages/survey-research?module_item_id=15132919 3/3

probability that each member of the population is likely to be part of the sample. Some examples of probability sampling are simple random sampling, stratified random sampling, and cluster sampling. These forms of sampling are further discussed in the chapter.

The following video describes the types of probability sampling and how they are conducted:

4.2 Probability Sampling Techniques (https://www.youtube.com/watch?v=jauhoR7w1YM)

(https://www.youtube.com/watch?v=jauhoR7w1YM)

Non-probability sampling refers to when the researcher does not know the probability that each member will be selected. One example of non-probability sampling is convenience sampling. Convenience sampling is when the researcher gathers whichever participants are the easiest to recruit.

This video from Pew Research discusses non-probability (online opt in) surveys – Methods 101: What are non-probability surveys? (https://www.youtube.com/watch?v=T9dmq6Lwh24)

One important concept to consider is sampling bias. Sampling bias occurs when a sample is not representative of the population. Sampling bias can create inaccurate results that cannot be generalized. Probability sampling was created as a means of reducing sampling bias.

The most common form of sampling bias is non-response bias. This refers to when individuals who do not take part in the survey differ in significant ways from the members of the population who did take part in the survey. Some methods of reducing non-response bias are to provide potential participants with reminders, notifying them ahead of time, incentivizing participation, and creating surveys that are short and easy to complete.

How can you distribute your survey? Lastly, the chapter summarizes the four main ways that surveys are distributed: through mail, over the phone, online, and in-person. In-person surveys yield the highest response rates but are the most expensive. Internet surveys and mail surveys are the least expensive, yet they have the lowest response rates. Internet surveys are rapidly gaining popularity and are projected to be the most commonly used way of obtaining survey data.

Below are some of the most commonly used websites to create and distribute online surveys:

http://www.qualtrics.com (http://www.qualtrics.com/) http://www.surveymonkey.com (http://www.surveymonkey.com/) https://requester.mturk.com/create/projects/new (https://requester.mturk.com/create/projects/new)