Survey Designs et al.

Designs

Surveys gather opinions or attitudes that can be analyzed to gain a general consensus, whereas experiments manipulate variables to produce an outcome, outcomes, or no outcome (Creswell, 2009). Both survey and experimental strategies of inquiry indentify characteristics of a sample and generalize to a wider populations, although the goal of the experimental design is to test how a treatment or intervention affects outcomes in a controlled setting, although not necessarily within a laboratory. The goal of experimental designs is usually to establish how variables are related. This can be accomplished by applying different variables to one group and not to a control group (Creswell, 2009). In a survey, researchers would not manipulate variables as they would in an experimental design .

For example, If I were attempting to identify attitudes of Japanese women recently integrated into the majority American culture, I would utilize a survey method. If I sought to determine the effects of American culture on newly integrated Japanese women who participated in an orientation class specifically designed for Japanese women, I would utilize an experimental design. In the experimental design, I would have an experimental group (the group that took the class) and a control group (the group that did not take the class). Experimental designs have the potential to answering cause and effect questions, however, a survey does not have that potential (Creswell, 2009).

Populations, Samples, and Participants

In a survey design, a population of interest is chosen by the researcher, and its characteristics described (size, gender, and other descriptive information). Similar information is needed for participants in an experimental design. Both can use randomly chosen individuals, however an experimental design considers statistical significance as well as the effect size when determining the size of a group studied.

Instrumentation, Variables, Data Analysis and Interpretation, and Experimental Procedures

In an experimental design, one or more groups of participants receive the independent variable or variables, and the control group does not. Variables in a survey design may be related to specific questions on the survey or questionnaire. In survey designs, the researcher reports the number of responders and non-responders. Further this type of research design must discuss response bias and provide information on the descriptive analysis utilized for the variables.

Both designs utilize a variety of statistical analyses to interpret and make inferences about the population characteristics or relationships between treatment and control groups. Both rely on inferential and descriptive statistics whereas . In the survey design, the primary instrument or means of collecting data is the survey or questionnaire, whereas in the experimental design, data is collected by observing the treatment group(s) as compared to the control group, and taking measurements at specified times during the experiment, (e.g., pre-test and/or post-test). Different from the survey design that utilizes the survey or questionnaire as its process of inquiry, experimental designs require the identification and description of procedures utilized as well as the rationale for the choice of procedures.

Reliability and Validity

In a survey design, it is important to prevent bias and confusion by using carefully chosen wording. Shortcomings in survey design include the challenge of obtaining the depth of information needed from a survey. Further, the survey relies on self-report, which has inherent shortcomings as well. Bowling (2005) found individuals provide answers to questions according to their immediate environment and transient emotional states. Although experiments are best evidence of causality, they have shortcomings as well. For example, some experiments are not possible and in many circumstances, individuals cannot be assigned to one group or another. In experimental designs, external and internal issues may threaten the validity of the experiment. These issues can occur as a result of participant characteristics or flaws in other aspect of the study (methods, measurements, inaccurate inferences and others). These threats must be identified, explained, and mitigated, when possible. Reliability and validity are cornerstones of scientific investigation. In empirical research, they provide a level of usability and practical application (Whiston, 2009). Without reliability and validity, research is of no consequence.

Terminology

Experiments measure relationships and use words such as causation, control, and variability, confounding variables, placebo, participants, pre-test, post-test, and procedures, whereas survey designs use the words and concepts of population, response bias, samples, questionnaire, survey, target populations, samples, scales, choice, open and closed-ended questions (Creswell, 2009; Whiston, 2009). Health psychology seems to utilize quantitative methods as well as qualitative methods.

References

Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281-291. doi: 10.1093/pubmed/fdi031

Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage Publications.

Whiston, S. C. (2009). Principles and applications of assessment in counseling (3rd ed.). Belmont, CA: Brooks/Cole, Cengage Learning.