Assessing and Recommending Quantitative Research Designs

Determining the most appropriate research design involves taking into account a variety of factors related to the study. The researcher must consider the research questions, the hypotheses, and whether variables will be utilized. Although making the choice of design can seem daunting, the characteristics of the study will determine its design (Fort-Nachmias & Nachmias, 2008). The purpose of this paper is to assess the strengths and limitations of research designs, and to recommend a quantitative design for my research plan, with a rationale for that recommendation. In addition, its purpose is to explain why other research designs were not chosen.

Strengths and Limitations of Research Designs

Experimental Design
Strengths. Experimental designs are the original, perhaps exemplary models of research. This design allows the researcher to control independent variables and eliminate extraneous variables to a greater extent than in other research designs. Experimental design makes it possible to determine causal relationships since it involves manipulating exposure to independent variables. This allows researchers to observe cause and effect, or the effect of the independent variable on the dependent variable (Fort-Nachmias & Nachmias, 2008). The controlled conditions of experimental design allow for replication, which enables the researcher to verify results. This element of experimental design is crucial for future research as well; when other researchers can replicate an experiment, it provides the research community with a higher level of confidence in the study's results (Fort-Nachmias & Nachmias, 2008).

Limitations. Although experimental design seeks to eliminate or control extraneous variables, this is not always practical or possible. When the intention of research is to study real-world circumstances, an experimental design may not be applicable because of the inability to replicate a natural environment (Levy, Ellis, & Cohen, 2011). This has the potential to weaken external validity. In addition, the selection process necessary to maintain control over variables may not be random. The sample chosen may not be representative of a wider population, and as such, the results may not be generalizeable to the population of interest (Fort-Nachmias & Nachmias, 2008). In many cases, it can be impossible, impractical, or unethical to utilize random assignment to treatment and control groups.

Quasi-Experimental Design

Strengths. Quasi-experimental design allows researchers to investigate behavior in natural settings that may not be amenable to an experimental design. Using naturally existing samples increases the external validity of a study (Fort-Nachmias & Nachmias, 2008; Trochim, 2006). For example, Marques and Lima (2011) studied the effects of living in industrial neighborhoods. For this research, a quasi-experimental design did not require the researchers to utilize random assignment to comparison groups. In a study such as Marques and Lima, researchers could not control who lived in 

Each participant will be assessed with the Beck Depression Inventory Fast Screen for Medical patients.  Comparisons of the results will be made to determine if patients with rare cancer types who participate in online disease specific support groups experience lower levels of depression than patients with rare cancer types who participate in face-to-face support groups.

             

When researchers make group comparisons utilizing quasi-experimental designs, they are creating the comparison groups through non-random processes.  Quasi-experimental design is an alternative to experimental design, which always utilizes a random assignment to groups.  Because quasi-experimental design does not utilize a random selection process for assigning participants to comparison groups, the researcher must be aware of how the non-random selection process might affect the results of the study (Shannon, Goldenhar, & Hale, 2001).  For example, the researcher might ask whether there are inherent differences between participants in the different groups because of age, gender, marital status, socio-economic status, level of education, or other undeterminable differences.  In my study, some of these differences may include length of time since cancer diagnosis, age, geographical location, the particular cancer diagnosis, presence or severity of depression, and other factors that may be difficult to determine.  It will be necessary to note these differences because they have the potential to affect how participants respond to the support group to which they are affiliated (Shannon et al., 2001).  In addition, it will be important to describe and list information on these differences and account for them in the statistical analysis (Shannon et al., 2001). 

Designs not Chosen for Research Plan

Experimental Design
A primary difference between experimental designs and quasi-experimental designs is that the former always involves a control group to which participants are randomly assigned, exposed to treatments imposed by the researcher, and followed by an assessment of the effects of the treatment (Levy, Ellis, & Cohen, 2011). Experimental designs are investigations in which researchers control the impact of independent variables on the dependant variables (Goba, Balfour, & Nkambule, 2011; Levy et al., 2011). Further, experimental designs measure the relationship between the independent and dependent variables (Fort-Nachmias & Nachmias, 2008). The variables in my study cannot be controlled, only observed. My study would not be effectively accomplished with an experimental design because I cannot randomly choose participants. They have already been diagnosed with rare cancer types, and are either utilizing face-to-face support groups or online disease-specific support groups. I cannot exert control over the assignment of participants to comparison groups.

My research question is whether and to what extent participation in online disease-specific support groups evokes a greater sense of control over disease than face-to-face support group participation in patients with rare cancer types. My comparison is between two intact groups. I cannot randomly assign participants to a control and a treatment group. My research question requires that I seek individuals with a rare cancer diagnosis, then assign individuals who participate exclusively in online disease-specific support groups to one group, and those who only participate in face-to-face support groups in a second group. Participants cannot be randomly assigned to one group or the other, and as such, the design must be a quasi-experimental one. The variables, participation in face-to-face support groups and participation in online, disease-specific support groups cannot be supported by random assignment, therefore, an experimental design is not appropriate for my study. The hypothesis of my study, that participation in online, disease-specific support groups evokes a greater sense of control over disease than participation in face-to-face support groups in patients with rare cancer types is not amenable to an experimental and a control group. The study will measure sense of control in participants in each group and identify which independent variable is more effective at evoking a sense of control in participants.

Cross-Sectional Designs

Cross-sectional designs are observational, utilize a random sample of participants, and are often associated with survey research (Fort-Nachmias & Nachmias, 2008). Researchers record information obtained from the survey, although they would not manipulate variables or expose one or more groups to a treatment. For example, a researcher might utilize a cross-sectional design to measure inflammation in exercisers and non-exercisers. Since a cross-sectional design can provide researchers with an effective method of examining several characteristics simultaneously, a cross-sectional study might record characteristics such as age, socio-economic status, and gender of the exercisers and the non-exercisers. Rather than determining causal direction or relationships, this type of research design is descriptive. Unlike experimental design, variables are not manipulated in cross-sectional research designs.

This type of research design would not be amenable to my research question, my dependent and independent variables, and hypothesis. This research design does not support variables. In addition, my hypothesis is not amenable to a survey, and cross-sectional research design does not support observing how the independent variable affects the dependent variable. Cross-sectional designs do not suggest cause-and-effect relationships. I do not intend to manipulate variables, but instead to determine the effects of the independent variable on the dependent variable. This cannot be accomplished in a cross-sectional design.

Conclusion

Choosing a research design depends on the research problems, whether the study utilizes a treatment to which a group will be exposed, its dependent and independent variables, and its hypotheses. Each research design has strengths and limitations inherent in its design. The design which is chosen guides the researcher in the data gathering process, as well as in the analysis and interpretation of the observations that take place throughout the study. Although experimental control and randomization are cornerstones of experimental design, not all inquiries into the social sciences are amenable to such control (Fort-Nachmias & Nachmias, 2008). Quasi-experimental and cross-sectional designs provide alternative means of inquiry when it is not plausible, ethical, or possible to conduct a study in an experimental design.

References

Frankfort-Nachmias, C., & Nachmias, D. (2008). Research methods in the social sciences (7th ed.). New York: Worth.

Goba, B. B., Balfour, R. J., & Nkambule, T. T. (2011). The Nature of Experimental and Quasi- Experimental Research in Postgraduate Education Research in South Africa: 1995-2004. South African Journal Of Higher Education, 25(2), 269-286.

Levy, Y., Ellis, T. J., & Cohen, E. (2011). A Guide for Novice Researchers on Experimental and Quasi-Experimental Studies in Information Systems Research. Interdisciplinary Journal Of Information, Knowledge & Management, 6151-161.

Marques, S., & Lima, M. L. (2011). Living in grey areas: Industrial activity and psychological health. Journal of Environmental Psychology, 31(4), 314-322. doi:10.1016/j.jenvp.2010.12.002

Morgan, G. A., Gliner, J. A., & Harmon, R. J. (2000). Quasi-Experimental Designs. Journal of the American Academy of Child & Adolescent Psychiatry, 39(6), 794-796. doi: 10.1097/00004583-200006000-00020

Shannon, H. S., Goldenhar, L. M., & Hale, A. R. (2001). Chapter 4 Quasi-experimental and experimental designs: More powerful evaluation designs. In L. S. Robson (Author), Guide to evaluating the effectiveness of strategies for preventing work injuries: How to show whether a safety intervention really works. (pp. 29-42). Cincinnati: NIOSH.

Trochim, W. M. (2006). Quasi-experimental design. The Research Methods Knowledge Base. Retrieved December 21, 2013, from http://www.socialresearchmethods.net/kb/quasiexp.php

Measuring Variables

Measuring variables can be a complicated task, and selecting valid measures is a complex undertaking (Sechrest, 2005). Some levels of measurement can be transformed to other levels, although others cannot. Special problems are posed by applying measurements in the social sciences, especially as they play a significant role in the inferences we, as social scientists make. Sechrest (2005) suggests the validity of the data is more important than the validity of a specific instrument, although selecting a valid instrument contributes to valid data.

The four levels of measurement include nominal, ordinal, interval, and ratio (Fort-Nachmias & Nachmias, 2008). The level of measurement determines the analysis appropriate for the numbers. Higher level measurements can be measured at lower levels. For example, a ratio can also be transformed to a nominal measurement, although the opposite is not true; a nominal measurement cannot be transformed to an interval or a ratio measurement. Nominal data, such as male versus female cannot be transformed to an interval, ordinal, or ratio scale. There is not enough information in the measurement; people fit into one category or the other. Being male or female neither captures information on rank order, as in ordinal scales, nor does being in a male or female category indicate meaningful differences between the categories. Male or female categories do not convey information related to having a zero point, as in a ratio scale. Therefore, male or female data, which is nominal, cannot be transformed to any of the other three levels of measurement.

On the other hand, ages of people is ratio level data. It can be given a number, there is an order to age, there are meaningful difference between the data (a 75 year old man is 2.083 years older than a 36 year old woman), ratios can be calculated because an individual cannot be any younger than 0. Data can be transformed according to the amount of information captured within the measurement. Whereas nominal levels of measurement capture a limited amount of information, ratio level data conveys a higher complexity of information. The higher levels of measurement have the same properties as the lower levels of measurement and can be transformed to lower levels, although the lower levels of measurement do not have the same properties so cannot be transformed to higher levels of measurement.

Whereas the concept of design validity is the extent to which research is sound (Fort-Nachmias & Nachmias, 2008), measurement validity is the extent to which the instrument measures what it intends to measure. A valid research design utilizes methods that result in relatively true findings. A valid design relies on internal and external validity to give credence to its results and the inferences made from them. In studies that aim to establish causal relationships, internal validity is the strength of the inferences made regarding these relationships (Trochim, 2006). External validity is the study's generalizeability to wider populations (Fort-Nachmias & Nachmias, 2008; Trochim, 2006).

Within the concept of measurement validity, three basic types of validity exist: content validity is the extent to which the instrument utilized for measurement measures the salient characteristics of the subject being measured. Empirical validity determines the degree to which the results produced by the measurement tool are similar to real world relationships between the variables (Fort-Nachmias & Nachmias, 2008). The extent to which a measure "correlates with some other phenomenon in which one is interested" (para. 1) determines its empirical validity (Sechrest, 2005). Construct validity isthe extent to which an instrument measures what it claims it is measuring and not something else (Cronbach & Meehl, 1955; Trochim, 2006). It is necessary for the researcher to find information appropriate to each of the three types of validity to validate the measurement.

References

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. doi: 10.1037/h0040957

Frankfort-Nachmias, C., & Nachmias, D. (2008). Research methods in the social sciences (7th ed.). New York: Worth.

Marques, S., & Lima, M. L. (2011). Living in grey areas: Industrial activity and psychological health. Journal of Environmental Psychology, 31(4), 314-322. doi:10.1016/j.jenvp.2010.12.002

Sechrest, L. (2005). Validity of Measures Is No Simple Matter. Health Services Research, 40(5p2), 1584-1604. doi: 10.1111/j.1475-6773.2005.00443.x

Trochim, W. M. (2006). Quasi-experimental design. The Research Methods Knowledge Base.
Retrieved December 21, 2013, from http://www.socialresearchmethods.net/kb/quasiexp.php

Online Disease Specific Support: The Effects on Rare Cancer Types

Since the beginning of the internet age, researchers have been interested in the psychosocial effects of online experiences. As a result of the growing evidence of the complex, but poorly understood, mind body relationship, the psychophysiological effects of online support as it effects disease have become a focus of interest (Juneau & Remolino, 2000). The growing field of psychneuroimmunology has provided ample empirical evidence that psychosocial experiences, play a salient, perhaps primary, role in immunological functioning, susceptibility to disease, disease progression, and overall survival (Goodwin et al., 2001). Huber et al. (2011) found online peer-to-peer interaction provided the same support as traditional face-to-face support groups, and online forums provided an ease of use in that direct personal contact was circumvented.

There is, however, a paucity of research on the effects of disease-specific online support for rare cancer types. This paucity is regrettable since the addition of disease-specific online support is readily available for many cancer patients, at least those in developed nations. As an adjunct therapy in the treatment of rare cancer types, oncologists can prescribe this effective addition to a new or established therapeutic treatment plan. Since social support, in the form of face-to-face support groups has been somewhat effective in the survival of cancer patients (Goodwin et al., 2001), it seems prudent to determine the role of disease-specific online support for rare cancer types. Maintaining a sense of control over disease is a salient, and perhaps critical component of contending with the emotional experience of a cancer diagnosis (Huber, 2011; Seckin, 2009; Sen, 2008). It is prudent and imperative to determine the most effective means of evoking this sense of control in cancer patients, and to understand the dynamics of peer-to-peer influences on gaining a sense of control derived from online support (Huber, 2011).

An abundance of research exists on the positive effects of social support for a diverse range of cancer patients of both genders (Goodwin et al., 2001). There is a handful of studies on the effects of online group support for cancer patients (Huber et al., 2011); however, far less on the psychosocial effects of disease-specific online support groups for patients with rare cancer types. If the addition of online support specifically related to the patient's cancer type is an effective addition to medical treatment, understanding the effects of this type of support may be a valuable tool for patients as well as their healthcare professionals. The benefits of online support have been documented for cancer patients; however, patients with rare cancer diagnoses may experience challenges exclusive to their particular cancer type. For example, fewer specialists may be available locally, and online support enhances information exchange between patients who utilize specialized oncological care and those who do not

Mortimer, & Spiegel, 2000), and benefits to the cardiovascular system (Uchino, 2006). The maintenance and support of these physiological functions may be valuable, even critical in cancer patients.  Seckin (2009) found patient participation in online support groups perceived their participation as a salient component of their ability to manage the stress associated with a cancer diagnosis, and it instilled optimism and a sense of control. Further, participation increased their knowledge about the disease, and it helped them find meaning in coping (Seckin, 2009; Sen, 2008). If online support participation increases these benefits, then the possibility exists that online support specific to an individual's cancer type has the potential to provide disease information and psychosocial support more specific to the patient's needs. If these benefits have a significant effect on patients with rare cancer diagnoses, including disease-specific online support participation may be an effective component of disease treatment.

References

Cohen, S., & Wills, T. A. (1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98(2), 310-357. doi: 10.1037//0033-2909.98.2.310

Goodwin, P.J., Leszcz, M., Ennis, M., Koopmans, J., Vincent, L., Guther, H., Drysdale, E.,
Hundleby, M., Chochinov, H. M., Navarro, M., Speca, M., & Hunter, J. (2001). The
effect of group psychosocial support on survival in metastatic breast cancer. New England Journal of Medicine, 345 , 1719–26.

Herbert, T., & Cohen, S. (1993). Stress and immunity in humans: a meta-analytic review. Psychosomatic Medicine, 55(4), 364-379.

Huber, J., Ihrig, A., Peters, T., Huber, C., Kessler, A., Hadaschik, B., & ... Hohenfellner, M. (2011). Decision-making in localized prostate cancer: lessons learned from an online support group. British Journal of Urology International, 107(10), 1570-1575. doi:10.1111/j.1464- 410X.2010.09859.x

Juneau, J. M., & Remolino, L. (2000). Online support groups: Nuts and bolts, benefits, limitations and future directions. (pp. 1-6, Rep. No. ED446330). Greensboro, NC: ERIC Clearinghouse on Counseling and Student Services. (ERIC Document Reproduction Service No. EDO-CG-00-7).

Owen, J., Boxley, L., Goldstein, M., Lee, J., Breen, N., & Rowland, J. (2010). Use of Health- Related Online Support Groups: Population Data from the California Health Interview Survey Complementary and Alternative Medicine Study. Journal Of Computer-Mediated Communication, 15(3), 427-446. Doi: 10.1111/j.1083-6101.2010.01501

Seckin, G. (2009). Medical Conversations in Technology Enabled Communities: Perceived Benefits of Participation in Online Patient Communities. Conference Papers -- American Sociological Association, 1.

Sen, N. (2008). Online Social Support for Cancer Patients: Effects of Stress, Shyness, and Family Support. Conference Papers -- National Communication Association, 1.

Tehrani, A., Farajzadegan, Z., Rajabi, F., & Zamani, A. (2011). Belonging to a peer support group enhances the quality of life and adherence rate in patients affected by breast cancer: a non-randomized controlled clinical trial. Journal Of Research In Medical Sciences, 16(5), 658-665.

Thoits, P. A. (1986). Social support as coping assistance. Journal of Consulting and Clinical Psychology, 54(4), 416-423. doi: 10.1037//0022-006X.54.4.416

Turner-Cobb, J., Sephton, S., Koopman, C., Blake-Mortimer, J., & Spiegel, D. (2000). Social support and salivary cortisol in women with metastatic breast cancer. Psychosomatic Medicine, 62(3), 337-345.

Uchino, B. N. (2006). Social support and health: A review of the physiological processes potentially underlying links to disease outcomes. Journal of Behavioral Medicine, 29(4), 377-387. doi: 10.1007/s10865-006-9056-5

Uchino, B. N. (2009). Understanding the links between social support and physical health: A life-span perspective with emphasis on the separability of perceived and received support. Perspectives on Psychological Science, 4(3), 236-255. doi: 10.1111/j.1745- 6924.2009.01122.x

Evaluating Burger (2009)

Burger (2009) chose an experimental design that compared an experimental group and a control group. Burger's choice of experimental design was appropriate because it made a comparison between two groups; the first, the group exposed to the treatment, and the second, the control group . In effect, an experimental design is a planned interference of responses (Frankfort-Nachmias & Nachmias, 2008). In the Burger study, the dependent variable was the point at which the participants stopped shocking the learner. He wanted to see if manipulating the participants' exposure to a refusing individual (independent variable), produced a change in the point at which participants stopped shocking the learner.

An experimental design administers a treatment to one group of participants for the purpose of observing their responses (Frankfort-Nachmias & Nachmias, 2008). This type of design manipulates the conditions imposed upon one group of individuals and maintains a control group, upon which the conditions are not imposed. For the Burger (2009) study, an experimental design was more appropriate than a cross-sectional or a quasi-experimental design because of the need to manipulate variables. This manipulation allows the researcher to determine the direction of causation (Frankfort-Nachmias & Nachmias, 2008). A cross-sectional or quasi-experimental design is chosen when the researcher seeks to make observations rather than determine causality. Pre-experimental designs have no control group and are not appropriate for determining causation (Frankfort-Nachmias & Nachmias, 2008).

Threats to Validity

Experimental designs can be weak in external validity because real-life circumstances cannot be replicated and laboratory environments can be construed and difficult to generalize to larger populations (Frankfort-Nachmias & Nachmias, 2008). The Burger (2009) study was somewhat weak in external validity for this reason. In addition, in a laboratory environment, participants may not be representative of a population about which the research may wish to generalize (Frankfort-Nachmias & Nachmias, 2008). The Burger study did not randomly select its participants; they were selected according to several criteria. This was important, however, because of ethical considerations. At the start, participants were self-selected, in that they responded to a call for participants. The lack of randomization is a threat to external validity.

Typically, when seeking evidence of causation (internal validity), which is typical in experimental design, generalizeability/external validity is sacrificed (Frankfort-Nachmias & Nachmias, 2008). Burger extended great control over the independent variable, which strengthened internal validity. He was careful to control internal validity by controlling for the effects of extraneous variables, such as gender, personality traits, education, and ethnicity.

Other threats to internal validity include maturation, issues in testing, such as when the pre-test affects the results of the post-test, using imprecise or inappropriate measures, non-random selection (selection bias), and participant drop-out, altered behavior of participant(s) from being studied.

Burger, J. M. (2009). Replicating Milgram: Would people still obey today? American Psychologist, 64(1), 1-11. doi:10.1037/a0010932

Frankfort-Nachmias, C., & Nachmias, D. (2008). Research methods in the social sciences (7th ed.). New York: Worth.

Research Proposal

My research problem is determining whether and to what extent participation in online disease-specific support groups evokes a stronger sense of control over disease than participation in face-to-face support groups in cancer patients with rare cancer types. The units of analysis are individuals as cancer patients. Individuals are reasonable for comparison, since the foundational research problem seeks to identify the experiences of individuals with rare cancer types. The experiences of individuals have been measured in a variety of quantitative and qualitative research inquiries. Bulsara, Ward, and Joske (2004) evaluated the experiences of individual cancer patients, as did Lavdaniti, Barbas, Fratzana, and Zyga (2012) and Sahin and Tan (2012).

Ecological fallacies are errors made in drawing conclusions about individuals based on information gained from group analysis (O'Dowd, 2003). Distortion from ecological fallacies will be avoided because this study will make inferences about individuals based upon data gained from individuals. Group data will not be used to make deductions or inferences about individuals (Frankfort-Nachmias & Nachmias, 2008). In regards to individualistic fallacies, because this research seeks to determine, interpret, and identify behaviors in individuals, inferences about groups, societies, or nations will not be made based on information gained from individuals participating in this study (Frankfort-Nachmias & Nachmias, 2008).

Two researchable hypotheses are:

1. Patients with rare cancer types who participate in online disease-specific support groups experience a greater sense of control than patients with rare cancer types who participate in face-to face support groups.

In this research hypothesis, sense of control is the dependent variable and the independent variables are participation in disease specific support groups and participation in face-to-face support groups. The control variable is the stage of the patient's cancer, so that they are not recently diagnosed, or in the terminal stages of disease. I anticipate a positive relation between the independent variable (participation in an online disease specific support group). dependent variable and the independent variable, participation in disease specific support groups. As disease-specific support group participation increases, sense of control will increase as well. Regarding magnitude, I expect a positive, but not perfect relation between the dependent variable and the independent variable. Not all patients with rare cancer types will gain a sense of control by participating in online disease-specific support groups.

2. Disease-related anxiety in patients with rare cancer types decreases as a result of participation in online disease-specific support groups.

The dependent variable is disease related anxiety; the independent variable is online disease specific support group participation. The control variable is the stage of the patient's cancer, so that they are not recently diagnosed, or in the terminal stages of disease. I anticipate a negative or inverse relation between anxiety and the independent variable: as participation in online disease specific support groups increases, anxiety will decrease I expect a negative, but not perfect relation between the dependent variable and the independent variable. Not all patients with rare cancer types will experience a decrease in disease-related anxiety from participation in online disease-specific support groups.

References

Bulsara, C., Ward, A., & Joske, D. (2004). Haematological cancer patients: achieving a sense of empowerment by use of strategies to control illness. Journal Of Clinical Nursing, 13(2), 251-258. doi:10.1046/j.1365-2702.2003.00886.x

Frankfort-Nachmias, C., & Nachmias, D. (2008). Research methods in the social sciences (7th ed.). New York: Worth.

Lavdaniti, M., Barbas, G., Fratzana, A., & Zyga, S. (2012). Evaluation of depression in colon cancer patients. Health Science Journal, 6(4), 681-692.

O'Dowd, L. (2003). Ecological fallacy. In R. L. Miller & J. D. Brewer (Ed.), The A-Z of social research: A dictionary of key social science research concepts. London: SAGE. doi: 10.4135/9780857020024

Sahin, Z., & Tan, M. (2012). Loneliness, Depression, and Social Support of Patients With Cancer and Their Caregivers. Clinical Journal Of Oncology Nursing, 16(2), 145-149.