Random Assignment to Conditions
Experimental vs. Statistical Control
No Random Assignment to Conditions
Other Designs Relevant to Aging
Home Page Course Calendar Course Syllabus
Chapter 3 Research Methods
The goal of this chapter is to give you a rudimentary knowledge of research methods. This chapter is not designed to be a definitive source on research methodology but instead explains many concepts and designs that you may encounter as you read the material for the course.
A correlation is
a relationship between two variables. For example, we could look at the
relationship between age and health and ask whether health improves or
declines with age. We could also look at the relationship between age and
cognitive decline and see if these two variables are related. Below is
another example that shows a relationship between two variables:
This example shows that there is a relationship between watching violence on TV and aggressive behavior. However, when we look at correlations we must be aware of three issues:
Correlations does not equal causation
We cannot infer a causal relationship. We cannot say that watching violent TV programs causes people to be more aggressive. All we can say is that there is a relationship between these two variables.
Correlations are bi-directional
The reason we cannot infer causation is that correlations are bi-directional (as represented by the two arrow heads between TV violence and aggression). It is not clear whether watching TV causes aggression or whether aggressive people simply like to watch violent programs.
Third-variable or Alternative Explanations
As can be seen in
the above diagram, a relationship between two variables does not rule out
the possibility that other variables may be important in determining TV
watching and aggression. Other variables such as family background, parenting
styles, socio-economic-status, the person's personality, or peers may be
affecting this relationship.
Research that uses
an experimental design has an advantage over correlational designs in that
an experimental design can control for or rule out third variable explanations.
How this is done is by random
assignment to conditions. Random assignment means that every participant
in a study has an equal chance of being in any of the conditions in the
study. For example, a researcher might be interested in how engaging in
mental activities, such as puzzle solving, might improve memory. The researcher
might want to compare a group that engages in puzzle solving (the experimental
group) to a group that does not solve puzzles (the control group) and see
if memory improves. The independent
variable is the variable being manipulated by the experimenter (i.e.,
solving puzzles or not) and the dependent
variable is the variable being measured (i.e., an increase in memory).
How random assignment to conditions rules out third variable explanations
is that everyone has an equal chance regardless of say, health or interest
in puzzle solving (potential third variable explanations) to be in either
condition. If we did not randomly assign participants to conditions we
might get more people in the control condition in poorer health than those
in the experimental condition. Then, if we find better memory for the experimental
condition than the control condition we are not sure whether it was the
prior activity (puzzle solving) or the participant's health that affected
the outcome. Random assignment to conditions is considered to be the defining
factor of an experiment. Without random assignment you cannot rule out
third variable explanations and you do not have an experiment! You have
something else, more than likely a correlational finding.
Experimental vs. Statistical Control
Random assignment
to conditions allows us to gain experimental control over our variables.
However, there are instances where we cannot randomly assign people to
conditions either because of ethical or logistical reasons. For example
one cannot ethically assign someone to smoke cigarettes for 25 years to
determine whether smoking causes cancer nor can we change someone's gender,
age, or ethnicity. Gender, age, and ethnicity are considered quasi-independent
variables (see below) that occur naturally. Controlling for these potential
third-variable explanations is attained by using statistical, rather than
experimental techiques. The third variable might present a potential confound,
but statistical procedures (e.g., multiple regression analyses) are used
to attempt to statistically eliminate this confound. The use of statistical
control is also known as partialing, controlling for, residualizing,
holding
constant, and covarying. Thus, if we were interested in the
relationship between puzzle solving and memory we could also control for
the person's age. This means that once we statistically account for any
effects of age on memory we can then see if puzzle solving in and of itself
affects memory.
A large amount of research on aging uses quasi-experimental designs because they include quasi-independent variables such as age, personality variables (e.g., self-esteem, obsessive-compulsive), ethnicity, or gender to name a few. We cannot control participant's exposure to or randomly assign them to these variables. As a result, there can be no random assignment of participants to these particular conditions. The use of quasi-experiments requires then that we statistically control for third variable explanations.
The table below summarizes the distinctions between experimental and quasi-experimental designs.
Table 1
Experimental and Quasi-experimental
Designs
|
|
|
| Independent variables are manipulated by the experimenter | Uses quasi-independent variables |
| Random assignment to conditions | No random assignment to conditions |
| Gets at cause and effect | Correlational--Does not get at cause and effect because alternative or third variable explanations are not controlled for |
| Experimental Control | Statistical Control |
Other Designs Relevant to Aging
A cross sectional design is one that compares different groups of people. These comparisons could be based on age, gender, ethnicity, or any variable that would place people into different groups. A cross sectional design can also look at different cohorts (i.e., people roughly born at the same time) with the goal of comparing these cohorts on some dependent variable. Below are the results of two studies that use a cross sectional design:
Figure 1
The Relationship Between Participant Age and Intelligence
This is a summary
of a number of studies that looked at differences in intelligence as measured
by IQ tests between younger and older adults. As can be seen, younger adults
seem to perform better on IQ tests than do older adults.
Figure 2
A Comparison of Just World Beliefs
Between Younger and Older Adults
(Madey & Chasteen, 2000)
This finding is from a study by Madey and Chasteen (2000). These researchers found that just world beliefs (i.e., believing that people usually get what they deserve) was higher for older adults (65-85 years old) than for younger adults (19-22 years).
Both findings are intriguing but we must be careful in how we interpret these data. For example, do the findings mean that younger adults are smarter than older adults? Or do older adults believe that the world is a just place and younger adults do not? In order to interpret these findings, we must first be aware of some points related to cross sectional designs.
One point is that
cross sectional designs cannot tell us about changes
as we get older. All cross sectional designs tell us is that there is a
difference
between the groups. We cannot infer that intelligence declines with age
or that a belief in a just world increases with age using a cross sectional
design. A second point is that other variables may be affecting the performance
of the two groups. For example, younger adults have had more experience
with the educational system than have older adults. Younger adults may
have had more practice taking tests than have older adults. Thus, with
cross sectional designs, alternative explanations for the findings cannot
be ruled out (unless we statistically control for these alternatives).
A third point, is that difference between the groups may reflect cohort
differences. People born at the same time share similar experiences and
possibly similar attitudes. Also, different cohorts are affected by historical
events in different ways. An older person who was draft eligible during
the Vietnam War may perceive politics, for instance differently than a
younger person living at a time when military service is voluntary. Baby
boomers would possibly differ from Generation X-ers on many values and
beliefs. Our example of differences in just world beliefs may reflect cohort
differences.
The advantage of a longitudinal design over a cross sectional design is that it can determine if a particular variable actually changes with age. This design is not without its disadvantages, however. For example, we can not be sure whether a change is due to getting older or to important historical events at different points in time. These period effects can be quite subtle and complex. For example, if you sample older workers over time about their attitudes toward retirement one must also consider that changes in retirement laws may affect the participants responses. Period effects may be particularly evident if you assess the participants' attitude toward retirement before mandatory retirement was introduced, then during the time when mandatory retirement was law, and later when mandatory retirement laws were eliminated. Another example is if we wished to determine if belief in a just world changes as people get older. That is, do people as they get older, come to believe more strongly that the world is a just place and that everyone gets what they deserve? We can use a longitudinal design and follow people for a period of 60 or 70 years and periodically measure their just world beliefs. However, we must also consider any important historical events that may have occurred over that period of time that may influence belief in a just world.
Other issues to
consider in using longitudinal designs is that the instruments that are
used may become outdated. Items on an anxiety scale used in the 1950s may
have significantly changed in the 1990s or become outdated. Therefore,
the instruments may not be consistent over time. Another issue is attrition.
Participants in a study, particularly if it is ongoing for around 50 years
or so, may do a number of things. They may get bored or sick and drop out
of the study, or they may die. Attrition creates problems for generalizing
the findings. The survivors, indeed, may be a different group from those
that were originally in the study.
Table 2
Comparing Cross Sectional and Longitudinal
Designs
| Design | Definition | Examples | Issues |
| Cross Sectional | Two or more groups from a different cohort are studied | Grandparenting
Styles in 50 vs. 70 year olds
Intelligence between 22 and 85 year olds Difference in a belief in a just world in younger compared to older adults |
Cohort Effects
Can only talk about Age Differences Other variables may be affecting the outcome |
| Longitudinal | A group is studied at several points in time | Intelligence over a 60 year period | Gets at age
changes
But also: Period Effects Changes in instruments
Attrition |
eQuestion#1: In 1969 college students and 45-year-old adults took the same personality test. Scores indicated that the middle-aged adults felt themselves to be more competent and better adjusted than did the college students (example taken from Perlmutter & Hall,1992, p. 44). What conclusions do you draw from this finding? Evaluate the validity of your conclusions.
eQuestion#2: What are the main strengths and weaknesses of experimental designs, quasiexperimental designs, cross-sectional, longitudinal, and correlational designs? How important is it to know how a study was conducted?
eQuestion#3: Why is it important to obtain information about adult development and aging from empirical research, rather than relying on intuition and common beliefs? (questions 2 and 3 adapted from Schulz & Ewen, 1993).
eQuestion #4: Is age a researchable
variable or is it simply a proxy for other variables?