|
This is potentially another
time-consuming, difficult task within the capstone experience.
Hopefully, you've taken the Tests and Measurement class
offered by the department. If so, you know what to look
for in a good measurement instrument, and you know how
to find one. If you haven't, you need to brush up on
some concepts important for evaluating instruments (specifically,
their psychometric properties), they are:
- Reliability: basically
measures the error in the instrument. It is a percentage
that tells us how much of the observed variations
are due to variations in true scores. Therefore, you
want this percentage to be high. For example, a good
reliability in psychological research would be r =
.80 or above. That tells us that only 20% of the variability
in scores is measurement error, the rest of the variability
is a reflection of true variability. You probably
don't want to use an instrument that reports reliabilities
lower than .80. If you can't find any for your variable,
talk to your instructor. There are 2 main types of
reliability coefficients you'll read about tests:
test-retest reliability, which is a correlation between
scores at one time and scores at a second time, and
coefficient alpha (sometimes seen as a) is a measure
of internal consistency (i.e. how well the various
items in the test 'hang together' versus the test-retest
that measures temporal stability). Coefficient alpha
is probably most commonly found because the researcher
only needs to administer the test once to get the
reliability coefficient.
- Validity: assesses
whether the test measures what it is supposed to measure.
Usually you don't get a 'number' for validity, what
you get is information from the test manufacturer
or the author of the manuscript where the test is
introduced telling you something like 'validity was
determined to be adequate/good/excellent based on….."
(there are a number of ways to test validity). If
there is no information on validity, you should be
suspicious of the test.
- Norm group: the
norm group is the group of individuals that were tested
and used as a 'guide' to determine what scores on
the test signify. You need to make sure you identify
a norm group that is similar to your population. If
a test for depression was normed with the elderly,
for example, the mean score for the norm group might
not be the mean score for your group, or cutoff scores
might be different also.
So where do you find the questionnaires you are looking
for? First off, determine what your instructor's expectations
are. Some instructors will allow you, or even want you,
to create your own instrument. Others do not allow you
to do this. In both cases, however, you will want to
have a few instruments that measure your variable, either
to choose from or to use as models to develop your own.
Before we get to 'where to find' the questionnaire
and its psychometrics, let's look at some tips about
'what to look for.'
- By now you've become quite an expert
in your research area. You've read dozens of articles
on the subject and summarized them in an eloquent
introduction. Hopefully you've found more than one
study similar to what you are planning to do. Use
these resources! Go over your literature and write
down the name of the instruments used by other, seasoned,
published researchers. Make a list and mark on that
list each time an instrument has been used more than
once. You may end up with a list, for example, of
5 depression questionnaires, one of which has been
used by 70% of the researchers. That probably gives
you a clue as to which to consider most strongly.
- Other factors come into play also.
If the questionnaire that is used the most is an 'interview
format' questionnaire, that will probably be out of
the question for you, since it will take hours and
previous training to administer it. Go for the next
most used that is also shorter, self-report, and still
has good 'psychometric qualities' (reliability, validity,
etc.)
- You may find that an instrument
you are considering is not appropriate based on language
use, questions used, etc. Remember you may have a
population to work with that may be different from
what others use (college age versus elderly, for example).
- We are supposing that questionnaires
used by published authors have excellent reliability
and validity, and that they are probably among the
best available. That is more than likely an appropriate
supposition since these researchers have been evaluated
a number of times by their peers (grant funding, publishing).
However, you still need to check to make sure the
instrument is psychometrically valid and appropriate
for use by you for your study, so ultimately, the
decision is yours as to what instrument to choose.
- And be aware that this is one of
the most important decisions you will make in designing
your study. A poorly selected instrument may result,
at the end of the semester, in findings that don't
make sense and don't match the existing literature,
leaving you at a loss as to how to make heads or tails
of your study. Researchers spend a great deal of time
selecting the appropriate instrument for their studies,
you do the same!
Useful links to find measures online:
Carver University of Miami
IUPUI Library Sources for Psychological Tests
|