Selecting a Dependent Variable

You have read at several places in this volume how important it is to select a dependent variable or an outcome measure with a great deal of care. It is the link between all the hard preparation and thinking you have done and the actual behavior you want to measure. Even if you have a terrific idea for a research project and your hypothesis is right on target, a poorly chosen dependent variable will result in disaster.The following nine items are important to remember when selecting such a variable. Use the following as a checklist when you search through previous studies to find what you need.

• • Try to find measures that have been used before. This gives them credibility and allows you to support your choice by citing previous use in other research studies.
• • Ensure that the validity of the measure has been established. Simply put, don’t select dependent variables whose validity either has not yet been established or is low. Doing so will raise too many questions about the integrity of your entire study. Remember, you can find out if a test has been shown to be valid through a review of other studies where the test has been used or through an examination of any manuals that accompany the test or assessment tool.
• • Ensure that the reliability of the measure has been established. As with validity, reliability is a crucial characteristic of a useful dependent variable.
• • If the test requires special training, consider the time and the commitment it will take to learn how to use it. This does not mean simply reading the instructions and practicing the administration of a test. It means undergoing intense training such as that required for the administration of intelligence tests and several personality scales.
• • Be sure you can get a sample of the test before you make any decision about whether you will use it. You might have read about it in a previous study, but you should not make a final decision until you examine its guidelines on the intended testing population, requirements for administration, costs, and so on. You can usually get a sample packet either at no cost or at a minimal cost from the test developer or publisher (although you may need a letter from your adviser because several test companies will not send materials to just anyone who requests it).
• • If you will need them, be sure that norms are available. Some tests do not require the use of norms, but if your intention is to compare the performance of different samples with scores from a more general population, you must have something to compare it with. As you will see later, norms are especially important for norm-referenced tests.
• • Obtain the latest version of the test. Publishers are always changing test materials, whether it is a repackaging of the materials or a change in the actual normative or reliability and validity data. Just ask the simple question, “Is this the latest version available?”
• • The test needs to be appropriate for the age group with which you are working. If a test measures something at age 10, it does not mean it will be equally reliable and valid at age 20, or even that it will measure the same underlying construct or behavior at that age. Look for other forms of the same test or another test that measures the same construct for the intended age group.
• • Finally, look for reviews of the test in various journals and reference sources, such as at the Buros Institute (www.unl.edu/buros), which lists thousands of tests on just about everything, and the Mental Measurement Yearbook (14th ed.), which is also published by the Buros Institute. Both these publications contain extensive information about different types of tests including administration procedures, costs, critical reviews of the tests by outside experts, and so on. Examine these critical reviews before you decide to adopt an instrument.

Reviewing a Test

What follows is more about selecting dependent variables (or screening measures for assignment to groups as independent variables). At best, with all things going in your favor, it is difficult to find exactly the test you want to use to diagnose, evaluate, determine effects, use as a placement tool, and so on. The dependent variable you select may not even be a test in the formal sense of the word. But if it is, you need to be concerned about many different characteristics and qualities of the instrument.With that in mind, the following outline of criteria will help you compare and contrast various tests. For each test you want to consider, complete the outline to the extent possible and then use this information to make a decision. Be sure to weigh each of the criteria accordingly. For example, although a test might be appropriate as far as its design and purpose, if it is prohibitively expensive or requires special training (which you do not have) to administer it, it is not likely that you will be able to use it.

Basic Information

• 1. Name of the test
• 2. Date of publication
• 3. Test author(s)
• 4. Publisher
• 5. Cost of all needed test materials
• 6. Cost of sample packet

General Test Information

• 7. Purpose of the test as stated by author(s)
• 8. Purpose of the test as used in other studies
• 9. Age levels included
• 10. Grades included
• 11. Special populations included
• 12. Method of administration (individual or group)
• 13. Method of scoring (manual or computer)
• 14. Amount of time required for administration
• 15. Ease of administration
• 16. Ease of scoring
• 17. Amount of training required for administration
• 18. Adequacy of test manual and other materials

Design and Appearance

• 19. Clear and straightforward directions
• 20. Design and production satisfactory
• 21. Arrangement of items on page
• 22. Ease of reading

Reliability

• 23. Reliability data provided
• 24. Type of reliability established (test–retest, parallel forms, etc.)
• 25. Independent studies used to establish reliability

Validity

• 26. Validity data provided
• 27. Type of validity established
• 28. Independent studies used to establish validity

Norms

• 29. Norms available
• 30. Description of norm groups
• 31. How norm groups were selected
• 32. Appropriateness of norm groups for your purpose

Evaluation

• 33. How used in the past
• 34. Summary of outside review(s)
• 35. Other evaluative information