1.  Determine the question asked and the conclusion reached. Keep these CLEARLY in mind as you work through the following questions.

2.  Whowas included in the study and how were they selected? Consider gender, ethnicity, class, age, geographical location, and any other characteristic that might be pertinent to the question.

1. POINT OF CRITIQUE: Is the sample biased (convenient sample, self-selected sample, selective under-coverage)? Are the conditions so specialized that other groups wouldn’t experience the conditions?
2. IMPORTANCE OF CRITIQUE: To whom and under what conditions can the conclusions be generalized? That is, does the study have external validity?

4.  If the study is observational, determine if it is a simple description (may include mean, median, standard deviation, percentages) or comparative (same kinds of descriptors but compares one or more groups based on these descriptors). If it is simple description, no generalizations should be made. If it is comparative, consider issues in Point #2. Are the conclusions reasonable based on the procedures and samples?

5.  If the study is a survey, see page 61 in your statistics text for questions to guide your critique of the study.

6.  If the study is correlational, check the conclusions and ask:

1. Have the researchers confined themselves to explaining possible relationships? If the conclusions suggest that one variable caused the other, the conclusion is not valid and should be questioned in your critique.
2. Is the correlation large enough to matter?

1. determine the design. It is useful to actually graph the design using O, X, R, and - - - - -. See the link I posted of different designs.
2. consider internal validity by examining the design and determining whether it controls for history, maturation, testing, instrumentation, statistical regression, differential selection, and differential mortality. Again, see the link I posted for the relative strengths and weaknesses of each design. In your critique, comment on whether these factors are controlled for and the implications for the conclusions if they are not controlled.
3. as in all quantitative research, review Point #2 for external validity/generalization.
4. are the results statistically significant? At what level of confidence?
5. even if the results are statistically significant, are the differences in the actual data large enough to mean anything on a practical level?
6. if the measurement of the results relied on tests, were the tests reliable and valid? How do you know? If observers were used, what was the inter-rater reliability?