Often one wants to compare two treatments or populations and determine if there is a difference. This can be done either with or without matching. Matching produces dependence between the two samples and will be discuss after the unmatched/independent case.
The various assumptions made in this situation primarily affects the appropriate number of degrees of freedom to be used. This can range from one less than the smallest sample to two less than the sum of the sample sizes with various values inbetween possible. The choice reflects the conservativeness of the researcher and the care taken in evaluating the underlying assumptions. There seems to be conflict between various sources which I have yet to resolve.
|t = ((1 - 2) - (µ1 - µ2)) ÷ sqrt(s12/n1 + s22/n2)|
The expression in the denominator reflects the way variances sum (standard deviations do not sum). There are two options for obtaining a value for the degrees of freedom. Calculate a fractional degrees of freedom as given below, or use the smaller of n1-1 or n2-1. This latter value always results in conservative results. As sample size increases, this latter procedure also becomes more accurate. The two-sample t procedures are more robust than the one-sample methods, especially when the distributions are not symmetric. If the two sample sizes are equal and the two distributions have similar shapes, it can be accurate down to sample sizes as small as n1 = n2 = 5. The two-sample t procedure is most robust against nonnormality when the two samples are of equal size. Thus when planning such a study, you should make them equal.
The fractional degrees of freedom formula is as follows:
|d.f.= (s12/n1 + s22/n2) ÷ (((s12/n1)2 ÷ (n1-1)) + ((s22/n2)2 ÷ (n2-1)))|
Suppose instead of two distinct populations we randomly select our sample and then randomly assign half the subjects to an experimental (treatment) group and the other half to a control group. In this case and others, the population variances are equal and the estimated standard error of the difference used in the formula above: sqrt(s12/n1 + s22/n2) simplifies to sqrt(s2(1/n1 + 1/n2)). However, s2 is the pooled estimate of the population variance which comes from the sum of the sums of squares for the two groups divided by n1 + n2 - 2, which is the degrees of freedom in this case. It can also be obtained from the two sample variances as s2 = ((n1 - 1)s12 + (n2 - 1)s22)/ (n1 + n2 - 2)
Note: pooling assumes equal variances for the two populations. If in doubt, a statistical test should be performed.
Confidence intervals are constructed in the usual way using standard error of the difference between the mean just like we used the standard error of the mean before.
Effect size gives an alternate indication of the magnitude of a difference to help distinguish between statistical significance and practical importance when the sample size can muddy the waters. Some groups have recommended the reporting of effect size for published research.
Note also that the estimated standard error of the difference given in the blue box above and the fractional degrees of freedom given in the blue box above are the same as those given by Hinkle for use when the population variances are unequal. (Hinkle gives a source (Satterthwaite, 1946) for the fractional degrees of freedom formula.)
An example might be before and after SAT scores for a high-priced course of study. Or your typical freshman practice EXPO project where peas, corn, or other seeds are grown with and without (control) a treatment. Some Biology instructors and EXPO judges have expected our freshmen to perform these t tests!
Note: Hinkle uses different symbols for the dependent situation than for the independent situation to emphasize the difference.
Suppose a teacher wonders if there is a statistical difference between two pages of a test after noting similar means and standard deviations for the pages and decides to do a matched pair test.
|Page 3||Page 4||diff||d2|