8. t-Test: Is the Difference in Means Significant?

The new method produced an average score of 75 and the old one 69. Is that a real effect, or is it chance: with another group of students the numbers could have come out the other way round? Student's t-test answers how large the difference in means is compared with the spread inside the groups.

The logic of any hypothesis test:

1. The null hypothesis H₀: there is no difference, and the population means are equal;
2. We calculate a test statistic. Here t = (mean₂ - mean₁) / standard error of the difference;
3. The p-value is the probability of getting a difference this large or larger if H₀ is true;
4. If p is below a significance level α chosen in advance (usually 5%), we reject H₀: the difference is significant.

For two independent groups with similar spread, use the pooled variance s² = ((n₁-1)·s₁² + (n₂-1)·s₂²) / (n₁+n₂-2), the standard error √(s²·(1/n₁ + 1/n₂)), and n₁+n₂-2 degrees of freedom.

• T.DIST.2T(|t|, deg_freedom) gives the two-tailed p-value from a finished statistic;
• T.INV.2T(probability, deg_freedom) gives the critical t: if |t| is larger, the difference is significant;
• T.TEST(array1, array2, tails, type) gives the p-value straight from the raw data. Tails: 2 tests for a difference in either direction, 1 only in one direction. Type: 1 is paired data (the same people before and after), 2 is independent groups with equal variances, and 3 is independent groups with unequal variances (Welch's test).

If you aren't sure the variances are equal, use type 3, which is the safer choice. And "not significant" doesn't mean "no effect": there may just not be enough data.