SOURCEBOOK

Examples Articles

Learning Activities | Post Hoc Comparisons

Learning Outcomes

  • Explain how post hoc comparisons clarify which group means differ after an omnibus analysis.
  • Interpret pairwise differences in context, including their direction, magnitude, and uncertainty.
  • Evaluate how accounting for multiple comparisons affects conclusions and the strength of evidence for individual group differences.

Learning Task

Study: A researcher measured three groups of four participants each on an Outcome.

  • Group 1 Outcome: 0, 0, 3, 5
  • Group 2 Outcome: 4, 7, 4, 9
  • Group 3 Outcome: 9, 6, 4, 9
  1. Use statistical software to obtain the descriptive statistics, the ANOVA source table, and the multiple comparisons using Tukey’s HSD procedure.
  2. On your own, calculate the mean difference and the standard error elements of the table of multiple comparisons using the descriptive statistics given by the software. Verify that your calculations match the statistics provided by the software.
  3. Using appropriate APA Style, report your results from this set of analyses.

Discussion

  • Why does a significant omnibus ANOVA not identify which group means differ, and how do pairwise comparisons address that question?
  • How does Tukey’s HSD account for the risk of false positives across several comparisons, and what does that adjustment mean for interpreting the results?
  • How do the direction and size of a pairwise difference, along with its uncertainty, help distinguish a statistically detectable difference from a meaningful one?