{{Short description|Statistical analyses that were not specified before the data were seen}} {{Distinguish|Post hoc theorizing}} In a scientific study, '''post hoc analysis''' (from Latin ''post hoc'', "after this") consists of statistical analyses that were specified after the data were seen.<ref>{{Cite web |title=What is the significance and use of post-hoc analysis studies? |url=https://www.cwauthors.com/article/significance-and-use-of-post-hoc-analysis-studies |access-date=2022-12-09 |website=www.cwauthors.com}}</ref><ref>{{Cite web |date=2019-11-12 |title=11.8: Post Hoc Tests |url=https://stats.libretexts.org/Bookshelves/Applied_Statistics/Book%3A_An_Introduction_to_Psychological_Statistics_(Foster_et_al.)/11%3A_Analysis_of_Variance/11.08%3A_Post_Hoc_Tests |access-date=2022-12-09 |website=Statistics LibreTexts |language=en}}</ref><ref>{{Cite web |title=Post Hoc |url=https://forrt.org/glossary/english/post_hoc/ |access-date=2025-11-02 |website=FORRT - Framework for Open and Reproducible Research Training |language=en-us}}</ref> A post hoc analysis is usually used to explore specific, statistically significant differences between the means of three or more independent groups-- differences detected with an analysis of variance (ANOVA).<ref>{{Cite web |title=SAGE Research Methods - The SAGE Encyclopedia of Communication Research Methods |url=https://methods.sagepub.com/reference/the-sage-encyclopedia-of-communication-research-methods/i11008.xml |access-date=2022-12-09 |website=methods.sagepub.com |language=en}}</ref> An ANOVA does not identify the group(s); for that, a post hoc analysis is required.<ref>{{Cite web |date=2019-11-12 |title=11.8: Post Hoc Tests |url=https://stats.libretexts.org/Bookshelves/Applied_Statistics/An_Introduction_to_Psychological_Statistics_(Foster_et_al.)/11%3A_Analysis_of_Variance/11.08%3A_Post_Hoc_Tests |access-date=2025-11-02 |website=Statistics LibreTexts |language=en}}</ref>
Because each post hoc analysis is effectively a statistical test, conducting multiple post hoc comparisons introduces a family-wise error rate problem, which is a type of multiple testing problem. This increases the likelihood of false positives unless corrected.
Post hoc tests are follow-up tests performed after a significant ANOVA result<ref>{{Cite web |last=Bobbitt |first=Zach |date=2019-04-14 |title=A Guide to Using Post Hoc Tests with ANOVA |url=https://www.statology.org/anova-post-hoc-tests/ |access-date=2025-11-02 |website=Statology |language=en-US}}</ref> to identify where the differences lie (which specific groups differ). To compensate, multiple post hoc testing procedures are sometimes used, but that is often difficult or impossible to do precisely. Post hoc analysis that is conducted and interpreted without adequate consideration of this problem is sometimes called ''data dredging'' (''p''-hacking) by critics because the statistical associations that it finds are often spurious.<ref>{{Cite journal |last=Zhang |first=Yiran |last2=Hedo |first2=Rita |last3=Rivera |first3=Anna |last4=Rull |first4=Rudolph |last5=Richardson |first5=Sabrina |last6=Tu |first6=Xin M. |date=2019-08-01 |title=Post hoc power analysis: is it an informative and meaningful analysis? |url=https://gpsych.bmj.com/content/32/4/e100069 |journal=General Psychiatry |language=en |volume=32 |issue=4 |article-number=e100069 |doi=10.1136/gpsych-2019-100069 |issn=2517-729X|pmc=6738696 }}</ref> In other words, findings from data dredging are invalid or not trustworthy.
Post hoc analyses are acceptable when transparently reported as exploratory. In other words, post hoc analyses are not inherently unethical.<ref name="APA-Manual-7e">{{cite book |year=2020 |author=American Psychological Association |title=Publication Manual of the American Psychological Association: the Official Guide to APA Style |publisher=American Psychological Association |location=Washington, DC |isbn=978-1-4338-3217-8 |edition=7th}}</ref> The main requirement for their ethical use is simply that their results not be mispresented as the original hypothesis.<ref name="APA-Manual-7e" /> Modern editions of scientific manuals have clarified this point; for example, APA style now specifies that "hypotheses should now be stated in three groupings: preplanned–primary, preplanned–secondary, and exploratory (post hoc). Exploratory hypotheses are allowable, and there should be no pressure to disguise them as if they were preplanned."<ref name="APA-Manual-7e" />
== Types of post hoc analysis == Types or categories of post hoc analyses include:<ref>{{Cite journal |last=Beaton |first=Albert E. |last2=Keppel |first2=Geoffrey |date=1975 |title=Design and Analysis: A Researcher's Handbook |url=https://doi.org/10.2307/1162588 |journal=American Educational Research Journal |volume=12 |issue=1 |pages=101 |doi=10.2307/1162588 |issn=0002-8312|url-access=subscription }}</ref>
* '''Pairwise comparisons:''' Tests all possible pairs * '''Trend analysis:''' Tests for linear or quadratic trends across ordered groups * '''Simple effects analysis:''' Examines effects within factorial ANOVA * '''Interaction probing:''' Analyzes interaction constraints within factorial ANOVA
* '''Restricted Sets of Contrasts:''' Testing smaller families of comparisons
In addition, a '''subgroup analysis'''<ref>{{Cite journal |last=Andrade |first=Chittaranjan |date=2023-11-01 |title=Types of Analysis: Planned (prespecified) vs Post Hoc, Primary vs Secondary, Hypothesis-driven vs Exploratory, Subgroup and Sensitivity, and Others |url=https://doi.org/10.1177/02537176231216842 |journal=Indian Journal of Psychological Medicine |language=EN |volume=45 |issue=6 |pages=640–641 |doi=10.1177/02537176231216842 |issn=0253-7176 |pmc=10964884 |pmid=38545527}}</ref> examines whether findings differ between discrete categories of subjects in the sample. This approach is common in clinical and observational studies.
== Common post hoc tests == Common post hoc tests include:<ref>{{Cite web |title=Post Hoc Definition and Types of Tests |url=https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/post-hoc/ |access-date=2022-12-09 |website=Statistics How To |language=en-US}}</ref><ref>{{Cite web |last=Pamplona |first=Fabricio |date=2022-07-28 |title=Post Hoc Analysis: Process and types of tests |url=https://mindthegraph.com/blog/post-hoc-analysis/ |access-date=2022-12-09 |website=Mind the Graph Blog |language=en-US}}</ref> * Fisher's least significant difference * Holm-Bonferroni Procedure * Newman-Keuls * Rodger's Method * Scheffé's Method * Tukey's Test and Honestly Significance Difference (HSD) (see also: Studentized Range Distribution)
However, with the exception of Scheffès Method, these tests should be specified "a priori" despite being called "post-hoc" in conventional usage. For example, a difference between means could be significant with the Holm-Bonferroni method but not with the Turkey Test and vice versa. It would be poor practice for a data analyst to choose which of these tests to report based on which gave the desired result.
== Causes == Sometimes the temptation to engage in post hoc analysis is motivated by a desire to produce positive results or see a project as successful. In the case of pharmaceutical research, there may be significant financial consequences to a failed trial.{{citation needed|date=November 2023}}
== See also == * HARKing * Testing hypotheses suggested by the data * Nemenyi test * Outcome switching
==References== <references />
Category:Data analysis Category:Multiple comparisons Category:Clinical research Category:Medical statistics