Clinical significance

{{short description|Practical importance of a treatment}} In medicine and psychology, '''clinical significance''' is the practical importance of a treatment effect—whether it has a real genuine, palpable, noticeable effect on daily life.<ref>{{cite journal | vauthors = Kazdin AE | title = The meanings and measurement of clinical significance | journal = Journal of Consulting and Clinical Psychology | volume = 67 | issue = 3 | pages = 332–9 | date = June 1999 | pmid = 10369053 | doi = 10.1037/0022-006x.67.3.332 | url = http://homepage.psy.utexas.edu/homepage/class/Psy394Q/Research%20Design%20Class/Assigned%20Readings/Clinical%20Trials/Kazdin99.pdf | access-date = 3 November 2013 | url-status = dead | citeseerx = 10.1.1.595.9231 | archive-url = https://web.archive.org/web/20131106103031/http://homepage.psy.utexas.edu/homepage/class/Psy394Q/Research%20Design%20Class/Assigned%20Readings/Clinical%20Trials/Kazdin99.pdf | archive-date = 6 November 2013 }}</ref>

==Types of significance== ===Statistical significance=== {{main|statistical significance}} Statistical significance is used in hypothesis testing, whereby the null hypothesis (that there is no relationship between variables) is tested.<ref name="Nursing Research">{{cite book | vauthors = Polit DF, Beck CT |title=Nursing Research: Generating Evidence for Nursing Practice |edition=9th |location=Philadelphia |publisher=Wolters Klower/Lippincott Williams & Wilkins |year=2012 |isbn=978-1-60547-782-4 }}</ref> A level of significance is selected (most commonly ''α'' = 0.05 or 0.01), which signifies the probability of incorrectly rejecting a true null hypothesis.<ref name="Nursing Research"/> If there is a significant difference between two groups at ''α'' = 0.05, it means that there is only a 5% probability of obtaining the observed results under the assumption that the difference is entirely due to chance (i.e., the null hypothesis is true); it gives no indication of the magnitude or clinical importance of the difference.<ref>{{cite journal | vauthors = Haase RF, Ellis MV, Ladany N | year = 1989 | title = Multiple Criteria for Evaluating the Magnitude of Experimental Effects | journal = Journal of Counseling Psychology | volume = 36 | issue = 4| pages = 511–516 | doi=10.1037/0022-0167.36.4.511}}</ref> When statistically significant results are achieved, they favor rejection of the null hypothesis, but they do not prove that the null hypothesis is false. Likewise, non-significant results do not prove that the null hypothesis is true; they also give no evidence of the truth or falsity of the hypothesis the researcher has generated.<ref name="Nursing Research"/> Statistical significance relates only to the compatibility between observed data and what would be expected under the assumption that the null hypothesis is true.

===Practical significance=== {{main|effect size}}

In broad usage, the "practical clinical significance" answers the question, ''how effective'' is the intervention or treatment, or how much change does the treatment cause. In terms of testing clinical treatments, practical significance optimally yields quantified information about the importance of a finding, using metrics such as effect size, number needed to treat (NNT), and preventive fraction.<ref name="ReferenceB">{{cite journal | vauthors = Shabbir SH, Sanders AE | title = Clinical significance in dementia research: a review of the literature | journal = American Journal of Alzheimer's Disease and Other Dementias | volume = 29 | issue = 6 | pages = 492–7 | date = September 2014 | pmid = 24526758 | doi = 10.1177/1533317514522539 | doi-access = free | pmc = 10852744 }}</ref> Practical significance may also convey semi-quantitative, comparative, or feasibility assessments of utility.

Effect size is one type of practical significance.<ref name="ReferenceB"/><ref name="Peterson_2008"/> It quantifies the extent to which a sample diverges from expectations.<ref>{{cite journal | vauthors = Vacha-Haase T, Nilsson JE, Reetz DR, Lance TS, Thompson B | date = June 2000 | title = Reporting practices and APA editorial policies regarding statistical significance and effect size. | journal = Theory & Psychology | volume = 10 | issue = 3 | pages = 413–425 | doi = 10.1177/0959354300103006 }}</ref> Effect size can provide important information about the results of a study, and are recommended for inclusion in addition to statistical significance. Effect sizes have their own sources of bias, are subject to change based on population variability of the dependent variable, and tend to focus on group effects, not individual changes.<ref name="Peterson_2008"/><ref name="Cohen">{{cite journal | vauthors = Cohen J | date = 1997 | title = The earth is round (p < 0.05) | journal = The American Psychologist | volume = 49 | issue = 12 | pages = 997–1003 | doi = 10.1037/0003-066X.49.12.997 }}</ref><ref>{{cite journal | vauthors = Wilkinson L | year = 1999 | title = Statistical methods in psychology journals: Guidelines and explanations | journal = American Psychologist | volume = 54 | issue = 8| pages = 594–604 | doi=10.1037/0003-066x.54.8.594 | bibcode = 1999AmPsy..54..594W }}</ref>

Although clinical significance and practical significance are often used synonymously, a more technical restrictive usage denotes this as erroneous.<ref name="Peterson_2008">{{cite conference | vauthors = Peterson L | title = Clinical" Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. | conference = Annual Meeting of the Southwest Educational Research Association | location = New Orleans, LA | date = 7 February 2008 }}</ref> This technical use within psychology and psychotherapy not only results from a carefully drawn precision and particularity of language, but it enables a shift in perspective from group effects to the specifics of change(s) within an individual.{{citation needed|date=August 2023}}

===Specific usage=== In contrast, when used as a technical term within psychology and psychotherapy, clinical significance yields information on whether a treatment was effective enough to change a patient's diagnostic label. In terms of clinical treatment studies, clinical significance answers the question "Is a treatment effective enough to cause the patient to be normal [with respect to the diagnostic criteria in question]?"{{citation needed|date=August 2023}}

For example, a treatment might significantly change depressive symptoms (statistical significance), the change could be a large decrease in depressive symptoms (practical significance- effect size), and 40% of the patients no longer met the diagnostic criteria for depression (clinical significance). It is very possible to have a treatment that yields a significant difference and medium or large effect sizes, but does not move a patient from dysfunctional to functional.{{citation needed|date=August 2023}}

Within psychology and psychotherapy, '''clinical significance''' was first proposed by Jacobson, Follette, and Revenstorf<ref>{{cite journal | vauthors = Jacobson NS, Follette WC, Revenstorf D | title = Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. | journal = Behavior Therapy | date = September 1984 | volume = 15 | issue = 4 | pages = 336–52 | doi = 10.1016/S0005-7894(84)80002-7 }}</ref> as a way to answer the question, is a therapy or treatment effective enough such that a client does not meet the criteria for a diagnosis? Jacobson and Truax later defined clinical significance as "the extent to which therapy moves someone outside the range of the dysfunctional population or within the range of the functional population."<ref name="Jacobson_1991">{{cite journal | vauthors = Jacobson NS, Truax P | title = Clinical significance: a statistical approach to defining meaningful change in psychotherapy research | journal = Journal of Consulting and Clinical Psychology | volume = 59 | issue = 1 | pages = 12–9 | date = February 1991 | pmid = 2002127 | doi = 10.1037/0022-006x.59.1.12 | s2cid = 28125243 }}</ref> They proposed two components of this index of change: the status of a patient or client after therapy has been completed, and "how much change has occurred during the course of therapy."<ref name="Jacobson_1991"/>

Clinical significance is also a consideration when interpreting the results of the psychological assessment of an individual. Frequently, there will be a difference of scores or subscores that is statistically significant, unlikely to have occurred purely by chance. However, not all of those statistically significant differences are clinically significant, in that they do not either explain existing information about the client, or provide useful direction for intervention. Differences that are small in magnitude typically lack practical relevance and are unlikely to be clinically significant. Differences that are common in the population are also unlikely to be clinically significant, because they may simply reflect a level of normal human variation. Additionally, clinicians look for information in the assessment data and the client's history that corroborates the relevance of the statistical difference, to establish the connection between performance on the specific test and the individual's more general functioning.<ref>{{cite book | vauthors = Sattler JM | date = 2008 | title = Assessment of children: Cognitive foundations | edition = 5th | location = San Diego | publisher = Sattler Publications | isbn = 978-0-9702671-6-0}}</ref><ref>{{cite book |title=Assessing Adolescent and Adult Intelligence | vauthors = Kaufman AS, Lichtenberger E |author-link1=Alan S. Kaufman |edition=3rd |year=2006|publisher=Wiley |location=Hoboken (NJ) |isbn=978-0-471-73553-3}}</ref>

==Calculation of clinical significance== Just as there are many ways to calculate statistical significance and practical significance, there are a variety of ways to calculate clinical significance. Five common methods are the Jacobson-Truax method, the Gulliksen-Lord-Novick method, the Edwards-Nunnally method, the Hageman-Arrindell method, and hierarchical linear modeling.<ref name="Peterson_2008"/>

===Jacobson-Truax=== Jacobson-Truax is common method of calculating clinical significance. It involves calculating a Reliability Change Index (RCI).<ref name="Jacobson_1991"/> The RCI equals the difference between a participant's pre-test and post-test scores, divided by the standard error of the difference. Cutoff scores are established for placing participants into one of four categories: recovered, improved, unchanged, or deteriorated, depending on the directionality of the RCI and whether the cutoff score was met.{{citation needed|date=August 2023}}

===Gulliksen-Lord-Novick=== The Gulliksen-Lord-Novick method<ref>{{cite journal | vauthors = Hsu LM | title = A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell | journal = Behaviour Research and Therapy | volume = 37 | issue = 12 | pages = 1195–202; discussion 1219–33 | date = December 1999 | pmid = 10596465 | doi = 10.1016/S0005-7967(99)00033-9 }}</ref> is similar to Jacobson-Truax, except that it takes into account regression to the mean. This is done by subtracting the pre-test and post-test scores from a population mean, and dividing by the standard deviation of the population.<ref name="Peterson_2008"/>

===Edwards-Nunnally=== The Edwards-Nunnally method<ref>{{cite journal | vauthors = Speer DC, Greenbaum PE | title = Five methods for computing significant individual client change and improvement rates: support for an individual growth curve approach | journal = Journal of Consulting and Clinical Psychology | volume = 63 | issue = 6 | pages = 1044–8 | date = December 1995 | pmid = 8543708 | doi = 10.1037/0022-006x.63.6.1044 }}</ref> of calculating clinical significance is a more stringent alternative to the Jacobson-Truax method.<ref name="Peterson_2008" /> Reliability scores are used to bring the pre-test scores closer to the mean, and then a confidence interval is developed for this adjusted pre-test score. Confidence intervals are used when calculating the change from pre-test to post-test, so greater actual change in scores is necessary to show clinical significance, compared to the Jacobson-Truax method.{{citation needed|date=August 2023}}

===Hageman-Arrindell=== The Hageman-Arrindell<ref>{{cite journal | vauthors = Hageman WJ, Arrindell WA | title = Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis | journal = Behaviour Research and Therapy | volume = 37 | issue = 12 | pages = 1169–93 | date = December 1999 | pmid = 10596464 | doi = 10.1016/s0005-7967(99)00032-7 }}</ref> calculation of clinical significance involves indices of group change and of individual change. The reliability of change indicates whether a patient has improved, stayed the same, or deteriorated. A second index, the clinical significance of change, indicates four categories similar to those used by Jacobson-Truax: deteriorated, not reliably changed, improved but not recovered, and recovered.{{citation needed|date=August 2023}}

===Hierarchical linear modeling (HLM)=== {{main|Hierarchical linear modeling}} HLM involves growth curve analysis instead of pre-test post-test comparisons, so three data points are needed from each patient, instead of only two data points (pre-test and post-test).<ref name="Peterson_2008" /> A computer program, such as Hierarchical Linear and Nonlinear Modeling<ref>{{cite web |url=http://www.ssicentral.com/hlm/index.html | title=SSI - Scientific Software International, Inc | access-date=19 July 2009 | archive-url=https://web.archive.org/web/20090602193811/http://www.ssicentral.com/hlm/index.html | archive-date=2 June 2009 | url-status=dead }}</ref> is used to calculate change estimates for each participant. HLM also allows for analysis of growth curve models of dyads and groups.{{citation needed|date=August 2023}}

== References == {{reflist|30em}}

{{DEFAULTSORT:Clinical Significance}} Category:Clinical research Category:Clinical trials Category:Biostatistics