{{short description|Statistical confidence interval for success counts}} In statistics, a '''binomial proportion confidence interval''' is a confidence interval for the probability of success calculated from the outcome of a series of success–failure experiments (Bernoulli trials). In other words, a binomial proportion confidence interval is an interval estimate of a success probability <math> p </math> when only the number of experiments <math> n </math> and the number of successes <math> n_\mathsf{s} </math> are known.
There are several formulas for a binomial confidence interval, but all of them rely on the assumption of a binomial distribution. In general, a binomial distribution applies when an experiment is repeated a fixed number of times, each trial of the experiment has two possible outcomes (success and failure), the probability of success is the same for each trial, and the trials are statistically independent. Because the binomial distribution is a discrete probability distribution (i.e., not continuous) and difficult to calculate for large numbers of trials, a variety of approximations are used to calculate this confidence interval, all with their own tradeoffs in accuracy and computational intensity.
A simple example of a binomial distribution is the set of various possible outcomes, and their probabilities, for the number of heads observed when a coin is flipped ten times. The observed binomial proportion is the fraction of the flips that turn out to be heads. Given this observed proportion, the confidence interval for the true probability of the coin landing on heads is a range of possible proportions, which may or may not contain the true proportion. A 95% confidence interval for the proportion, for instance, will contain the true proportion 95% of the times that the procedure for constructing the confidence interval is employed.<ref name=Sullivan> {{cite web |last = Sullivan |first = Lisa |date = 2017-10-27 |title = Confidence Intervals |id = BS704 |type = course notes |website = sphweb.bumc.bu.edu |place = Boston, MA |publisher = Boston University School of Public Health |url = http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Confidence_Intervals/ }} </ref>
== Problems with using a normal approximation or "Wald interval" {{anchor|Normal approximation interval|Wald interval}} == [[File:Normal_approx_interval_and_logistic_example.png|thumb|300px|Plotting the normal approximation interval on an arbitrary logistic curve reveals problems of ''overshoot'' and ''zero-width intervals''.<ref name=Newcombe-1998/>]]
A commonly used formula for a binomial confidence interval relies on approximating the distribution of error about a binomially-distributed observation, <math>\ \hat p\ ,</math> with a normal distribution.<ref name="Wallis2013"> {{cite journal |last = Wallis |first = Sean A. |year = 2013 |title = Binomial confidence intervals and contingency tests: Mathematical fundamentals and the evaluation of alternative methods |journal = Journal of Quantitative Linguistics |volume = 20 |issue = 3 |pages = 178–208 |doi = 10.1080/09296174.2013.799918 |s2cid = 16741749 |url = http://www.ucl.ac.uk/english-usage/staff/sean/resources/binomialpoisson.pdf }} </ref> The normal approximation depends on the de Moivre–Laplace theorem (the original, binomial-only version of the central limit theorem) and becomes unreliable when it violates the theorems' premises, as the sample size becomes small or the success probability grows close to either {{math|0}} or {{nobr| {{math|1}} .}}<ref name=Brown2001> {{cite journal | last1 = Brown | first1 = Lawrence D. | author1-link = Lawrence D. Brown | last2 = Cai | first2 = T. Tony | author-link2 = T. Tony Cai | last3 = DasGupta | first3 = Anirban | year = 2001 | title = Interval estimation for a binomial proportion | journal = Statistical Science | volume = 16 | issue = 2 | pages = 101–133 | doi = 10.1214/ss/1009213286 | mr = 1861069 | zbl = 1059.62533 | citeseerx = 10.1.1.50.3025 }}</ref>
Using the normal approximation, the success probability <math>\ p\ </math> is estimated by
<math display="block"> p ~ \approx ~ \hat p \pm \frac{ z_\alpha }{ \sqrt{n\ } }\sqrt{ \hat p \left(1 - \hat p \right)\ }\ ,</math>
where <math>\ \hat p \equiv \frac{ n_\mathsf{s} }{ n }\ </math> is the proportion of successes in a Bernoulli trial process and an estimator for <math> p </math> in the underlying Bernoulli distribution. The equivalent formula in terms of observation counts is
<math display="block">p ~ \approx ~ \frac{ n_\mathsf{s} }{ n } \pm \frac{ z_\alpha }{ \sqrt{n\ } } \sqrt{ \frac{ n_\mathsf{s} }{ n } \frac{ n_\mathsf{f} }{ n }\ }\ ,</math>
where the data are the results of <math>\ n\ </math> trials that yielded <math>\ n_\mathsf{s}\ </math> successes and <math>\ n_\mathsf{f} = n - n_\mathsf{s}\ </math> failures. The distribution function argument <math>\ z_\alpha\ </math> is the <math>\ 1 - \tfrac{ \alpha }{2}\ </math> quantile of a standard normal distribution (i.e., the probit) corresponding to the target error rate <math>\ \alpha ~.</math> For a 95% confidence level, the error <math>\ \alpha ~=~ 1 - 0.95 ~=~ 0.05\ ,</math> so that <math>\ 1 - \tfrac{ \alpha }{ 2 } = 0.975\ </math> and <math>\ z_{.05} = 1.96 ~.</math>
When using the Wald formula to estimate <math>\ p\ ,</math> or just considering the possible outcomes of this calculation, two problems immediately become apparent: * First, for <math>\ \hat p\ </math> approaching either {{math|1}} or {{math|0}}, the interval narrows to zero width (falsely implying certainty). * Second, for values of <math>\ \hat p < \frac{ 1 }{ 1 + n / z_\alpha^2 }\ </math> (probability too low / too close to {{math|0}}), the interval boundaries exceed <math>\ [0 , 1]\ </math> (''overshoot''). (Another version of the second, overshoot problem, arises when instead <math>\ 1 - \hat p\ </math> falls below the same upper bound: probability too high / too close to {{math|1}} .)
An important theoretical derivation of this confidence interval involves the inversion of a hypothesis test. Under this formulation, the confidence interval represents those values of the population parameter that would have large {{mvar|p}}-values if they were tested as a hypothesized population proportion.{{clarification needed|date=January 2024|reason=Hypothesis tests assume the population parameter value and derive <math> p </math>values, probabilities, for observed estimates under the assumption of that hypothesis. Confidence interval gives those values such that, assuming the center of the confidence interval as null hypothesis, estimates falling within the interval would be assigned <math> p </math>values too large to reject. Which is kind of what the above is saying but also not, hence why clarification is needed.}} The collection of values, <math>\ \theta\ ,</math> for which the normal approximation is valid can be represented as
<math display="block"> \left\{\quad \theta \quad \Bigg\vert \quad y_{\alpha} ~ \le ~ \frac{\hat p - \theta}{\sqrt{\tfrac{ 1 }{ n }\ \hat p \left(1 - \hat p\right)\ } } ~ \le ~ z_{\alpha} \quad\right\}\ ,</math>
where <math>\ y_{\alpha}\ </math> is the lower <math>\ \tfrac{ \alpha }{ 2 }\ </math> quantile of a standard normal distribution, vs. <math>\ z_{\alpha}\ ,</math> which is the ''upper'' {{big|(}} i.e., <math>\ 1 - \tfrac{ \alpha }{ 2 }\ </math>{{big|)}} quantile.
Since the test in the middle of the inequality is a Wald test, the normal approximation interval is sometimes called the '''Wald interval''' or '''Wald method''', after Abraham Wald, but it was first described by Laplace (1812).<ref> {{cite book |last = Laplace |first=P.S. |author-link = Pierre-Simon Laplace |year = 1812 |title = Théorie analytique des probabilités |lang=fr |trans-title = Analyitic Probability Theory |publisher = Ve. Courcier |page = 283 |url = https://archive.org/details/thorieanalytiqu00laplgoog }} </ref>
=== Bracketing the confidence interval === Extending the normal approximation and Wald-Laplace interval concepts, Michael Short has shown that inequalities on the approximation error between the binomial distribution and the normal distribution can be used to accurately bracket the estimate of the confidence interval around <math>\ p\ :</math><ref name=Short-2021> {{cite journal |last = Short |first = Michael |date = 2021-11-08 |title = On binomial quantile and proportion bounds: With applications in engineering and informatics |journal = Communications in Statistics - Theory and Methods |volume = 52 |issue = 12 |pages = 4183–4199 |doi = 10.1080/03610926.2021.1986540 |doi-access = free |s2cid = 243974180 |issn = 0361-0926 |df=dmy-all }} </ref>
<math display="block"> \frac{ k + C_\mathsf{L1} - z_\alpha \widehat{W} }{ n + z_\alpha^2 } ~\le~ p ~\le~ \frac{ k + C_\mathsf{U1} + z_\alpha \widehat{W} }{ n + z_\alpha^2 } </math>
with <math display="block"> \widehat{W} ~\equiv~ \sqrt{ \frac{ n k - k^2 + C_\mathsf{L2} n - C_\mathsf{L3} k + C_\mathsf{L4} }{ n }\ }\ ,</math>
and where <math>\ p\ </math> is again the (unknown) proportion of successes in a Bernoulli trial process (as opposed to <math>\ \hat p \equiv \frac{ n_\mathsf{s} }{ n }\ </math> that estimates it) measured with <math>\ n\ </math> trials yielding <math>\ k\ </math> successes, <math>\ z_\alpha\ </math> is the <math>\ 1 - \tfrac{\alpha}{2}\ </math> quantile of a standard normal distribution (i.e., the probit) corresponding to the target error rate <math>\ \alpha\ ,</math> and the constants <math>\ C_\mathsf{L1}\ ,</math> <math>C_\mathsf{L2}\ ,</math> <math>\ C_\mathsf{L3}\ ,</math> <math>\ C_\mathsf{L4}\ ,</math> <math>\ C_\mathsf{U1}\ ,</math> <math>\ C_\mathsf{U2}\ ,</math> <math>\ C_\mathsf{U3}\ ,</math> and <math>\ C_\mathsf{U4}\ </math> are simple algebraic functions of <math>\ z_\alpha ~.</math><ref name=Short-2021/> For a fixed <math>\ \alpha\ </math> (hence fixed <math>\ z_\alpha\ </math>), the above inequalities give easily computed one- or two-sided intervals which bracket the exact binomial upper and lower confidence limits corresponding to the error rate <math>\ \alpha ~.</math>
===Standard error of a proportion estimation when using weighted data===
To apply the principles of the weighted data correctly, you need to understand the type of the weight <ref>{{cite web |url=https://stats.oarc.ucla.edu/other/mult-pkg/faq/what-types-of-weights-do-sas-stata-and-spss-support |website=UCLA Office of Advanced Research Computing, Statistical Methods and Data Analytics |title=What types of weights do statistical packages support?}})</ref> you are dealing with. For analytic weights, the following calculation can be applied.
Let <math>\ X_1,\ \ldots,\ X_n\ </math> be such that each <math>\ X_i\ </math> is i.i.d from a Bernoulli( {{mvar|p}} ) distribution and weight <math>\ w_i\ </math> is the weight for each observation, with the (positive) weights <math>\ w_i\ </math> normalized so they sum to {{nobr| {{math|1}} .}} The weighted sample proportion is: <math display="inline">\ \hat p = \sum_{i=1}^n w_i X_i ~.</math> Since each of the <math>\ X_i\ </math> is independent from all the others, and each one has variance <math>\ \operatorname{var}\{\ X_i\ \} = p \left(1 - p\right)\ </math> for every <math>\ i = 1 ,\ \ldots ,\ n\ ;</math> the '''sampling variance of the proportion''' therefore is:<ref>{{cite web |title=How to calculate the standard error of a proportion using weighted data? |website = stats.stackexchange.com |id = 159220 / 253 |url=https://stats.stackexchange.com/a/159220/253 }}</ref>
<math display="block"> \operatorname{var}\left\{\ \hat p\ \right\} ~=~ \sum_{i=1}^n \operatorname{var}\left\{\ w_i X_i\ \right\} ~=~ p\left( 1 - p \right) \sum_{i=1}^n w_i^2 ~.</math>
The '''standard error''' of <math>\ \hat p\ </math> is the square root of this quantity. Because we do not know <math>\ p \left(1 - p\right)\ ,</math> we have to estimate it. Although there are many possible estimators, a conventional one is to use <math>\ \hat p\ ,</math> the sample mean, and plug this into the formula. That gives:
<math display="block"> \operatorname{SE}\left\{\ \hat p\ \right\} ~\approx~ \sqrt{ \hat p \left(1 - \hat p\right) \sum_{i=1}^n w_i^2 ~} ~.</math>
For otherwise unweighted data, the effective weights are uniform <math display="inline">\ w_i = \frac{ 1 }{ n }\ ,</math> giving <math display="inline">\ \sum_{i=1}^n w_i^2 = \frac{ 1 }{ n } ~.</math> The <math>\ \operatorname{SE}\ </math> becomes <math display="inline">\ \sqrt{ \tfrac{ 1 }{ n }\ \hat p \left( 1 - \hat p \right)\ }\ ,</math> leading to the familiar formulas, showing that the calculation for weighted data is a direct generalization of them.
If the weights in question are the complex sampling design weights, appropriate statistical software (R package {{mono|survey}}<ref name=Lumley-survey>{{cite web |last=Lumley |first=Thomas |title=survey: Analysis of Complex Survey Samples |url=https://cran.r-project.org/web/packages/survey/index.html |access-date=2026-04-01 |website=The Comprehensive R Archive Network}}</ref>; Python package {{mono|svy}}<ref>{{cite web |last=Diallo |first=Mamadou |title=svy: Python Package for Complex Survey Design and Analysis |url=https://svylab.com/docs/svy/ |access-date=2026-04-01}}</ref>) needs to be used to obtain standard errors corrected for the survey sampling design. An extension of the {{section link||Clopper-Pearson interval}} presented below to the complex survey data is also known as the Korn-Graubard confidence interval<ref>{{cite journal |last1 = Ward |first1 = Brian W. |title = kg_nchs: A command for Korn–Graubard confidence intervals and National Center for Health Statistics’ Data Presentation Standards for Proportions |url = https://doi.org/10.1177/1536867X19874 |journal = The Stata Journal |volume = 19 |issue = 3 |publisher = Stata Corp |pages = 510-522 |doi = 10.1177/1536867X19874 |access-date = 2026-04-01 }}</ref>.
==Wilson score interval== thumb|300px|Wilson score intervals plotted on a logistic curve, revealing asymmetry and good performance for small {{mvar|n}} and where {{mvar|p}} is at or near 0 or 1. The '''Wilson score interval''' was developed by E.B. Wilson (1927).<ref name=Wilson1927> {{cite journal | last = Wilson | first = E.B. | author-link = Edwin Bidwell Wilson | year = 1927 | title = Probable inference, the law of succession, and statistical inference | journal = Journal of the American Statistical Association | volume = 22 | issue = 158 | pages = 209–212 | jstor = 2276774 | doi = 10.1080/01621459.1927.10502953 }} </ref> It is an improvement over the normal approximation interval in multiple respects: Unlike the symmetric normal approximation interval (above), the Wilson score interval is ''asymmetric'', and it doesn't suffer from problems of ''overshoot'' and ''zero-width intervals'' that afflict the normal interval. It can be safely employed with small samples and skewed observations.<ref name=Wallis2013/> The observed coverage probability is consistently closer to the nominal value, <math>\ 1 - \alpha ~.</math><ref name=Newcombe-1998/><ref name=Dean-Pagano-2015/>.
Like the normal interval, the interval can be computed directly from a formula.
Wilson started with the normal approximation to the binomial: <math display="block"> z_\alpha\ \approx\ \frac{ p - \hat{p}}{ \sigma_n } </math> where <math>\ z_\alpha\ </math> is the standard normal interval half-width corresponding to the desired confidence <math>\ 1 - \alpha ~.</math> The analytic formula for a binomial sample standard deviation is <math display="block"> \sigma_n = \sqrt{\frac{ p \left( 1 - p \right) }{n} } ~.</math> Combining the two, and squaring out the radical, gives an equation that is quadratic in <math>\ p\ :</math> <!-- \hat{p} = p \pm z_\alpha \sqrt{\frac{p\left(1 - p\right)}{n}} \qquad\Longrightarrow\qquad --> <math display="block"> \left( p - \hat{p} \right)^2 ~=~ \frac{\ z_\alpha^2 }{ n }\ p \left( 1 - p \right) </math> or <math display="block"> p^2\ -\ 2\ p\ \hat{p}\ +\ {\hat{p}}^2 ~=~ p\ \frac{ z_\alpha^2 }{ n }\ -\ p^2\ \frac{ z_\alpha^2 }{ n } ~. </math> Transforming the relation into a standard-form quadratic equation for <math>\ p\ ,</math> treating <math>\ \hat p\ </math> and <math>\ n\ </math> as known values from the sample (see prior section), and using the value of <math>\ z_\alpha\ </math> that corresponds to the desired confidence <math>\ 1 - \alpha\ </math> for the estimate of <math>\ p\ </math> gives this: <math display="block"> \left( 1 + \frac{ z_\alpha^2 }{ n } \right) p^2 - \left( 2 {\hat p} + \frac{ z_\alpha^2 }{ n } \right) p + \biggl(\ {\hat p}^2 \biggr) = 0\ , </math> where all of the values bracketed by parentheses are known quantities. The solution for <math>\ p\ </math> estimates the upper and lower limits of the confidence interval for <math>\ p ~.</math> Hence the probability of success <math>\ p\ </math> is estimated by <math>\ \hat p\ </math> and with <math>\ 1 - \alpha\ </math> confidence bracketed in the interval <math display="block"> p ~~ \underset{\approx}{\in}_\alpha \; \left(\ w^- , w^+\ \right) ~=~ \frac{ 1 }{ 1 + z_\alpha^2 / n } \left(\ \hat p + \frac{ z_\alpha^2 }{ 2\ n }\ \pm\ \frac{ z_\alpha }{ 2\ n } \sqrt{ 4\ n\ \hat p \left(1 - \hat p\right) + z_\alpha^2\ }\ \right) </math>
where <math>\ \underset{\approx}{\in}_\alpha\ </math> is an abbreviation for
<math display="block"> \operatorname{\mathbb P} \left\{\ p \in \left( w^- , w^+ \right)\ \right\} = 1 - \alpha ~.</math>
An equivalent expression using the observation counts <math>\ n_\mathsf{s}\ </math> and <math>\ n_\mathsf{f}\ </math> is <math display="block"> p ~~ \underset{\approx}{\in}_\alpha \; \frac{\ n_\mathsf{s} + \tfrac{ 1 }{ 2 }\ z_\alpha^2 }{\ n + z_\alpha^2 } ~\pm~ \frac{\ z_\alpha }{~ n + z_\alpha^2\ } \sqrt{ \frac{\ n_\mathsf{s}\ n_\mathsf{f} }{ n } + \frac{\ z_\alpha^2 }{ 4 } ~}\ , </math>
with the counts as above: <math>\ n_\mathsf{s} \equiv </math> the count of observed "successes", <math>\ n_\mathsf{f} \equiv </math> the count of observed "failures", and their sum is the total number of observations <math>\ n = n_\mathsf{s} + n_\mathsf{f} ~.</math>
In practical tests of the formula's results, users find that this interval has good properties even for a small number of trials and / or the extremes of the probability estimate, <math>\ \hat p \equiv \frac{ n_\mathsf{s} }{ n } ~.</math><ref name=Newcombe-1998/><ref name=Wallis2013/><ref name=Wallis2021/>
Intuitively, the center value of this interval is the weighted average of <math>\ \hat{p}\ </math> and <math display="inline">\ \tfrac{ 1 }{ 2 }\ ,</math> with <math>\ \hat{p}\ </math> receiving greater weight as the sample size increases. Formally, the center value corresponds to using a pseudocount of <math>\ \tfrac{ 1 }{ 2 } z_\alpha^2\ ,</math> the number of standard deviations of the confidence interval: Add this number to both the count of successes and of failures to yield the estimate of the ratio. For the common two standard deviations in each direction interval (approximately 95% coverage, which itself is approximately 1.96 standard deviations), this yields the estimate <math>\ \frac{n_\mathsf{s} + 2 }{ n + 4 }\ ,</math> which is known as the "plus four rule".
Although the quadratic can be solved explicitly, in most cases Wilson's equations can also be solved numerically using the fixed-point iteration <math display="block"> p_{k+1} = \hat{p} \pm z_\alpha\sqrt{ \tfrac{1}{ n }p_k\left(1 - p_k\right) ~} </math> with <math>\ p_0 = \hat{p} ~.</math>
The Wilson interval can also be derived from the single sample z-test or Pearson's chi-squared test with two categories. The resulting interval,
<math display="block"> \left\{~ \theta \quad \Bigg| \quad y_\alpha ~~ \le ~~ \frac{\hat{p} - \theta}{\sqrt{ \tfrac{\!1\!}{ n }\theta\left( 1 - \theta\right) ~}} ~~ \le ~~ z_\alpha ~ \right\}\ , </math>
(with <math> y_\alpha </math> the lower <math> \alpha </math> quantile) can then be solved for <math> \theta </math> to produce the Wilson score interval. The test in the middle of the inequality is a score test.
===The interval equality principle=== [[File:Wilson score pdf and interval equality.png|thumb|300px|The probability density function ({{sc|pdf}}) for the Wilson score interval, plus {{sc|pdf}}s at interval bounds. Tail areas are equal.]] Since the interval is derived by solving from the normal approximation to the binomial, the Wilson score interval <math>\ \left(w^-,w^+\right)\ </math> has the property of being guaranteed to obtain the same result as the equivalent z-test or chi-squared test.
This property can be visualised by plotting the probability density function for the Wilson score interval (''see'' Wallis).<ref name="Wallis2021"> {{cite book | last = Wallis | first = Sean A. | year = 2026 | title = Statistics in Corpus Linguistics: A new approach | place = New York, NY | publisher = Routledge | isbn = 9781003622697 | url = https://www.routledge.com/Statistics-in-Corpus-Linguistics-Research-A-New-Approach/Wallis/p/book/9781041031871 |edition=2nd}} </ref>{{rp|style=ama|pp=399-421}} After that, then also plotting a normal {{sc|pdf}} across each bound. The tail areas of the resulting Wilson and normal distributions represent the chance of a significant result, in that direction, must be equal.
The continuity-corrected Wilson score interval and the Clopper-Pearson interval are also compliant with this property. The practical import is that these intervals may be employed as significance tests, with identical results to the source test, and new tests may be derived by geometry.<ref name=Wallis2021/>
===Wilson score interval with continuity correction=== The Wilson interval may be modified by employing a continuity correction, in order to align the minimum coverage probability, rather than the average coverage probability, with the nominal value, <math>\ 1 - \alpha ~.</math>
Just as the Wilson interval mirrors Pearson's chi-squared test, the Wilson interval with continuity correction mirrors the equivalent Yates' chi-squared test.
The following formulae for the lower and upper bounds of the Wilson score interval with continuity correction <math>\ \left( w_\mathsf{cc}^- , w_\mathsf{cc}^+ \right)\ </math> are derived from Newcombe:<ref name=Newcombe-1998> {{cite journal | last = Newcombe | first = R.G. | year = 1998 | title = Two-sided confidence intervals for the single proportion: Comparison of seven methods | journal = Statistics in Medicine | volume = 17 | issue = 8 | pages = 857–872 | pmid = 9595616 | doi = 10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E }} </ref><math display="block">\begin{align} w_\mathsf{cc}^- &= \max \left\{ ~ 0, ~ \frac{2n\hat{p} + z_\alpha^2 - \left[z_\alpha\sqrt{ z_\alpha^2 - \frac{1}{ n } + 4n\hat{p}\left(1 - \hat{p}\right) + \left( 4\hat{p} - 2 \right) ~}+1\right]}{2 \left( n + z_\alpha^2 \right) ~} \right\}\ ,\\ w_\mathsf{cc}^+ &= \min \left\{ ~~ 1, ~~ \frac{2n\hat{p} + z_\alpha^2 + \left[z_\alpha\sqrt{ z_\alpha^2 - \frac{1}{ n } + 4n\hat{p}\left( 1 - \hat{p} \right) - \left( 4\hat{p} - 2 \right) ~}+1\right]}{2 \left( n + z_\alpha^2 \right) ~~} \right\}\ , \end{align} </math> for <math>\ \hat p \ne 0\ </math> and <math>\ \hat p \ne 1 ~.</math>
If <math>\ \hat p = 0\ ,</math> then <math>\ w_\mathsf{cc}^-\ </math> must instead be set to <math>\ 0\ ;</math> if <math>\ \hat p = 1\ ,</math> then <math>\ w_\mathsf{cc}^+\ </math> must be instead set to <math>\ 1 ~.</math>
Wallis (2026)<ref name="Wallis2021" /> identifies a simpler method for computing continuity-corrected Wilson intervals that employs a special function based on Wilson's formula: In Wallis' notation, for the lower bound, let <math display="block"> \mathsf{Wilson_{lower}}\!\left(\ \hat{p},\ n,\ \alpha \right) ~\equiv~ w^- ~=~ \frac{ 1 }{~ 1 + z_\alpha^2/ n ~} \Biggl(\ \hat p + \frac{\; z_\alpha^2}{ 2n } ~~ - ~~ \frac{\ z_\alpha }{ 2n } \sqrt{4n\hat p(1 - \hat p) + z_\alpha^2 ~} ~ \Biggr)\ , </math>where <math>\ \alpha\ </math> is the selected tolerable error level for <math>\ z_\alpha</math>. The upper bound, <math>\mathsf{Wilson_{upper}}</math> is obtained from<math display="block"> \mathsf{Wilson_{upper}}\!\left(\ \hat{p},\ n,\ \alpha \right) ~\equiv~ w^+ ~=~ \frac{ 1 }{~ 1 + z_\alpha^2/ n ~} \Biggl(\ \hat p + \frac{\; z_\alpha^2}{ 2n } ~~ + ~~ \frac{\ z_\alpha }{ 2n } \sqrt{4n\hat p(1 - \hat p) + z_\alpha^2 ~} ~ \Biggr)\ .</math>Then <math display="block"> w_\mathsf{cc}^- ~=~ \mathsf{Wilson_{lower}}\!\left(\ \max\left\{\ \hat{p} - \tfrac{ 1 }{ 2 n },\ 0\ \right\},\ n,~~ \alpha \right) ~,</math>and<math display="block"> w_\mathsf{cc}^+ ~=~ \mathsf{Wilson_{upper}}\!\left(\ \min\left\{\ \hat{p} + \tfrac{ 1 }{ 2 n },\ 1\ \right\},\ n,~~ \alpha \right) ~.</math>This method has the advantage of being further decomposable.
To apply a continuity correction to the Wilson interval with such an adjusted <math>\tilde n</math>, we modify the second parameter of Wallis' Wilson formulae above, but leave the continuity adjustment term <math>\tfrac 1 {2n}</math> unaltered (Wallis 2026).<ref name="Wallis2021" />
===Wilson interval with complex survey data===
For complex survey data, such as when sampling from a finite population, or sampling with unequal weighting and clustering, the sample size <math>n</math> needs to be replaced with the effective sample size <math>\tilde n = n/{\rm DEFF}</math> that accounts for the design effect of sampling.
It may also be reasonable to replace the normal quantiles <math>z_\alpha</math> with the Student distribution quantiles <math>t_\alpha(d)</math> where <math>d</math> is the sampling design degrees of freedom.
==Jeffreys interval== thumb|300px|Jeffreys intervals plotted on a logistic curve, revealing asymmetry and good performance for small {{mvar|n}} and where {{mvar|p}} is at or near 0 or 1. The ''Jeffreys interval'' has a Bayesian derivation, but good frequentist properties (outperforming most frequentist constructions). In particular, it has coverage properties that are similar to those of the Wilson interval, but it is one of the few intervals with the advantage of being ''equal-tailed'' (e.g., for a 95% confidence interval, the probabilities of the interval lying above or below the true value are both close to 2.5%). In contrast, the Wilson interval has a systematic bias such that it is centred too close to {{nowrap|<math> p = 0.5 </math>.}}<ref> {{cite journal | last = Cai | first = T.T. | author-link = T. Tony Cai | year = 2005 | title = One-sided confidence intervals in discrete distributions | journal = Journal of Statistical Planning and Inference | volume = 131 | issue = 1 | pages = 63–88 | doi=10.1016/j.jspi.2004.01.005 }} </ref>
The Jeffreys interval is the Bayesian credible interval obtained when using the non-informative Jeffreys prior for the binomial proportion <math> p ~.</math> The Jeffreys prior for this problem is a Beta distribution with parameters <math> \left( \tfrac{\!1\!}{ 2 }, \tfrac{\!1\!}{ 2 } \right),</math> a conjugate prior. After observing <math> x </math> successes in <math> n </math> trials, the posterior distribution for <math> p </math> is a Beta distribution with parameters <math> \left( x + \tfrac{\!1\!}{ 2 }, n - x + \tfrac{\!1\!}{ 2 } \right) ~.</math>
When <math> x \ne 0 </math> and <math> x \ne n,</math> the Jeffreys interval is taken to be the <math> 100\left( 1 - \alpha \right)\mathrm{%} </math> equal-tailed posterior probability interval, i.e., the <math> \tfrac{\!1\!}{ 2 }\alpha </math> and <math> 1 - \tfrac{\!1\!}{ 2 }\alpha </math> quantiles of a Beta distribution with parameters <math> \left(x + \tfrac{1}{ 2 },n - x + \tfrac{1}{ 2 }\right) ~.</math>
In order to avoid the coverage probability tending to zero when <math> p \to 0 </math> or {{math|1}} , when <math> x = 0 </math> the upper limit is calculated as before but the lower limit is set to {{math|0}} , and when <math> x = n </math> the lower limit is calculated as before but the upper limit is set to {{math|1}} .<ref name=Brown2001/>
Jeffreys' interval can also be thought of as a frequentist interval based on inverting the p-value from the G-test after applying the Yates correction to avoid a potentially-infinite value for the test statistic.{{cn|date=March 2025}}
==Clopper–Pearson interval==
The Clopper–Pearson interval is an early and very common method for calculating binomial confidence intervals.<ref> {{Cite journal | last1 = Clopper | first1 = C. | last2 = Pearson | first2 = E.S. | author-link2 = Egon Pearson | year = 1934 | title = The use of confidence or fiducial limits illustrated in the case of the binomial | journal = Biometrika | volume = 26 | issue = 4 | pages = 404–413 | doi = 10.1093/biomet/26.4.404 }} </ref> This is often called an 'exact' method, as it attains the nominal coverage level in an exact sense, meaning that the coverage level is never less than the nominal {{nowrap|<math> 1 - \alpha </math>.}}<ref name=Newcombe-1998/>
The Clopper–Pearson interval can be written as
<math display="block">S_{\le} \cap S_{\ge} </math>
or equivalently,
<math display="block">\left( \inf S_{\ge},\sup S_{\le} \right) </math>
with
<math display="block"> S_{\le} ~ \equiv ~ \left\{ ~ p ~ \Big| ~ \operatorname{\mathbb P} \left\{~ \operatorname{Bin}\left( n; p \right) \le x \right\} > \tfrac{\alpha}{ 2 } ~ \right\} ~</math> and <math display="block"> S_{\ge} ~ \equiv ~ \left\{ ~ p ~ \Big| ~ \operatorname{\mathbb P} \left\{~ \operatorname{Bin}\left( n; p \right) \ge x \right\} > \tfrac{\alpha}{ 2 } ~ \right\},</math>
where <math> 0 \le x \le n </math> is the number of successes observed in the sample and <math> \mathsf{Bin}\left( n; p \right) </math> is a binomial random variable with <math> n </math> trials and probability of success <math> p ~.</math>
Equivalently we can say that the Clopper–Pearson interval is <math display="inline">\left(\frac{\!x\!}{ n } - \varepsilon_1,\frac{\!x\!}{ n } + \varepsilon_2\right) </math> with confidence level <math> 1 - \alpha </math> if <math> \varepsilon_i </math> is the infimum of those such that the following tests of hypothesis succeed with significance <math display="inline">\frac{\!\alpha\!}{2}:</math>
# H<sub>0</sub>: <math> p = \frac{\!x\!}{ n } - \varepsilon_1 </math> with H<sub>A</sub>: <math> p > \frac{\!x\!}{ n } - \varepsilon_1</math> # H<sub>0</sub>: <math> p = \frac{\!x\!}{ n } + \varepsilon_2 </math> with H<sub>A</sub>: <math> p < \frac{\!x\!}{ n } + \varepsilon_2 ~.</math>
Because of a relationship between the binomial distribution and the beta distribution, the Clopper–Pearson interval is sometimes presented in an alternate format that uses quantiles from the beta distribution.<ref name=Thulin2014> {{cite journal |last = Thulin |first = Måns |date = 2014-01-01 |title = The cost of using exact confidence intervals for a binomial proportion |journal = Electronic Journal of Statistics |volume = 8 |issue = 1 |pages = 817–840 |doi = 10.1214/14-EJS909 |issn = 1935-7524 |arxiv = 1303.1288 |s2cid = 88519382 |lang = en }} </ref>
<math display="block">B\!\left(\tfrac{\!\alpha\!}{ 2 };x,n - x + 1 \right) ~ < ~ p ~ < ~ B\!\left(1 - \tfrac{\!\alpha\!}{ 2 };x + 1,n - x\right) </math>
where <math> x </math> is the number of successes, <math> n </math> is the number of trials, and <math> B\!\left(p;v,w\right) </math> is the {{mvar|p}}th quantile from a beta distribution with shape parameters <math> v </math> and <math> w ~.</math>
Thus, <math> p_{\min} ~ < ~ p < ~ p_{\max},</math> where: <math display="block"> \tfrac{\Gamma(n+1)}{\Gamma\!( x )\Gamma\!( n-x+1 )}\int_0^{ p_{\min}}t^{x-1}(1-t)^{n-x}\mathrm{d}\!t ~~ = ~~ \tfrac{\!\alpha\!}{ 2 }, </math> <math display="block"> \tfrac{\Gamma(n+1)}{\Gamma(x+1)\Gamma(n-x)}\int_0^{ p_{\max}}t^{x}(1-t)^{n-x-1}\mathrm{d}\!t ~ = ~ 1 - \tfrac{\!\alpha\!}{ 2 } ~. </math> The binomial proportion confidence interval is then <math> \left(p_{\min},p_{\max} \right),</math> as follows from the relation between the Binomial distribution cumulative distribution function and the regularized incomplete beta function.
When <math> x </math> is either {{math|0}} or <math> n,</math> closed-form expressions for the interval bounds are available: when <math> x = 0 </math> the interval is <math display="block" display="inline">\left( 0,1 - \left(\tfrac{\!\alpha\!}{ 2 }\right)^{ 1/n } \right) </math> and when <math> x = n </math> it is <math display="block" display="inline">\left(\left( \tfrac{\!\alpha\!}{ 2 }\right)^{ 1 / n },1\right) ~.</math><ref name=Thulin2014/>
The beta distribution is, in turn, related to the F-distribution so a third formulation of the Clopper–Pearson interval can be written using F quantiles:
<math display="block"> \left(1 + \frac{n - x + 1}{x F\!\left[\tfrac{\alpha}{ 2 };2x,2(n - x + 1)\right]}\right)^{-1} ~ < ~ p ~ < ~ \left(1 + \frac{n - x}{(x + 1)F\!\left[1 - \tfrac{\alpha}{ 2 };2(x + 1),2(n - x)\right]}\right)^{-1} </math>
where <math> x </math> is the number of successes, <math> n </math> is the number of trials, and <math> F\!\left(c; d_1, d_2\right) </math> is the <math> c </math> quantile from an F-distribution with <math> d_1 </math> and <math> d_2 </math> degrees of freedom.<ref name=AgrestiCoull1998/>
The Clopper–Pearson interval is an 'exact' interval, since it is based directly on the binomial distribution rather than any approximation to the binomial distribution. This interval never has less than the nominal coverage for any population proportion, but that means that it is usually conservative. For example, the true coverage rate of a 95% Clopper–Pearson interval may be well above 95%, depending on <math> n </math> and <math> p ~.</math><ref name=Brown2001/> Thus the interval may be wider than it needs to be to achieve 95% confidence, and wider than other intervals. In contrast, it is worth noting that other confidence interval may have coverage levels that are lower than the nominal <math> 1 - \alpha,</math> i.e., the normal approximation (or "standard") interval, Wilson interval,<ref name=Wilson1927/> Agresti–Coull interval,<ref name=AgrestiCoull1998> {{cite journal | last1 = Agresti | first1 = Alan | author-link1 = Alan Agresti | last2 = Coull | first2 = Brent A. | year = 1998 | title = Approximate is better than 'exact' for interval estimation of binomial proportions | journal = The American Statistician | volume = 52 | issue = 2 | pages = 119–126 | mr = 1628435 | jstor = 2685469 | doi=10.2307/2685469 }} </ref> etc., with a nominal coverage of 95% may in fact cover less than 95%,<ref name=Brown2001/> even for large sample sizes.<ref name=Thulin2014/>
The definition of the Clopper–Pearson interval can also be modified to obtain exact confidence intervals for different distributions. For instance, it can also be applied to the case where the samples are drawn without replacement from a population of a known size, instead of repeated draws of a binomial distribution. In this case, the underlying distribution would be the hypergeometric distribution.
The interval boundaries can be computed with numerical functions {{mono|qbeta}}<ref> {{cite web |title = The Beta distribution |series = R Manual |type = software doc |website = stat.ethz.ch |url=https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Beta.html |access-date=2023-12-02 }} </ref> in R and <code>scipy.stats.beta.ppf</code><ref> {{cite report |section= scipy.stats.beta |title= SciPy Manual |edition = 1.11.4 |website = docs.scipy.org |type = software doc |section-url = https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.beta.html |access-date = 2023-12-02 }} </ref> in Python.
<syntaxhighlight lang="python"> from scipy.stats import beta import numpy as np
k = 20 n = 400 alpha = 0.05 p_u, p_o = beta.ppf([alpha / 2, 1 - alpha / 2], [k, k + 1], [n - k + 1, n - k]) if np.isnan(p_o): p_o = 1 if np.isnan(p_u): p_u = 0 </syntaxhighlight>
==Agresti–Coull interval==
The Agresti–Coull interval is also another approximate binomial confidence interval.<ref name=AgrestiCoull1998/>
Given <math> n_\mathsf{s} </math> successes in <math> n </math> trials, define <math display="block">\tilde n \equiv n + z^2_\alpha </math>
and <math display="block">\tilde p = \frac{ 1 }{\tilde n}\left(\!n_\mathsf{s} + \tfrac{z^2_\alpha }{ 2 }\!\right) </math>
Then, a confidence interval for <math> p </math> is given by
<math display="block"> p ~~ \approx ~~ \tilde p ~ \pm ~ z_\alpha\sqrt{ \frac{\!\tilde p\!}{ \tilde n }\left(\!1 - \tilde p \!\right) ~} </math>
where <math> z_\alpha = \operatorname{\Phi^{-1}}\!\!\left(1 - \tfrac{\!\alpha\!}{ 2 }\right) </math> is the quantile of a standard normal distribution, as before (for example, a 95% confidence interval requires <math> \alpha = 0.05,</math> thereby producing <math> z_{.05} = 1.96 </math>). According to Brown, Cai, & DasGupta (2001),<ref name=Brown2001/> taking <math> z = 2 </math> instead of 1.96 produces the "add 2 successes and 2 failures" interval previously described by Agresti & Coull.<ref name=AgrestiCoull1998/>
This interval can be summarised as employing the centre-point adjustment, <math> \tilde p,</math> of the Wilson score interval, and then applying the Normal approximation to this point.<ref name=Wallis2013/><ref name=Brown2001/>
<math display="block">\tilde p = \frac{\quad \hat p + \frac{z^2_\alpha\!}{2n} \quad}{\quad 1 + \frac{z^2_\alpha }{ n } \quad} </math>
==Arcsine transformation== {{main|Arcsine transformation}} {{further|Cohen's h}}
The arcsine transformation has the effect of pulling out the ends of the distribution.<ref> {{cite web |last=Holland|first=Steven |title=Transformations of proportions and percentages |url=http://strata.uga.edu/8370/rtips/proportions.html |access-date=2020-09-08 |website=strata.uga.edu }} </ref> While it can stabilize the variance (and thus confidence intervals) of proportion data, its use has been criticized in several contexts.<ref> {{cite journal |last1 = Warton |first1 = David I. |last2 = Hui |first2 = Francis K.C. |date = January 2011 |title = The arcsine is asinine: The analysis of proportions in ecology |journal = Ecology |volume = 92 |issue = 1 |pages = 3–10 |doi = 10.1890/10-0340.1 |pmid = 21560670 |bibcode = 2011Ecol...92....3W |issn = 0012-9658 |hdl = 1885/152287 |hdl-access = free |url = http://doi.wiley.com/10.1890/10-0340.1 |lang = en }} </ref>
Let <math> X </math> be the number of successes in <math> n </math> trials and let <math> p = \tfrac{ 1 }{\!n\!}\!X ~.</math> The variance of <math> p </math> is
<math display="block"> \operatorname{var}\{~~ p ~~\} = \tfrac{ 1 }{\!n\!}p(1 - p) ~.</math>
Using the arc sine transform, the variance of the arcsine of <math> \sqrt{p ~} </math> is<ref name=Shao1998> {{cite book |last = Shao |first = J. |year = 1998 |title = Mathematical Statistics |publisher = Springer |place = New York, NY }} </ref>
<math display="block">\operatorname{var} \left\{~~ \arcsin \sqrt{ p ~} ~~\right\} ~ \approx ~ \frac{\operatorname{var}\{~~ p ~~\}}{4p(1 - p)} = \frac{p(1 - p)}{4np(1 - p)} = \frac{1}{4n} ~.</math>
So, the confidence interval itself has the form
<math display="block">\sin^2 \left(-\frac{ z_\alpha }{2\sqrt{n}} + \arcsin\sqrt{p} ~\right) ~ < ~ \theta ~ < ~ \sin^2 \left(+\frac{z_\alpha}{2\sqrt{n}}+ \arcsin\sqrt{p} ~\right),</math>
where <math> z_\alpha </math> is the <math> 1 -\tfrac{\!\alpha\!}{2} </math> quantile of a standard normal distribution.
This method may be used to estimate the variance of <math> p </math> but its use is problematic when <math> p </math> is close to {{math|0}} or {{math|1}} .
==''t''<sub>''a''</sub> transform== {{unreferenced section|date=July 2017}} Let <math> p </math> be the proportion of successes. For <math> 0 \le a \le 2,</math>
<math display="block">t_a=\log\left(\frac{p^a}{(1 - p)^{2-a}}\right)=a\log p - (2-a)\log(1 - p) </math>
This family is a generalisation of the logit transform which is a special case with ''a'' = 1 and can be used to transform a proportional data distribution to an approximately normal distribution. The parameter ''a'' has to be estimated for the data set.
==Rule of three — for when no successes are observed== The rule of three is used to provide a simple way of stating an approximate 95% confidence interval for <math>p</math>, in the special case that no successes (<math>\hat p = 0</math>) have been observed.<ref> {{cite web |first = Steve |last = Simon |year = 2010 |title = Confidence interval with zero events |series= Ask Professor Mean |place = Kansas City, MO |publisher = The Children's Mercy Hospital |url = http://www.pmean.com/01/zeroevents.html |archive-url = https://web.archive.org/web/20111015182854/http://www.childrensmercy.org/stats/ |archive-date=15 October 2011 }} [http://www.childrensmercy.org/stats/ Stats topics on Medical Research] </ref> The interval is <math>\left(0, \tfrac{3}{n} \right)</math>.
By symmetry, in the case of only successes (<math>\hat p = 1</math>), the interval is <math>\left(1 - \tfrac{3}{n},1 \right)</math>.
==Comparison and discussion==
There are several research papers that compare these and other confidence intervals for the binomial proportion.<ref name=Wallis2013/><ref name=Newcombe-1998/><ref> {{cite conference |last1 = Sauro |first1 = J. |last2 = Lewis |first2 = J.R. |year = 2005 |title = Comparison of Wald, Adj-Wald, exact, and Wilson intervals calculator |conference = Human Factors and Ergonomics Society, 49th Annual Meeting (HFES 2005) |place = Orlando, FL |pages = 2100–2104 |url = http://www.measuringusability.com/papers/sauro-lewisHFES.pdf |archive-url = https://web.archive.org/web/20120618053914/http://www.measuringusability.com/papers/sauro-lewisHFES.pdf |archive-date = 2012-06-18 |df = dmy-all }} </ref><ref> {{cite journal | last = Reiczigel | first = J. | year = 2003 | title = Confidence intervals for the binomial parameter: Some new considerations | url = http://www.zoologia.hu/qp/Reiczigel_conf_int.pdf | journal = Statistics in Medicine | volume = 22 | issue = 4 | pages = 611–621 | doi=10.1002/sim.1320 | pmid = 12590417 | s2cid = 7715293 }} </ref><ref name=Dean-Pagano-2015>{{cite journal |last1 = Dean |first1 = Natalie |last2 = Pagano |first2 = Marcello |date = December 2015 |title = Evaluating Confidence Interval Methods for Binomial Proportions in Clustered Surveys |url = https://doi.org/10.1093/jssam/smv024 |journal = Journal of Survey Statistics and Methodology |volume = 3 |issue = 4 |publisher = American Association for Public Opinion Research |pages = 484–503 |doi = 10.1093/jssam/smv024 }} </ref><ref name=Franco-et-al-2019>{{cite journal |last1 = Franco |first1 = Carolina |last2 = Little |first2 = Roderick J A |last3 = Louis |first3 = Thomas A |last4 = Slud |first4 = Eric V |date = September 2019 |title = Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys |url = https://doi.org/10.1093/jssam/smy019 |journal = Journal of Survey Statistics and Methodology |volume = 7 |issue = 3 |publisher = American Association for Public Opinion Research |pages = 334–364 |doi = 10.1093/jssam/smy019 }} </ref>
Both Ross (2003)<ref> {{Cite journal | last = Ross | first = T.D. | year = 2003 | title = Accurate confidence intervals for binomial proportion and Poisson rate estimation | journal = Computers in Biology and Medicine | volume = 33 | issue = 6 | pages = 509–531 | doi = 10.1016/S0010-4825(03)00019-2 | pmid = 12878234 | url = https://zenodo.org/record/1259565 }} </ref> and Agresti & Coull (1998)<ref name=AgrestiCoull1998/> point out that exact methods such as the Clopper–Pearson interval may not work as well as some approximations. The normal approximation interval and its presentation in textbooks has been heavily criticised, with many statisticians advocating that it not be used.<ref name=Brown2001/> The principal problems are ''overshoot'' (bounds exceed {{closed-closed|0, 1}}), ''zero-width intervals'' at <math> \hat p = 0 </math> or {{math|1}} (falsely implying certainty),<ref name=Newcombe-1998/>, inaccurate coverage (the actual coverage is not equal to the nominal level such as 95%) and overall inconsistency with significance testing.<ref name=Wallis2013/>
Of the approximations listed above, Wilson score interval methods (with or without continuity correction) have been shown to be the most accurate and the most robust,<ref name=Wallis2013/><ref name=Brown2001/><ref name=Newcombe-1998/><ref name=Dean-Pagano-2015/><ref name=Franco-et-al-2019/> though some prefer Agresti & Coull's approach for larger sample sizes.<ref name=Brown2001/> Wilson and Clopper–Pearson methods obtain consistent results with source significance tests,<ref name=Wallis2021/> and this property is decisive for many researchers.
Many of these intervals can be calculated in R using packages like {{mono|binom}}<ref> {{cite report |last = Dorai-Raj |first = Sundar |date = 2022-05-02 |title = binom: Binomial confidence intervals for several parameterizations |type = software doc. |url = https://cran.r-project.org/web/packages/binom/index.html |access-date = 2023-12-02 |df=dmy-all }} </ref> and {{mono|survey}}<ref name=Lumley-survey/>; and in Python using {{mono|statsmodel}} function {{mono|proportion_confint()}}<ref>{{cite web |title=statsmodels online documentation |author=<!-- not stated --> |url=https://www.statsmodels.org/stable/generated/statsmodels.stats.proportion.proportion_confint.html |access-date=2026-05-14}}</ref>
==See also== * Binomial distribution * Estimation theory * pseudocount * {{slink|CDF-based nonparametric confidence interval#Pointwise band}} * {{slink|Z-test#Comparing the Proportions of Two Binomials}}
==References== {{reflist|25em}}
{{DEFAULTSORT:Binomial Proportion Confidence Interval}} Category:Statistical approximations Category:Statistical intervals