Logarithmically concave function

{{Short description|Type of mathematical function}} In convex analysis, a non-negative function {{math|''f'' : '''R'''''n'' → '''R'''+}} is '''logarithmically concave''' (or '''log-concave''' for short) if its domain is a convex set, and if it satisfies the inequality : <math> f(\theta x + (1 - \theta) y) \geq f(x)^{\theta} f(y)^{1 - \theta} </math> for all {{math|''x'',''y'' ∈ dom ''f''}} and {{math|0 < ''θ'' < 1}}. If {{math|''f''}} is strictly positive, this is equivalent to saying that the logarithm of the function, {{math|log ∘ ''f''}}, is concave; that is, : <math> \log f(\theta x + (1 - \theta) y) \geq \theta \log f(x) + (1-\theta) \log f(y) </math> for all {{math|''x'',''y'' ∈ dom ''f''}} and {{math|0 < ''θ'' < 1}}.

Examples of log-concave functions are the 0-1 indicator functions of convex sets (which requires the more flexible definition), and the Gaussian function.

Similarly, a function is ''log-convex'' if it satisfies the reverse inequality : <math> f(\theta x + (1 - \theta) y) \leq f(x)^{\theta} f(y)^{1 - \theta} </math> for all {{math|''x'',''y'' ∈ dom ''f''}} and {{math|0 < ''θ'' < 1}}.

For non-negative discrete functions {{math|''f'' : '''Z''' → '''R'''+}}, it is log-concave <ref>Johnson, O., 2007. Log-concavity and the maximum entropy property of the Poisson distribution. Stochastic Processes and their Applications, 117(6), pp.791-802.</ref> if : <math> f(k)^2 \geq f(k+1)f(k-1) </math>

==Properties== * A log-concave function is also quasi-concave. This follows from the fact that the logarithm is monotone implying that the superlevel sets of this function are convex.<ref name=":0" /> * Every concave function that is nonnegative on its domain is log-concave. However, the reverse does not necessarily hold. An example is the Gaussian function {{math|''f''(''x'')}} = {{math|exp(−''x''2/2)}} which is log-concave since {{math|log ''f''(''x'')}} = {{math|−''x''2/2}} is a concave function of {{math|''x''}}. But {{math|''f''}} is not concave since the second derivative is positive for |{{math|''x''}}| > 1:

::<math>f''(x)=e^{-\frac{x^2}{2}} (x^2-1) \nleq 0</math> * From above two points, concavity <math>\Rightarrow</math> log-concavity <math>\Rightarrow</math> quasiconcavity. * A twice differentiable, nonnegative function with a convex domain is log-concave if and only if for all {{math|''x''}} satisfying {{math|''f''(''x'') > 0}},

::<math>f(x)\nabla^2f(x) \preceq \nabla f(x)\nabla f(x)^T</math>,<ref name=":0">{{cite book |first1=Stephen |last1=Boyd |author-link=Stephen P. Boyd |first2=Lieven |last2=Vandenberghe |chapter=Log-concave and log-convex functions |title=Convex Optimization |publisher=Cambridge University Press |year=2004 |isbn=0-521-83378-7 |chapter-url=https://web.stanford.edu/~boyd/cvxbook/ |pages=104–108 }}</ref>

:i.e.

::<math>f(x)\nabla^2f(x) - \nabla f(x)\nabla f(x)^T</math> is

:negative semi-definite. For functions of one variable, this condition simplifies to

::<math>f(x)f''(x) \leq (f'(x))^2</math>

==Operations preserving log-concavity==

* Products: The product of log-concave functions is also log-concave. Indeed, if {{math|''f''}} and {{math|''g''}} are log-concave functions, then {{math|log ''f''}} and {{math|log ''g''}} are concave by definition. Therefore

::<math>\log\,f(x) + \log\,g(x) = \log(f(x)g(x))</math>

:is concave, and hence also {{math|''f'' ''g''}} is log-concave.

* Marginals: if {{math|''f''(''x'',''y'')}} : {{math|'''R'''''n''+''m'' → '''R'''}} is log-concave, then

::<math>g(x)=\int f(x,y) dy</math>

:is log-concave (see Prékopa–Leindler inequality).

* This implies that convolution preserves log-concavity, since {{math|''h''(''x'',''y'')}} = {{math|''f''(''x''-''y'') ''g''(''y'')}} is log-concave if {{math|''f''}} and {{math|''g''}} are log-concave, and therefore

::<math>(f*g)(x)=\int f(x-y)g(y) dy = \int h(x,y) dy</math>

:is log-concave.

==Log-concave distributions== Log-concave distributions are necessary for a number of algorithms, e.g. adaptive rejection sampling. Every distribution with log-concave density is a maximum entropy probability distribution with specified mean ''μ'' and Deviation risk measure ''D''.<ref name="Grechuk1">{{cite journal | last1=Grechuk | first1=Bogdan | last2=Molyboha | first2=Anton | last3=Zabarankin | first3=Michael | date=May 2009 | title=Maximum Entropy Principle with General Deviation Measures | journal=Mathematics of Operations Research | volume=34 | issue=2 | pages=445–467 | doi=10.1287/moor.1090.0377 | url=https://www.researchgate.net/profile/Bogdan-Grechuk/publication/220442393_Maximum_Entropy_Principle_with_General_Deviation_Measures/links/59132b61a6fdcc963e7ed4fd/Maximum-Entropy-Principle-with-General-Deviation-Measures.pdf}}</ref> As it happens, many common probability distributions are log-concave. Some examples:<ref name=":1">See {{cite journal |first1=Mark |last1=Bagnoli |first2=Ted |last2=Bergstrom |year=2005 |title=Log-Concave Probability and Its Applications |journal=Economic Theory |volume=26 |issue=2 |pages=445–469 |doi=10.1007/s00199-004-0514-4 |s2cid=1046688 |url=http://www.econ.ucsb.edu/~tedb/Theory/delta.pdf }}</ref> *the normal distribution and multivariate normal distributions, *the exponential distribution, *the uniform distribution over any convex set, *the binomial distribution, *the logistic distribution, *the extreme value distribution, *the Laplace distribution, *the chi distribution, *the hyperbolic secant distribution, *the Wishart distribution, if ''n'' ≥ ''p'' + 1,<ref name="prekopa">{{cite journal | last1 = Prékopa | first1 = András | author-link = András Prékopa | year = 1971 | title = Logarithmic concave measures with application to stochastic programming | journal = Acta Scientiarum Mathematicarum | volume = 32 | issue = 3-4 | pages = 301–316 | url = http://rutcor.rutgers.edu/~prekopa/SCIENT1.pdf}}</ref> *the Dirichlet distribution, if all parameters are ≥ 1,<ref name="prekopa"/> *the gamma distribution if the shape parameter is ≥ 1, *the chi-square distribution if the number of degrees of freedom is ≥ 2, *the beta distribution if both shape parameters are ≥ 1, and *the Weibull distribution if the shape parameter is ≥ 1.

Note that all of the parameter restrictions have the same basic source: The exponent of non-negative quantity must be non-negative in order for the function to be log-concave.

The following distributions are non-log-concave for all parameters: *the Student's t-distribution, *the Cauchy distribution, *the Pareto distribution, *the log-normal distribution, and *the F-distribution.

Note that the cumulative distribution function (CDF) of all log-concave distributions is also log-concave. However, some non-log-concave distributions also have log-concave CDF's: *the log-normal distribution, *the Pareto distribution, *the Weibull distribution when the shape parameter < 1, and *the gamma distribution when the shape parameter < 1.

The following are among the properties of log-concave distributions: *If a density is log-concave, so is its cumulative distribution function (CDF). *If a multivariate density is log-concave, so is the marginal density over any subset of variables. *The sum of two independent log-concave random variables is log-concave. This follows from the fact that the convolution of two log-concave functions is log-concave. *The product of two log-concave functions is log-concave. This means that joint densities formed by multiplying two probability densities (e.g. the normal-gamma distribution, which always has a shape parameter ≥ 1) will be log-concave. This property is heavily used in general-purpose Gibbs sampling programs such as BUGS and JAGS, which are thereby able to use adaptive rejection sampling over a wide variety of conditional distributions derived from the product of other distributions. * If a density is log-concave, so is its survival function.<ref name=":1" /> * If a density is log-concave, it has a monotone hazard rate (MHR), and is a regular distribution since the derivative of the logarithm of the survival function is the negative hazard rate, and by concavity is monotone i.e. ::<math>\frac{d}{dx}\log\left(1-F(x)\right) = -\frac{f(x)}{1-F(x)}</math> which is decreasing as it is the derivative of a concave function.

==Notes== {{Reflist}}

==References== * {{cite book|author-link=Ole Barndorff-Nielsen|last=Barndorff-Nielsen|first=Ole|title=Information and exponential families in statistical theory|series=Wiley Series in Probability and Mathematical Statistics|publisher=John Wiley \& Sons, Ltd.|location=Chichester|year=1978|pages=ix+238 pp|isbn=0-471-99545-2|mr=489333}} * {{cite book|title=Unimodality, convexity, and applications |last1=Dharmadhikari|first1=Sudhakar |last2=Joag-Dev |first2=Kumar|series=Probability and Mathematical Statistics |publisher=Academic Press, Inc. |location=Boston, MA |year=1988 |pages=xiv+278 |isbn=0-12-214690-5|mr=954608}}

* {{cite book|title=Parametric Statistical Theory | last1=Pfanzagl | first1=Johann |author-link=  |last2=with the assistance of R. Hamböker |year=1994|publisher=Walter de Gruyter |isbn=3-11-013863-8 |mr=1291393}}

* {{cite book|title=Convex functions, partial orderings, and statistical applications|last1=Pečarić|first1=Josip E.|last2=Proschan|first2=Frank|last3=Tong|first3=Y. L. |author-link2=Frank Proschan |series=Mathematics in Science and Engineering|volume=187 |publisher=Academic Press, Inc. |location=Boston, MA |year=1992|pages=xiv+467 pp |isbn=0-12-549250-2 |mr=1162312}}

{{DEFAULTSORT:Logarithmically Concave Function}} Category:Mathematical analysis Category:Convex analysis