Lagrange polynomial

{{Short description|Polynomials used for interpolation}} {{Distinguish|text = Legendre polynomials (the orthogonal basis of function space)}} thumb|upright=1.5|This image shows, for four data points ({{color|#5e81B5|{{math|(−9, 5)}}}}, {{color|#e19c24|{{math|(−4, 2)}}}}, {{color|#8FB131|{{math|(−1, −2)}}}}, {{color|#EC6235|{{math|(7, 9)}}}}), the (cubic) interpolation polynomial {{color|black|{{math|''L''(''x'')}}}} (dashed, black), which is the sum of the ''scaled'' basis polynomials {{color|#5e81B5|{{math|''y''{{sub|0}}''ℓ''{{sub|0}}(''x'')}}}}, {{color|#e19c24|{{math|''y''{{sub|1}}''ℓ''{{sub|1}}(''x'')}}}}, {{color|#8FB131|{{math|''y''{{sub|2}}''ℓ''{{sub|2}}(''x'')}}}} and {{color|#EC6235|{{math|''y''{{sub|3}}''ℓ''{{sub|3}}(''x'')}}}}. The interpolation polynomial passes through all four control points, and each ''scaled'' basis polynomial passes through its respective control point and is {{math|0}} where {{mvar|x}} corresponds to the other three control points.

In numerical analysis, the '''Lagrange interpolating polynomial''' is the unique polynomial of lowest degree that interpolates a given set of data.

Given a data set of coordinate pairs {{tmath|\textstyle (x_j, y_j)}}, the {{tmath|\textstyle x_j}} are called ''nodes'' and the {{tmath|\textstyle y_j}} are called ''values''. The Lagrange polynomial {{tmath|L(x)}} which interpolates the data assumes each value at the corresponding node, {{tmath|1=\textstyle L(x_j) = y_j}}. If there are {{tmath|k+1}} data pairs, the Lagrange polynomial has degree {{tmath|\leq k}}.

Although named after Joseph-Louis Lagrange, who published it in 1795,<ref>{{cite book |last=Lagrange |first=Joseph-Louis |author-link= Joseph-Louis Lagrange |title=Leçons Elémentaires sur les Mathématiques |language=fr |year=1795 |chapter=Leçon Cinquième. Sur l'usage des courbes dans la solution des problèmes |place=Paris}} Republished in {{cite book |last=Lagrange |first=Joseph-Louis |editor-last=Serret |editor-first=Joseph-Alfred |editor-link=Joseph-Alfred Serret |display-authors=0 |title=Oeuvres de Lagrange |year=1877 |volume=7 |publisher=Gauthier-Villars |pages=[https://archive.org/details/oeuvresdelagrang07lagr/page/271 271–287] }} Translated as {{cite book |last=Lagrange |first=Joseph-Louis |display-authors=0 |translator-last=McCormack |translator-first=Thomas J. |title=Lectures on Elementary Mathematics |edition=2nd |publisher=Open Court |year=1901 |chapter=Lecture V. On the Employment of Curves in the Solution of Problems |chapter-url=https://archive.org/details/lecturesonelemen00lagriala/page/127 |pages=127–149}}</ref> the method was first discovered in 1779 by Edward Waring.<ref>{{cite journal |title=Problems concerning interpolations |first=Edward |last=Waring |author-link=Edward Waring |journal=Philosophical Transactions of the Royal Society |year=1779 |volume=69 |pages=59–67 |url=https://archive.org/details/philosophicaltra6917roya/page/59 |doi=10.1098/rstl.1779.0008 }}</ref> It is also an easy consequence of a formula published in 1783 by Leonhard Euler.<ref>{{Cite journal | last1=Meijering | first1=Erik | title=A chronology of interpolation: from ancient astronomy to modern signal and image processing | doi=10.1109/5.993400 | year=2002 | journal=Proceedings of the IEEE | volume=90 | issue=3 | pages=319–342 | url = http://bigwww.epfl.ch/publications/meijering0201.pdf}}</ref>

Uses of Lagrange polynomials include the Newton–Cotes method of numerical integration, Shamir's secret sharing scheme in cryptography, and Reed–Solomon error correction in coding theory.

For equispaced nodes, Lagrange interpolation is susceptible to Runge's phenomenon of large oscillation.

==Definition== Given a set of {{tmath|k + 1}} nodes {{nowrap|<math>\{x_0, x_1, \ldots, x_k\}</math>,}} which must all be distinct, {{tmath|\textstyle x_j \neq x_m}} for indices {{tmath|j \neq m}}, the '''Lagrange basis''' for polynomials of degree {{tmath|\leq k}} for those nodes is the set of polynomials <math>\textstyle \{\ell_0(x), \ell_1(x), \ldots, \ell_k(x)\}</math> each of degree {{tmath|k}} which take values {{tmath|1=\textstyle \ell_j(x_m) = 0}} if {{tmath|m \neq j}} and {{tmath|1=\textstyle \ell_j(x_j) = 1}}. Using the Kronecker delta this can be written {{tmath|1=\textstyle \ell_j(x_m) = \delta_{jm} }}. Each basis polynomial can be explicitly described by the product:

<math display=block>\begin{aligned} \ell_j(x) &= \frac{(x-x_0)}{(x_j-x_0)} \cdots \frac{(x-x_{j-1})}{(x_j-x_{j - 1})} \frac{(x-x_{j+1})}{(x_j-x_{j+1})} \cdots \frac{(x-x_k)}{(x_j-x_k)} \\[8mu] &= \prod_{\begin{smallmatrix}0\le m\le k\\ m\neq j\end{smallmatrix}} \frac{x-x_m}{x_j-x_m} \vphantom\Bigg|. \end{aligned}</math>

Notice that the numerator {{tmath|\textstyle \prod_{m \neq j}(x - x_m)}} has {{tmath|k}} roots at the nodes <math>\textstyle \{x_m\}_{m \neq j}</math> while the denominator {{tmath|\textstyle \prod_{m \neq j}(x_j - x_m)}} scales the resulting polynomial so that {{tmath|1=\textstyle \ell_j(x_j) = 1}}.

The Lagrange interpolating polynomial for those nodes through the corresponding ''values'' <math>\{y_0, y_1, \ldots, y_k\}</math> is the linear combination:

Each basis polynomial has degree {{tmath|k}}, so the sum {{tmath|L(x)}} has degree {{tmath|\leq k}}, and it interpolates the data because {{tmath|1=\textstyle L(x_m) = \sum_{j=0}^{k} y_j \ell_j(x_m) = \sum_{j=0}^{k} y_j \delta_{mj} = y_m}}.

The interpolating polynomial is unique. Proof: assume some polynomial {{tmath|M(x)}} of degree {{tmath|\leq k}} interpolates the data. Then the difference {{tmath|M(x) - L(x)}} is zero at {{tmath|k + 1}} distinct nodes {{nowrap|<math display=inline>\{x_0, x_1, \ldots, x_k\}</math>.}} But the only polynomial of degree {{tmath|\leq k}} with more than {{tmath|k}} roots is the constant zero function, so {{tmath|1= M(x) - L(x) = 0}}, or {{tmath|1= M(x) = L(x)}}.

==Barycentric form==

Each Lagrange basis polynomial {{tmath|\textstyle \ell_j(x)}} can be rewritten as the product of three parts, a function {{tmath|1=\textstyle \ell(x) = \prod_m (x - x_m)}} common to every basis polynomial, a node-specific constant {{tmath|1=\textstyle w_j = \prod_{m\neq j}(x_j - x_m)^{-1} }} (called the ''barycentric weight''), and a part representing the displacement from {{tmath|\textstyle x_j}} to {{tmath|x}}:<ref>{{cite journal | first1 = Jean-Paul | last1 = Berrut | first2 = Lloyd N. | last2 = Trefethen |author-link = Lloyd N. Trefethen | year = 2004 | title = Barycentric Lagrange Interpolation | journal = SIAM Review | volume = 46 | issue = 3 | pages = 501–517 | doi = 10.1137/S0036144502417715 | bibcode = 2004SIAMR..46..501B | url = https://people.maths.ox.ac.uk/trefethen/barycentric.pdf | doi-access = free }}</ref>

<math display=block>\ell_j(x) = \ell(x) \dfrac{w_j}{x - x_j}</math>

By factoring {{tmath|\ell(x)}} out from the sum, we can write the Lagrange polynomial in the so-called ''first barycentric form'':

If the weights {{tmath|\textstyle w_j}} have been pre-computed, this requires only {{tmath|\mathcal O(k)}} operations compared to {{tmath|\textstyle \mathcal O(k^2)}} for evaluating each Lagrange basis polynomial {{tmath|\textstyle \ell_j(x)}} individually. (See Big O notation.)

The barycentric interpolation formula can also easily be updated to incorporate a new node {{tmath|\textstyle x_{k+1} }} by dividing each of the {{tmath|\textstyle w_j}}, {{tmath|1= j = 0 \dots k}} by {{tmath|\textstyle (x_j - x_{k+1})}} and constructing the new {{tmath|\textstyle w_{k+1} }} as above.

For any {{mvar|x}}, <math display=inline>\sum_{j=0}^k \ell_j(x) = 1</math> because the constant function <math display="inline">g(x) = 1</math> is the unique polynomial of degree <math>\leq k</math> interpolating the data {{nowrap|<math display=inline>\{(x_0, 1), (x_1, 1), \ldots, (x_k, 1) \}</math>.}} We can thus further simplify the barycentric formula by dividing {{{nowrap|<math>L(x) = L(x) / g(x)</math>:}}

<math display="block">\begin{aligned} L(x) &= \ell(x) \sum_{j=0}^k \frac{w_j}{x-x_j}y_j \Bigg/ \ell(x) \sum_{j=0}^k \frac{w_j}{x-x_j} \\[10mu] &= \sum_{j=0}^k \frac{w_j}{x-x_j}y_j \Bigg/ \sum_{j=0}^k \frac{w_j}{x-x_j}. \end{aligned}</math>

This is called the ''second form'' or ''true form'' of the barycentric interpolation formula.

This second form has advantages in computation cost and accuracy: it avoids evaluation of <math>\ell(x)</math>; the work to compute each term in the denominator <math>w_j/(x-x_j)</math> has already been done in computing <math>\bigl(w_j/(x-x_j)\bigr)y_j</math> and so computing the sum in the denominator costs only <math display=inline>k</math> addition operations; for evaluation points <math display=inline>x</math> which are close to one of the nodes <math display=inline>x_j</math>, catastrophic cancelation would ordinarily be a problem for the value <math display=inline>(x-x_j)</math>, however this quantity appears in both numerator and denominator and the two cancel leaving good relative accuracy in the final result.

Using this formula to evaluate <math>L(x)</math> at one of the nodes <math>x_j</math> will result in the indeterminate <math>\infty y_j/\infty</math>; computer implementations must replace such results by <math>L(x_j) = y_j.</math>

Each Lagrange basis polynomial can also be written in barycentric form:

==A perspective from linear algebra==

Solving an interpolation problem leads to a problem in linear algebra amounting to inversion of a matrix. Using a standard monomial basis for our interpolation polynomial <math display="inline">L(x) = \sum_{j=0}^k x^j m_j</math>, we must invert the Vandermonde matrix <math>(x_i)^j</math> to solve <math>L(x_i) = y_i</math> for the coefficients <math>m_j</math> of <math>L(x)</math>. By choosing a better basis, the Lagrange basis, <math display="inline">L(x) = \sum_{j=0}^k l_j(x) y_j</math>, we merely get the identity matrix, <math>\delta_{ij}</math>, which is its own inverse: the Lagrange basis automatically ''inverts'' the analog of the Vandermonde matrix.

This construction is analogous to the Chinese remainder theorem. Instead of checking for remainders of integers modulo prime numbers, we are checking for remainders of polynomials when divided by linears.

Furthermore, when the order is large, Fast Fourier transformation can be used to solve for the coefficients of the interpolated polynomial.

==Example== We wish to interpolate <math>f(x) = x^2</math> over the domain <math>1 \leq x \leq 3</math> at the three nodes {{nobr|<math>\{1,\, 2,\, 3\}</math>:}}

<math display="block">\begin{align} x_0 & = 1, & & & y_0 = f(x_0) & = 1, \\[3mu] x_1 & = 2, & & & y_1 = f(x_1) & = 4, \\[3mu] x_2 & = 3, & & & y_2 = f(x_2) & =9. \end{align}</math>

The node polynomial <math>\ell</math> is <math display="block">\ell(x) = (x-1)(x-2)(x-3) = x^3 - 6x^2 + 11x - 6.</math>

The barycentric weights are <math display="block">\begin{align} w_0 &= (1-2)^{-1}(1-3)^{-1} = \tfrac12, \\[3mu] w_1 &= (2-1)^{-1}(2-3)^{-1} = -1, \\[3mu] w_2 &= (3-1)^{-1}(3-2)^{-1} = \tfrac12. \end{align}</math>

The Lagrange basis polynomials are

<math display="block">\begin{align} \ell_0(x) &= \frac{x - 2}{1 - 2}\cdot\frac{x - 3}{1 - 3} = \tfrac12x^2 - \tfrac52x + 3, \\[5mu] \ell_1(x) &= \frac{x - 1}{2 - 1}\cdot\frac{x - 3}{2 - 3} = -x^2 + 4x - 3, \\[5mu] \ell_2(x) &= \frac{x - 1}{3 - 1}\cdot\frac{x - 2}{3 - 2} = \tfrac12x^2 - \tfrac32x + 1. \end{align}</math>

The Lagrange interpolating polynomial is: <math display="block"> \begin{align} L(x) &= y_0 \cdot \ell_0(x) + y_1 \cdot \ell_1(x) + y_2 \cdot \ell_2(x) = x^2. \end{align} </math>

In (second) barycentric form,

<math display="block"> L(x) = \frac {\displaystyle \sum_{j=0}^2 \frac{w_j}{x-x_j}y_j} {\displaystyle \sum_{j=0}^2 \frac{w_j}{x-x_j}} = \frac {\displaystyle \frac{\tfrac12}{x - 1} + \frac{-4}{x - 2} + \frac{\tfrac92}{x - 3}} {\displaystyle \frac{\tfrac12}{x - 1} + \frac{-1}{x - 2} + \frac{\tfrac12}{x - 3}}. </math>

==Notes== thumb|upright=1.5|Example of interpolation divergence for a set of Lagrange polynomials.

The Lagrange form of the interpolation polynomial shows the linear character of polynomial interpolation and the uniqueness of the interpolation polynomial. Therefore, it is preferred in proofs and theoretical arguments. Uniqueness can also be seen from the invertibility of the Vandermonde matrix, due to the non-vanishing of the Vandermonde determinant.

But, as can be seen from the construction, each time a node {{math|''x''<sub>''k''</sub>}} changes, all Lagrange basis polynomials have to be recalculated. A better form of the interpolation polynomial for practical (or computational) purposes is the barycentric form of the Lagrange interpolation (see below) or Newton polynomials.

Lagrange and other interpolation at equally spaced points, as in the example above, yield a polynomial oscillating above and below the true function. This behaviour tends to grow with the number of points, leading to a divergence known as Runge's phenomenon; the problem may be eliminated by choosing interpolation points at Chebyshev nodes.<ref>{{cite book|title=Scientific Computing with MATLAB|volume=2|series=Texts in computational science and engineering |author-link1= Alfio Quarteroni |first1=Alfio|last1=Quarteroni|first2=Fausto|last2=Saleri|publisher=Springer|year=2003|isbn=978-3-540-44363-6|page=66|url=https://books.google.com/books?id=fE1W5jsU4zoC&pg=PA66}}.</ref>

The Lagrange basis polynomials can be used in numerical integration to derive the Newton–Cotes formulas.

==Remainder in Lagrange interpolation formula== When interpolating a given function ''f'' by a polynomial of degree {{mvar|k}} at the nodes <math>x_0,\dots, x_k</math> we get the remainder <math>R(x) = f(x) - L(x)</math> which can be expressed as<ref>{{AS ref|25, eqn 25.2.3|878}}</ref>

<math display="block"> \begin{align} R(x) &= f[x_0,\ldots,x_k,x] \ell(x) \\[1ex] &= \ell(x) \frac{f^{(k+1)}(\xi)}{(k+1)!}, & x_0 < \xi < x_k, \end{align}</math>

where <math>f[x_0,\ldots,x_k,x]</math> is the notation for divided differences. Alternatively, the remainder can be expressed as a contour integral in complex domain as

<math display="block">\begin{align} R(x) &= \frac{\ell(x)}{2\pi i} \int_C \frac{f(t)}{(t-x)(t-x_0) \cdots (t-x_k)} dt \\[1ex] &= \frac{\ell(x)}{2\pi i} \int_C \frac{f(t)}{(t-x)\ell(t)} dt. \end{align}</math>

The remainder can be bound as

=== Derivation=== Clearly, <math>R(x) </math> is zero at nodes. To find <math>R(x)</math> at a point {{nowrap|<math>x_p </math>,}} define a new function <math>F(x) = R(x)-\tilde{R}(x) = f(x)-L(x)-\tilde{R}(x)</math> and choose <math display="inline">\tilde{R}(x) = C\cdot\prod_{i=0}^k(x-x_i)</math> where <math>C</math> is the constant we are required to determine for a given <math>x_p</math>. We choose <math>C</math> so that <math>F(x)</math> has <math>k+2</math> zeroes (at all nodes and <math>x_p</math>) between <math>x_0</math> and <math>x_k</math> (including endpoints). Assuming that <math>f(x)</math> is <math>k+1</math>-times differentiable, since <math>L(x)</math> and <math>\tilde{R}(x)</math> are polynomials, and therefore, are infinitely differentiable, <math>F(x)</math> will be <math>k+1</math>-times differentiable. By Rolle's theorem, <math>F^{(1)}(x)</math> has <math>k+1</math> zeroes, <math>F^{(2)}(x)</math> has <math>k</math> zeroes... <math>F^{(k+1)}</math> has 1 zero, say {{nowrap|<math>\xi</math>,}} where {{nowrap|<math>x_0 < \xi < x_k</math>.}} Explicitly writing <math>F^{(k+1)}(\xi)</math>:

<math display="block">F^{(k+1)}(\xi)=f^{(k+1)}(\xi)-L^{(k+1)}(\xi)-\tilde{R}^{(k+1)}(\xi)</math> <math display="block">L^{(k+1)}=0,\tilde{R}^{(k+1)}=C\cdot(k+1)!</math> (Because the highest power of <math>x</math> in <math>\tilde{R}(x)</math> is <math>k+1</math>)

The equation can be rearranged as<ref>{{Cite web|url=https://sam.nitk.ac.in/sites/default/Numerical_Methods/Interpolation/interpolation.pdf|title=Interpolation|pages=12–15|archive-url=https://web.archive.org/web/20170215114507/http://www.sam.nitk.ac.in/sites/default/Numerical_Methods/Interpolation/interpolation.pdf|archive-date=2017-02-15}}</ref>

<math display="block">C=\frac{f^{(k+1)}(\xi)}{(k+1)!}</math> Since <math>F(x_p) = 0</math> we have <math>R(x_p)=\tilde{R}(x_p) = \frac{f^{k+1}(\xi)}{(k+1)!} \prod_{i=0}^k(x_p-x_i)</math>

==Derivatives== The {{mvar|d}}th derivative of a Lagrange interpolating polynomial can be written in terms of the derivatives of the basis polynomials,

Recall (see {{slink|#Definition}} above) that each Lagrange basis polynomial is

<math display=block>\begin{aligned} \ell_j(x) &= \prod_{\begin{smallmatrix}m = 0\\ m\neq j\end{smallmatrix}}^k \frac{x-x_m}{x_j-x_m}. \end{aligned}</math>

The first derivative can be found using the product rule:

<math display="block">\begin{align} \ell_j'(x) &= \sum_{\begin{smallmatrix}i=0 \\ i\not=j\end{smallmatrix}}^k \Biggl[ \frac{1}{x_j-x_i}\prod_{\begin{smallmatrix}m=0 \\ m\not = (i , j)\end{smallmatrix}}^k \frac{x-x_m}{x_j-x_m} \Biggr] \\[5mu] &= \ell_j(x)\sum_{\begin{smallmatrix}i=0 \\i\not=j\end{smallmatrix}}^k \frac{1}{x-x_i}. \end{align}</math>

The second derivative is

<math display="block">\begin{align} \ell_j''(x) &= \sum_{\begin{smallmatrix}i=0 \\ i\ne j\end{smallmatrix}}^{k} \frac{1}{x_j-x_i} \Biggl[ \sum_{\begin{smallmatrix}m=0 \\ m\ne(i,j)\end{smallmatrix}}^{k} \Biggl( \frac{1}{x_j-x_m} \prod_{\begin{smallmatrix}n=0 \\ n\ne(i,j,m)\end{smallmatrix}}^{k} \frac{x-x_n}{x_j-x_n} \Biggr) \Biggr] \\[10mu] &= \ell_j(x) \sum_{0 \leq i < m \leq k} \frac{2}{(x-x_i)(x - x_m)} \\[10mu] &= \ell_j(x)\Biggl[\Biggl(\sum_{\begin{smallmatrix}i=0 \\i\not=j\end{smallmatrix}}^k \frac{1}{x-x_i}\Biggr)^2 - \sum_{\begin{smallmatrix}i=0 \\i\not=j\end{smallmatrix}}^k \frac{1}{(x-x_i)^2}\Biggr]. \end{align}</math>

The third derivative is

<math display="block">\begin{align} \ell_j'''(x) &= \ell_j(x) \sum_{0 \leq i < m < n \leq k} \frac{3!}{(x-x_i)(x - x_m)(x - x_n)} \end{align}</math>

and likewise for higher derivatives.

Note that all of these formulas for derivatives are invalid at or near a node. A method of evaluating all orders of derivatives of a Lagrange polynomial efficiently at all points of the domain, including the nodes, is converting the Lagrange polynomial to power basis form and then evaluating the derivatives.

==Finite fields== The Lagrange polynomial can also be computed in finite fields. This has applications in cryptography, such as in Shamir's Secret Sharing scheme.

==See also== {{div col}} * Neville's algorithm * Newton form of the interpolation polynomial * Bernstein polynomial * Carlson's theorem * Lebesgue constant * The Chebfun system * Table of Newtonian series * Frobenius covariant * Sylvester's formula * Finite difference coefficient * Hermite interpolation {{div col end}}

==References== {{reflist}}

==External links== {{wikibooks|Algorithm Implementation|Mathematics/Polynomial interpolation|Polynomial interpolation}} * {{springer|title=Lagrange interpolation formula|id=p/l057170}} * [https://www.alglib.net/interpolation/polynomial.php ALGLIB] has an implementations in C++ / C# / VBA / Pascal. * [https://www.gnu.org/software/gsl/ GSL] has a polynomial interpolation code in C * [https://stackoverflow.com/questions/11029615/lagrange-interpolation-method/11552763 SO] has a MATLAB example that demonstrates the algorithm and recreates the first image in this article * [https://nm.mathforcollege.com/chapter-05-04-lagrange-method/ Lagrange Method of Interpolation — Notes, PPT, Mathcad, Mathematica, MATLAB, Maple] *[http://www.math-linux.com/spip.php?article71 Lagrange interpolation polynomial] on www.math-linux.com * {{MathWorld|urlname=LagrangeInterpolatingPolynomial|title=Lagrange Interpolating Polynomial}} * [http://mathformeremortals.wordpress.com/2013/01/15/bicubic-interpolation-excel-worksheet-function/ Excel Worksheet Function for Bicubic Lagrange Interpolation] * [https://learn.64bitdragon.com/articles/computer-science/numerical-analysis/lagrange-interpolation Lagrange polynomials in Python]

Category:Interpolation Category:Polynomials Category:Articles containing proofs