Legendre transformation

File:Legendre transformation.png

The function f(x) is defined on the interval [a, b]. The difference px − f(x) takes a maximum at x'. Thus, f*(p) = px' − f(x').

In mathematics and physics, the Legendre transformation or Legendre transform, named after Adrien-Marie Legendre, is an involutive transformation on the real-valued convex functions of one real variable. Its generalization to convex functions of affine spaces is sometimes called the Legendre-Fenchel transformation.

It is commonly used in thermodynamics and to derive the Hamiltonian formalism of classical mechanics out of the Lagrangian formulation, as well as in the solution of differential equations of several variables.

For sufficiently smooth functions on the real line, the Legendre transform Template:Mvar of a function Template:Mvar can be specified, up to an additive constant, by the condition that the functions' first derivatives are inverse functions of each other,

$Df=\left(Dg\right)^{-1}~.$

Definition

Let I ⊂ R be an interval, and f : I → R a convex function; then its Legendre transform is the function f* : I* → R defined by

$f^{*}(x^{*})=\sup _{x\in I}(x^{*}x-f(x)),\quad x^{*}\in I^{*}$

with domain

$I^{*}=\left\{x^{*}:\sup _{x\in I}(x^{*}x-f(x))<\infty \right\}~.$

The transform is always well-defined when f(x) is convex.

The generalization to convex functions f : X → R on a convex set X ⊂ Rⁿ is straightforward: f* : X* → R has domain

$X^{*}=\left\{x^{*}:\sup _{x\in X}(\langle x^{*},x\rangle -f(x))<\infty \right\}$

and is defined by

$f^{*}(x^{*})=\sup _{x\in X}(\langle x^{*},x\rangle -f(x)),\quad x^{*}\in X^{*}~,$

where $\langle x^{*},x\rangle$ denotes the dot product of x* and Template:Mvar.

The function f * is called the convex conjugate function of Template:Mvar. For historical reasons (rooted in analytic mechanics), the conjugate variable is often denoted Template:Mvar, instead of x*. If the convex function Template:Mvar is defined on the whole line and is everywhere differentiable, then

$f^{*}(p)=\sup _{x\in I}(px-f(x))$

can be interpreted as the negative of the y-intercept of the tangent line to the graph of Template:Mvar that has slope Template:Mvar.

The Legendre transformation is an application of the duality relationship between points and lines. The functional relationship specified by Template:Mvar can be represented equally well as a set of (x, y) points, or as a set of tangent lines specified by their slope and intercept values.

Properties

The Legendre transform of a convex function is convex.

Let us show this for the case of a doubly differentiable Template:Mvar with a non zero (and hence positive, due to convexity) double derivative.

For a fixed Template:Mvar, let Template:Mvar maximize px − f(x). Then f *(p) = px − f(x), noting that Template:Mvar depends on Template:Mvar. Thus,

$f^{\prime }(x)=p~.$

The derivative of Template:Mvar is itself differentiable with a positive derivative and hence strictly monotonic and invertible. Thus x = g(p) where $g\equiv (f^{\prime })^{-1}(p)$, meaning that g is defined so that $f'(g(p))=p$.

Note that Template:Mvar is also differentiable with the following derivative,

${\frac {dg(p)}{dp}}={\frac {1}{f''(g(p))}}~.$

Thus f *(p) = pg(p) − f(g(p)) is the composition of differentiable functions, hence differentiable.

Applying the product rule and the chain rule yields

${\begin{aligned}{\frac {d(f^{*})}{dp}}&=g(p)+\left(p-f'(g(p))\right)\cdot {\frac {dg(p)}{dp}}\\&=g(p),\end{aligned}}~,$

giving

${\begin{aligned}{\frac {d^{2}(f^{*})}{dp^{2}}}&={\frac {dg(p)}{dp}}\\&{}={\frac {1}{f''(g(p))}}\\&{}>0,\end{aligned}}$

so f * is convex.

It follows that the Legendre transformation is an involution, i.e., f ** = f:

By using the above equalities for g(p), f *(p) and its derivative,

${\begin{aligned}f^{**}(x)&{}={\left(x\cdot p_{s}-f^{*}(p_{s})\right)}_{|{\frac {d}{dp}}f^{*}(p=p_{s})=x}\\&{}=g(p_{s})\cdot p_{s}-f^{*}(p_{s})\\&{}=f(g(p_{s}))\\&{}=f(x)~.\end{aligned}}$

Examples

Example 1

Let f(x) = cx² defined on R, where c > 0 is a fixed constant.

For x* fixed, the function x*x – f(x) = x*x – cx² of Template:Mvar has the first derivative x* – 2cx and second derivative −2c; there is one stationary point at x = x*/2c, which is always a maximum. Thus, I* = R and

$f^{*}(x^{*})=c^{*}{x^{*}}^{2}~,$

where c* = 1/4c.

Clearly,

$f^{**}(x)={\frac {1}{4c^{*}}}x^{2}=cx^{2},$

namely f ** = f.

Example 2

Let f(x) = x² for x ∈ I = [2, 3].

For x* fixed, x*x − f(x) is continuous on Template:Mvar compact, hence it always takes a finite maximum on it; it follows that I* = R. The stationary point at x = x*/2 is in the domain [2, 3] if and only if 4 ≤ x* ≤ 6, otherwise the maximum is taken either at x = 2, or x = 3. It follows that

$f^{*}(x^{*})={\begin{cases}2x^{*}-4,\quad &x^{*}<4\\{\frac {{x^{*}}^{2}}{4}},&4\leqslant x^{*}\leqslant 6,\\3x^{*}-9,&x^{*}>6\end{cases}}$ .

Example 3

The function f(x) = cx is convex, for every Template:Mvar (strict convexity is not required for the Legendre transformation to be well defined). Clearly x*x − f(x) = (x* − c)x is never bounded from above as a function of Template:Mvar, unless x* − c = 0. Hence f* is defined on I* = {c} and f*(c) = 0.

One may check involutivity: of course x*x − f*(x*) is always bounded as a function of x* ∈ {c}, hence I ** = R. Then, for all Template:Mvar one has

$\sup _{x^{*}\in \{c\}}(xx^{*}-f^{*}(x^{*}))=xc,$

and hence f **(x) = cx = f(x).

Example 4 (many variables)

Let

$f(x)=\langle x,Ax\rangle +c$

be defined on X = Rⁿ, where Template:Mvar is a real, positive definite matrix. Then Template:Mvar is convex, and

$\langle p,x\rangle -f(x)=\langle p,x\rangle -\langle x,Ax\rangle -c,$

has gradient p − 2Ax and Hessian −2A, which is negative; hence the stationary point x = A^-1p/2 is a maximum. We have X* = Rⁿ, and

$f^{*}(p)={\frac {1}{4}}\langle p,A^{-1}p\rangle -c$ .

An equivalent definition in the differentiable case

Equivalently, two convex functions Template:Mvar and Template:Mvar defined on the whole line are said to be Legendre transforms of each other if their first derivatives are inverse functions of each other,

$Df=\left(Dg\right)^{-1}~,$

in which case one writes equivalently f* = g and g* = f. We can see this by first taking the derivative of f*,

${df^{\star }(p) \over dp}={d \over dp}(xp-f(x))=x+p{dx \over dp}-{df \over dx}{dx \over dp}=x~.$

This equation, taken together with the previous equation resulting from the maximization condition, results in the following pair of reciprocal equations,

$p={df \over dx}(x),$

$x={df^{\star } \over dp}(p).$

From these, it is evident that Df and Df* are inverses, as stated. One may exemplify this by considering f(x) = exp x and hence g(p) = p log p − p.

They are unique, up to an additive constant, which is fixed by the additional requirement that

$f(x)+f^{\star }(p)=x\,p~.$

The symmetry of this expression underscores that the Legendre transformation is its own inverse (involutive).

In practical terms, given f(x), the parametric plot of xf'(x) − f(x) versus f '(x) amounts to the graph of g(p) versus Template:Mvar.

In some cases (e.g. thermodynamic potentials, below), a non-standard requirement is used, amounting to an alternative definition of f* with a minus sign,

$f(x)-f^{\star }(p)=xp.$

Behavior of differentials under Legendre transforms

The Legendre transform is linked to integration by parts, pdx = d(px) − xdp.

Let Template:Mvar be a function of two independent variables Template:Mvar and Template:Mvar, with the differential

$df={\partial f \over \partial x}dx+{\partial f \over \partial y}dy=pdx+vdy$ .

Assume that it is convex in Template:Mvar for all Template:Mvar, so that one may perform the Legendre transform in Template:Mvar, with Template:Mvar the variable conjugate to Template:Mvar. Since the new independent variable is Template:Mvar, the differentials dx and dy devolve to dp and dy, i.e., we build another function with its differential expressed in terms of the new basis dp and dy. We thus consider the function g(p, y) = f − px so that

$dg=df-pdx-xdp=pdx+vdy-pdx-xdp=-xdp+vdy$

$x=-{\frac {\partial g}{\partial p}}$

$v={\frac {\partial g}{\partial y}}.$

The function g(p, y) is the Legendre transform of f(x, y), where only the independent variable Template:Mvar has been supplanted by Template:Mvar. This is widely used in thermodynamics, as illustrated below.

Applications

Hamilton-Lagrange mechanics

A Legendre transform is used in classical mechanics to derive the Hamiltonian formulation from the Lagrangian formulation, and conversely. A typical Lagrangian has the form

$L(v,q)={\tfrac {1}{2}}\langle v,Mv\rangle -V(q)$,

where $(v,q)$ are coordinates on Rⁿ × Rⁿ, Template:Mvar is a positive real matrix, and

$\langle x,y\rangle =\sum _{j}x_{j}y_{j}.$

For every Template:Mvar fixed, $L(v,q)$ is a convex function of $v$, while $V(q)$ plays the role of a constant.

Hence the Legendre transform of $L(v,q)$ as a function of Template:Mvar is the Hamiltonian function,

$H(p,q)={\tfrac {1}{2}}\langle p,M^{-1}p\rangle +V(q)$.

In a more general setting, $(v,q)$ are local coordinates on the tangent bundle $T{\mathcal {M}}$ of a manifold ${\mathcal {M}}$. For each Template:Mvar, $L(v,q)$ is a convex function of the tangent space V_q. The Legendre transform gives the Hamiltonian $H(p,q)$ as a function of the coordinates (p, q) of the cotangent bundle $T^{*}{\mathcal {M}}$; the inner product used to define the Legendre transform is inherited from the pertinent canonical symplectic structure.

Thermodynamics

The strategy behind the use of Legendre transforms in thermodynamics is to shift from a function that depends on a variable to a new (conjugate) function that depends on a new variable, the conjugate of the original one. The new variable is the partial derivative of the original function with respect to the original variable. The new function is the difference between the original function and the product of the old and new variables. Typically, this transformation is useful because it shifts the dependence of, e.g., the energy from an extensive variable to its conjugate intensive variable, which can usually be controlled more easily in a physical experiment.

For example, the internal energy is an explicit function of the extensive variables entropy, volume, and chemical composition

$U=U\left(S,V,\{N_{i}\}\right),$

which has a total differential

$dU=TdS-PdV+\sum \mu _{i}dN_{i}$.

By using the (non standard) Legendre transform of the internal energy, Template:Mvar, with respect to volume, Template:Mvar, it is possible to define the enthalpy as

$H=U+PV\,=H\left(S,P,\{N_{i}\}\right),$

which is an explicit function of the pressure, Template:Mvar. The enthalpy contains all of the same information as the internal energy, but is often easier to work with in situations where the pressure is constant.

It is likewise possible to shift the dependence of the energy from the extensive variable of entropy, Template:Mvar, to the (often more convenient) intensive variable Template:Mvar, resulting in the Helmholtz and Gibbs free energies. The Helmholtz free energy, Template:Mvar, and Gibbs energy, Template:Mvar, are obtained by performing Legendre transforms of the internal energy and enthalpy, respectively,

$A=U-TS~,$

$G=H-TS=U+PV-TS~.$

The Helmholtz free energy is often the most useful thermodynamic potential when temperature and volume are held constant, while the Gibbs energy is often the most useful when temperature and pressure are held constant.

An example – variable capacitor

As another example from physics, consider a parallel-plate capacitor, in which the plates can move relative to one another. Such a capacitor would allow transfer of the electric energy which is stored in the capacitor into external mechanical work, done by the force acting on the plates. One may think of the electric charge as analogous to the "charge" of a gas in a cylinder, with the resulting mechanical force exerted on a piston.

Compute the force on the plates as a function of x, the distance which separates them. To find the force, compute the potential energy, and then apply the definition of force as the gradient of the potential energy function.

The energy stored in a capacitor of capacitance C(x) and charge Q is

$U(Q,\mathbf {x} )={\frac {1}{2}}QV={\frac {1}{2}}{\frac {Q^{2}}{C(\mathbf {x} )}}~$ ,

where the dependence on the area of the plates, the dielectric constant of the material between the plates, and the separation x are abstracted away as the capacitance C(x). (For a parallel plate capacitor, this is proportional to the area of the plates and inversely proportional to the separation.)

The force F between the plates due to the electric field is then

$\mathbf {F} (\mathbf {x} )=-{\frac {dU}{d\mathbf {x} }}~.$

If the capacitor is not connected to any circuit, then the charges on the plates remain constant as they move, and the force is the negative gradient of the electrostatic energy

$\mathbf {F} (\mathbf {x} )={\frac {1}{2}}{\frac {dC}{d\mathbf {x} }}{\frac {Q^{2}}{C^{2}}}.$

However, suppose, instead, that the voltage between the plates V is maintained constant by connection to a battery, which is a reservoir for charge at constant potential difference; now the charge is variable instead of the voltage, its Legendre conjugate. To find the force, first compute the non-standard Legendre transform,

$U^{*}=U-QV={\frac {1}{2}}QV-QV=-{\frac {1}{2}}QV=-{\tfrac {1}{2}}V^{2}C(\mathbf {x} ).$

The force now becomes the negative gradient of this Legendre transform, still pointing in the same direction,

$\mathbf {F} (\mathbf {x} )=-{\frac {dU^{*}}{d\mathbf {x} }}~.$

The two conjugate energies happen to stand opposite to each other, only because of the linearity of the capacitance—except now Q is no longer a constant. They reflect the two different pathways of storing energy into the capacitor, resulting in, for instance, the same "pull" between a capacitor's plates.

Probability theory

In large deviations theory, the rate function is defined as the Legendre transformation of the logarithm of the moment generating function of a random variable. An important application of the rate function is in the calculation of tail probabilities of sums of i.i.d. random variables.

Geometric interpretation

For a strictly convex function, the Legendre transformation can be interpreted as a mapping between the graph of the function and the family of tangents of the graph. (For a function of one variable, the tangents are well-defined at all but at most countably many points, since a convex function is differentiable at all but at most countably many points.)

The equation of a line with slope p and y-intercept b is given by y = px + b. For this line to be tangent to the graph of a function Template:Mvar at the point (x₀, f(x₀)) requires

$f\left(x_{0}\right)=px_{0}+b$

and

$p={\dot {f}}\left(x_{0}\right).$

f' is strictly monotone as the derivative of a strictly convex function. The second equation can be solved for x₀, allowing elimination of x₀ from the first, giving the y-intercept b of the tangent as a function of its slope p,

$b=f\left({\dot {f}}^{-1}\left(p\right)\right)-p\cdot {\dot {f}}^{-1}\left(p\right)=-f^{\star }(p).$

Here, $f^{\star }$ denotes the Legendre transform of Template:Mvar.

The family of tangents of the graph of Template:Mvar parameterized by Template:Mvar is therefore given by

$y=px-f^{\star }(p)$ ,

or, written implicitly, by the solutions of the equation

$F(x,y,p)=y+f^{\star }(p)-px=0~.$

The graph of the original function can be reconstructed from this family of lines as the envelope of this family by demanding

${\partial F(x,y,p) \over \partial p}={\dot {f}}^{\star }(p)-x=0~.$

Eliminating p from these two equations gives

$y=x\cdot {\dot {f}}^{\star -1}(x)-f^{\star }\left({\dot {f}}^{\star -1}(x)\right).$

Identifying y with f(x) and recognizing the right side of the preceding equation as the Legendre transform of f*, yields

$f(x)=f^{\star \star }(x)~.$

Legendre transformation in more than one dimension

For a differentiable real-valued function on an open subset Template:Mvar of Rⁿ the Legendre conjugate of the pair (U, f) is defined to be the pair (V, g), where Template:Mvar is the image of Template:Mvar under the gradient mapping Df, and Template:Mvar is the function on Template:Mvar given by the formula

$g(y)=\left\langle y,x\right\rangle -f(x),\qquad x=\left(Df\right)^{-1}(y)$

where

$\left\langle u,v\right\rangle =\sum _{k=1}^{n}u_{k}\cdot v_{k}$

is the scalar product on Rⁿ. The multidimensional transform can be interpreted as an encoding of the convex hull of the function's epigraph in terms of its supporting hyperplanes.[1]

Alternatively, if Template:Mvar is a vector space and Y is its dual vector space, then for each point Template:Mvar of X and y of Y, there is a natural identification of the cotangent spaces T*X_x with Y and T*Y_y with X. If Template:Mvar is a real differentiable function over X, then ∇f is a section of the cotangent bundle T*X and as such, we can construct a map from X to Y. Similarly, if Template:Mvar is a real differentiable function over Y, ∇g defines a map from Y to X'. If both maps happen to be inverses of each other, we say we have a Legendre transform.

When the function is not differentiable, the Legendre transform can still be extended, and is known as the Legendre-Fenchel transformation. In this more general setting, a few properties are lost: for example, the Legendre transform is no longer its own inverse (unless there are extra assumptions, like convexity).

Further properties

Scaling properties

The Legendre transformation has the following scaling properties: For a > 0,

$f(x)=a\cdot g(x)\Rightarrow f^{\star }(p)=a\cdot g^{\star }\left({\frac {p}{a}}\right)$

$f(x)=g(a\cdot x)\Rightarrow f^{\star }(p)=g^{\star }\left({\frac {p}{a}}\right).$

It follows that if a function is [[homogeneous function|homogeneous of degree Template:Mvar]] then its image under the Legendre transformation is a homogeneous function of degree Template:Mvar, where 1/r + 1/s = 1. (Since f(x) = x^r/r, with r > 1, implies f*(p) = p^s/s.) Thus, the only monomial whose degree is invariant under Legendre transform is the quadratic.

Behavior under translation

$f(x)=g(x)+b\Rightarrow f^{\star }(p)=g^{\star }(p)-b$

$f(x)=g(x+y)\Rightarrow f^{\star }(p)=g^{\star }(p)-p\cdot y$

Behavior under inversion

$f(x)=g^{-1}(x)\Rightarrow f^{\star }(p)=-p\cdot g^{\star }\left({\frac {1}{p}}\right)$

Behavior under linear transformations

Let A : Rⁿ → R^m be a linear transformation. For any convex function Template:Mvar on Rⁿ, one has

$(Af)^{\star }=f^{\star }A^{\star }$

where A* is the adjoint operator of Template:Mvar defined by

$\left\langle Ax,y^{\star }\right\rangle =\left\langle x,A^{\star }y^{\star }\right\rangle ,$

and Af is the push-forward of Template:Mvar along Template:Mvar

$(Af)(y)=\inf\{f(x):x\in X,Ax=y\}.$

A closed convex function Template:Mvar is symmetric with respect to a given set Template:Mvar of orthogonal linear transformations,

$f(Ax)=f(x),\;\forall x,\;\forall A\in G$

if and only if f* is symmetric with respect to Template:Mvar.

Infimal convolution

The infimal convolution of two functions Template:Mvar and Template:Mvar is defined as

$\left(f\star _{\inf }g\right)(x)=\inf \left\{f(x-y)+g(y)\,|\,y\in \mathbf {R} ^{n}\right\}.$

Let f₁, ..., f_m be proper convex functions on Rⁿ. Then

$\left(f_{1}\star _{\inf }\cdots \star _{\inf }f_{m}\right)^{\star }=f_{1}^{\star }+\cdots +f_{m}^{\star }.$

Fenchel's inequality

For any function Template:Mvar and its convex conjugate f* Fenchel's inequality (also known as the Fenchel–Young inequality) holds for every x ∈ X and p ∈ X*, i.e., independent x, p pairs,

$\left\langle p,x\right\rangle \leq f(x)+f^{\star }(p).$

See also

• Dual curve
• Projective duality
• Young's inequality
• Convex conjugate
• Moreau's theorem
• Integration by parts
• Fenchel's duality theorem

References

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• Fenchel, W. (1949). "On conjugate convex functions", Canad. J. Math 1: 73-77.
• {{#invoke:citation/CS1|citation

|CitationClass=book }}

• Template:Cite doi

Further reading

• Template:Cite web
• Template:Cite web

External links

• Legendre transform with figures at maze5.net
• Legendre and Legendre-Fenchel transforms in a step-by-step explanation at onmyphd.com