|
|
| Line 1: |
Line 1: |
| {{unreferenced|date=February 2010}}
| | I'm Delilah and I live with my husband and our two children in Fort Saskatchewan, in the AB south part. My hobbies are Seaglass collecting, Badminton and Drawing.<br><br>My website - [http://www.gamecookie.com/profile/fawalden buy an essay] |
| In [[probability theory]], '''Hoeffding's lemma''' is an [[inequality (mathematics)|inequality]] that bounds the [[moment-generating function]] of any [[bounded function|bounded]] [[random variable]]. It is named after the [[Finnish people|Finnish]]–[[United States|American]] [[mathematical statistics|mathematical statistician]] [[Wassily Hoeffding]].
| |
| | |
| The proof of Hoeffding's lemma uses [[Taylor's theorem]] and [[Jensen's inequality]]. Hoeffding's lemma is itself used in the proof of [[McDiarmid's inequality]].
| |
| | |
| ==Statement of the lemma==
| |
| | |
| Let ''X'' be any real-valued random variable with [[expected value]] '''E'''[''X''] = 0 and such that ''a'' ≤ ''X'' ≤ ''b'' [[almost surely]]. Then, for all ''λ'' ∈ '''R''',
| |
| | |
| :<math>\mathbf{E} \left[ e^{\lambda X} \right] \leq \exp \left( \frac{\lambda^2 (b - a)^2}{8} \right).</math>
| |
| | |
| ==Proof of the lemma==
| |
| | |
| Since <math> e^{\lambda x}</math> is a convex function, we have
| |
| | |
| :<math>e^{\lambda x}\leq \frac{b-x}{b-a}e^{\lambda a}+\frac{x-a}{b-a}e^{\lambda b}\qquad \forall a\leq x\leq b</math>
| |
| | |
| So, <math> \mathbf{E}\left[e^{\lambda X}\right] \leq \frac{b-EX}{b-a}e^{\lambda a}+\frac{EX-a}{b-a}e^{\lambda b}.</math>
| |
| | |
| Let <math> h=\lambda(b-a)</math>, <math> p=\frac{-a}{b-a}</math> and <math> L(h)=-hp+\ln(1-p+pe^h)</math>
| |
| | |
| Then, <math>\frac{b-EX}{b-a}e^{\lambda a}+\frac{EX-a}{b-a}e^{\lambda b}=e^{L(h)}</math> since <math> EX=0</math>
| |
| | |
| Taking derivative of <math> L(h)</math>,
| |
| :<math> L(0)=L^{'}(0)=0\text{ and } L^{''}(h)\leq \frac{1}{4}</math>
| |
| | |
| By Tayor's expansion,
| |
| | |
| <math> L(h)\leq \frac{1}{8}h^2=\frac{1}{8}\lambda^2(b-a)^2</math>
| |
| | |
| Hence, <math> \mathbf{E}\left[e^{\lambda X}\right] \leq e^{\frac{1}{8}\lambda^2(b-a)^2}</math>
| |
| | |
| ==See also==
| |
| *[[Hoeffding's inequality]]
| |
| *[[Bennett's inequality]]
| |
| | |
| [[Category:Probabilistic inequalities]]
| |
| | |
| | |
| {{probability-stub}}
| |
I'm Delilah and I live with my husband and our two children in Fort Saskatchewan, in the AB south part. My hobbies are Seaglass collecting, Badminton and Drawing.
My website - buy an essay