|
|
| Line 1: |
Line 1: |
| In [[information theory]], the '''typical set''' is a set of sequences whose [[probability]] is close to two raised to the negative power of the [[Information entropy|entropy]] of their source distribution. That this set has total [[probability]] close to one is a consequence of the [[asymptotic equipartition property]] (AEP) which is a kind of [[law of large numbers]]. The notion of typicality is only concerned with the probability of a sequence and not the actual sequence itself.
| | Some folks need a high-calorie, high-protein diet, even for a short period of time. This applies for people whom have newly lost weight, have a poor appetite. Perhaps certain are recovering from disease or like to build muscle or maintain a heavy exercise or function system. Others merely desire to place on weight. High calorie, significant protein diets are usually suggested for children. While there is a great deal of information available on calories and protein contents of foods, there is comparatively surprisingly small data on foods which have the mixture of excellent calories plus excellent protein.<br><br>What is Body Mass Index? Body Mass Index, more commonly well-known as "BMI", is an indicator of the amount of body fat a person has. The BMI essentially divides your fat by your height to give we the body fat analysis. The greater amount of body fat a person carries, the more likely they are to have wellness associated illnesses. "Weight for height" is an important measuring considering when you're inside the ideal range of weight for height, you can expect to live the longest lifetime possible with limited ailments. When a individual steps out of this range, on either side; having too small fat or having too much weight, is when health problems occur.<br><br>Fats which are superior for you are those that are called unsaturated, polyunsaturated, plus monounsaturated. We'll receive waist to height ratio into what all which signifies inside future articles, yet for now, only look found on the food labels for "un", "mono", and "poly" before the word "saturated". These are the "superior guys" which help you stay well.<br><br>While skinfold is calculated at house with a wise caliper, it ought to be done by a trained fitness professional in purchase to avoid error. Skinfold uses the total of measured subcutaneous fat and age as factors to predict an total body fat percentage. Error 4% (ACSM, 107).<br><br>The chart given earlier was for men above the age of 20 plus below the age of 60. However, men above the age of 50 should make a note of the fact that, regardless of your body frame kind, it happens to be important for you to cut down found on the fat element of the body weight. This can help in protecting you from age-related fat disorders, heart ailments plus other health issues. The BMI (Body Mass Index) is a good means of finding out what the perfect weight range is, or ought to be. A BMI range of 19 to 25 is considered to be healthy. Anything above 25 would place you inside the obese category.<br><br>Then, coming back to what I was discussing earlier which after striving all these conventional techniques of fat reduction for the last 4 years, I have come to the conclusion which [http://safedietplansforwomen.com/waist-to-height-ratio waist to height ratio] there could be a technique that you can rely on. I mean to say that there must be a means to lose body fat quick. I researched on this for several months plus then found a permanent solution. Guys, I am referring to Dr. Charles (famous nutritionist and fitness expert) techniques to get rid of weight. He teaches how to burn body fat quickly. He additionally tells the number one techniques to get rid of body fat.<br><br>Your ideal waist size will be between 45%-47% of your height. In other words, when you're 70 inches tall you're perfect waist is between 31.5 - 33 inches.<br><br>Percentage body fat is anything which is performed at the doctors or by experts in the region. It is done by measuring fat beneath folds of skin. Usually at the waist, hips and thighs. This may not be truly accurate as individuals age because the fat distribution inside the body changes because you receive older. Its more exact for individuals beneath 40 specifically however, it nevertheless functions OK until age of 55. The accuracy begins to decrease from the age of 40 onwards. |
| | |
| This has great use in [[data compression|compression]] theory as it provides a theoretical means for compressing data, allowing us to represent any sequence ''X''<sup>''n''</sup> using ''nH''(''X'') bits on average, and, hence, justifying the use of entropy as a measure of information from a source.
| |
| | |
| The AEP can also be proven for a large class of [[stationary ergodic process]]es, allowing typical set to be defined in more general cases. | |
| | |
| ==(Weakly) typical sequences (weak typicality, entropy typicality)==
| |
| If a sequence ''x''<sub>1</sub>, ..., ''x''<sub>''n''</sub> is drawn from an [[Independent identically-distributed random variables|i.i.d. distribution]] ''X'' defined over a finite alphabet <math>\mathcal{X}</math>, then the typical set, ''A''<sub>''ε''</sub><sup>(''n'')</sup><math>\in\mathcal{X}</math><sup>(''n'')</sup> is defined as those sequences which satisfy:
| |
| | |
| :<math>
| |
| 2^{-n(H(X)+\varepsilon)} \leqslant p(x_1, x_2, \dots , x_n) \leqslant 2^{-n(H(X)-\varepsilon)}
| |
| </math>
| |
| | |
| Where
| |
| | |
| : <math> H(X) = - \sum_{y \isin \mathcal{X}}p(y)\log_2 p(y) </math>
| |
| | |
| is the information entropy of ''X''. The probability above need only be within a factor of 2<sup>''n''ε''</sup>.
| |
| | |
| It has the following properties if ''n'' is sufficiently large, <math>\epsilon>0</math> can be chosen arbitrarily small so that:
| |
| #The probability of a sequence from ''X'' being drawn from ''A''<sub>''ε''</sup><sup>(''n'')</sup> is greater than 1 − ''ε'', i.e. <math>Pr[x^{(n)} \in A_\epsilon^{(n)}] \geq 1 - \epsilon </math>
| |
| #<math>\left| {A_\varepsilon}^{(n)} \right| \leqslant 2^{n(H(X)+\varepsilon)}</math>
| |
| #<math>\left| {A_\varepsilon}^{(n)} \right| \geqslant (1-\varepsilon)2^{n(H(X)-\varepsilon)}</math>
| |
| #Most sequences are not typical. If the distribution over <math>\mathcal{X}</math> is not uniform, then the fraction of sequences that are typical is
| |
| ::<math>\frac{|A_\epsilon^{(n)}|}{|\mathcal{X}^{(n)}|} \equiv \frac{2^{nH(X)}}{2^{n\log|\mathcal{X}|}} = 2^{-n(\log|\mathcal{X}|-H(X))} \rightarrow 0 </math>
| |
| | |
| ::as ''n'' becomes very large, since <math>H(X) < \log|\mathcal{X}|.</math>
| |
| | |
| For a general stochastic process {''X''(''t'')} with AEP, the (weakly) typical set can be defined similarly with ''p''(''x''<sub>1</sub>, ''x''<sub>2</sub>, ..., ''x''<sub>''n''</sub>) replaced by ''p''(''x''<sub>0</sub><sup>''τ''</sup>) (i.e. the probability of the sample limited to the time interval [0, ''τ'']), ''n'' being the [[degrees of freedom (physics and chemistry)|degree of freedom]] of the process in the time interval and ''H''(''X'') being the [[entropy rate]]. If the process is continuous-valued, [[differential entropy]] is used instead.
| |
| | |
| Counter-intuitively, most likely sequence is often not a member of the typical set. For example, suppose that ''X'' is an i.i.d Bernoulli random variable with ''p''(0)=0.1 and ''p''(1)=0.9. In ''n'' independent trials, since ''p''(1)>''p''(0), the most likely sequence of outcome is the sequence of all 1's, (1,1,...,1). Here the entropy of ''X'' is ''H''(''X'')=0.469, while <math> -\frac{1}{n}\log p(x^{(n)}=(1,1,\ldots,1)) = -\frac{1}{n}\log (0.9)^n = 0.152</math>
| |
| | |
| So this sequence is not in the typical set because its average logarithmic probability cannot come arbitrarily close to the entropy of the random variable ''X'' no matter how large we take the value of ''n''. For Bernoulli random variables, the typical set consists of sequences with average numbers of 0s and 1s in ''n'' independent trials. For this example, if ''n''=10, then the typical set consist of all sequences that has a single 0 in the entire sequence. In case ''p''(0)=''p''(1)=0.5, then every possible binary sequences belong to the typical set.
| |
| | |
| ==Strongly typical sequences (strong typicality, letter typicality)==
| |
| If a sequence ''x''<sub>1</sub>, ..., ''x''<sub>''n''</sub> is drawn from some specified joint distribution defined over a finite or an infinite alphabet <math>\mathcal{X}</math>, then the strongly typical set, ''A''<sub>ε,strong</sub><sup>(''n'')</sup><math>\in\mathcal{X}</math> is defined as the set of sequences which satisfy
| |
| | |
| :<math>
| |
| \left|\frac{N(x_i)}{n}-p(x_i)\right| < \frac{\varepsilon}{\|\mathcal{X}\|}.
| |
| </math>
| |
| | |
| where <math>{N(x_i)}</math> is the number of occurrences of a specific symbol in the sequence.
| |
| | |
| It can be shown that strongly typical sequences are also weakly typical (with a different constant ε), and hence the name. The two forms, however, are not equivalent. Strong typicality is often easier to work with in proving theorems for memoryless channels. However, as is apparent from the definition, this form of typicality is only defined for random variables having finite support.
| |
| | |
| ==Jointly typical sequences==
| |
| Two sequences <math>x^n</math> and <math>y^n</math> are jointly ε-typical if the pair <math>(x^n,y^n)</math> is ε-typical with respect to the joint distribution <math>p(x^n,y^n)=\prod_{i=1}^n p(x_i,y_i)</math> and both <math>x^n</math> and <math>y^n</math> are ε-typical with respect to their marginal distributions <math>p(x^n)</math> and <math>p(y^n)</math>. The set of all such pairs of sequences <math>(x^n,y^n)</math> is denoted by <math>A_{\varepsilon}^n(X,Y)</math>. Jointly ε-typical ''n''-tuple sequences are defined similarly.
| |
| | |
| Let <math>\tilde{X}^n</math> and <math>\tilde{Y}^n</math> be two independent sequences of random variables with the same marginal distributions <math>p(x^n)</math> and <math>p(y^n)</math>. Then for any ε>0, for sufficiently large ''n'', jointly typical sequences satisfy the following properties:
| |
| #<math> P\left[ (X^n,Y^n) \in A_{\varepsilon}^n(X,Y) \right] \geqslant 1 - \epsilon </math>
| |
| #<math> \left| A_{\varepsilon}^n(X,Y) \right| \leqslant 2^{n (H(X,Y) + \epsilon)} </math>
| |
| #<math> \left| A_{\varepsilon}^n(X,Y) \right| \geqslant (1 - \epsilon) 2^{n (H(X,Y) - \epsilon)} </math>
| |
| #<math> P\left[ (\tilde{X}^n,\tilde{Y}^n) \in A_{\varepsilon}^n(X,Y) \right] \leqslant 2^{-n (I(X;Y) - 3 \epsilon)} </math>
| |
| #<math> P\left[ (\tilde{X}^n,\tilde{Y}^n) \in A_{\varepsilon}^n(X,Y) \right] \geqslant (1 - \epsilon) 2^{-n (I(X;Y) + 3 \epsilon)}</math>
| |
| | |
| {{Expand section|date=December 2009}}
| |
| | |
| ==Applications of typicality==
| |
| {{Expand section|date=December 2009}}
| |
| | |
| ===Typical set encoding===
| |
| In [[information theory]], typical set encoding encodes only the typical set of a stochastic source with fixed length block codes. Asymptotically, it is, by the AEP, lossless and achieves the minimum rate equal to the entropy rate of the source.
| |
| | |
| {{Expand section|date=December 2009}}
| |
| | |
| ===Typical set decoding===
| |
| In [[information theory]], typical set decoding is used in conjunction with [[random coding]] to estimate the transmitted message as the one with a codeword that is jointly ε-typical with the observation. i.e.
| |
| :<math>\hat{w}=w \iff (\exists w)( (x_1^n(w),y_1^n)\in A_{\varepsilon}^n(X,Y)) </math>
| |
| where <math>\hat{w},x_1^n(w),y_1^n</math> are the message estimate, codeword of message <math>w</math> and the observation respectively. <math>A_{\varepsilon}^n(X,Y)</math> is defined with respect to the joint distribution <math>p(x_1^n)p(y_1^n|x_1^n)</math> where <math>p(y_1^n|x_1^n)</math> is the transition probability that characterizes the channel statistics, and <math>p(x_1^n)</math> is some input distribution used to generate the codewords in the random codebook.
| |
| | |
| {{Expand section|date=December 2009}}
| |
| | |
| ===Universal null-hypothesis testing===
| |
| {{Empty section|date=December 2009}}
| |
| | |
| ===Universal channel code===
| |
| {{Expand section|date=December 2009}}
| |
| {{See also|algorithmic complexity theory}}
| |
| | |
| ==See also==
| |
| * [[Asymptotic equipartition property]]
| |
| * [[Source coding theorem]]
| |
| * [[Noisy-channel coding theorem]]
| |
| | |
| ==References==
| |
| * [[C. E. Shannon]], "[http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf A Mathematical Theory of Communication]", ''[[Bell System Technical Journal]]'', vol. 27, pp. 379–423, 623-656, July, October, 1948
| |
| * {{Cite book
| |
| | last = Cover
| |
| | first = Thomas M.
| |
| | title = Elements of Information Theory
| |
| | chapter = Chapter 3: Asymptotic Equipartition Property, Chapter 5: Data Compression, Chapter 8: Channel Capacity
| |
| | year = 2006
| |
| | publisher = John Wiley & Sons
| |
| | isbn = 0-471-24195-4 }}
| |
| * [[David J. C. MacKay]]. ''[http://www.inference.phy.cam.ac.uk/mackay/itila/book.html Information Theory, Inference, and Learning Algorithms]'' Cambridge: Cambridge University Press, 2003. ISBN 0-521-64298-1
| |
| | |
| {{DEFAULTSORT:Typical Set}}
| |
| [[Category:Information theory]]
| |
| [[Category:Probability theory]]
| |
Some folks need a high-calorie, high-protein diet, even for a short period of time. This applies for people whom have newly lost weight, have a poor appetite. Perhaps certain are recovering from disease or like to build muscle or maintain a heavy exercise or function system. Others merely desire to place on weight. High calorie, significant protein diets are usually suggested for children. While there is a great deal of information available on calories and protein contents of foods, there is comparatively surprisingly small data on foods which have the mixture of excellent calories plus excellent protein.
What is Body Mass Index? Body Mass Index, more commonly well-known as "BMI", is an indicator of the amount of body fat a person has. The BMI essentially divides your fat by your height to give we the body fat analysis. The greater amount of body fat a person carries, the more likely they are to have wellness associated illnesses. "Weight for height" is an important measuring considering when you're inside the ideal range of weight for height, you can expect to live the longest lifetime possible with limited ailments. When a individual steps out of this range, on either side; having too small fat or having too much weight, is when health problems occur.
Fats which are superior for you are those that are called unsaturated, polyunsaturated, plus monounsaturated. We'll receive waist to height ratio into what all which signifies inside future articles, yet for now, only look found on the food labels for "un", "mono", and "poly" before the word "saturated". These are the "superior guys" which help you stay well.
While skinfold is calculated at house with a wise caliper, it ought to be done by a trained fitness professional in purchase to avoid error. Skinfold uses the total of measured subcutaneous fat and age as factors to predict an total body fat percentage. Error 4% (ACSM, 107).
The chart given earlier was for men above the age of 20 plus below the age of 60. However, men above the age of 50 should make a note of the fact that, regardless of your body frame kind, it happens to be important for you to cut down found on the fat element of the body weight. This can help in protecting you from age-related fat disorders, heart ailments plus other health issues. The BMI (Body Mass Index) is a good means of finding out what the perfect weight range is, or ought to be. A BMI range of 19 to 25 is considered to be healthy. Anything above 25 would place you inside the obese category.
Then, coming back to what I was discussing earlier which after striving all these conventional techniques of fat reduction for the last 4 years, I have come to the conclusion which waist to height ratio there could be a technique that you can rely on. I mean to say that there must be a means to lose body fat quick. I researched on this for several months plus then found a permanent solution. Guys, I am referring to Dr. Charles (famous nutritionist and fitness expert) techniques to get rid of weight. He teaches how to burn body fat quickly. He additionally tells the number one techniques to get rid of body fat.
Your ideal waist size will be between 45%-47% of your height. In other words, when you're 70 inches tall you're perfect waist is between 31.5 - 33 inches.
Percentage body fat is anything which is performed at the doctors or by experts in the region. It is done by measuring fat beneath folds of skin. Usually at the waist, hips and thighs. This may not be truly accurate as individuals age because the fat distribution inside the body changes because you receive older. Its more exact for individuals beneath 40 specifically however, it nevertheless functions OK until age of 55. The accuracy begins to decrease from the age of 40 onwards.