Karplus equation - Revision history

en>Tom.Reding: Gen fixes (page/s, endash, , et al., unicodify, and/or concising wikilinks, etc.), & ref cleanup using AWB

2015-01-06T18:31:32Z

Gen fixes (page/s, endash, &nbsp, et al., unicodify, and/or concising wikilinks, etc.), & ref cleanup using AWB

← Older revision		Revision as of 20:31, 6 January 2015
Line 1:		Line 1:
	~~{{context\|date=February 2012}}~~		Hi there. Allow [http://netwk.hannam.ac.kr/xe/data_2/85669 psychic readers] me begin by introducing the author, her title is Sophia. Ohio is where my home is but my husband desires us to transfer. What me and my family members adore is to climb but I'm thinking on starting some thing new. Office supervising is where her primary income arrives from but she's already utilized for another 1.<br><br>Also visit my web page [http://www.article-galaxy.com/profile.php?a=143251 online psychic] reader [[http://ltreme.com/index.php?do=/profile-127790/info/ http://ltreme.com/]]
	~~'''Pointwise mutual information''' ('''PMI'''), or '''point mutual information''', is a measure of association used in [[information theory]] and [[statistics]]~~.

	~~==Definition==~~
	~~The PMI of a pair of~~ [[probability space\|outcomes]] ''x'' and ''y'' belonging to [[discrete random variable]]s ''X'' and ''Y'' quantifies the discrepancy between the probability of their coincidence given their joint distribution and their individual distributions, assuming independence. Mathematically:

	~~: <math>~~
	~~\operatorname{pmi}(x;y) \equiv \log\frac{p(x,y)}{p(x)p(y)} = \log\frac{p(x\|y)}{p(x)} = \log\frac{p(y\|x)}{p(y)}.~~
	</~~math>~~

	~~The [[mutual information]] (MI) of the random variables ''X'' and ''Y'' is the expected value of the PMI over all possible outcomes (with respect to the joint distribution <math>p(x,y)<~~/~~math>)~~.

	~~The measure is symmetric (<math>\operatorname{pmi}(x;y)=\operatorname{pmi}(y;x)</math>)~~. ~~It can take positive or negative values, but is zero if ''X'' and ''Y'' are [[statistical independence\|independent]]~~. ~~PMI maximizes when ''X'' and ''Y'' are [[perfectly associated~~]~~], yielding~~ the ~~following bounds:~~

	~~:<math>~~
	~~-\infty \leq \operatorname{pmi}(x;y) \leq \min\left[ -\log p(x)~~, ~~-\log p(y) \right]~~ .
	~~</math>~~

	~~Finally, <math>\operatorname{pmi}(x;y)</math> will increase if <math>p(x\|y)</math>~~ is ~~fixed~~ but ~~<math>p(x)</math>decreases~~.

	~~Here~~ is ~~an example~~ to ~~illustrate:~~
	~~{\| border="1" cellpadding="2" class="wikitable"~~
	!''~~x''!!''y''!!''p''(''x'', ''y'')~~
	\|-
	~~\|0\|\|0\|\|0.1~~
	\|-
	~~\|0\|\|1\|\|0.7~~
	\|-
	~~\|1\|\|0\|\|0.15~~
	\|-
	~~\|1\|\|1\|\|0.05~~
	\|}
	~~Using this table we can marginalize to get the following additional table~~ for ~~the individual distributions:~~
	~~{\| border="1" cellpadding="2" class="wikitable"~~
	~~! !!''p''(''x'')!!''p''(''y'')~~
	\|-
	~~\|0\|\|.8\|\|0.25~~
	\|-
	\|1~~\|\|.2\|\|0~~.75
	\|}
	~~With this example, we can compute four values for~~ <~~math~~>~~pmi(x;y)~~<~~/math~~>~~. Using base-2 logarithms~~:
	~~{\| cellpadding="2"~~
	\|-
	~~\|pmi(x=0;y=0)\|\|−1~~
	\|-
	~~\|pmi(x=0;y=1)\|\|0~~.~~222392421~~
	\|-
	~~\|pmi(x=1;y=0)\|\|1~~.~~584962501~~
	\|-
	~~\|pmi(x=1;y=1)\|\|−1.584962501~~
	\|-
	\|}

	~~(For reference, the [[mutual information]] <math>\operatorname{I}(X;Y)<~~/~~math> would then be 0~~.~~214170945)~~

	=~~=Similarities to mutual information==~~
	~~Pointwise Mutual Information has many of the same relationships as the mutual information. In particular,~~

	~~<math>~~
	~~\begin{align}~~
	~~\operatorname{pmi}(x;y) &=& h(x) + h(y) - h(x,y) \\~~
	~~&=& h(x) - h(x\|y) \\~~
	~~&=& h(y) - h(y\|x)~~
	~~\end{align}~~
	~~</math>~~

	~~Where <math>h(x)</math> is the [[self-information]~~]~~, or <math>-\log_2 p(X=x)</math>.~~

	~~==Normalized pointwise mutual information (npmi)==~~
	~~Pointwise mutual information can be normalized between~~ [~~-1,+1] resulting in -1 (in the limit) for never occurring together, 0 for independence, and +1 for complete~~ [~~[co-occurrence]].~~

	~~<math>~~

	~~\operatorname{npmi}(x;y) = \frac{\operatorname{pmi}(x;y)}{-\log \left[ p(x, y) \right] }~~

	~~</math>~~

	~~==Chain-rule for pmi==~~
	~~Pointwise mutual information follows the [[Chain_rule_%28disambiguation%29\|chain rule]], that is,~~
	:~~<math>\operatorname{pmi}(x;yz) = \operatorname{pmi}(x;y) + \operatorname{pmi}(x;z\|y)<~~/~~math>~~

	~~This is easily proven by:~~
	~~:<math>~~
	~~\begin{align}~~
	~~\operatorname{pmi}(x;y) + \operatorname{pmi}(x;z\|y) & {} = \log\frac{p(x,y)}{p(x)p(y)} + \log\frac{p(x,z\|y)}{p(x\|y)p(z\|y)} \\~~
	~~& {} = \log \left[ \frac{p(x,y)}{p(x)p(y)} \frac{p(x,z\|y)}{p(x\|y)p(z\|y)} \right] \\~~
	~~& {} = \log \frac{p(x\|y)p(y)p(x,z\|y)}{p(x)p(y)p(x\|y)p(z\|y)} \\~~
	~~& {} = \log \frac{p(x,yz)}{p(x)p(yz)} \\~~
	~~& {} = \operatorname{pmi}(x;yz)~~
	~~\end{align}~~
	</~~math>~~

	~~{{inline\|date=February 2012}}~~

	~~==References==~~
	* {{cite web\|title=Normalized (Pointwise) Mutual Information in Collocation Extraction\|url=https:/~~/svn~~.~~spraakdata.gu.se~~/~~repos/gerlof/pub/www~~/~~Docs~~/~~npmi-pfd.pdf\|last1=Bouma\|first1=Gerloff\|year=2009\|publisher=Proceedings of the Biennial GSCL Conference}}~~
	* {{cite book\|last1=Fano\|first1=R M\|year=1961\|title=Transmission of Information: A Statistical Theory of Communications\|publisher=MIT Press, Cambridge, MA\|chapter=chapter 2}}

	~~==External links==~~
	* [http://~~cwl-projects.cogsci.rpi~~.~~edu~~/~~msr/ Demo at Rensselaer MSR Server] (PMI values normalized to be between 0 and 1)~~


	~~[[Category:Information theory]]~~
	~~[[Category:Statistical dependence]]~~
	~~[[Category:Entropy and information~~]]

en>Mark viking: Added wl

2013-10-10T07:44:30Z

Added wl

← Older revision		Revision as of 09:44, 10 October 2013
Line 1:		Line 1:
	~~Marvella~~ is ~~what you can call her but it~~'s ~~not~~ the ~~most female title out there~~. ~~To play baseball~~ is the ~~hobby he~~ will ~~never quit performing~~. ~~Managing people~~ is ~~what~~ I ~~do and~~ the ~~salary has been truly satisfying~~. ~~North Dakota~~ is ~~where me~~ and ~~my spouse live~~.<br><br>~~Also visit my web blog~~ - [~~http~~://~~www~~.~~adosphere~~.~~com~~/~~poyocum~~ www.~~adosphere~~.~~com~~]		{{context\|date=February 2012}}
			'''Pointwise mutual information''' ('''PMI'''), or '''point mutual information''', is a measure of association used in [[information theory]] and [[statistics]].

			==Definition==
			The PMI of a pair of [[probability space\|outcomes]] ''x'' and ''y'' belonging to [[discrete random variable]]s ''X'' and ''Y'' quantifies the discrepancy between the probability of their coincidence given their joint distribution and their individual distributions, assuming independence. Mathematically:

			: <math>
			\operatorname{pmi}(x;y) \equiv \log\frac{p(x,y)}{p(x)p(y)} = \log\frac{p(x\|y)}{p(x)} = \log\frac{p(y\|x)}{p(y)}.
			</math>

			The [[mutual information]] (MI) of the random variables ''X'' and ''Y'' is the expected value of the PMI over all possible outcomes (with respect to the joint distribution <math>p(x,y)</math>).

			The measure is symmetric (<math>\operatorname{pmi}(x;y)=\operatorname{pmi}(y;x)</math>). It can take positive or negative values, but is zero if ''X'' and ''Y'' are [[statistical independence\|independent]]. PMI maximizes when ''X'' and ''Y'' are [[perfectly associated]], yielding the following bounds:

			:<math>
			-\infty \leq \operatorname{pmi}(x;y) \leq \min\left[ -\log p(x), -\log p(y) \right] .
			</math>

			Finally, <math>\operatorname{pmi}(x;y)</math> will increase if <math>p(x\|y)</math> is fixed but <math>p(x)</math>decreases.

			Here is an example to illustrate:
			{\| border="1" cellpadding="2" class="wikitable"
			!''x''!!''y''!!''p''(''x'', ''y'')
			\|-
			\|0\|\|0\|\|0.1
			\|-
			\|0\|\|1\|\|0.7
			\|-
			\|1\|\|0\|\|0.15
			\|-
			\|1\|\|1\|\|0.05
			\|}
			Using this table we can marginalize to get the following additional table for the individual distributions:
			{\| border="1" cellpadding="2" class="wikitable"
			! !!''p''(''x'')!!''p''(''y'')
			\|-
			\|0\|\|.8\|\|0.25
			\|-
			\|1\|\|.2\|\|0.75
			\|}
			With this example, we can compute four values for <math>pmi(x;y)</math>. Using base-2 logarithms:
			{\| cellpadding="2"
			\|-
			\|pmi(x=0;y=0)\|\|−1
			\|-
			\|pmi(x=0;y=1)\|\|0.222392421
			\|-
			\|pmi(x=1;y=0)\|\|1.584962501
			\|-
			\|pmi(x=1;y=1)\|\|−1.584962501
			\|-
			\|}

			(For reference, the [[mutual information]] <math>\operatorname{I}(X;Y)</math> would then be 0.214170945)

			==Similarities to mutual information==
			Pointwise Mutual Information has many of the same relationships as the mutual information. In particular,

			<math>
			\begin{align}
			\operatorname{pmi}(x;y) &=& h(x) + h(y) - h(x,y) \\
			&=& h(x) - h(x\|y) \\
			&=& h(y) - h(y\|x)
			\end{align}
			</math>

			Where <math>h(x)</math> is the [[self-information]], or <math>-\log_2 p(X=x)</math>.

			==Normalized pointwise mutual information (npmi)==
			Pointwise mutual information can be normalized between [-1,+1] resulting in -1 (in the limit) for never occurring together, 0 for independence, and +1 for complete [[co-occurrence]].

			<math>

			\operatorname{npmi}(x;y) = \frac{\operatorname{pmi}(x;y)}{-\log \left[ p(x, y) \right] }

			</math>

			==Chain-rule for pmi==
			Pointwise mutual information follows the [[Chain_rule_%28disambiguation%29\|chain rule]], that is,
			:<math>\operatorname{pmi}(x;yz) = \operatorname{pmi}(x;y) + \operatorname{pmi}(x;z\|y)</math>

			This is easily proven by:
			:<math>
			\begin{align}
			\operatorname{pmi}(x;y) + \operatorname{pmi}(x;z\|y) & {} = \log\frac{p(x,y)}{p(x)p(y)} + \log\frac{p(x,z\|y)}{p(x\|y)p(z\|y)} \\
			& {} = \log \left[ \frac{p(x,y)}{p(x)p(y)} \frac{p(x,z\|y)}{p(x\|y)p(z\|y)} \right] \\
			& {} = \log \frac{p(x\|y)p(y)p(x,z\|y)}{p(x)p(y)p(x\|y)p(z\|y)} \\
			& {} = \log \frac{p(x,yz)}{p(x)p(yz)} \\
			& {} = \operatorname{pmi}(x;yz)
			\end{align}
			</math>

			{{inline\|date=February 2012}}

			==References==
			* {{cite web\|title=Normalized (Pointwise) Mutual Information in Collocation Extraction\|url=https://svn.spraakdata.gu.se/repos/gerlof/pub/www/Docs/npmi-pfd.pdf\|last1=Bouma\|first1=Gerloff\|year=2009\|publisher=Proceedings of the Biennial GSCL Conference}}
			* {{cite book\|last1=Fano\|first1=R M\|year=1961\|title=Transmission of Information: A Statistical Theory of Communications\|publisher=MIT Press, Cambridge, MA\|chapter=chapter 2}}

			==External links==
			* [http://cwl-projects.cogsci.rpi.edu/msr/ Demo at Rensselaer MSR Server] (PMI values normalized to be between 0 and 1)


			[[Category:Information theory]]
			[[Category:Statistical dependence]]
			[[Category:Entropy and information]]

en>Bibcode Bot: Adding 0 arxiv eprint(s), 1 bibcode(s) and 0 doi(s). Did it miss something? Report bugs, errors, and suggestions at User talk:Bibcode Bot

2012-08-18T15:11:05Z

Adding 0 arxiv eprint(s), 1 bibcode(s) and 0 doi(s). Did it miss something? Report bugs, errors, and suggestions at User talk:Bibcode Bot

New page

Marvella is what you can call her but it's not the most female title out there. To play baseball is the hobby he will never quit performing. Managing people is what I do and the salary has been truly satisfying. North Dakota is where me and my spouse live.<br><br>Also visit my web blog - [http://www.adosphere.com/poyocum www.adosphere.com]