Main Page: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
← Older edit
mNo edit summary
No edit summary
 
(288 intermediate revisions by more than 100 users not shown)
Line 1: Line 1:
{{context|date=February 2012}}
This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users.
'''Pointwise mutual information''' ('''PMI''')<ref name="Church1990">{{cite journal|url=http://dl.acm.org/citation.cfm?id=89095|author=Kenneth Ward Church and Patrick Hanks|year=1990|title=Word association norms, mutual information, and lexicography|journal=Comput. Linguist.|volume=16|issue=1|date=March 1990|pages=22-29}}</ref> , or '''point mutual information''', is a measure of association used in [[information theory]] and [[statistics]].


==Definition==
If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]]
The PMI of a pair of [[probability space|outcomes]] ''x'' and ''y'' belonging to [[discrete random variable]]s ''X'' and ''Y'' quantifies the discrepancy between the probability of their coincidence given their joint distribution and their individual distributions, assuming independence. Mathematically:
* Only registered users will be able to execute this rendering mode.
* Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.


: <math>
Registered users will be able to choose between the following three rendering modes:  
\operatorname{pmi}(x;y) \equiv \log\frac{p(x,y)}{p(x)p(y)} = \log\frac{p(x|y)}{p(x)} = \log\frac{p(y|x)}{p(y)}.
</math>


The [[mutual information]] (MI) of the random variables ''X'' and ''Y'' is the expected value of the PMI over all possible outcomes (with respect to the joint distribution <math>p(x,y)</math>).
'''MathML'''
:<math forcemathmode="mathml">E=mc^2</math>


The measure is symmetric (<math>\operatorname{pmi}(x;y)=\operatorname{pmi}(y;x)</math>).  It can take positive or negative values, but is zero if ''X'' and ''Y'' are [[statistical independence|independent]].  PMI maximizes when ''X'' and ''Y'' are [[perfectly associated]], yielding the following bounds:
<!--'''PNG'''  (currently default in production)
:<math forcemathmode="png">E=mc^2</math>


:<math>
'''source'''
-\infty \leq \operatorname{pmi}(x;y) \leq \min\left[ -\log p(x), -\log p(y) \right] .
:<math forcemathmode="source">E=mc^2</math> -->
</math>


Finally, <math>\operatorname{pmi}(x;y)</math> will increase if <math>p(x|y)</math> is fixed but <math>p(x)</math>decreases.
<span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples].


Here is an example to illustrate:
==Demos==
{| border="1" cellpadding="2" class="wikitable"
!''x''!!''y''!!''p''(''x'',&nbsp;''y'')
|-
|0||0||0.1
|-
|0||1||0.7
|-
|1||0||0.15
|-
|1||1||0.05
|}
Using this table we can marginalize to get the following additional table for the individual distributions:
{| border="1" cellpadding="2" class="wikitable"
! !!''p''(''x'')!!''p''(''y'')
|-
|0||.8||0.25
|-
|1||.2||0.75
|}
With this example, we can compute four values for <math>pmi(x;y)</math>.  Using base-2 logarithms:
{| cellpadding="2"
|-
|pmi(x=0;y=0)||&minus;1
|-
|pmi(x=0;y=1)||0.222392421
|-
|pmi(x=1;y=0)||1.584962501
|-
|pmi(x=1;y=1)||&minus;1.584962501
|-
|}


(For reference, the [[mutual information]] <math>\operatorname{I}(X;Y)</math> would then be 0.214170945)
Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]:


==Similarities to mutual information==
Pointwise Mutual Information has many of the same relationships as the mutual information.  In particular,


<math>
* accessibility:
\begin{align}
** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]]
\operatorname{pmi}(x;y) &=& h(x) + h(y) - h(x,y) \\
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)]
&=& h(x) - h(x|y) \\
** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)]
&=& h(y) - h(y|x)
** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)]
\end{align}
** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]].
</math>
** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]].
** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode.


Where <math>h(x)</math> is the [[self-information]], or <math>-\log_2 p(X=x)</math>.
==Test pages ==


==Normalized pointwise mutual information (npmi)==
To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages:
Pointwise mutual information can be normalized between [-1,+1] resulting in -1 (in the limit) for never occurring together, 0 for independence, and +1 for complete [[co-occurrence]].
*[[Displaystyle]]
*[[MathAxisAlignment]]
*[[Styling]]
*[[Linebreaking]]
*[[Unique Ids]]
*[[Help:Formula]]


<math>
*[[Inputtypes|Inputtypes (private Wikis only)]]
 
*[[Url2Image|Url2Image (private Wikis only)]]
\operatorname{npmi}(x;y) = \frac{\operatorname{pmi}(x;y)}{-\log \left[ p(x, y) \right] }
==Bug reporting==
 
If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de .
</math>
 
==Chain-rule for pmi==
Pointwise mutual information follows the [[Chain_rule_%28disambiguation%29|chain rule]], that is,
:<math>\operatorname{pmi}(x;yz) = \operatorname{pmi}(x;y) + \operatorname{pmi}(x;z|y)</math>
 
This is easily proven by:
:<math>
\begin{align}
\operatorname{pmi}(x;y) + \operatorname{pmi}(x;z|y) & {} = \log\frac{p(x,y)}{p(x)p(y)} + \log\frac{p(x,z|y)}{p(x|y)p(z|y)} \\
& {} = \log \left[ \frac{p(x,y)}{p(x)p(y)} \frac{p(x,z|y)}{p(x|y)p(z|y)} \right] \\
& {} = \log \frac{p(x|y)p(y)p(x,z|y)}{p(x)p(y)p(x|y)p(z|y)} \\
& {} = \log \frac{p(x,yz)}{p(x)p(yz)} \\
& {} = \operatorname{pmi}(x;yz)
\end{align}
</math>
 
{{inline|date=February 2012}}
 
==References==
{{reflist}}
* {{cite web|title=Normalized (Pointwise) Mutual Information in Collocation Extraction|url=https://svn.spraakdata.gu.se/repos/gerlof/pub/www/Docs/npmi-pfd.pdf|last1=Bouma|first1=Gerloff|year=2009|publisher=Proceedings of the Biennial GSCL Conference}}
* {{cite book|last1=Fano|first1=R M|year=1961|title=Transmission of Information: A Statistical Theory of Communications|publisher=MIT Press, Cambridge, MA|chapter=chapter 2|isbn=978-0262561693}}
 
==External links==
* [http://cwl-projects.cogsci.rpi.edu/msr/ Demo at Rensselaer MSR Server] (PMI values normalized to be between 0 and 1)
 
 
[[Category:Information theory]]
[[Category:Statistical dependence]]
[[Category:Entropy and information]]

Latest revision as of 22:52, 15 September 2019

This is a preview for the new MathML rendering mode (with SVG fallback), which is availble in production for registered users.

If you would like use the MathML rendering mode, you need a wikipedia user account that can be registered here [[1]]

  • Only registered users will be able to execute this rendering mode.
  • Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere.

Registered users will be able to choose between the following three rendering modes:

MathML

E=mc2


Follow this link to change your Math rendering settings. You can also add a Custom CSS to force the MathML/SVG rendering or select different font families. See these examples.

Demos

Here are some demos:


Test pages

To test the MathML, PNG, and source rendering modes, please go to one of the following test pages:

Bug reporting

If you find any bugs, please report them at Bugzilla, or write an email to math_bugs (at) ckurs (dot) de .

Retrieved from "https://en.formulasearchengine.com/index.php?title=Main_Page&oldid=329754"