|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| {{refimprove|date=October 2011}}
| | Hi there, I am Yoshiko Villareal but I never really liked that name. Bookkeeping is how he supports his family members and his salary has been really satisfying. To keep birds is one of the issues he enjoys most. I've usually cherished living in Idaho.<br><br>Here is my web blog :: [http://89.1.13.189/index.php?mod=users&action=view&id=17137 http://89.1.13.189] |
| | |
| The '''Ugly Duckling theorem''' is an [[argument]] asserting that classification is impossible without some sort of [[bias (statistics) | bias]]. It is named for [[Hans Christian Andersen]]'s famous story of "[[The Ugly Duckling]]." It gets its name because it shows that, all things being equal, an ugly duckling is just as similar to a [[swan]] as two swans are to each other, although it is only a [[theorem]] in a very informal sense. It was proposed by [[Satosi Watanabe]] in 1969.<ref>{{cite book
| |
| | last = Watanabe
| |
| | first = Satosi
| |
| | title = Knowing and Guessing: A Quantitative Study of Inference and Information
| |
| | publisher = Wiley
| |
| | year = 1969
| |
| | location = New York
| |
| | pages=376–377}}</ref>
| |
| | |
| ==Basic idea==
| |
| | |
| Suppose there are <var>n</var> things in the universe, and one wants to put them into classes or categories. One has no preconceived ideas or [[bias]]es about what sorts of categories are "natural" or "normal" and what are not. So one has to consider all the possible classes that could be, all the possible ways of making sets out of the <var>n</var> objects. There are <math>2^n</math> such ways, the size of the [[power set]] of <var>n</var> objects. One can use that to measure the similarity between two objects: and one would see how many sets they have in common. However one can not. Any two objects have exactly the same number of classes in common if they are only distinguished by their names{{clarification needed|date=March 2011}}<!-- what means "in common"? if it is a∈C ⇔ b∈C , then really 2^{n-1}, but I understand this word as a∈C & b∈C , which leads to 2^{n-2}. --> with one another, namely <math>2^{n-1}</math> (half the total number of classes there are). To see this is so, one may imagine each class is a represented by an <var>n</var>-bit [[bit array|string]] (or [[binary numeral system|binary encoded]] integer), with a zero for each element not in the class and a one for each element in the class. As one finds, there are <math>2^n</math> such strings.
| |
| | |
| As all possible choices of zeros and ones are there, any two bit-positions will agree exactly half the time. One may pick two elements and reorder the bits so they are the first two, and imagine the numbers sorted lexicographically. The first <math>2^n/2</math> numbers will have bit #1 set to zero, and the second <math>2^n/2</math> will have it set to one. Within each of those blocks, the top <math>2^n/4</math> will have bit #2 set to zero and the other <math>2^n/4</math> will have it as one, so they agree on two blocks of <math>2^n/4</math> or on half of all the cases. No matter which two elements one picks. So if we have no preconceived bias about which categories are better, everything is then equally similar (or equally dissimilar). The number of [[propositional function|predicates]] simultaneously satisfied by two non-identical elements is constant over all such pairs and is the same{{citation needed|date=March 2011}} as the number of those satisfied by one.<!-- I calculated these as 2^{n-2} and 2^{n-1} respectively. --> Thus, some kind of inductive{{citation needed|date=March 2011}} bias is needed to make judgements; i.e. to prefer certain categories over others.
| |
| | |
| ==As a statement about Boolean functions== | |
| Let <math>x_1, x_2, \dots, x_n</math> be a set of vectors of <math>k</math> booleans each. The ugly duckling is the vector which is least like the others. Given the booleans, this can be computed using [[Hamming distance]].
| |
| | |
| However, the choice of boolean features to consider could have been somewhat arbitrary. Perhaps there were features derivable from the original features that were important for identifying the ugly duckling. The set of booleans in the vector can be extended with new features computed as boolean functions of the <math>k</math> original features. The only canonical way to do this is to extend it with ''all'' possible Boolean functions. The resulting completed vectors have <math>2^k</math> features. The Ugly Duckling Theorem states that there is no ugly duckling because any two completed vectors will either be equal or differ in exactly half of the features.
| |
| | |
| Proof. Let x and y be two vectors. If they are the same, then their completed vectors must also be the same because any Boolean function of x will agree with the same Boolean function of y. If x and y are different, then there exists a coordinate <math>i</math> where the <math>i</math>-th coordinate of <math>x</math> differs from the <math>i</math>-th coordinate of <math>y</math>. Now the completed features contain every Boolean function on <math>k</math> Boolean variables, with each one exactly once. Viewing these Boolean functions as polynomials in <math>k</math> variables over GF(2), segregate the functions into pairs <math>(f,g)</math> where <math>f</math> contains the <math>i</math>-th coordinate as a linear term and <math>g</math> is <math>f</math> without that linear term. Now, for every such pair <math>(f,g)</math>, <math>x</math> and <math>y</math> will agree on exactly one of the two functions. If they agree on one, they must disagree on the other and vice versa. (This proof is believed to be due to Watanabe.)
| |
| | |
| ==See also==
| |
| * [[No free lunch in search and optimization]]
| |
| * [[No free lunch theorem]]
| |
| | |
| ==Notes==
| |
| <references/>
| |
| | |
| {{The Ugly Duckling}}
| |
| | |
| [[Category:Theorems]]
| |
| [[Category:Arguments]]
| |
| [[Category:Machine learning]]
| |
| [[Category:Ontology]]
| |
Hi there, I am Yoshiko Villareal but I never really liked that name. Bookkeeping is how he supports his family members and his salary has been really satisfying. To keep birds is one of the issues he enjoys most. I've usually cherished living in Idaho.
Here is my web blog :: http://89.1.13.189