Test theories of special relativity

The entropic vector or entropic function is a concept arising in information theory. Shannon's information entropy measures and their associated identities and inequalities (both constrained and unconstrained) have received a lot of attention over the past from the time Shannon introduced his concept of Information Entropy. A lot of inequalities and identities have been found and are available in standard Information Theory texts. But recent researchers have laid focus on trying to find all possible identities and inequalities (both constrained and unconstrained) on such entropies and characterize them. Entropic vector lays down the basic framework for such a study.

Definition

Let $X_1,X_2,\dots ,X_n$ to be random variables, with $n\in N$

will tell you that h a element in $R^{2^n-1}$ is entropic vector of order n if and only if there a tuple $\overrightarrow{X}=X_1,X_2,\dots ,X_n$ with associated vector $h_{\overrightarrow{X}}$ defined by $h_{\overrightarrow{X}}(I)=H(X_I)=H(X_{i_1},X_{i_2},\dots ,X_{i_k})$ where $I=\{i_1,i_2,\dots ,i_k\}$ y $h=h_{\overrightarrow{x}}$. the set of all entropic vectors of order n is denoted by ${\mathrm{\Gamma }}_n^{*}$

All the properties for entropic functions can be used in vectors.

$H:P_n\to R^{+}$ is continuous

Given $x$ deterministic random variable we have $H(x)=0$

Given $\alpha \in R^{+}$, exist $x$ random variable such as $H(x)=\alpha$

Given $P$ a probalility distribution on $[n]$ we have $H(P)\le {\log }_{2}n$

Example

Let X,Y be two independent random variables with discrete uniform distribution over the set $\{0,1\}$. Then

$H\left(X\right)=H(Y)=1,I(X;Y)=0$

In Cosequence be obtain that

$H(X,Y)=H(X)+H(Y)-I(X;Y)=2$

The entropic vector is thus

$v={(1,1,2)}^T\in {\mathrm{\Gamma }}_2^{*}$

The region Γ_n^*

The Shannon inequality and Γ_n

The entropy satisfies the properties

$1)H(\mathrm{\varnothing })=0$

$2)\alpha \subseteq \beta :H(\alpha )\le H(\beta )$

The Shannon inequality is

$3)H(X_{\alpha })+H(X_{\beta })\le H(X_{\alpha \cup \beta })+H(X_{\alpha \cap \beta })$

The entropy vector that satisfies the linear combination of this region is called ${\mathrm{\Gamma }}_n$.

The region ${\mathrm{\Gamma }}_n^{*}$ has been studied recently, the cases for n = 1, 2, 3

$L_n={\mathrm{\Gamma }}_n={\mathrm{\Gamma }}_n^{*}=\stackrel{*}{\overline{{\mathrm{\Gamma }}_n}}$

$L_n^o={\mathrm{\Gamma }}_n^o=\stackrel{*o}{\overline{{\mathrm{\Gamma }}_n}}=⟨{\mathrm{S}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{o}\mathrm{n}}_n{⟩}^{+}$

if and only if n ∈ {1, 2, 3}

File:Cone diagram.jpg

Cone diagram

It is difficult harder con the case $n\ge 4$, the number of inequalities given by monotone and submodularity properties increase when we increase n, however the relationship among entropic vectors, polymatroids, are an object of study for the information theory and there are other ways to characterize those relationships mentioned

The most important results for the characterization of ${\mathrm{\Gamma }}_n^{*}$ is not precisely about these set, but its topological clousure i.e. the set $\overrightarrow{{\mathrm{\Gamma }}_n^{*}}$, which says that $\overrightarrow{{\mathrm{\Gamma }}_n^{*}}$ is a convex cone, other interesing characterization is that $\overrightarrow{{\mathrm{\Gamma }}_n^{*}}={\mathrm{\Gamma }}_n$ (${\mathrm{\Gamma }}_n$ is the set of vectors that satisfy Shannon-type inequalities) for $n\le 3$, in other words the set of entropy vector is completely characterized by Sahnnon's Inequalities, for the case n = 4 fails this property, particularly by the Ingleton's inequality.

$L_n\subseteq \stackrel{*}{\overline{{\mathrm{\Gamma }}_n}}\subseteq {\mathrm{\Gamma }}_n$

${\mathrm{\Gamma }}_n^o\subseteq \stackrel{*o}{\overline{{\mathrm{\Gamma }}_n}}\subseteq L_n^o$

${\mathrm{\Gamma }}_n^o=⟨{\mathrm{S}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{o}\mathrm{n}}_n{⟩}^{+}$

The Matus theorem

On the year 1998 the Senior Member IEEE Zhen Zhang and Raymond W. Yeung

.

show a new non-Shannon's inequality

$I(X_3,X_4)=I(X4,X_2)+I(X_1:X_3,X_4)+3I(X_3:X_4|X_1)+I(X_3:X_4|X_2)$

On the year 2007 Matus proved

$\overline{{\mathrm{\Gamma }}_4^{*}}$ is not polihedral.

Entropy and groups

Group-charactizable vectors and quasi-uniform distribution

One way to charactize ${\mathrm{\Gamma }}_n^{*}$ is by looking at some special distributions.\\ Definition: A group characterizable vector h is also denoted to be$2^n\to R$

such that there exists a group $G$ and subgroups $G_1,G_2,\dots ,G_n$ and for $\alpha \subset n$

$H(\alpha )=\frac{\left|Gi\right|}{\left|G\right|}$

if $G_{\alpha }$ is not $\mathrm{\varnothing }$ and 0 otherwise. $G={\cap }_{i\in \alpha }{G}_i$ .

Definition: ${\gamma }^n$ is the set of all group charactizable vectors is $n$, and we can describe better the set ${\mathrm{\Gamma }}^n$

Theorem: ${\gamma }^n\subset {\mathrm{\Gamma }}^n$

Open problem

Given a vector $v\in R^{2^n-1}$, is it possible to say if there exists $n$ random variables such that their joint entropies are given by $v$? It turns out that for $n=2,3$ the problem has been solved. But for $n\ge 4$, it still remains unsolved. Defining the set of all such vectors $v\in R^{2^n-1}$ that can be constructed from a set of $n$ random variables as ${\mathrm{\Gamma }}_n^{*}$, we see that a complete characterization of this space remains an unsolved mystery.

References

• Thomas M. Cover, Joy A. Thomas. Elements of information theory New York: Wiley, 1991. ISBN 0-471-06259-6
• Raymond Yeung. A First Course in Information Theory, Chapter 12, Information Inequalities, 2002, Print ISBN 0-306-46791-7