Anisometropia: Difference between revisions

Revision as of 08:18, 20 July 2013

My name is Winnie and I am studying Anthropology and Sociology and Modern Languages and Classics at Rillieux-La-Pape / France.

Also visit my web site ... hostgator1centcoupon.info In the theory of stochastic processes, the Karhunen–Loève theorem (named after Kari Karhunen and Michel Loève), also known as the Kosambi–Karhunen–Loève theorem^[1]^[2] is a representation of a stochastic process as an infinite linear combination of orthogonal functions, analogous to a Fourier series representation of a function on a bounded interval. Stochastic processes given by infinite series of this form were first^[3] considered by Damodar Dharmananda Kosambi.^[4] There exist many such expansions of a stochastic process: if the process is indexed over [a, b], any orthonormal basis of L²([a, b]) yields an expansion thereof in that form. The importance of the Karhunen–Loève theorem is that it yields the best such basis in the sense that it minimizes the total mean squared error.

In contrast to a Fourier series where the coefficients are real numbers and the expansion basis consists of sinusoidal functions (that is, sine and cosine functions), the coefficients in the Karhunen–Loève theorem are random variables and the expansion basis depends on the process. In fact, the orthogonal basis functions used in this representation are determined by the covariance function of the process. One can think that the Karhunen–Loève transform adapts to the process in order to produce the best possible basis for its expansion.

In the case of a centered stochastic process {X_t}_{t ∈ [a, b]} (where centered means that the expectations E(X_t) are defined and equal to 0 for all values of the parameter t in [a, b]) satisfying a technical continuity condition, X_t admits a decomposition

X_{t} = \sum_{k = 1}^{\infty} Z_{k} e_{k} (t)

where Z_{k are pairwise uncorrelated random variables and the functions e_k are continuous real-valued functions on [a, b] that are pairwise orthogonal in L²[a, b]. It is therefore sometimes said that the expansion is bi-orthogonal since the random coefficients Z_{k are orthogonal in the probability space while the deterministic functions e_k are orthogonal in the time domain. The general case of a process X_t that is not centered can be brought back to the case of a centered process by considering (X_t − E(X_t)) which is a centered process.}}

Moreover, if the process is Gaussian, then the random variables Z_k are Gaussian and stochastically independent. This result generalizes the Karhunen–Loève transform. An important example of a centered real stochastic process on [0,1] is the Wiener process; the Karhunen–Loève theorem can be used to provide a canonical orthogonal representation for it. In this case the expansion consists of sinusoidal functions.

The above expansion into uncorrelated random variables is also known as the Karhunen–Loève expansion or Karhunen–Loève decomposition. The empirical version (i.e., with the coefficients computed from a sample) is known as the Karhunen–Loève transform (KLT), principal component analysis, proper orthogonal decomposition (POD), Empirical orthogonal functions (a term used in meteorology and geophysics), or the Hotelling transform.

Formulation

Throughout this article, we will consider a square integrable zero-mean random process X_t defined over a probability space (Ω,F,P) and indexed over a closed interval [a, b], with covariance function K_X(s,t). We thus have:

\forall t \in [a, b], X_{t} \in L^{2} (Ω, ℱ, P),

\forall t \in [a, b], E [X_{t}] = 0,

\forall t, s \in [a, b], K_{X} (s, t) = E [X_{s} X_{t}] .

We associate to K_X a linear operator T_{K_X} defined in the following way:

\begin{array}{rrl} T_{K_{X}} : L^{2} ([a, b]) & \to & L^{2} ([a, b]) \\ f (t) & \mapsto & \int_{[a, b]} K_{X} (s, t) f (s) d s \end{array}

Since T_{K_X} is a linear operator, it makes sense to talk about its eigenvalues λ_k and eigenfunctions e_k, which are found solving the homogeneous Fredholm integral equation of the second kind

\int_{[a, b]} K_{X} (s, t) e_{k} (s) d s = λ_{k} e_{k} (t)

Statement of the theorem

Theorem. Let X_t be a zero-mean square integrable stochastic process defined over a probability space (Ω,F,P) and indexed over a closed and bounded interval [a, b], with continuous covariance function K_X(s,t).

Then K_X(s,t) is a Mercer kernel and letting e_k be an orthonormal basis of L²([a, b]) formed by the eigenfunctions of T_{K_X} with respective eigenvalues λ_k, X_t admits the following representation

X_{t} = \sum_{k = 1}^{\infty} Z_{k} e_{k} (t)

where the convergence is in L², uniform in t and

Z_{k} = \int_{[a, b]} X_{t} e_{k} (t) d t

Furthermore, the random variables Z_k have zero-mean, are uncorrelated and have variance λ_k

E [Z_{k}] = 0, \forall k \in ℕ and E [Z_{i} Z_{j}] = δ_{i j} λ_{j}, \forall i, j \in ℕ

Note that by generalizations of Mercer's theorem we can replace the interval [a, b] with other compact spaces C and the Lebesgue measure on [a, b] with a Borel measure whose support is C.

Proof

The covariance function K_X satisfies the definition of a Mercer kernel. By Mercer's theorem, there consequently exists a set {λ_k,e_k(t)} of eigenvalues and eigenfunctions of T_{K_X} forming an orthonormal basis of L²([a,b]), and K_X can be expressed as

K_{X} (s, t) = \sum_{k = 1}^{\infty} λ_{k} e_{k} (s) e_{k} (t)

The process X_t can be expanded in terms of the eigenfunctions e_k as:

X_{t} = \sum_{k = 1}^{\infty} Z_{k} e_{k} (t)

where the coefficients (random variables) Z_k are given by the projection of X_t on the respective eigenfunctions

Z_{k} = \int_{[a, b]} X_{t} e_{k} (t) d t

We may then derive

E [Z_{k}] = E [\int_{[a, b]} X_{t} e_{k} (t) d t] = \int_{[a, b]} E [X_{t}] e_{k} (t) d t = 0

and:

\begin{aligned} E [Z_{i} Z_{j}] & = E [\int_{[a, b]} \int_{[a, b]} X_{t} X_{s} e_{j} (t) e_{i} (s) d t d s] \\ = \int_{[a, b]} \int_{[a, b]} E [X_{t} X_{s}] e_{j} (t) e_{i} (s) d t d s \\ = \int_{[a, b]} \int_{[a, b]} K_{X} (s, t) e_{j} (t) e_{i} (s) d t d s \\ = \int_{[a, b]} e_{i} (s) (\int_{[a, b]} K_{X} (s, t) e_{j} (t) d t) d s \\ = λ_{j} \int_{[a, b]} e_{i} (s) e_{j} (s) d s \\ = δ_{i j} λ_{j} \end{aligned}

where we have used the fact that the e_k are eigenfunctions of T_{K_X} and are orthonormal.

Let us now show that the convergence is in L²:
let $S_{N} = \sum_{k = 1}^{N} Z_{k} e_{k} (t)$ .

\begin{aligned} E [| X_{t} - S_{N} |^{2}] & = E [X_{t}^{2}] + E [S_{N}^{2}] - 2 E [X_{t} S_{N}] \\ = K_{X} (t, t) + E [\sum_{k = 1}^{N} \sum_{l = 1}^{N} Z_{k} Z_{l} e_{k} (t) e_{l} (t)] - 2 E [X_{t} \sum_{k = 1}^{N} Z_{k} e_{k} (t)] \\ = K_{X} (t, t) + \sum_{k = 1}^{N} λ_{k} e_{k} (t)^{2} - 2 E [\sum_{k = 1}^{N} \int_{a}^{b} X_{t} X_{s} e_{k} (s) e_{k} (t) d s] \\ = K_{X} (t, t) - \sum_{k = 1}^{N} λ_{k} e_{k} (t)^{2} \end{aligned}

which goes to 0 by Mercer's theorem.

Properties of the Karhunen–Loève transform

Special case: Gaussian distribution

Since the limit in the mean of jointly Gaussian random variables is jointly Gaussian, and jointly Gaussian random (centered) variables are independent if and only if they are orthogonal, we can also conclude:

Theorem. The variables Z_i have a joint Gaussian distribution and are stochastically independent if the original process {X_t}_t is Gaussian.

In the gaussian case, since the variables Z_i are independent, we can say more:

\lim_{N \to \infty} \sum_{i = 1}^{N} e_{i} (t) Z_{i} (ω) = X_{t} (ω)

almost surely.

The Karhunen–Loève transform decorrelates the process

This is a consequence of the independence of the Z_k.

The Karhunen–Loève expansion minimizes the total mean square error

In the introduction, we mentioned that the truncated Karhunen–Loeve expansion was the best approximation of the original process in the sense that it reduces the total mean-square error resulting of its truncation. Because of this property, it is often said that the KL transform optimally compacts the energy.

More specifically, given any orthonormal basis {f_k} of L²([a, b]), we may decompose the process X_t as:

X_{t} (ω) = \sum_{k = 1}^{\infty} A_{k} (ω) f_{k} (t)

where $A_{k} (ω) = \int_{[a, b]} X_{t} (ω) f_{k} (t) d t$

and we may approximate X_t by the finite sum ${\hat{X}}_{t} (ω) = \sum_{k = 1}^{N} A_{k} (ω) f_{k} (t)$ for some integer N.

Claim. Of all such approximations, the KL approximation is the one that minimizes the total mean square error (provided we have arranged the eigenvalues in decreasing order).

[Proof]

Consider the error resulting from the truncation at the N-th term in the following orthonormal expansion:

ϵ_{N} (t) = \sum_{k = N + 1}^{\infty} A_{k} (ω) f_{k} (t)

The mean-square error ε_N²(t) can be written as:

\begin{aligned} ε_{N}^{2} (t) & = E [\sum_{i = N + 1}^{\infty} \sum_{j = N + 1}^{\infty} A_{i} (ω) A_{j} (ω) f_{i} (t) f_{j} (t)] \\ = \sum_{i = N + 1}^{\infty} \sum_{j = N + 1}^{\infty} E [\int_{[a, b]} \int_{[a, b]} X_{t} X_{s} f_{i} (t) f_{j} (s) d s d t] f_{i} (t) f_{j} (t) \\ = \sum_{i = N + 1}^{\infty} \sum_{j = N + 1}^{\infty} f_{i} (t) f_{j} (t) \int_{[a, b]} \int_{[a, b]} K_{X} (s, t) f_{i} (t) f_{j} (s) d s d t \end{aligned}

We then integrate this last equality over [a, b]. The orthonormality of the f_k yields:

\int_{[a, b]} ε_{N}^{2} (t) d t = \sum_{k = N + 1}^{\infty} \int_{[a, b]} \int_{[a, b]} K_{X} (s, t) f_{k} (t) f_{k} (s) d s d t

The problem of minimizing the total mean-square error thus comes down to minimizing the right hand side of this equality subject to the constraint that the f_k be normalized. We hence introduce β_k, the Lagrangian multipliers associated with these constraints, and aim at minimizing the following function:

E r [f_{k} (t), k \in {N + 1, \dots}] = \sum_{k = N + 1}^{\infty} \int_{[a, b]} \int_{[a, b]} K_{X} (s, t) f_{k} (t) f_{k} (s) d s d t - β_{k} (\int_{[a, b]} f_{k} (t) f_{k} (t) d t - 1)

Differentiating with respect to f_i(t) and setting the derivative to 0 yields:

\frac{\partial E r}{\partial f_{i} (t)} = \int_{[a, b]} (\int_{[a, b]} K_{X} (s, t) f_{i} (s) d s - β_{i} f_{i} (t)) d t = 0

which is satisfied in particular when $\int_{[a, b]} K_{X} (s, t) f_{i} (s) d s = β_{i} f_{i} (t)$ , in other words when the f_k are chosen to be the eigenfunctions of T_{K_X}, hence resulting in the KL expansion.

Explained variance

An important observation is that since the random coefficients Z_k of the KL expansion are uncorrelated, the Bienaymé formula asserts that the variance of X_t is simply the sum of the variances of the individual components of the sum:

\begin{aligned} Var [X_{t}] & = \sum_{k = 0}^{\infty} e_{k} (t)^{2} Var [Z_{k}] = \sum_{k = 1}^{\infty} λ_{k} e_{k} (t)^{2} \end{aligned}

Integrating over [a, b] and using the orthonormality of the e_k, we obtain that the total variance of the process is:

\int_{[a, b]} Var [X_{t}] d t = \sum_{k = 1}^{\infty} λ_{k}

In particular, the total variance of the N-truncated approximation is $\sum_{k = 1}^{N} λ_{k}$ . As a result, the N-truncated expansion explains $\sum_{k = 1}^{N} λ_{k} / \sum_{k = 1}^{\infty} λ_{k}$ of the variance; and if we are content with an approximation that explains, say, 95% of the variance, then we just have to determine an $N \in ℕ$ such that $\sum_{k = 1}^{N} λ_{k} / \sum_{k = 1}^{\infty} λ_{k} \geq 0.95$ .

The Karhunen–Loève expansion has the minimum representation entropy property

Template:Expand section

Linear Karhunen-Loeve Approximations

Let us consider a whole class of signals we want to approximate over the first M vectors of a basis. These signals are modeled as realizations of a random vector Y[n] of size N. To optimize the approximation we design a basis that minimizes the average approximation error. This section proves that optimal bases are karhunen-loeve bases that diagonalize the covariance matrix of Y. The random vector Y can be decomposed in an orthogonal basis ${g_{m}}_{0 \leq m \leq N} :$

$Y = \sum_{m = 0}^{N - 1} ⟨ Y, g_{m} ⟩ g_{m},$

where each $⟨ Y, g_{m} ⟩ = \sum_{n = 0}^{N - 1} Y [n] . g_{m}^{*} [n]$

is a random variable. The approximation from the first $M \leq N$ vectors of the basis is

$Y_{M} = \sum_{m = 0}^{M - 1} ⟨ Y, g_{m} ⟩ g_{m}$

The energy conservation in an orthogonal basis implies

$ε [M] = E {{‖ Y - Y_{M} ‖}^{2}} = \sum_{m = M}^{N - 1} E {{| ⟨ Y, g_{m} ⟩ |}^{2}}$

This error is related to the covariance of Y defined by

$R [n, m] = E {Y [n] Y^{*} [m]}$

For any vector x[n] we denote by K the covariance operator represented by this matrix,

$E {{| ⟨ Y, x ⟩ |}^{2}} = ⟨ K x, x ⟩ = \sum_{n = 0}^{N - 1} \sum_{m = 0}^{N - 1} R [n, m] x [n] x^{*} [m]$

The error $ε [M]$ is therefore a sum of the last N-M coefficients of the covariance operator

$ε [M] = \sum_{m = M}^{N - 1} ⟨ K g_{m}, g_{m} ⟩$

The covariance operator K is Hermitian and Positive and is thus diagonalized in an orthogonal basis called a Karhunen-Loeve basis. The following theorem states that a Karhunen-Loeve basis is optimal for linear approximations.

Theorem:(Optimality of Karhunen-Loeve basis) Let K be acovariance operator. For all $M \geq 1$ , the approximation error

$ε [M] = \sum_{m = M}^{N - 1} ⟨ K g_{m}, g_{m} ⟩$

is minimum if and only if ${g_{m}}_{0 \leq m < N}$ is a Karhunen-Loeve basis ordered by decreasing eigenvalues.

$⟨ K g_{m}, g_{m} ⟩ \geq ⟨ K g_{m + 1}, g_{m + 1} ⟩ f o r, 0 \leq m < N - 1$

Non-Linear Approximation in Bases

Linear approximations project the signal on M vectors a priori. The approximation can be made more precise by choosing the M orthogonal vectors depending on the signal properties. This section analyzes the general performance of these non-linear approximations. A signal $f \in H$ is approximated with M vectors selected adaptively in an orthonormal basis $B = {g_{m}}_{m \in ℕ}$ of $H$ . Let $f_{M}$ be the projection of f over M vectors whose indices are in $I_{M}$ :

$f_{M} = \sum_{m \in I_{M}}^{} ⟨ f, g_{m} ⟩ g_{m}$

The approximation error is the sum of the remaining coefficients

$ε [M] = {{‖ f - f_{M} ‖}^{2}} = \sum_{m \notin I_{M}}^{N - 1} {{| ⟨ f, g_{m} ⟩ |}^{2}}$

To minimize this error, the indices in $I_{M}$ must correspond to the M vectors having the largest inner product amplitude $| ⟨ f, g_{m} ⟩ |$ . These are the vectors that best correlate f. They can thus be interpreted as the main features of f. The resulting error is necessarily smaller than the error of a linear approximation which selects the M approximation vectors independently of f. Let us sort ${| ⟨ f, g_{m} ⟩ |}_{m \in ℕ}$ in decreasing order : $| ⟨ f, g_{m_{k}} ⟩ | \geq | ⟨ f, g_{m_{k + 1}} ⟩ |$ . The best non-linear approximation is

$f_{M} = \sum_{k = 1}^{M} ⟨ f, g_{m_{k}} ⟩ g_{m_{k}}$

It can also be written as inner product thresholding:

$f_{M} = \sum_{m = 0}^{\infty} θ_{T} (⟨ f, g_{m} ⟩) g_{m}$

with $T = | ⟨ f, g_{m_{M}} ⟩ |$ and

$θ_{T} (x) = {\begin{aligned} x, i f | x | \geq T \\ 0, i f | x | < T \end{aligned}$

The non-linear error is

$ε [M] = {{‖ f - f_{M} ‖}^{2}} = \sum_{k = M + 1}^{\infty} {{| ⟨ f, g_{m_{k}} ⟩ |}^{2}}$

this error goes quickly to zero as M increases,if the sorted values of $| ⟨ f, g_{m_{k}} ⟩ |$ have a fast decay as k increases. This decay is quantified by computing the $I^{P}$ norm of the signal inner products in B:

${‖ f ‖}_{B, p} = {(\sum_{m = 0}^{\infty} {| ⟨ f, g_{m} ⟩ |}^{p})}^{1 / p}$

The following theorem relates the decay of $ε [M]$ to ${‖ f ‖}_{B, p}$

Theorem:(decay of error) If ${‖ f ‖}_{B, p} < + \infty$ with $p < 2$ then

$ε [M] \leq \frac{{‖ f ‖}_{B, p}^{2}}{\frac{2}{p} - 1} M^{1 - \frac{2}{p}}$

and $ε [M] = o (M^{1 - \frac{2}{p}})$ . Conversely, if $ε [M] = o (M^{1 - \frac{2}{p}})$ then

${‖ f ‖}_{B, q} < + \infty$ for any $q > p$ .

Non-optimality of Karhunen-Loéve Bases

To further illustrate the differences between linear and non -linear approximations, we study the decomposition of a simple non-Gaussian random vector in a Karhunen-Loéve basis. Processes whose realizations have a random translation are stationary. The Karhunen-Loéve basis is then a Fourier basis and we study its performance. To simplify the analysis, consider a random vector Y[n] of size N that is random shift modulo N of a deterministic signal f[n] of zero mean $\sum_{n = 0}^{N - 1} f [n] = 0$ :

$Y [n] = f [(n - p) mod N]$

The random shift P is uniformly distributed on [0,N-1]:

$\Pr (P = p) = \frac{1}{N} f o r, 0 \leq p < N$

Clearly

$E {Y [n]} = \frac{1}{N} \sum_{p = 0}^{N - 1} f [(n - p) mod N] = 0$

and

$R [n, k] = E {Y [n] Y [k]} = \frac{1}{N} \sum_{p = 0}^{N - 1} f [(n - p) mod N] f [(k - p) mod N]$

$= \frac{1}{N} f Θ \bar{f} [n - k] w i t h, \bar{f} [n] = f [- n]$

Hence $R [n, k] = R_{Y} [n - k] w i t h,$

$R_{Y} [k] = \frac{1}{N} f Θ \bar{f} [k]$

Since R_Y is N periodic, Y is a circular stationary random vector. The covariance operator is a circular convolution with R_Y and is therefore diagonalized in the discrete Fourier Karhunen-Loéve basis ${\frac{1}{\sqrt{N}} e^{\frac{i 2 π m n}{N}}}_{0 \leq m < N}$ . The power spectrum is Fourier Transform of R_Y:

$P_{Y} [m] = {\hat{R}}_{Y} [m] = \frac{1}{N} {| \hat{f} [m] |}^{2}$

Example: Consider an extreme case where $f [n] = δ [n] - δ [n - 1]$ A theorem stated above guarantees that the Fourier Karhunen-Loéve basis produces a smaller expected approximation error than a canonical basis of Diracs ${g_{m} [n] = δ [n - m]}_{0 \leq m < N}$ . Indeed we do not know a priori the abscissa of the non-zero coefficients of Y, so there is no particular Dirac that is better adapted to perform the approximation . But the Fourier vectors cover the whole support of Y and thus absorb a part of the signal energy.

$E {{| ⟨ Y [n], \frac{1}{\sqrt{N}} e^{\frac{i 2 π m n}{N}} ⟩ |}^{2}} = P_{Y} [m] = \frac{4}{N} \sin^{2} (\frac{π k}{N})$

Selecting higher frequency Fourier coefficients yields a better mean-square approximation than choosing a priori a few Dirac vectors to perform the approximation. The situation is totally different for non-linear approximations. If $f [n] = δ [n] - δ [n - 1]$ then the discrete Fourier basis is extremely inefficient because because f and hence Y have an energy that is almost uniformly spread among all Fourier vectors. In contrast, since f has only two non-zero coefficients in the Dirac basis, a non-linear approximation of Y with $M \geq 2$ gives zero error. ^[5]

Principal component analysis

Mining Engineer (Excluding Oil ) Truman from Alma, loves to spend time knotting, largest property developers in singapore developers in singapore and stamp collecting. Recently had a family visit to Urnes Stave Church.

We have established the Karhunen–Loève theorem and derived a few properties thereof. We also noted that one hurdle in its application was the numerical cost of determining the eigenvalues and eigenfunctions of its covariance operator through the Fredholm integral equation of the second kind $\int_{[a, b]} K_{X} (s, t) e_{k} (s) d s = λ_{k} e_{k} (t)$ .

However, when applied to a discrete and finite process ${(X_{n})}_{n \in {1, \dots, N}}$ , the problem takes a much simpler form and standard algebra can be used to carry out the calculations.

Note that a continuous process can also be sampled at N points in time in order to reduce the problem to a finite version.

We henceforth consider a random N-dimensional vector $X = {(X_{1} X_{2} \dots X_{N})}^{T}$ . As mentioned above, X could contain N samples of a signal but it can hold many more representations depending on the field of application. For instance it could be the answers to a survey or economic data in an econometrics analysis.

As in the continuous version, we assume that X is centered, otherwise we can let $X : = X - μ_{X}$ (where $μ_{X}$ is the mean vector of X) which is centered.

Let us adapt the procedure to the discrete case.

Covariance matrix

Recall that the main implication and difficulty of the KL transformation is computing the eigenvectors of the linear operator associated to the covariance function, which are given by the solutions to the integral equation written above.

Define Σ, the covariance matrix of X. Σ is an N by N matrix whose elements are given by:

Σ_{i j} = E [X_{i} X_{j}], \forall i, j \in {1, \dots, N}

Rewriting the above integral equation to suit the discrete case, we observe that it turns into:

\begin{aligned} \sum_{i = 1}^{N} Σ_{i j} e_{j} = λ e_{i} \\ \Leftrightarrow & Σ e = λ e \end{aligned}

where $e = (e_{1} e_{2} \dots e_{N})^{T}$ is an N-dimensional vector.

The integral equation thus reduces to a simple matrix eigenvalue problem, which explains why the PCA has such a broad domain of applications.

Since Σ is a positive definite symmetric matrix, it possesses a set of orthonormal eigenvectors forming a basis of $ℝ^{N}$ , and we write ${λ_{i}, ϕ_{i}}_{i \in {1, \dots, N}}$ this set of eigenvalues and corresponding eigenvectors, listed in decreasing values of λ_i. Let also $Φ$ be the orthonormal matrix consisting of these eigenvectors:

\begin{aligned} Φ & : = {(ϕ_{1} ϕ_{2} \dots ϕ_{N})}^{T} \\ Φ^{T} Φ & = I \end{aligned}

Principal component transform

It remains to perform the actual KL transformation, called the principal component transform in this case. Recall that the transform was found by expanding the process with respect to the basis spanned by the eigenvectors of the covariance function. In this case, we hence have:

\begin{aligned} X & = \sum_{i = 1}^{N} ⟨ ϕ_{i}, X ⟩ ϕ_{i} \\ = \sum_{i = 1}^{N} ϕ_{i}^{T} X ϕ_{i} \end{aligned}

In a more compact form, the principal component transform of X is defined by:

{\begin{aligned} Y & = Φ^{T} X \\ X & = Φ Y \end{aligned}

The i-th component of Y is $Y_{i} = ϕ_{i}^{T} X$ , the projection of X on $ϕ_{i}$ and the inverse transform $X = Φ Y$ yields the expansion of $X$ on the space spanned by the $ϕ_{i}$ :

X = \sum_{i = 1}^{N} Y_{i} ϕ_{i} = \sum_{i = 1}^{N} ⟨ ϕ_{i}, X ⟩ ϕ_{i}

As in the continuous case, we may reduce the dimensionality of the problem by truncating the sum at some $K \in {1, \dots, N}$ such that $\frac{\sum_{i = 1}^{K} λ_{i}}{\sum_{i = 1}^{N} λ_{i}} \geq α$ where α is the explained variance threshold we wish to set.

We can also reduce the dimensionality through the use of multilevel dominant eigenvector estimation (MDEE).^[6]

Examples

The Wiener process

There are numerous equivalent characterizations of the Wiener process which is a mathematical formalization of Brownian motion. Here we regard it as the centered standard Gaussian process W_t with covariance function

K_{W} (t, s) = Cov (W_{t}, W_{s}) = \min (s, t) .

We restrict the time domain to [a,b]=[0,1] without loss of generality.

The eigenvectors of the covariance kernel are easily determined. These are

e_{k} (t) = \sqrt{2} \sin ((k - \frac{1}{2}) π t)

and the corresponding eigenvalues are

λ_{k} = \frac{1}{(k - \frac{1}{2})^{2} π^{2}} .

[Proof]

In order to find the eigenvalues and eigenvectors, we need to solve the integral equation:

\begin{aligned} \int_{[a, b]} K_{W} (s, t) e (s) d s & = λ e (t) \forall t, 0 \leq t \leq 1 \\ \int_{0}^{1} \min (s, t) e (s) d s & = λ e (t) \forall t, 0 \leq t \leq 1 \\ \int_{0}^{t} s e (s) d s + t \int_{t}^{1} e (s) d s & = λ e (t) \forall t, 0 \leq t \leq 1 \end{aligned}

differentiating once with respect to t yields:

\int_{t}^{1} e (s) d s = λ e^{'} (t)

a second differentiation produces the following differential equation:

- e (t) = λ e^{″} (t)

The general solution of which has the form:

e (t) = A \sin (\frac{t}{\sqrt{λ}}) + B \cos (\frac{t}{\sqrt{λ}})

where A and B are two constants to be determined with the boundary conditions. Setting t=0 in the initial integral equation gives e(0)=0 which implies that B=0 and similarly, setting t=1 in the first differentiation yields e' (1)=0, whence:

\cos (\frac{1}{\sqrt{λ}}) = 0

which in turn implies that eigenvalues of T_{K_X} are:

λ_{k} = {(\frac{1}{(k - \frac{1}{2}) π})}^{2}, k \geq 1

The corresponding eigenfunctions are thus of the form:

e_{k} (t) = A \sin ((k - \frac{1}{2}) π t), k \geq 1

A is then chosen so as to normalize e_k:

\int_{0}^{1} e_{k}^{2} (t) d t = 1 ⟹ A = \sqrt{2}

This gives the following representation of the Wiener process:

Theorem. There is a sequence {Z_i}_i of independent Gaussian random variables with mean zero and variance 1 such that

W_{t} = \sqrt{2} \sum_{k = 1}^{\infty} Z_{k} \frac{\sin ((k - \frac{1}{2}) π t)}{(k - \frac{1}{2}) π} .

Note that this representation is only valid for $t \in [0, 1] .$ On larger intervals, the increments are not independent. As stated in the theorem, convergence is in the L² norm and uniform in t.

The Brownian bridge

Similarly the Brownian bridge $B_{t} = W_{t} - t W_{1}$ which is a stochastic process with covariance function

K_{B} (t, s) = \min (t, s) - t s

can be represented as the series

B_{t} = \sum_{k = 1}^{\infty} Z_{k} \frac{\sqrt{2} \sin (k π t)}{k π}

Applications

Template:Expand section Adaptive optics systems sometimes use K–L functions to reconstruct wave-front phase information (Dai 1996, JOSA A). Karhunen–Loève expansion is closely related to the Singular Value Decomposition. The latter has myriad applications in image processing, radar, seismology, and the like. If one has independent vector observations from a vector valued stochastic process then the left singular vectors are maximum likelihood estimates of the ensemble KL expansion.

Applications in signal estimation and detection

Detection of a known continuous signal S(t)

In communication, we usually have to decide whether a signal from a noisy channel contains valuable information. The following hypothesis testing is used for detecting continuous signal s(t) from channel output X(t), N(t) is the channel noise, which is usually assumed zero mean gaussian process with correlation function $R_{N} (t, s) = E [N (t) N (s)]$

H : X (t) = N (t)

,

K : X (t) = N (t) + s (t), t \in (0, T)

.

Signal detection in white noise

When the channel noise is white, its correlation function is

R_{N} (t) = \frac{N_{0}}{2} δ (t)

,

and it has constant power spectrum density. In physically practical channel, the noise power is finite, so:

S_{N} (f) = \frac{N_{0}}{2} for | f | < w, 0 for | f | > w

.

Then the noise correlation function is sinc function with zeros at $\frac{n}{2 ω}, n = . . . - 1, 0, 1, . . .$ . Since are uncorrelated and gaussian, they are independent. Thus we can take samples from X(t) with time spacing

Δ t = \frac{n}{2 ω}

within (0,T).

Let $X_{i} = X (i Δ t)$ . We have a total of $n = \frac{T}{Δ t} = T (2 ω) = 2 ω T$ i.i.d samples ${X_{1}, X_{2}, . . ., X_{n}}$ to develop the likelihood-ratio test. Define signal $S_{i} = S (i Δ t)$ , the problem becomes,

H : X_{i} = N_{i}

,

K : X_{i} = N_{i} + S_{i}, i = 1, 2 . . . n .

The log-likelihood ratio

ℒ (\underline{x}) = \log \frac{\sum_{i = 1}^{n} (2 S_{i} x_{i} - S_{i}^{2})}{2 σ^{2}} \Leftrightarrow Δ t Σ_{i = 1}^{n} S_{i} x_{i} = \sum_{i = 1}^{n} S (i Δ t) x (i Δ t) Δ t ≷ λ_{2}

.

As $t \to 0, let G = \int_{0}^{T} S (t) x (t) d t$ .

Then G is the test statistics and the Neyman–Pearson optimum detector is: $G (\underline{x}) > G_{0} \Rightarrow K, < G_{0} \Rightarrow H$ . As G is gaussian, we can characterize it by finding its mean and variances. Then we get

H : G \sim N (0, \frac{N_{0} E}{2})

K : G \sim N (E, \frac{N_{0} E}{2})

,

where $E = \int_{0}^{T} S^{2} (t) d t$ is the signal energy.

The false alarm error

α = \int_{G_{0}}^{\infty} N (0, \frac{N_{0} E}{2}) d G \Rightarrow G_{0} = \sqrt{\frac{N_{0} E}{2}} Φ^{- 1} (1 - α)

And the probability of detection:

β = \int_{G_{0}}^{\infty} N (E, \frac{N_{0} E}{2}) d G = 1 - Φ (\frac{G_{0} - E}{\sqrt{\frac{N_{0} E}{2}}}) = Φ [\sqrt{\frac{2 E}{N_{0}}} - Φ^{- 1} (1 - α)], Φ (\cdot)

is the cdf of standard normal gaussian variable.

Signal detection in colored noise

When N(t) is colored (correlated in time) gaussian noise with zero mean and covariance function $R_{N} (t, s) = E [X (t) X (s)],$ we cannot sample independent discrete observations by evenly spacing the time. Instead, we can use K–L expansion to uncorrelate the noise process and get independent gaussian observation 'samples'. The K–L expansion of N(t):

N (t) = \sum_{i = 1}^{\infty} N_{i} Φ_{i} (t), 0 < t < T

,

where $N_{i} = \int N (t) Φ_{i} (t) d t$ and the orthonormal bases ${Φ_{i} t}$ are generated by kernal $R_{N} (t, s)$ , i.e., solution to

\int_{0}^{T} R_{N} (t, s) Φ_{i} (s) d s = λ_{i} Φ_{i} (t), v a r [N_{i}] = λ_{i}

.

Do the expansion:

S (t) = \sum_{i = 1}^{\infty} S_{i} Φ_{i} (t)

,

where $S_{i} = \int_{0}^{T} S (t) Φ_{i} (t) d t, 0 < t < T .$ , then

X_{i} = \int_{0}^{T} X (t) Φ_{i} (t) d t = N_{i}

under H and $N_{i} + S_{i}$ under K. Let $\overline{X} = {X_{1}, X_{2}, \dots}$ , we have

N_{i}

are independent gaussian r.v's with variance

λ_{i}

under H:

{X_{i}}

are independent gaussian r.v's.

f_{H} [x (t) | 0 < t < T] = f_{H} (\underline{x}) = \prod_{i = 1}^{\infty} \frac{1}{\sqrt{2 π λ_{i}}} e x p [- \frac{x_{i}^{2}}{2 λ_{i}}]

under K:

{X_{i} - S_{i}}

are independent gaussian r.v's.

f_{K} [x (t) | 0 < t < T] = f_{K} (\underline{x}) = \prod_{i = 1}^{\infty} \frac{1}{\sqrt{2 π λ_{i}}} e x p [- \frac{(x_{i} - S_{i})^{2}}{2 λ_{i}}]

Hence, the log-LR is given by

ℒ (\underline{x}) = \sum_{i = 1}^{\infty} \frac{2 S_{i} x_{i} - S_{i}^{2}}{2 λ_{i}}

and the optimum detector is

G = \sum_{i = 1}^{\infty} S_{i} x_{i} λ_{i} > G_{0} \Rightarrow K, < G_{0} \Rightarrow H .

Define

k (t) = \sum_{i = 1}^{\infty} λ_{i} S_{i} Φ_{i} (t), 0 < t < T,

then $G = \int_{0}^{T} k (t) x (t) d t$ .

How to find k(t)

Since

\int_{0}^{T} R_{N} (t, s) k (s) d s = \sum_{i = 1}^{\infty} λ_{i} S_{i} \int_{0}^{T} R_{N} (t, s) Φ_{i} (s) d s = \sum_{i = 1}^{\infty} S_{i} Φ_{i} (t) = S (t)

,

k(t) is the solution to

\int_{0}^{T} R_{N} (t, s) k (s) d s = S (t)

.

If N(t)is wide-sense stationary,

\int_{0}^{T} R_{N} (t - s) k (s) d s = S (t)

,

which is known as the Wiener–Hopf equation. The equation can be solved by taking fourier transform, but not practically realizable since infinite spectrum needs spatial factorization. A special case which is easy to calculate k(t) is white gaussian noise.

\int_{0}^{T} \frac{N_{0}}{2} δ (t - s) k (s) d s = S (t) \Rightarrow k (t) = C S (t), 0 < t < T

.

The corresponding impulse response is h(t) = k(T-t) = C S(T-t). Let C = 1, this is just the result we arrived at in previous section for detecting of signal in white noise.

Test threshold for Neyman–Pearson detector

Since X(t)is gaussian process, $G = \int_{0}^{T} k (t) x (t) d t$ is a gaussian random variable that can be characterized by its mean and variance.

E [G | H] = \int_{0}^{T} k (t) E [x (t) | H] d t = 0

E [G | K] = \int_{0}^{T} k (t) E [x (t) | K] d t = \int_{0}^{T} k (t) S (t) d t \equiv ρ

E [G^{2} | H] = \int_{0}^{T} \int_{0}^{T} k (t) k (s) R_{N} (t, s) d t d s = \int_{0}^{T} k (t) (\int_{0}^{T} k (s) R_{N} (t, s) d s) = \int_{0}^{T} k (t) S (t) d t = ρ

v a r [G | H] = E [G^{2} | H] - (E [G | H])^{2} = ρ

E [G^{2} | K] = \int_{0}^{T} \int_{0}^{T} k (t) k (s) E [x (t) x (s)] d t d s = \int_{0}^{T} \int_{0}^{T} k (t) k (s) (R_{N} (t, s) + S (t) S (s)) d t d s = ρ + ρ^{2}

v a r [G | K] = E [G^{2} | K] - (E [G | K])^{2} = ρ + ρ^{2} - ρ^{2} = ρ .

Hence, we obtain the distributions of H and K:

H : G \sim N (0, ρ)

K : G \sim N (ρ, ρ)

The false alarm error is

α = \int_{G_{0}}^{\infty} N (0, ρ) d G = 1 - Φ (\frac{G_{0}}{\sqrt{ρ}}) .

So the test threshold for the Neyman–Pearson optimum detector is

G_{0} = \sqrt{ρ} Φ^{- 1} (1 - α)

.

Its power of detection is

β = \int_{G_{0}}^{\infty} N (ρ, ρ) d G = Φ [\sqrt{ρ} - Φ^{- 1} (1 - α)]

.

When the noise is white gaussian process, the signal power is

ρ = \int_{0}^{T} k (t) S (t) d t = \int_{0}^{T} S (t)^{2} d t = E

.

Prewhitening

For some type of colored noise, a typical practise is to add a prewhiterning filter before the matched filter to transform the colored noise into white noise. For example, N(t) is a wide-sense stationary colored noise with correlation function

ℝ_{N} (τ) = \frac{B N_{0}}{4} e^{- B | τ |}

and

S_{N} (f) = \frac{N_{0}}{2 (1 + (\frac{w}{B})^{2})}

.

The transfer function of prewhitening filter is $H (f) = 1 + j \frac{w}{B}$ .

Detection of a gaussian random signal in AWGN

When the signal we want to detect from the noisy channel is also random, for example, a white gaussian process X(t), we can still implement K–L expansion to get independent sequence of observation. In this case, the detection problem is described as follows:

H_{0} : Y (t) = N (t)

H_{1} : Y (t) = N (t) + X (t), 0 < t < T .

X(t) is a random process with correlation function $R_{X} (t, s) = E {X [t] X [s]}$

The K–L expansion of X(t) is

X (t) = \sum_{i = 1}^{\infty} X_{i} Φ_{i} (t)

,

where

X_{i} = \int_{0}^{T} X (t) Φ_{i} (t) . Φ (t)

are solutions to

\int_{0}^{T} R_{X} (t, s) Φ_{i} (s) d s = λ_{i} Φ_{i} (t)

.

So $X_{i}$ 's are independent sequence of r.v's with zero mean and variance $λ_{i}$ . Expanding Y(t) and N(t) by $Φ_{i} (t)$ , we get

Y_{i} = \int_{0}^{T} Y (t) Φ_{i} (t) d t = \int_{0}^{T} [N (t) + X (t)] Φ_{i} (t) = N_{i} + X_{i}

,

where $N_{i} = \int_{0}^{T} N (t) Φ_{i} (t) d t .$

As N(t) is gaussian white noise, $N_{i}$ 's are i.i.d sequence of r.v with zero mean and variance $\frac{N_{0}}{2}$ , then the problem is simplified as follows,

H_{0} : Y_{i} = N_{i}

H_{1} : Y_{i} = N_{i} + X_{i}

The Neyman–Pearson optimal test:

Λ = \frac{f_{Y} | H_{1}}{f_{Y} | H_{0}} = C e^{- \sum_{i = 1}^{\infty} \frac{y_{i}^{2}}{2} \frac{λ_{i}}{\frac{N_{0}}{2} (\frac{N_{0}}{2} + λ_{i})}}

,

so the log-likelihood ratio

ℒ = l n (Λ) = K - \sum_{i = 1}^{\infty} \frac{y_{i}^{2}}{2} \frac{λ_{i}}{\frac{N_{0}}{2} (\frac{N_{0}}{2} + λ_{i})}

.

Since

\hat{X_{i}} = \frac{λ_{i}}{\frac{N_{0}}{2} (\frac{N_{0}}{2} + λ_{i})}

is just the minimum-mean-square estimate of $X_{i}$ given $Y_{i}$ 's,

ℒ = K + \frac{1}{N_{0}} \sum_{i = 1}^{\infty} Y_{i} \hat{X_{i}}

.

K–L expansion has the following property: If

f (t) = \sum f_{i} Φ_{i} (t), g (t) = \sum g_{i} Φ_{i} (t)

,

where

f_{i} = \int_{0}^{T} f (t) Φ_{i} (t), g_{i} = \int_{0}^{T} g (t) Φ_{i} (t) .

,

then

\sum_{i = 1}^{\infty} f_{i} g_{i} = \int_{0}^{T} g (t) f (t) d t

.

So let

\hat{X (t | T)} = \sum_{i = 1}^{\infty} \hat{X_{i}} Φ_{i} (t)

,

ℒ = K + \frac{1}{N_{0}} \int_{0}^{T} Y (t) \hat{X (t | T)} d t

.

Noncausal filter Q(t, s) can be used to get the estimate through

\hat{X (t | T)} = \int_{0}^{T} Q (t, s) Y (s) d s

.

By orthogonality principle, Q(t,s) satisfies

\int_{0}^{T} Q (t, s) R_{X} (s, t) d s + \frac{N_{0}}{2} Q (t, λ) = R_{X} (t, λ), 0 < λ < T, 0 < t < T .

.

However for practical reason, it's necessary to further derive the causal filter h(t, s), where h(t, s) = 0 for s > t, to get estimate $\hat{X (t | t)}$ . Specifically,

Q (t, s) = h (t, s) + h (s, t) - \int_{0}^{T} h (λ, t) h (s, λ) d λ

.

Notes

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

References

20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.

My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.

My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.

My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.

My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting

In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang

Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules

Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.

A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running

The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more

There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang
20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.

My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting

In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang

Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules

Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.

A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running

The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more

There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang
Wu B., Zhu J., Najm F.(2005) "A Non-parametric Approach for Dynamic Range Estimation of Nonlinear Systems". In Proceedings of Design Automation Conference(841-844) 2005
Wu B., Zhu J., Najm F.(2006) "Dynamic Range Estimation". IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 25 Issue:9 (1618-1636) 2006
Template:Cite arXiv
One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting

In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang

Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules

Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.

A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running

The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more

There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang
Template:Cite arXiv
One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting

In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang

Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules

Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.

A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running

The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more

There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang

External links

Mathematica KarhunenLoeveDecomposition function.
E161: Computer Image Processing and Analysis notes by Pr. Ruye Wang at Harvey Mudd College [1]

fr:Transformée de Karhunen-Loève ru:Теорема Кархунена-Лоэва sv:Karhunen-Loeve-transformen

↑ Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010
↑ Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010
↑ Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010
↑ Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010.
↑ A wavelet tour of signal processing-Stéphane Mallat
↑ X. Tang, “Texture information in run-length matrices,” IEEE Transactions on Image Processing, vol. 7, No. 11, pp. 1602- 1609, Nov. 1998

[sapatnekar-1] Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010

[ghoman-2] Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010

[Raju-3] Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010

[Kosambi-4] Many property agents need to declare for the PIC grant in Singapore. However, not all of them know find out how to do the correct process for getting this PIC scheme from the IRAS. There are a number of steps that you need to do before your software can be approved.

Naturally, you will have to pay a safety deposit and that is usually one month rent for annually of the settlement. That is the place your good religion deposit will likely be taken into account and will kind part or all of your security deposit. Anticipate to have a proportionate amount deducted out of your deposit if something is discovered to be damaged if you move out. It's best to you'll want to test the inventory drawn up by the owner, which can detail all objects in the property and their condition. If you happen to fail to notice any harm not already mentioned within the inventory before transferring in, you danger having to pay for it yourself.

In case you are in search of an actual estate or Singapore property agent on-line, you simply should belief your intuition. It's because you do not know which agent is nice and which agent will not be. Carry out research on several brokers by looking out the internet. As soon as if you end up positive that a selected agent is dependable and reliable, you can choose to utilize his partnerise in finding you a home in Singapore. Most of the time, a property agent is taken into account to be good if he or she locations the contact data on his website. This may mean that the agent does not mind you calling them and asking them any questions relating to new properties in singapore in Singapore. After chatting with them you too can see them in their office after taking an appointment.

Have handed an trade examination i.e Widespread Examination for House Brokers (CEHA) or Actual Property Agency (REA) examination, or equal; Exclusive brokers are extra keen to share listing information thus making certain the widest doable coverage inside the real estate community via Multiple Listings and Networking. Accepting a severe provide is simpler since your agent is totally conscious of all advertising activity related with your property. This reduces your having to check with a number of agents for some other offers. Price control is easily achieved. Paint work in good restore-discuss with your Property Marketing consultant if main works are still to be done. Softening in residential property prices proceed, led by 2.8 per cent decline within the index for Remainder of Central Region

Once you place down the one per cent choice price to carry down a non-public property, it's important to accept its situation as it is whenever you move in – faulty air-con, choked rest room and all. Get round this by asking your agent to incorporate a ultimate inspection clause within the possibility-to-buy letter. HDB flat patrons routinely take pleasure in this security net. "There's a ultimate inspection of the property two days before the completion of all HDB transactions. If the air-con is defective, you can request the seller to repair it," says Kelvin.

15.6.1 As the agent is an intermediary, generally, as soon as the principal and third party are introduced right into a contractual relationship, the agent drops out of the image, subject to any problems with remuneration or indemnification that he could have against the principal, and extra exceptionally, against the third occasion. Generally, agents are entitled to be indemnified for all liabilities reasonably incurred within the execution of the brokers´ authority.

To achieve the very best outcomes, you must be always updated on market situations, including past transaction information and reliable projections. You could review and examine comparable homes that are currently available in the market, especially these which have been sold or not bought up to now six months. You'll be able to see a pattern of such report by clicking here It's essential to defend yourself in opposition to unscrupulous patrons. They are often very skilled in using highly unethical and manipulative techniques to try and lure you into a lure. That you must also protect your self, your loved ones, and personal belongings as you'll be serving many strangers in your home. Sign a listing itemizing of all of the objects provided by the proprietor, together with their situation. HSR Prime Recruiter 2010.

[5] A wavelet tour of signal processing-Stéphane Mallat

[6] X. Tang, “Texture information in run-length matrices,” IEEE Transactions on Image Processing, vol. 7, No. 11, pp. 1602- 1609, Nov. 1998

[1]

[2]

[3]

[4]

[5]

[6]

Anisometropia: Difference between revisions

Revision as of 08:18, 20 July 2013

Contents

Formulation

Statement of the theorem

Proof

Properties of the Karhunen–Loève transform

Special case: Gaussian distribution

The Karhunen–Loève transform decorrelates the process

The Karhunen–Loève expansion minimizes the total mean square error

Explained variance

The Karhunen–Loève expansion has the minimum representation entropy property

Linear Karhunen-Loeve Approximations

Non-Linear Approximation in Bases

Non-optimality of Karhunen-Loéve Bases

Principal component analysis

Covariance matrix

Principal component transform

Examples

The Wiener process

The Brownian bridge

Applications

Applications in signal estimation and detection

Detection of a known continuous signal S(t)

Signal detection in white noise

Signal detection in colored noise

How to find k(t)

Test threshold for Neyman–Pearson detector

Prewhitening

Detection of a gaussian random signal in AWGN

See also

Notes

References

External links

Navigation menu

@@ Line 1: / Line 1: @@
-Invest in a rechargeable battery in your wireless gaming controller. You can buy re-chargeable power supplies for your controller. If you prefer to play video games regularly, you will be overeating through a small wealth in the batteries used to run your controllers. A rechargeable battery can help you save a lot of funds in the long run.<br><br>If you have got to reload one specific arms when playing conflict of clans hack this is certainly shooting entailed, always get cover first. This process is common for players to be gunned affordable while a reload might be happening, and you take a look at helplessly. Do Not even let it happen for! Find somewhere to assist you conceal before you start up to reload.<br><br>A great [http://circuspartypanama.com clash of clans hack] equipment and thus tricks could be exceedingly handy in the position that you are an [http://Search.Huffingtonpost.com/search?q=enthusiastic+player&s_it=header_form_v1 enthusiastic player] of the diversion. The finest hack apparatuses tackle just regarding any doing work framework - Android and Apple iOS. You possibly can exploit the obtain that can be used on your apple i phone or Android telephones.<br><br>Explore the evaluations and see the trailers before buying another video game. Ensure it is one thing you are looking for before you get it. These video games aren't low-cost, and also you will get nearly as a long way cash whenever you  inside a employed video game which you have only utilized several times.<br><br>Few some online games provde the comfort of adding a true-entire world timepiece accessible in the playstation game itself. Is actually a usually a downside in full-monitor game titles. You don't want these using up even a good deal more of your time and even energy than within some budget place a instant clock of your in close proximity to to your display computer monitor to be able to help you monitor just how big you've been enjoying.<br><br>A meaningful tutorial will guide you thru your first few raids, constructions, and upgrades, yet unfortunately youre left to your wiles pretty quickly. Your buildings take real time to construct and upgrade, your army units take time to recruit, and your supplier buildings take time to create food and gold. Like all of his or her genre cousins, Throne Hasten is meant to played in multiple short bursts in the daytlight. This type of obsessive gaming definitely works more complete on mobile devices that always with you that will send push notifications  when timed tasks are finalized. Then again, the success of a lot of hit Facebook games over the years indicates that people consult Facebook often enough different short play sessions accomplish the task there too.<br><br>To assist you to conclude, clash of clans hack tool no piece of research must not be encouraged to get in means of the bigger question: what makes we in this? Putting this aside the of great importance. It replenishes the self, provides financial security  always chips in.
+{{technical|date=June 2012}}
+In the theory of [[stochastic process]]es, the '''Karhunen–Loève theorem'''  (named after [[Kari Karhunen]] and [[Michel Loève]]), also known as the '''Kosambi–Karhunen–Loève theorem'''<ref name="sapatnekar">{{Citation |last=Sapatnekar |first=Sachin |title= Overcoming variations in nanometer-scale technologies|journal= IEEE Journal on Emerging and Selected Topics in Circuits and Systems|volume= 1|year= 2011 |issue= 1|pages= 5–18}}</ref><ref name="ghoman">{{Citation |last=Ghoman |first=Satyajit |last2= Wang|first2= Zhicun|last3=Chen |first3=PC |last4=Kapania|first4=Rakesh|title= A POD-based Reduced Order Design Scheme for Shape Optimization of Air Vehicles|booktitle=Proc of 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA-2012-1808, Honolulu, Hawaii |year=2012 }}</ref> is a representation of a stochastic process as an infinite linear combination of [[orthogonal function]]s, analogous to a [[Fourier series]] representation of a function on a bounded interval. Stochastic processes given by infinite series of this form were first<ref name="Raju">{{Citation |first=C.K. |last=Raju |title=Kosambi the Mathematician |journal=Economic and Political Weekly |volume=44 |year=2009 |issue=20 |pages=33–45 }}</ref> considered by [[Damodar Dharmananda Kosambi]].<ref name="Kosambi">{{Citation |first=D. D. |last=Kosambi |title=Statistics in Function Space |journal=Journal of the Indian Mathematical Society |volume=7 |year=1943 |issue= |pages=76–88 |id={{MathSciNet|9816}} }}.</ref>  There exist many such expansions of a stochastic process: if the process is indexed over [''a'', ''b''], any [[orthonormal basis]] of ''L''<sup>2</sup>([''a'', ''b'']) yields an expansion thereof in that form. The importance of the Karhunen–Loève theorem is that it yields the best such basis in the sense that it minimizes the total [[mean squared error]].
+In contrast to a Fourier series where the coefficients are real numbers and the expansion basis consists of [[trigonometric function|sinusoidal functions]] (that is, [[sine]] and [[cosine]] functions), the coefficients in the Karhunen–Loève theorem are [[random variable]]s and the expansion basis depends on the process. In fact, the orthogonal basis functions used in this representation are determined by the [[covariance function]] of the process. One can think that the Karhunen–Loève transform adapts to the process in order to produce the best possible basis for its expansion.
+In the case of a ''centered'' stochastic process {''X''<sub>''t''</sub>}<sub>''t'' ∈ [''a'', ''b'']</sub>  (where ''centered'' means that the expectations E(''X''<sub>''t''</sub>) are defined and equal to 0 for all values of the parameter ''t'' in [''a'', ''b'']) satisfying a technical continuity condition, ''X''<sub>''t''</sub> admits a decomposition
+:<math> X_t = \sum_{k=1}^\infty Z_k e_k(t)</math>
+where ''Z''<sub>''k''</sup> are pairwise [[uncorrelated]] random variables and the functions ''e''<sub>''k''</sub> are continuous real-valued functions on [''a'', ''b''] that are pairwise [[orthogonal]] in ''L''<sup>2</sup>[''a'', ''b'']. It is therefore sometimes said that the expansion is ''bi-orthogonal'' since the random coefficients ''Z''<sub>''k''</sup> are orthogonal in the probability space while the deterministic functions ''e''<sub>''k''</sub> are orthogonal in the time domain. The general case of a process ''X''<sub>''t''</sub> that is not centered can be brought back to the case of a centered process by considering (''X''<sub>''t''</sub> &minus; E(''X''<sub>''t''</sub>)) which is a centered process.
+Moreover,  if the process is [[Gaussian process|Gaussian]], then the random variables ''Z''<sub>''k''</sub> are Gaussian and [[stochastically independent]].  This result generalizes the ''Karhunen–Loève transform''.  An important example of a centered real stochastic process on [0,1] is the [[Wiener process]]; the Karhunen–Loève theorem can be used to provide a canonical orthogonal representation for it. In this case the expansion consists of sinusoidal functions.
+The above expansion into uncorrelated random variables is also known as the ''Karhunen–Loève expansion'' or ''Karhunen–Loève decomposition''. The [[statistic|empirical]] version (i.e., with the coefficients computed from a sample) is known as the ''Karhunen–Loève transform'' (KLT), ''[[principal component analysis]]'', ''proper orthogonal decomposition (POD)'', ''[[Empirical orthogonal functions]]'' (a term used in [[meteorology]] and [[geophysics]]), or the ''[[Harold Hotelling|Hotelling]] transform''.
+== Formulation ==
+*Throughout this article, we will consider a square integrable zero-mean random process ''X''<sub>''t''</sub> defined over a probability space (&Omega;,''F'','''P''') and indexed over a closed interval [''a'',&nbsp;''b''], with covariance function ''K''<sub>''X''</sub>(''s,t''). We thus have:
+::<math>\forall t\in [a,b], X_t\in L^2(\Omega,\mathcal{F},\mathrm{P}),</math>
+::<math>\forall t\in [a,b], \mathrm{E}[X_t]=0,</math>
+::<math>\forall t,s \in [a,b], K_X(s,t)=\mathrm{E}[X_s X_t].</math>
+*We associate to ''K''<sub>''X''</sub> a [[linear operator]] ''T''<sub>''K''<sub>''X''</sub></sub> defined in the following way:
+:<math>
+\begin{array}{rrl}
+T_{K_X}: L^2([a,b]) &\rightarrow & L^2([a,b])\\
+f(t) & \mapsto & \int_{[a,b]} K_X(s,t) f(s) ds
+\end{array}
+</math><br>Since ''T''<sub>''K''<sub>''X''</sub></sub> is a linear operator, it makes sense to talk about its eigenvalues &lambda;<sub>''k''</sub> and eigenfunctions ''e''<sub>''k''</sub>, which are found solving the homogeneous Fredholm [[integral equation]] of the second kind
+:<math>\int_{[a,b]} K_X(s,t) e_k(s)\,ds=\lambda_k e_k(t)</math>
+== Statement of the theorem ==
+'''Theorem'''.  Let ''X''<sub>''t''</sub> be a zero-mean square integrable stochastic process defined over a probability space (&Omega;,''F'','''P''') and indexed over a closed and bounded interval [''a'',&nbsp;''b''], with continuous covariance function ''K''<sub>''X''</sub>(''s,t'').
+Then ''K''<sub>''X''</sub>(''s,t'') is a [[Mercer's theorem|Mercer kernel]] and letting ''e''<sub>''k''</sub> be an orthonormal basis of ''L''<sup>2</sup>([''a'',&nbsp;''b'']) formed by the eigenfunctions of ''T''<sub>''K''<sub>''X''</sub></sub> with respective eigenvalues &lambda;<sub>''k''</sub>, ''X''<sub>''t''</sub> admits the following representation
+:<math>
+X_t=\sum_{k=1}^\infty Z_k e_k(t)
+</math>
+where the convergence is in [[Convergence of random variables#Convergence in mean|''L''<sup>2</sup>]], uniform in ''t'' and
+:<math>
+Z_k=\int_{[a,b]} X_t e_k(t)\, dt
+</math>
+Furthermore, the random variables ''Z''<sub>''k''</sub> have zero-mean, are uncorrelated and have variance &lambda;<sub>''k''</sub>
+:<math>
+\mathrm{E}[Z_k]=0,~\forall k\in\mathbb{N} \quad\quad\mbox{and}\quad\quad \mathrm{E}[Z_i Z_j]=\delta_{ij} \lambda_j,~\forall i,j\in \mathbb{N}
+</math>
+Note that by generalizations of Mercer's theorem we can replace the interval [''a'', ''b''] with other compact spaces ''C'' and the Lebesgue measure on [''a'', ''b''] with a Borel measure whose support is ''C''.
+==Proof==
+*The covariance function ''K''<sub>''X''</sub> satisfies the definition of a Mercer kernel. By [[Mercer's theorem]], there consequently exists a set {&lambda;<sub>''k''</sub>,''e''<sub>''k''</sub>(''t'')} of eigenvalues and eigenfunctions of T<sub>''K''<sub>''X''</sub></sub> forming an orthonormal basis of ''L''<sup>2</sup>([''a'',''b'']), and ''K''<sub>''X''</sub> can be expressed as
+:<math>K_X(s,t)=\sum_{k=1}^\infty \lambda_k e_k(s) e_k(t) </math>
+*The process ''X''<sub>''t''</sub> can be expanded in terms of the eigenfunctions ''e''<sub>''k''</sub> as:
+:<math>X_t=\sum_{k=1}^\infty Z_k e_k(t)</math><br> where the coefficients (random variables) ''Z''<sub>''k''</sub> are given by the projection of ''X''<sub>''t''</sub> on the respective eigenfunctions
+:<math>Z_k=\int_{[a,b]} X_t e_k(t) \,dt</math>
+*We may then derive
+:<math>\mathrm{E}[Z_k]=\mathrm{E}\left[\int_{[a,b]} X_t e_k(t) \,dt\right]=\int_{[a,b]} \mathrm{E}[X_t] e_k(t) dt=0</math><br>and:
+:<math>
+\begin{array}[t]{rl}
+\mathrm{E}[Z_i Z_j]&=\mathrm{E}\left[ \int_{[a,b]}\int_{[a,b]} X_t  X_s e_j(t)e_i(s) dt\, ds\right]\\
+&=\int_{[a,b]}\int_{[a,b]}  \mathrm{E}\left[X_t  X_s\right] e_j(t)e_i(s) dt\, ds\\
+&=\int_{[a,b]}\int_{[a,b]}  K_X(s,t) e_j(t)e_i(s) dt \, ds\\
+&=\int_{[a,b]} e_i(s)\left(\int_{[a,b]}  K_X(s,t) e_j(t) dt\right) ds\\
+&=\lambda_j \int_{[a,b]} e_i(s) e_j(s)  ds\\
+&=\delta_{ij}\lambda_j
+\end{array}
+</math><br>where we have used the fact that the ''e''<sub>''k''</sub> are eigenfunctions of ''T''<sub>''K''<sub>''X''</sub></sub> and are orthonormal.
+*Let us now show that the convergence is in ''L''<sup>2</sup>:<br>let <math>S_N=\sum_{k=1}^N Z_k e_k(t)</math>.
+:<math>
+\begin{align}
+\mathrm{E}[|X_t-S_N|^2]&=\mathrm{E}[X_t^2]+\mathrm{E}[S_N^2]-2\mathrm{E}[X_t S_N]\\
+&=K_X(t,t)+\mathrm{E}\left[\sum_{k=1}^N \sum_{l=1}^N Z_k Z_l e_k(t)e_l(t) \right] -2\mathrm{E}\left[X_t\sum_{k=1}^N Z_k e_k(t)\right]\\
+&=K_X(t,t)+\sum_{k=1}^N \lambda_k e_k(t)^2 -2\mathrm{E}\left[\sum_{k=1}^N \int_a^b X_t X_s e_k(s) e_k(t) ds\right]\\
+&=K_X(t,t)-\sum_{k=1}^N \lambda_k e_k(t)^2
+\end{align}
+</math><br>which goes to 0 by Mercer's theorem.
+== Properties of the Karhunen–Loève transform ==
+=== Special case: Gaussian distribution ===
+Since the limit in the mean of jointly Gaussian random variables is jointly Gaussian, and jointly Gaussian random (centered) variables are independent if and only if they are orthogonal, we can also conclude:
+'''Theorem'''.  The variables '''Z'''<sub>''i''</sub> have a joint Gaussian distribution and are stochastically independent if the original process {'''X'''<sub>''t''</sub>}<sub>''t''</sub> is Gaussian.
+In the gaussian case, since the variables '''Z'''<sub>''i''</sub> are independent, we can say more:
+:<math> \lim_{N \rightarrow \infty} \sum_{i=1}^N e_i(t) Z_i(\omega) = X_t(\omega) </math>
+almost surely.
+=== The Karhunen–Loève transform decorrelates the process ===
+This is a consequence of the independence of the ''Z''<sub>''k''</sub>.
+=== The Karhunen–Loève expansion minimizes the total mean square error ===
+In the introduction, we mentioned that the truncated Karhunen–Loeve expansion was the best approximation of the original process in the sense that it reduces the total mean-square error resulting of its truncation. Because of this property, it is often said that the KL transform optimally compacts the energy.
+More specifically, given any orthonormal basis {''f''<sub>''k''</sub>} of ''L''<sup>2</sup>([''a'', ''b'']), we may decompose the process ''X''<sub>''t''</sub> as:
+:<math>X_t(\omega)=\sum_{k=1}^\infty A_k(\omega) f_k(t)</math>
+where <math>A_k(\omega)=\int_{[a,b]} X_t(\omega) f_k(t)\,dt</math>
+and we may approximate ''X''<sub>''t''</sub> by the finite sum <math>\hat{X}_t(\omega)=\sum_{k=1}^N A_k(\omega) f_k(t)</math> for some integer ''N''.
+'''Claim'''.
+Of all such approximations, the KL approximation is the one that minimizes the total mean square error (provided we have arranged the eigenvalues in decreasing order).
+<div class="NavFrame collapsed">
+  <div class="NavHead">[Proof]</div>
+  <div class="NavContent" style="text-align:left">
+Consider the error resulting from the truncation at the ''N''-th term in the following orthonormal expansion:
+:<math>\epsilon_N(t)=\sum_{k=N+1}^\infty A_k(\omega) f_k(t)</math>
+The mean-square error &epsilon;<sub>''N''</sub><sup>2</sup>(''t'') can be written as:
+:<math>
+\begin{align}
+\varepsilon_N^2(t)&=\mathrm{E}\left[\sum_{i=N+1}^\infty \sum_{j=N+1}^\infty A_i(\omega) A_j(\omega) f_i(t) f_j(t)\right]\\
+&=\sum_{i=N+1}^\infty \sum_{j=N+1}^\infty \mathrm{E}\left[\int_{[a, b]}\int_{[a, b]} X_t X_s f_i(t)f_j(s) ds\, dt\right] f_i(t) f_j(t)\\
+&=\sum_{i=N+1}^\infty \sum_{j=N+1}^\infty f_i(t) f_j(t) \int_{[a, b]}\int_{[a, b]}K_X(s,t) f_i(t)f_j(s) ds\, dt
+\end{align}
+</math>
+We then integrate this last equality over [''a'', ''b'']. The orthonormality of the ''f''<sub>k</sub> yields:
+:<math>
+\int_{[a, b]} \varepsilon_N^2(t) dt=\sum_{k=N+1}^\infty \int_{[a, b]}\int_{[a, b]} K_X(s,t) f_k(t)f_k(s) ds\, dt
+</math>
+The problem of minimizing the total mean-square error thus comes down to minimizing the right hand side of this equality subject to the constraint that the ''f''<sub>''k''</sub> be normalized. We hence introduce &beta;<sub>k</sub>, the Lagrangian multipliers associated with these constraints, and aim at minimizing the following function:
+:<math>
+Er[f_k(t),k\in\{N+1,\ldots\}]=\sum_{k=N+1}^\infty \int_{[a, b]}\int_{[a, b]} K_X(s,t) f_k(t)f_k(s) ds dt-\beta_k \left(\int_{[a, b]} f_k(t) f_k(t) dt -1\right)
+</math>
+Differentiating with respect to ''f''<sub>''i''</sub>(''t'') and setting the derivative to 0 yields:
+:<math>
+\frac{\partial Er}{\partial f_i(t)}=\int_{[a, b]} \left(\int_{[a, b]} K_X(s,t) f_i(s) ds -\beta_i f_i(t)\right)dt=0
+</math>
+which is satisfied in particular when <math>\int_{[a, b]} K_X(s,t) f_i(s) \,ds =\beta_i f_i(t)</math>, in other words when the ''f''<sub>''k''</sub> are chosen to be the eigenfunctions of ''T''<sub>''K''<sub>''X''</sub></sub>, hence resulting in the KL expansion.
+  </div>
+</div>
+=== Explained variance ===
+An important observation is that since the random coefficients ''Z''<sub>''k''</sub> of the KL expansion are uncorrelated, the [[Variance#Sum of uncorrelated variables .28Bienaym.C3.A9 formula.29|Bienaymé formula]] asserts that the variance of ''X''<sub>''t''</sub> is simply the sum of the variances of the individual components of the sum:
+:<math>
+\begin{align}
+\mbox{Var}[X_t]&=\sum_{k=0}^\infty e_k(t)^2 \mbox{Var}[Z_k]=\sum_{k=1}^\infty \lambda_k e_k(t)^2
+\end{align}
+</math>
+Integrating over [''a'', ''b''] and using the orthonormality of the ''e''<sub>''k''</sub>, we obtain that the total variance of the process is:
+:<math>
+\int_{[a,b]} \mbox{Var}[X_t] dt=\sum_{k=1}^\infty \lambda_k
+</math>
+In particular, the total variance of the ''N''-truncated approximation is <math>\sum_{k=1}^N \lambda_k</math>. As a result, the ''N''-truncated expansion explains <math>\sum_{k=1}^N \lambda_k/\sum_{k=1}^\infty \lambda_k</math> of the variance; and if we are content with an approximation that explains, say, 95% of the variance, then we just have to determine an <math>N\in\mathbb{N}</math> such that <math>\sum_{k=1}^N \lambda_k/\sum_{k=1}^\infty \lambda_k \geq 0.95</math>.
+=== The Karhunen–Loève expansion has the minimum representation entropy property ===
+{{Expand section|date=May 2011}}
+== Linear Karhunen-Loeve Approximations ==
+Let us consider a whole class of signals we want to approximate over the first M vectors of a basis. These signals are modeled as realizations of a random vector Y[n] of size N. To optimize the approximation we design a basis that minimizes the average approximation error. This section proves that optimal bases are karhunen-loeve bases that diagonalize the covariance matrix of Y. The random vector Y can be decomposed in an orthogonal basis <math>{{\left\{ {{g}_{m}} \right\}}_{0\le m\le N}}:</math>
+<math>Y=\sum\limits_{m=0}^{N-1}{\left\langle Y,{{g}_{m}} \right\rangle {{g}_{m}},}</math>
+where each
+<math>\left\langle Y,{{g}_{m}} \right\rangle =\sum\limits_{n=0}^{N-1}{Y\left[ n \right]}.g_{m}^{*}\left[ n \right]</math>
+is a random variable. The approximation from the first <math>M\le N</math> vectors of the basis is
+<math>{{Y}_{M}}=\sum\limits_{m=0}^{M-1}{\left\langle Y,{{g}_{m}} \right\rangle {{g}_{m}}}</math>
+The energy conservation in an orthogonal basis implies
+<math>\varepsilon \left[ M \right]=E\left\{ {{\left\| Y-{{Y}_{M}} \right\|}^{2}} \right\}=\sum\limits_{m=M}^{N-1}{E\left\{ {{\left| \left\langle Y,{{g}_{m}} \right\rangle  \right|}^{2}} \right\}}</math>
+This error is related to the covariance of Y defined by
+<math>R\left[ n,m \right]=E\left\{ Y\left[ n \right]{{Y}^{*}}\left[ m \right] \right\}</math>
+For any vector x[n] we denote by K the covariance operator represented by this matrix,
+<math>E\left\{ {{\left| \left\langle Y,x \right\rangle  \right|}^{2}} \right\}=\left\langle Kx,x \right\rangle =\sum\limits_{n=0}^{N-1}{\sum\limits_{m=0}^{N-1}{R\left[ n,m \right]x\left[ n \right]{{x}^{*}}\left[ m \right]}}</math>
+The error <math>\varepsilon \left[ M \right]</math> is therefore a sum of the last N-M coefficients of the covariance operator
+<math>\varepsilon \left[ M \right]=\sum\limits_{m=M}^{N-1}{\left\langle K{{g}_{m}},{{g}_{m}} \right\rangle }</math>
+The covariance operator K is Hermitian and Positive and is thus diagonalized in an orthogonal basis called a Karhunen-Loeve basis. The following theorem states that a Karhunen-Loeve basis is optimal for linear approximations.
+'''Theorem:(Optimality of Karhunen-Loeve basis)'''  Let K be acovariance operator. For all <math>M\ge 1</math>, the approximation error
+<math>\varepsilon \left[ M \right]=\sum\limits_{m=M}^{N-1}{\left\langle K{{g}_{m}},{{g}_{m}} \right\rangle }</math>
+is minimum if and only if <math>{{\left\{ {{g}_{m}} \right\}}_{0\le m<N}}</math> is a Karhunen-Loeve basis ordered by decreasing eigenvalues.
+<math>\left\langle K{{g}_{m}},{{g}_{m}} \right\rangle \ge \left\langle K{{g}_{m+1}},{{g}_{m+1}} \right\rangle for,0\le m<N-1</math>
+== Non-Linear Approximation in Bases ==
+Linear approximations project the signal on M vectors a priori. The approximation can be made more precise by choosing the M orthogonal vectors depending on the signal properties. This section analyzes the general performance of these non-linear approximations.
+A signal <math>f\in \Eta </math> is approximated with M vectors selected adaptively in an orthonormal basis <math>\Beta ={{\left\{ {{g}_{m}} \right\}}_{m\in \mathbb{N}}}</math> of <math>\Eta </math>. Let <math>{{f}_{M}}</math> be the projection of f over M vectors whose indices are in <math>{{I}_{M}}</math>:
+<math>{{f}_{M}}=\sum\limits_{m\in {{I}_{M}}}^{{}}{\left\langle f,{{g}_{m}} \right\rangle {{g}_{m}}}</math>
+The approximation error is the sum of the remaining coefficients
+<math>\varepsilon \left[ M \right]=\left\{ {{\left\| f-{{f}_{M}} \right\|}^{2}} \right\}=\sum\limits_{m\notin {{I}_{M}}}^{N-1}{\left\{ {{\left| \left\langle f,{{g}_{m}} \right\rangle  \right|}^{2}} \right\}}</math>
+To minimize this error, the indices in <math>{{I}_{M}}</math> must correspond to the M vectors having the largest inner product amplitude
+<math>\left| \left\langle f,{{g}_{m}} \right\rangle  \right|</math>. These are the vectors that best correlate f. They can thus be interpreted as the main features of f. The resulting error is necessarily smaller than the error of a linear approximation which selects the M approximation vectors independently of f.
+Let us sort <math>{{\left\{ \left| \left\langle f,{{g}_{m}} \right\rangle  \right| \right\}}_{m\in \mathbb{N}}}</math> in decreasing order :<math>\left| \left\langle f,{{g}_{{{m}_{k}}}} \right\rangle  \right|\ge \left| \left\langle f,{{g}_{{{m}_{k+1}}}} \right\rangle  \right|</math>. The best non-linear approximation is
+<math>{{f}_{M}}=\sum\limits_{k=1}^{M}{\left\langle f,{{g}_{{{m}_{k}}}} \right\rangle {{g}_{{{m}_{k}}}}}</math>
+It can also be written as inner product thresholding:
+<math>{{f}_{M}}=\sum\limits_{m=0}^{\infty }{{{\theta }_{T}}\left( \left\langle f,{{g}_{m}} \right\rangle  \right){{g}_{m}}}</math>
+with <math>T=\left| \left\langle f,{{g}_{{{m}_{M}}}} \right\rangle  \right|</math> and
+<math>{{\theta }_{T}}\left( x \right)=\left\{ \begin{align}
+  & x,if\left| x \right|\ge T \\
+ & 0,if\left| x \right|<T \\
+\end{align} \right.</math>
+The non-linear error is
+<math>\varepsilon \left[ M \right]=\left\{ {{\left\| f-{{f}_{M}} \right\|}^{2}} \right\}=\sum\limits_{k=M+1}^{\infty }{\left\{ {{\left| \left\langle f,{{g}_{{{m}_{k}}}} \right\rangle  \right|}^{2}} \right\}}</math>
+this error goes quickly to zero as M increases,if the sorted values of <math>\left| \left\langle f,{{g}_{{{m}_{k}}}} \right\rangle  \right|</math> have a fast decay as k increases. This decay is quantified by computing the <math>{{\Iota }^{\Rho }}</math> norm of the signal inner products in B:
+<math>{{\left\| f \right\|}_{\Beta ,p}}={{\left( \sum\limits_{m=0}^{\infty }{{{\left| \left\langle f,{{g}_{m}} \right\rangle  \right|}^{p}}} \right)}^{1/p}}</math>
+The following theorem relates the decay of <math>\varepsilon \left[ M \right]</math> to <math>{{\left\| f \right\|}_{\Beta ,p}}</math>
+'''Theorem:(decay of error)'''  If <math>{{\left\| f \right\|}_{\Beta ,p}}<+\infty </math> with <math>p<2</math> then
+<math>\varepsilon \left[ M \right]\le \frac{\left\| f \right\|_{\Beta ,p}^{2}}{\frac{2}{p}-1}{{M}^{1-\frac{2}{p}}}</math>
+and <math>\varepsilon \left[ M \right]=o\left( {{M}^{1-\frac{2}{p}}} \right)</math>. Conversely, if <math>\varepsilon \left[ M \right]=o\left( {{M}^{1-\frac{2}{p}}} \right)</math> then
+<math>{{\left\| f \right\|}_{\Beta ,q}}<+\infty </math> for any <math>q>p</math>.
+=== Non-optimality of Karhunen-Loéve Bases ===
+To further illustrate the differences between linear and non -linear approximations, we study the decomposition of a simple non-Gaussian random vector in a Karhunen-Loéve basis. Processes whose realizations have a random translation are stationary. The Karhunen-Loéve basis is then a Fourier basis and we study its performance.
+To simplify the analysis, consider a random vector Y[n] of size N that is random shift modulo N of a deterministic signal f[n] of zero mean <math>\sum\nolimits_{n=0}^{N-1}{f\left[ n \right]=0}</math>:
+<math>Y\left[ n \right]=f\left[ \left( n-p \right)\bmod N \right]</math>
+The random shift P is uniformly distributed on [0,N-1]:
+<math>\Pr \left( P=p \right)=\frac{1}{N}for,0\le p<N</math>
+Clearly
+<math>E\left\{ Y\left[ n \right] \right\}=\frac{1}{N}\sum\limits_{p=0}^{N-1}{f\left[ \left( n-p \right)\bmod N \right]}=0</math>
+and
+<math>R\left[ n,k \right]=E\left\{ Y\left[ n \right]Y\left[ k \right] \right\}=\frac{1}{N}\sum\limits_{p=0}^{N-1}{f\left[ \left( n-p \right)\bmod N \right]}f\left[ \left( k-p \right)\bmod N \right]</math>
+<math>=\frac{1}{N}f\Theta \bar{f}\left[ n-k \right]with,\bar{f}\left[ n \right]=f\left[ -n \right]</math>
+Hence <math>R\left[ n,k \right]={{R}_{Y}}\left[ n-k \right]with,</math>
+<math>{{R}_{Y}}\left[ k \right]=\frac{1}{N}f\Theta \bar{f}\left[ k \right]</math>
+Since R<sub>Y</sub> is N periodic, Y is a circular stationary random vector. The covariance operator is a circular convolution with R<sub>Y</sub> and is therefore diagonalized in the discrete Fourier Karhunen-Loéve basis <math>{{\left\{ \frac{1}{\sqrt{N}}{{e}^{\frac{i2\pi mn}{N}}} \right\}}_{0\le m<N}}</math>. The power spectrum is Fourier Transform of R<sub>Y</sub>:
+<math>{{P}_{Y}}\left[ m \right]={{{\hat{R}}}_{Y}}\left[ m \right]=\frac{1}{N}{{\left| \hat{f}\left[ m \right] \right|}^{2}}</math>
+'''Example:'''  Consider an extreme case where <math>f\left[ n \right]=\delta \left[ n \right]-\delta \left[ n-1 \right]</math>
+A theorem stated above guarantees that the Fourier Karhunen-Loéve basis produces a smaller expected approximation error than a canonical basis of Diracs <math>{{\left\{ {{g}_{m}}\left[ n \right]=\delta \left[ n-m \right] \right\}}_{0\le m<N}}</math>.
+Indeed we do not know a priori the abscissa of the non-zero coefficients of Y, so there is no particular Dirac that is better adapted to perform the approximation . But the Fourier vectors cover the whole support of Y and thus absorb a part of the signal energy.
+<math>E\left\{ {{\left| \left\langle Y\left[ n \right],\frac{1}{\sqrt{N}}{{e}^{\frac{i2\pi mn}{N}}} \right\rangle  \right|}^{2}} \right\}={{P}_{Y}}\left[ m \right]=\frac{4}{N}{{\sin }^{2}}\left( \frac{\pi k}{N} \right)</math>
+Selecting higher frequency Fourier coefficients yields a better mean-square approximation than choosing a priori a few Dirac vectors to perform the approximation.
+The situation is totally different for non-linear approximations. If <math>f\left[ n \right]=\delta \left[ n \right]-\delta \left[ n-1 \right]</math> then the discrete Fourier basis is extremely inefficient because because f and hence Y have an energy that is almost uniformly spread among all Fourier vectors. In contrast, since f has only two non-zero coefficients in the Dirac basis, a non-linear approximation of Y with <math>M\ge 2</math> gives zero error.
+<ref>A wavelet tour of signal processing-Stéphane Mallat</ref>
+== Principal component analysis ==
+{{Main|Principal component analysis}}
+We have established the Karhunen–Loève theorem and derived a few properties thereof. We also noted that one hurdle in its application was the numerical cost of determining the eigenvalues and eigenfunctions of its covariance operator through the Fredholm integral equation of the second kind <math>\int_{[a,b]} K_X(s,t) e_k(s)\,ds=\lambda_k e_k(t)</math> .
+However, when applied to a discrete and finite process <math>\left(X_n\right)_{n\in\{1,\ldots,N\}}</math>, the problem takes a much simpler form and standard algebra can be used to carry out the calculations.
+Note that a continuous process can also be sampled at ''N'' points in time in order to reduce the problem to a finite version.
+We henceforth consider a random ''N''-dimensional vector <math>X=\left(X_1~X_2~\ldots~X_N\right)^T</math>. As mentioned above, ''X'' could contain ''N'' samples of a signal but it can hold many more representations depending on the field of application. For instance it could be the answers to a survey or economic data in an econometrics analysis.
+As in the continuous version, we assume that ''X'' is centered, otherwise we can let <math>X:=X-\mu_X</math> (where <math>\mu_X</math> is the [[mean vector]] of ''X'') which is centered.
+Let us adapt the procedure to the discrete case.
+=== Covariance matrix ===
+Recall that the main implication and difficulty of the KL transformation is computing the eigenvectors of the linear operator associated to the covariance function, which are given by the solutions to the integral equation written above.
+Define &Sigma;, the covariance matrix of ''X''. &Sigma; is an ''N'' by ''N'' matrix whose elements are given by:
+:<math>\Sigma_{ij}=E[X_i X_j],\qquad \forall i,j \in \{1,\ldots,N\}</math>
+Rewriting the above integral equation to suit the discrete case, we observe that it turns into:
+:<math>
+\begin{align}
+&\sum_{i=1}^N \Sigma_{ij} e_j=\lambda e_i\\
+\Leftrightarrow \quad& \Sigma e=\lambda e
+\end{align}
+</math>
+where <math>e=(e_1~e_2~\ldots~e_N)^T</math> is an ''N''-dimensional vector.
+The integral equation thus reduces to a simple matrix eigenvalue problem, which explains why the PCA has such a broad domain of applications.
+Since &Sigma; is a positive definite symmetric matrix, it possesses a set of orthonormal eigenvectors forming a basis of <math>\R^N</math>, and we write <math>\{\lambda_i,\phi_i\}_{i\in\{1,\ldots,N\}}</math> this set of eigenvalues and corresponding eigenvectors, listed in decreasing values of &lambda;<sub>''i''</sub>. Let also <math>\Phi</math> be the orthonormal matrix consisting of these eigenvectors:
+:<math>
+\begin{align}
+\Phi &:=\left(\phi_1~\phi_2~\ldots~\phi_N\right)^T\\
+\Phi^T \Phi &=I
+\end{align}
+</math>
+=== Principal component transform ===
+It remains to perform the actual KL transformation, called the ''principal component transform'' in this case. Recall that the transform was found by expanding the process with respect to  the basis spanned by the eigenvectors of the covariance function. In this case, we hence have:
+:<math>
+\begin{align}
+X &=\sum_{i=1}^N \langle \phi_i,X\rangle \phi_i\\
+&=\sum_{i=1}^N \phi_i^T X \phi_i
+\end{align}
+</math>
+In a more compact form, the principal component transform of ''X'' is defined by:
+:<math>
+\left\{
+\begin{array}{rl}
+Y&=\Phi^T X\\
+X&=\Phi Y
+\end{array}
+\right.
+</math>
+The ''i''-th component of ''Y'' is <math>Y_i=\phi_i^T X</math>, the projection of ''X'' on <math>\phi_i</math> and the inverse transform <math>X=\Phi Y</math> yields the expansion of <math>X</math> on the space spanned by the <math>\phi_i</math>:
+:<math>
+X=\sum_{i=1}^N Y_i \phi_i=\sum_{i=1}^N \langle \phi_i,X\rangle \phi_i
+</math>
+As in the continuous case, we may reduce the dimensionality of the problem by truncating the sum at some <math>K\in\{1,\ldots,N\}</math> such that <math>\frac{\sum_{i=1}^K \lambda_i}{\sum_{i=1}^N \lambda_i}\geq \alpha</math> where &alpha; is the explained variance threshold we wish to set.
+We can also reduce the dimensionality through the use of multilevel dominant eigenvector estimation (MDEE).<ref>X. Tang, “Texture information in run-length matrices,” IEEE Transactions on Image Processing, vol. 7, No. 11, pp. 1602- 1609, Nov. 1998</ref>
+== Examples ==
+=== The Wiener process ===
+There are numerous equivalent characterizations of the [[Wiener process]] which is a mathematical formalization of [[Brownian motion]].  Here we regard it as the centered standard Gaussian process  '''W'''<sub>''t''</sub> with covariance function
+:<math> K_{W}(t,s)  = \operatorname{Cov}(W_t,W_s) = \min (s,t). </math>
+We restrict the time domain to [''a'',''b'']=[0,1] without loss of generality.
+The eigenvectors of the covariance kernel are easily determined.  These are
+:<math> e_k(t) = \sqrt{2} \sin \left( \left(k - \textstyle\frac{1}{2}\right) \pi t \right)</math>
+and the corresponding eigenvalues are
+:<math> \lambda_k = \frac{1}{(k -\frac{1}{2})^2 \pi^2}. </math>
+<div class="NavFrame collapsed">
+  <div class="NavHead">[Proof]</div>
+  <div class="NavContent" style="text-align:left">
+In order to find the eigenvalues and eigenvectors, we need to solve the integral equation:
+:<math>
+\begin{align}
+\int_{[a,b]} K_W(s,t) e(s)ds&=\lambda e(t)\qquad \forall t, 0\leq t\leq 1\\
+\int_0^1\min(s,t) e(s)ds&=\lambda e(t)\qquad \forall t, 0\leq t\leq 1 \\
+\int_0^t s e(s) ds + t \int_t^1 e(s) ds &= \lambda e(t) \qquad \forall t, 0\leq t\leq 1
+\end{align}
+</math>
+differentiating once with respect to ''t'' yields:
+:<math>
+\int_{t}^1 e(s) ds=\lambda e'(t)
+</math>
+a second differentiation produces the following differential equation:
+:<math>
+-e(t)=\lambda e''(t)
+</math>
+The general solution of which has the form:
+:<math>
+e(t)=A\sin\left(\frac{t}{\sqrt{\lambda}}\right)+B\cos\left(\frac{t}{\sqrt{\lambda}}\right)
+</math>
+where ''A'' and ''B'' are two constants to be determined with the boundary conditions. Setting ''t''=0 in the initial integral equation gives ''e''(0)=0 which implies that ''B''=0 and similarly, setting ''t''=1 in the first differentiation yields ''e' ''(1)=0, whence:
+:<math>\cos\left(\frac{1}{\sqrt{\lambda}}\right)=0</math>
+which in turn implies that eigenvalues of ''T''<sub>''K''<sub>''X''</sub></sub> are:
+:<math>\lambda_k=\left(\frac{1}{(k-\frac{1}{2})\pi}\right)^2,\qquad k\geq 1</math>
+The corresponding eigenfunctions are thus of the form:
+:<math>e_k(t)=A \sin\left((k-\frac{1}{2})\pi t\right),\qquad k\geq 1</math>
+''A'' is then chosen so as to normalize ''e''<sub>''k''</sub>:
+:<math>\int_0^1 e_k^2(t) dt=1\quad \implies\quad A=\sqrt{2}</math>
+  </div>
+</div>
+This gives the following representation of the Wiener process:
+'''Theorem'''.  There is a sequence {''Z''<sub>''i''</sub>}<sub>''i''</sub> of independent Gaussian random variables with mean zero and variance 1 such that
+:<math> W_t = \sqrt{2} \sum_{k=1}^\infty Z_k \frac{\sin \left(\left(k - \frac{1}{2}\right) \pi t\right)}{ \left(k - \frac{1}{2}\right) \pi}. </math>
+Note that this representation is only valid for <math> t\in[0,1]. </math>  On larger intervals, the increments are not independent.  As stated in the theorem, convergence is in the L<sup>2</sup> norm and uniform in ''t''.
+=== The Brownian bridge ===
+Similarly the [[Brownian bridge]] <math>B_t=W_t-tW_1</math> which is a [[stochastic process]] with covariance function
+:<math>K_B(t,s)=\min(t,s)-ts</math>
+can be represented as the series
+:<math>B_t = \sum_{k=1}^\infty Z_k \frac{\sqrt{2} \sin(k \pi t)}{k \pi}</math>
+== Applications ==
+{{Expand section|date=July 2010}}
+[[Adaptive optics]] systems sometimes use K–L functions to reconstruct wave-front phase information (Dai 1996, JOSA A).
+Karhunen–Loève expansion is closely related to the [[Singular Value Decomposition]]. The latter has myriad applications in image processing, radar, seismology, and the like. If one has independent vector observations from a vector valued stochastic process then the left singular vectors are [[maximum likelihood]] estimates of the ensemble KL expansion.
+=== Applications in signal estimation and detection===
+====Detection of a known continuous signal S(t)====
+In communication, we usually have to decide whether a signal from a noisy channel contains valuable information. The following hypothesis testing is used for detecting continuous signal s(t) from channel output X(t), N(t) is the channel noise, which is usually assumed zero mean gaussian process with correlation function <math>R_{N} (t, s) = E[N(t)N(s)]</math>
+{|
+|-
+|<math>H: X(t) = N(t)</math>,
+|-
+|<math>K: X(t) = N(t)+s(t), t\in(0,T)</math>.
+|}
+====Signal detection  in white noise====
+When the channel noise is white, its correlation function is
+:<math>R_{N}(t) = \frac{N_0}{2} \delta (t)</math>,
+and it has constant power spectrum density. In physically practical channel, the noise power is finite, so:
+:<math>S_{N}(f) = \frac{N_{0}}{2} \text{ for } |f|<w, 0 \text{ for } |f|>w</math>.
+Then the noise correlation function is sinc function with zeros at  <math>\frac{n}{2\omega}, n = ...-1,0,1,...</math> .
+Since are uncorrelated and gaussian, they are independent. Thus we can take samples from X(t) with time spacing
+:<math> \Delta t = \frac{n}{2\omega}</math> within (0,T).
+Let <math>X_i = X(i\Delta t)</math>. We have a total of <math>n = \frac{T}{\Delta t} = T(2\omega) = 2\omega T</math> i.i.d samples  <math>\{X_1, X_2,...,X_n\}</math> to develop the likelihood-ratio test.  Define signal <math>S_i = S(i\Delta t)</math>, the problem becomes,
+:<math>H: X_i = N_i</math>,
+:<math>K: X_i = N_i + S_i, i = 1,2...n.</math>
+The log-likelihood ratio
+:<math>\mathcal{L}(\underline{x}) = \log\frac{\sum^n_{i=1} (2S_i x_i - S_i^2)}{2\sigma^2} \Leftrightarrow \Delta t \Sigma ^n_{i = 1} S_i x_i = \sum^n_{i=1} S(i\Delta t)x(i\Delta t)\Delta t \gtrless \lambda_2</math>.
+As <math> t \rightarrow 0, \text{ let } G = \int^T_0 S(t)x(t)dt</math>.
+Then G is the test statistics and the [[Neyman–Pearson lemma|Neyman–Pearson optimum detector]] is:<math>G(\underline{x}) > G_0 \Rightarrow K, < G_0 \Rightarrow H</math>. As G is gaussian, we can characterize it by finding its mean and variances. Then we get
+:<math>H: G \sim N(0,\frac{N_{0}E}{2})</math>
+:<math>K: G \sim N(E,\frac{N_{0}E}{2})</math>,
+where <math>E = \int^T_{0} S^2(t)dt</math> is the signal energy.
+The false alarm error
+:<math>\alpha = \int^{\infty}_{G_{0}} N(0,\frac{N_{0}E}{2})dG \Rightarrow G_0 = \sqrt{\frac{N_0 E}{2}} \Phi^{-1}(1-\alpha)</math>
+And the probability of detection:
+:<math>\beta = \int^{\infty}_{G_0} N(E, \frac{N_0 E}{2})dG = 1-\Phi(\frac{G_0 - E}{\sqrt{\frac{N_0 E}{2}}}) = \Phi [\sqrt{\frac{2E}{N_0}} - \Phi^{-1}(1-\alpha)] , \Phi(\cdot)</math> is the cdf of standard normal gaussian variable.
+====Signal detection in colored noise ====
+When N(t) is colored (correlated in time) gaussian noise  with zero mean and covariance function <math>R_N(t,s) = E[X(t)X(s)],</math>  we cannot sample independent discrete observations by evenly spacing the time. Instead, we can use K–L expansion to uncorrelate the noise process and get independent gaussian observation 'samples'.
+The K–L expansion of N(t):
+:<math>N(t) = \sum^{\infty}_{i=1} N_i \Phi_i(t), 0<t<T</math>,
+where <math>N_i =\int N(t)\Phi_i(t)dt</math> and the orthonormal bases <math>\{\Phi_i{t}\} </math>are generated by kernal <math>R_N(t,s)</math>, i.e., solution to
+:<math> \int ^T _0 R_N(t,s)\Phi_i(s)ds = \lambda_i \Phi_i(t), var[N_i] = \lambda_i</math>.
+Do the expansion:
+:<math>S(t) = \sum^{\infty}_{i = 1}S_i\Phi_i(t)</math>,
+where <math>S_i = \int^T _0 S(t)\Phi_i(t)dt, 0<t<T.</math>, then
+:<math>X_i = \int^T _0 X(t)\Phi_i(t) dt = N_i</math>
+under H and <math>N_i + S_i</math> under K. Let <math>\overline{X} = \{X_1,X_2,\dots\}</math>, we have
+:<math>{N_i}</math> are independent gaussian r.v's with variance <math>\lambda_i</math>
+:under H: <math>\{X_i\}</math> are independent gaussian r.v's. <math>f_H[x(t)|0<t<T] = f_H(\underline{x}) = \prod^{\infty} _{i=1} \frac{1}{\sqrt{2\pi \lambda_i}}exp[-\frac{x_i^2}{2 \lambda_i}]</math>
+:under K: <math>\{X_i - S_i\}</math> are independent gaussian r.v's. <math>f_K[x(t)|0<t<T] = f_K(\underline{x}) = \prod^{\infty} _{i=1} \frac{1}{\sqrt{2\pi \lambda_i}}exp[-\frac{(x_i - S_i)^2}{2 \lambda_i}]</math>
+Hence, the log-LR is given by
+:<math>\mathcal{L}(\underline{x}) = \sum^{\infty}_{i=1} \frac{2S_i x_i - S_i^2}{2\lambda_i}</math>
+and the optimum detector is
+:<math>G = \sum^{\infty}_{i=1} S_i x_i \lambda_i > G_0 \Rightarrow K, < G_0 \Rightarrow H.</math>
+Define
+:<math>k(t) = \sum^{\infty}_{i=1} \lambda_i S_i \Phi_i(t), 0<t<T,</math>
+then <math>G = \int^T _0 k(t)x(t)dt</math>.
+=====How to find ''k''(''t'')=====
+Since
+:<math>\int^T_0 R_N(t,s)k(s)ds = \sum^{\infty}_{i=1} \lambda_i S_i \int^T _0 R_N(t,s)\Phi_i (s) ds = \sum^{\infty}_{i=1} S_i \Phi_i(t) = S(t)</math>,
+k(t) is the solution to
+:<math>\int^T_0 R_N(t,s)k(s)ds = S(t)</math>.
+If N(t)is wide-sense stationary,
+:<math>\int^T_0 R_N(t-s)k(s)ds = S(t) </math>,
+which is known as the [[Wiener–Hopf equation]]. The equation can be solved by taking fourier transform, but not practically realizable since infinite spectrum needs spatial factorization.
+A special case which is easy to calculate k(t) is white gaussian noise.
+:<math>\int^T_0 \frac{N_0}{2}\delta(t-s)k(s)ds = S(t) \Rightarrow k(t) = C S(t), 0<t<T</math>.
+The corresponding impulse response is h(t) = k(T-t) = C S(T-t). Let C = 1, this is just the result we arrived at in previous section for detecting of signal in white noise.
+=====Test threshold for Neyman–Pearson detector=====
+Since X(t)is gaussian process, <math>G = \int^T_0 k(t)x(t)dt</math> is a gaussian random variable that can be characterized by its mean and variance.
+:<math>E[G|H] = \int^T_0 k(t)E[x(t)|H]dt = 0</math>
+:<math>E[G|K] = \int^T_0 k(t)E[x(t)|K]dt = \int^T_0 k(t)S(t)dt \equiv \rho</math>
+:<math>E[G^2|H] = \int^T_0\int^T_0 k(t)k(s) R_N(t,s)dtds = \int^T_0 k(t)(\int^T_0 k(s)R_N(t,s)ds)=\int^T_0 k(t)S(t)dt = \rho</math>
+:<math>var[G|H] = E[G^2|H] - (E[G|H])^2 = \rho</math>
+:<math>E[G^2|K]=\int^T_0\int^T_0k(t)k(s)E[x(t)x(s)]dtds = \int^T_0\int^T_0k(t)k(s)(R_N(t,s) +S(t)S(s))dtds = \rho + \rho^2</math>
+:<math>var[G|K] = E[G^2|K] - (E[G|K])^2 = \rho + \rho^2 -\rho^2 = \rho. </math>
+Hence, we obtain the distributions of ''H'' and ''K'':
+:<math>H: G \sim N(0,\rho)</math>
+:<math>K: G \sim N(\rho, \rho)</math>
+The false alarm error is
+:<math>\alpha = \int^{\infty}_{G_0} N(0,\rho)dG = 1 - \Phi(\frac{G_0}{\sqrt{\rho}}).</math>
+So the test threshold for the Neyman–Pearson optimum detector is
+:<math>G_0 = \sqrt{\rho} \Phi^{-1} (1-\alpha)</math>.
+Its power of detection is
+:<math>\beta = \int^{\infty}_{G_0} N(\rho, \rho)dG = \Phi [\sqrt{\rho} - \Phi^{-1}(1 - \alpha)]</math>.
+When the noise is white gaussian process, the signal power is
+:<math>\rho = \int^T_0 k(t)S(t)dt = \int^T_0 S(t)^2 dt = E</math>.
+=====Prewhitening=====
+For some type of colored noise, a typical practise is to add a prewhiterning filter before the matched filter to transform the colored noise into white noise. For example, N(t) is a wide-sense stationary colored noise with correlation function
+:<math>\R_N(\tau) = \frac{B N_0}{4} e^{-B|\tau|}</math>
+:and <math>S_N(f) = \frac{N_0}{2(1+(\frac{w}{B})^2)}</math>.
+The transfer function of prewhitening filter is <math>H(f) = 1 + j \frac{w}{B}</math>.
+====Detection of a gaussian random signal in AWGN====
+When the signal we want to detect from the noisy channel is also random, for example, a white gaussian process X(t), we can still implement K–L expansion to get independent sequence of observation. In this case, the detection problem is described as follows:
+:<math>H_0 : Y(t) = N(t)</math>
+:<math>H_1 : Y(t) = N(t) + X(t), 0<t<T. </math>
+X(t) is a random process with correlation function <math>R_X(t,s) = E\{X[t]X[s]\}</math>
+The K–L expansion of X(t) is
+:<math>X(t) = \sum^{\infty}_{i=1} X_i \Phi_i(t)</math>,
+where
+:<math>X_i =\int^T_0 X(t) \Phi_i(t). \Phi(t)</math>
+are solutions to
+:<math> \int^T_0 R_X(t,s)\Phi_i(s)ds= \lambda_i \Phi_i(t)</math>.
+So <math>X_i</math>'s are independent sequence of r.v's with zero mean and variance <math>\lambda_i</math>. Expanding Y(t) and N(t) by <math>\Phi_i(t)</math>, we get
+:<math>Y_i = \int^T_0 Y(t)\Phi_i(t)dt = \int^T_0 [N(t) + X(t)]\Phi_i(t) = N_i + X_i</math>,
+where <math>N_i = \int^T_0 N(t)\Phi_i(t)dt.</math>
+As N(t) is gaussian white noise, <math>N_i</math>'s are i.i.d sequence of r.v with zero mean and variance<math>\frac{N_0}{2}</math>, then the problem is simplified as follows,
+:<math>H_0: Y_i = N_i</math>
+:<math>H_1: Y_i = N_i + X_i</math>
+The Neyman–Pearson optimal test:
+:<math>\Lambda = \frac{f_Y|H_1}{f_Y|H_0} = Ce^{-\sum^{\infty}_{i=1}\frac{y_i^2}{2} \frac{\lambda_i}{\frac{N_0}{2}(\frac{N_0}{2} + \lambda_i)}}</math>,
+so the log-likelihood ratio
+:<math>\mathcal{L} = ln(\Lambda) = K -\sum^{\infty}_{i=1}\frac{y_i^2}{2} \frac{\lambda_i}{\frac{N_0}{2}(\frac{N_0}{2} + \lambda_i)}</math>.
+Since
+:<math>\hat{X_i} = \frac{\lambda_i}{\frac{N_0}{2}(\frac{N_0}{2} + \lambda_i)}</math>
+is just the minimum-mean-square estimate of <math>X_i</math> given <math>Y_i</math>'s,
+:<math>\mathcal{L} = K + \frac{1}{N_0} \sum^{\infty}_{i=1} Y_i \hat{X_i}</math>.
+K–L expansion has the following property:  If
+:<math>f(t) = \sum f_i \Phi_i(t), g(t) = \sum g_i \Phi_i(t)</math>,
+where
+:<math>f_i = \int_0^T f(t) \Phi_i(t), g_i = \int_0^T g(t)\Phi_i(t).</math>,
+then
+:<math>\sum^{\infty}_{i=1} f_i g_i = \int^T_0 g(t)f(t)dt</math>.
+So let
+:<math>\hat{X(t|T)} = \sum^{\infty}_{i=1} \hat{X_i}\Phi_i(t)</math>, <math>\mathcal{L} = K + \frac{1}{N_0} \int^T_0 Y(t) \hat{X(t|T)}dt</math>.
+Noncausal filter Q(t, s) can be used to get the estimate through
+:<math>\hat{X(t|T)} = \int^T_0 Q(t,s)Y(s)ds</math>.
+By [[orthogonality principle]], Q(t,s) satisfies
+:<math>\int^T_0 Q(t,s)R_X(s,t)ds + \frac{N_0}{2} Q(t, \lambda) = R_X(t, \lambda), 0 < \lambda < T, 0<t<T. </math>.
+However for practical reason, it's necessary to further derive the causal filter  h(t, s), where h(t, s) = 0 for s > t, to get estimate <math>\hat{X(t|t)}</math>.
+Specifically,
+:<math>Q(t,s) = h(t,s) + h(s, t) - \int^T_0 h(\lambda, t)h(s, \lambda)d\lambda</math>.
+==See also==
+*[[Principal component analysis]]
+*[[Proper orthogonal decomposition]]
+*[[Polynomial chaos]]
+==Notes==
+{{Reflist}}
+==References==
+* {{cite book
+|first1=Henry
+|last1=Stark
+|first2=John W.
+|last2=Woods
+|title=Probability, Random Processes, and Estimation Theory for Engineers
+|publisher=Prentice-Hall, Inc
+|year=1986
+|isbn=0-13-711706-X
+|url = http://openlibrary.org/books/OL21138080M/Probability_random_processes_and_estimation_theory_for_engineers
+}}
+*{{cite book
+|first1=Roger
+|last1=Ghanem
+|first2=Pol
+|last2=Spanos
+|publisher = Springer-Verlag
+|isbn = 0-387-97456-3
+|title = Stochastic finite elements: a spectral approach
+|url = http://openlibrary.org/books/OL1865197M/Stochastic_finite_elements
+|year = 1991
+}}
+* {{cite book
+|first1=I.
+|last1=Guikhman
+|first2=A.
+|last2=Skorokhod
+|title=Introduction a la Théorie des Processus Aléatoires
+|publisher=Éditions MIR
+|year=1977
+}}
+* {{cite book
+|first1=B.
+|last1=Simon
+|title=Functional Integration and Quantum Physics
+|publisher=Academic Press
+|year=1979
+}}
+* {{cite journal
+|last1=Karhunen
+|first1=Kari
+|title=Über lineare Methoden in der Wahrscheinlichkeitsrechnung
+|journal=Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys.
+|year=1947
+|volume=37
+|pages=1&ndash;79
+}}
+* {{cite book
+|first1=M.
+|last1=Loève
+|title=Probability theory.'' Vol. II, 4th ed.
+|series=Graduate Texts in Mathematics
+|volume=46
+|publisher=Springer-Verlag
+|year=1978
+|isbn=0-387-90262-7
+}}
+* {{cite journal
+|first1=G.
+|last1=Dai
+|title=Modal wave-front reconstruction with Zernike polynomials and Karhunen–Loeve functions
+|journal=JOSA A
+|volume=13
+|issue=6
+|page=1218
+|year=1996
+|doi=10.1364/JOSAA.13.001218
+|bibcode=1996JOSAA..13.1218D
+}}
+*Wu B., Zhu J., Najm F.(2005) "A Non-parametric Approach for Dynamic Range Estimation of Nonlinear Systems". In Proceedings of Design Automation Conference(841-844) 2005
+*Wu B., Zhu J., Najm F.(2006) "Dynamic Range Estimation". IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 25 Issue:9 (1618-1636) 2006
+* {{cite arXiv
+|title=Entropy Encoding, Hilbert Space and Karhunen–Loeve Transforms
+|first1=Palle E. T.
+|last1=Jorgensen
+|first2=Myung-Sin
+|last2=Song
+|eprint=math-ph/0701056
+|year=2007
+|bibcode=2007JMP....48j3503J
+}}
+<!--
+* {{cite arXiv
+|title=Synthesis of Taylor Phase Screens with Karhunen–Loeve Basis Functions
+|first1=Richard J.
+|last1=Mathar
+|eprint=0705.1700
+|class=astro-ph
+|year=2007
+|bibcode=2007arXiv0705.1700M
+}}
+-->
+* {{cite journal
+|first1=Richard J.
+|last1=Mathar
+|journal=Baltic Astronomy
+|title=Karhunen–Loeve basis functions of Kolmogorov turbulence in the sphere
+|year=2008
+|volume=17
+|issue=3/4
+|pages=383&ndash;398
+|bibcode=2008BaltA..17..383M
+|arxiv = 0805.3979 }}
+* {{cite arXiv
+|first1=Richard J.
+|last1=Mathar
+|title=Modal decomposition of the von-Karman covariance of atmospheric turbulence in the circular entrance pupil
+|eprint=0911.4710
+|class=astro-ph.IM
+|year=2009
+|bibcode=2009arXiv0911.4710M
+}}
+* {{cite journal
+|first1=Richard J.
+|last1=Mathar
+|journal=Waves in Random and Complex Media
+|title=Karhunen–Loeve basis of Kolmogorov phase screens covering a rectangular stripe
+|volume=20
+|issue=1
+|doi=10.1080/17455030903369677
+|year=2010
+|pages=23&ndash;35
+|bibcode=2020WRCM...20...23M
+}}
+==External links==
+* ''Mathematica'' [http://reference.wolfram.com/mathematica/ref/KarhunenLoeveDecomposition.html KarhunenLoeveDecomposition] function.
+* ''E161: Computer Image Processing and Analysis'' notes by Pr. Ruye Wang at [[Harvey Mudd College]] [http://fourier.eng.hmc.edu/e161/lectures/klt/klt.html]
+{{DEFAULTSORT:Karhunen-Loeve theorem}}
+[[Category:Estimation theory]]
+[[Category:Probability theorems]]
+[[Category:Signal processing]]
+[[Category:Stochastic processes]]
+[[Category:Statistical theorems]]
+[[fr:Transformée de Karhunen-Loève]]
+[[ru:Теорема Кархунена-Лоэва]]
+[[sv:Karhunen-Loeve-transformen]]