|
|
| Line 1: |
Line 1: |
| {{technical|date=November 2013}}
| | A superb tip to consider if you want to be sure you never obtain a candida albicans would be to always wear clothing that is manufactured from natural resources. Although she was treated, but if she still has a yeast infection, there is the possibility she can still spread the yeast infection is on the spouse. As a guy, having a yeast infection can be an embarrassing thing. The benefits of this are that the affected areas will. s immune system keeps the growth of yeast under control so that it doesn. <br><br>Do not let this build up of sweat around the body irritate the yeast condition. Numerous people say yes and that it can't be beat when it comes to a yeast infection home remedy. Many women have had a yeast infection in their lifetime. This is why you can feel so much worse than when you first start on your yeast treatment program. The tiny blood vessels under the skin break easily and leak fluid into the skin - because of this a bruise will appear. <br><br>You probably know how necessary it's to stop the causes and symptoms of the infection, if you have experienced Candidiasis or fungus infection. Add one cup of vinegar and same amount of salt into the bath tub and dip the infected areas in the tub and leave it for 20 to 30 minutes. The raw foods bring forth a sense of balance into the body. If you are using or doing one of the following below it may be the source of your yeast infection:. A burning sensation, mainly during intercourse or while urinating. <br><br>- Echinacea and Golden seal are herbs which help to boost the immune system and prevent infection. Steer clear of super tight jeans and synthetic fabrics including lycra and nylon. The resulting symptoms include red and irritated skin, which can be sometimes accompanied by a scaly rash. Don't make the mistake of treating your yeast infection yourself if you don't know what caused it. It too treats and soothes acne, burns, hemorrhoids but you can merely take the vital oil to handle blemishes. <br><br>Soon after using the prescription medication, it might take about a 7 days for that medication to start working. Every time that I have ever gotten a reacurring yeast infection I have always been able to cure it with hydrogen peroxide. But in all the cases EPD is not a lasting solution for Candida. Bright Hub authors have top picks for the best free antivirus software including AVG, Avast and Anti - Vir. The key is to get rid of all the candida in your system.<br><br>If you liked this information and you would such as to obtain additional info regarding [http://www.naturalcureyeastinfection.info/sitemap/ how do you treat a yeast infection] kindly browse through the webpage. |
| [[File:Bandlimited.svg|thumb|250px|Fig. 1: Fourier transform of a bandlimited function (amplitude vs frequency)|right]]
| |
| | |
| The '''Nyquist–Shannon sampling theorem''', after [[Harry Nyquist]] and [[Claude Shannon]],<ref>The theorem was also discovered independently by [[E. T. Whittaker]], by [[Vladimir Kotelnikov]], and by others. So it is also known as the ''Nyquist–Shannon–Kotelnikov'', ''Whittaker–Shannon–Kotelnikov'', ''Whittaker–Nyquist–Kotelnikov–Shannon'', ''WKS'', as well as the '''cardinal theorem of interpolation theory'''.</ref> in the literature more commonly referred to as the '''Nyquist sampling theorem''' or simply as '''the sampling theorem''', is a fundamental result in the field of [[information theory]], in particular [[telecommunication]]s and [[signal processing]]. [[Sampling (signal processing)|Sampling]] is the process of converting a [[signal (electrical engineering)|signal]] (for example, a function of continuous time or space) into a numeric sequence (a function of discrete time or space). [[Claude Shannon|Shannon's]] version of the theorem states:<ref name="Shannon49">[[Claude E. Shannon|C. E. Shannon]], "Communication in the presence of noise", [[Proceedings of the IRE|Proc. Institute of Radio Engineers]], vol. 37, no. 1, pp. 10–21, Jan. 1949. [http://www.stanford.edu/class/ee104/shannonpaper.pdf Reprint as classic paper in: ''Proc. IEEE'', vol. 86, no. 2, (Feb. 1998)]</ref>
| |
| | |
| <blockquote>If a function ''x''(''t'') contains no frequencies higher than ''B'' [[hertz]], it is completely determined by giving its ordinates at a series of points spaced 1/(2''B'') seconds apart.</blockquote>
| |
| | |
| In other words, a [[bandlimited]] function can be perfectly reconstructed from a [[countable]] sequence of samples if the bandlimit, ''B'', is no greater than half the [[sampling rate]] (samples per second). The theorem also leads to a formula for reconstruction of the original function from its samples. When the bandlimit is too high (or there is no bandlimit), the reconstruction exhibits imperfections known as [[aliasing]]. The [[Poisson summation formula]] provides a graphic understanding of aliasing and an alternative derivation of the theorem, using the perspective of the function's [[Fourier transform]].
| |
| | |
| In practice, infinite sequences, perfect sampling, and perfect interpolation are all replaced by approximations, deviating from the ideal mathematical reconstruction.
| |
| | |
| Modern statements of the theorem are sometimes careful to explicitly state that ''x''(''t'') must contain no [[Sine wave|sinusoidal]] component at exactly frequency ''B'', or that ''B'' must be strictly less than ½ the sample rate. The theorem does not preclude the possibility of perfect reconstruction under special circumstances that do not satisfy the sample-rate criterion. It is a sufficient, but not necessary, condition (for examples where perfect reconstruction is possible but the conditions of the sampling theorem are not satisfied, see [[#Sampling of non-baseband signals|Sampling of non-baseband signals]] below, and [[Compressed sensing]]).
| |
| | |
| ==Introduction==
| |
| To formalize the statements above, let ''X''(''f'') be the [[Fourier transform]] of bandlimited function ''x''(''t'')''':'''
| |
| | |
| :<math>X(f)\ \stackrel{\mathrm{def}}{=}\ \int_{-\infty}^{\infty} x(t) \ e^{- i 2 \pi f t} \ {\rm d}t,</math> and <math>X(f) = 0 \quad </math> for all <math>|f| > B.\,</math>
| |
| | |
| When ''x''(''t'') is sampled uniformly at intervals of ''T'' seconds, the resultant sequence is denoted by '''x(nT)''', for all integer values of ''n''. And the sample-rate (samples per second) is''':'''
| |
| | |
| :<math>f_s\ \stackrel{\mathrm{def}}{=}\ 1/T.\,</math>
| |
| | |
| A sufficient condition to reconstruct ''x''(''t'') from its samples is <math>\scriptstyle f_s > 2B\!</math> and equivalently <math>\scriptstyle B < f_s/2.</math> The two thresholds, <math>\scriptstyle 2B</math> and <math>\scriptstyle f_s/2,</math> are respectively called the '''[[Nyquist rate]]''' and '''[[Nyquist frequency]]'''. And respectively, they are attributes of ''x''(''t'') and of the sampling equipment. The condition described by these inequalities is called the '''Nyquist criterion''', or sometimes the ''Raabe condition.'' The theorem is also applicable to functions of other domains, such as ''space,'' in the case of a digitized image. The only change, in the case of other domains, is the units of measure applied to ''t'', ''T'', ''f'', ''f<sub>s</sub>'', and ''B''.
| |
| | |
| One way to ensure the Nyquist criterion is met (as an approximation) is to understand the physics of the origination of ''x''(''t''), and selecting an appropriate rate. For example, sounds that are made by the human voice normally contain relatively insignificant frequency components at or above 10 kHz. Sampling such an audio signal at 20k sam/sec, or more, provides an excellent approximation to meeting the criterion. But often the sample-rate is pre-determined by other considerations, such as an industry standard format (e.g. 8k sam/sec). In this situation, the human voice should be filtered, to reduce to an acceptable level frequency components above 4 kHz, before being sampled. The type of filter required is a [[lowpass filter]], and in this type of application it is referred to as an [[anti-aliasing filter]].
| |
| | |
| [[File:Sinc function (normalized).svg|thumb|right|250px|Fig. 2: The normalized [[sinc function]]: sin(π''x'') / (π''x'') ... showing the central peak at ''x''= 0, and zero-crossings at the other integer values of ''x''.]]
| |
| | |
| Methods that reconstruct a continuous function, from the ''x''(''nT'') sequence, are called [[interpolation]]. As demonstrated below, the mathematically ideal way to reconstruct ''x''(''t'') involves the use of [[sinc function]]s, like those shown in Fig 2. Each sample in the sequence is replaced by a sinc function, centered on the time axis at the original location of the sample (''nT''), with the amplitude of the sinc function scaled to the sample value, ''x''(''nT''). In the following step, all the sinc functions are summed into a continuous function. A mathematically equivalent method is to convolve one sinc function with a series of [[Dirac delta]] pulses, weighted by the sample values. Neither method is numerically practical. Instead, some type of approximation of the sinc functions, finite in length, should be utilized. The imperfections attributable to the approximation are known as ''interpolation error''.
| |
| | |
| Practical [[digital-to-analog converter]]s produce neither scaled and delayed sinc functions, nor ideal Dirac pulses. Instead they produce a [[piecewise-constant]] sequence of scaled and delayed [[rectangular function|rectangular pulses]], usually followed by a "shaping filter" to clean up spurious high-frequency content.
| |
| | |
| ==Aliasing==
| |
| {{main|Aliasing}}
| |
| [[File:CPT-sound-nyquist-thereom-1.5percycle.svg|thumb|250px|Fig. 3: The samples of several different sine waves can be identical, when at least one of them is at a frequency above half the sample rate.]]
| |
| | |
| The [[Poisson summation formula]] shows that the samples, ''x''(''nT''), of function ''x''(''t'') are sufficient to create a [[periodic summation]] of function ''X''(''f''). The result is (T=1/''f<sub>s</sub>''): | |
| | |
| {{NumBlk|:|<math>X_s(f)\ \stackrel{\mathrm{def}}{=} \sum_{k=-\infty}^{\infty} X\left(f - k f_s\right) = \sum_{n=-\infty}^{\infty} \underbrace{T\cdot x(nT)}_{x[n]}\ e^{-i 2\pi n T f},</math>|{{EquationRef|Eq.1}}}}
| |
| which is a periodic function and its equivalent representation as a [[Fourier series]], whose coefficients are ''x''[''n'']. This function is also known as the [[discrete-time Fourier transform]] (DTFT). As depicted in Figures 4 and 5, copies of ''X''(''f'') are shifted by multiples of ''f<sub>s</sub>'' and combined by addition.
| |
| | |
| If the Nyquist criterion is not satisfied, adjacent copies overlap, and it is not possible in general to discern an unambiguous ''X''(''f''). Any frequency component above ''f<sub>s</sub>''/2 is indistinguishable from a lower-frequency component, called an ''alias'', associated with one of the copies. In such cases, the reconstruction technique described below produces the alias, rather than the original component.
| |
| | |
| [[File:AliasedSpectrum.png|thumb|400px|Fig. 4: ''X''(''f'') (top blue) and ''X''<sub>A</sub>(''f'') (bottom blue) are continuous Fourier transforms of two <u>different</u> functions, ''x''(''t'') and ''x''<sub>A</sub>(''t'') (not shown). When the functions are sampled at rate ''f''<sub>s</sub>, the images (green) are added to the original transforms (blue) when one examines the discrete-time Fourier transforms (DTFT) of the sequences. In this hypothetical example, the DTFTs are identical, which means <u>the sampled sequences are identical</u>, even though the original continuous pre-sampled functions are not. If these were audio signals, ''x''(''t'') and ''x''<sub>A</sub>(''t'') might not sound the same. But their samples (taken at rate ''f''<sub>s</sub>) are identical and would lead to identical reproduced sounds; thus ''x''<sub>A</sub>(''t'') is an alias of ''x''(''t'') at this sample rate. In this example (of a bandlimited function), such aliasing can be prevented by increasing ''f''<sub>s</sub> such that the green images in the top figure do not overlap the blue portion.|right]]
| |
| | |
| [[File:ReconstructFilter.png|thumb|right|400px|Fig. 5: Spectrum, ''X<sub>s</sub>''(''f''), of a properly sampled bandlimited signal (blue) and the adjacent DTFT images (green) that do not overlap. A ''brick-wall'' low-pass filter, ''H''(''f''), removes the images, leaves the original spectrum, ''X''(''f''), and recovers the original signal from its samples.]]
| |
| | |
| ==Derivation as a special case of Poisson summation==
| |
| From Figure 5, it is apparent that when there is no overlap of the copies (aka "images") of ''X''(''f''), the ''k'' = 0 term of ''X<sub>s</sub>''(''f'') can be recovered by the product''':'''
| |
| | |
| :<math>X(f) = H(f) \cdot X_s(f),\,</math> where''':'''
| |
| | |
| :<math>H(f)\ \stackrel{\mathrm{def}}{=}\ \begin{cases}1 & |f| < B \\ 0 & |f| > f_s - B. \end{cases}</math>
| |
| | |
| At this point, the sampling theorem is proved, since ''X''(''f'') uniquely determines ''x''(''t'').
| |
| | |
| All that remains is to derive the formula for reconstruction. ''H''(''f'') need not be precisely defined in the region {{nowrap|[''B'', ''f<sub>s</sub>'' − ''B'']}} because ''X''<sub>s</sub>(''f'') is zero in that region. However, the worst case is when ''B'' = ''f<sub>s</sub>''/2, the Nyquist frequency. A function that is sufficient for that and all less severe cases is''':'''
| |
| | |
| :<math>H(f) = \mathrm{rect} \left(\frac{f}{f_s} \right) = \begin{cases}1 & |f| < \frac{f_s}{2} \\ 0 & |f| > \frac{f_s}{2}, \end{cases}</math>
| |
| | |
| where rect(•) is the [[rectangular function]]. Therefore''':'''
| |
| | |
| :<math>X(f) = \mathrm{rect} \left(\frac{f}{f_s} \right) \cdot X_s(f)\ </math>
| |
| :::<math> = \mathrm{rect}(Tf)\cdot \sum_{n=-\infty}^{\infty} T\cdot x(nT)\ e^{-i 2\pi n T f}</math> (from {{EquationNote|Eq.1}}, above).
| |
| :::<math> = \sum_{n=-\infty}^{\infty} x(nT)\cdot \underbrace{T\cdot \mathrm{rect} (Tf) \cdot e^{-i 2\pi n T f}}_{
| |
| \mathcal{F}\left \{
| |
| \mathrm{sinc} \left( \frac{t - nT}{T} \right)
| |
| \right \}
| |
| }.</math> <ref>The sinc function follows from rows 202 and 102 of the [[Table of Fourier transforms|transform tables]]</ref>
| |
| | |
| The inverse transform of both sides produces the [[Whittaker–Shannon interpolation formula]]''':'''
| |
| | |
| :<math>x(t) = \sum_{n=-\infty}^{\infty} x(nT)\cdot \mathrm{sinc} \left( \frac{t - nT}{T}\right),</math>
| |
| | |
| which shows how the samples, ''x''(''nT''), can be combined to reconstruct ''x''(''t'').
| |
| | |
| *From Figure 5, it is clear that larger-than-necessary values of ''f<sub>s</sub>'' (smaller values of ''T''), called ''oversampling'', have no effect on the outcome of the reconstruction and have the benefit of leaving room for a ''transition band'' in which ''H''(''f'') is free to take intermediate values. [[Undersampling]], which causes aliasing, is not in general a reversible operation.
| |
| *Theoretically, the interpolation formula can be implemented as a [[low pass filter]], whose impulse response is sinc(''t''/''T'') and whose input is <math>\textstyle\sum_{n=-\infty}^{\infty} x(nT)\cdot \delta(t - nT),</math> which is a [[Dirac comb]] function modulated by the signal samples. Practical [[digital-to-analog converter]]s (DAC) implement an approximation like the [[zero-order hold]]. In that case, oversampling can reduce the approximation error.
| |
| | |
| ==Shannon's original proof==
| |
| Poisson shows that the Fourier series in {{EquationNote|Eq.1}} produces the periodic summation of ''X''(''f''), regardless of ''f<sub>s</sub>'' and ''B''. Shannon, however, only derives the series coefficients for the case ''f<sub>s</sub>'' = ''2B''. Quoting Shannon's original paper, which uses ''f'' for the function, ''F'' for the spectrum, and ''W'' instead of ''B''''':'''
| |
| | |
| :Let <math>\scriptstyle F(\omega)</math> be the spectrum of <math>\scriptstyle f(t).</math> Then
| |
| | |
| ::{|
| |
| |-
| |
| |<math>f(t)\,</math>
| |
| |<math>= {1 \over 2\pi} \int_{-\infty}^{\infty} F(\omega) e^{i\omega t}\;{\rm d}\omega \ </math>
| |
| |-
| |
| |
| |
| |<math>= {1 \over 2\pi} \int_{-2\pi W}^{2\pi W} F(\omega) e^{i\omega t}\;{\rm d}\omega \ </math>
| |
| |}
| |
| | |
| :since <math>\scriptstyle F(\omega)</math> is assumed to be zero outside the band ''W''. If we let
| |
| | |
| ::<math>t = {n \over {2W}}\,</math>
| |
| | |
| :where ''n'' is any positive or negative integer, we obtain
| |
| | |
| ::<math>f \left({n \over {2W}} \right) = {1 \over 2\pi} \int_{-2\pi W}^{2\pi W} F(\omega) e^{i\omega {n \over {2W}}}\;{\rm d}\omega.</math>
| |
| | |
| :On the left are values of <math>\scriptstyle f(t)</math> at the sampling points. The integral on the right will be recognized as essentially<ref group="note">The actual coefficient formula contains an additional factor of <math>\scriptstyle 1/2W = T.</math> So Shannon's coefficients are <math>\scriptstyle T\cdot f(nT),</math> which agrees with {{EquationNote|Eq.1}}.</ref> the ''n''<sup>th</sup> coefficient in a Fourier-series expansion of the function <math>\scriptstyle F(\omega),</math> taking the interval –''W'' to ''W'' as a fundamental period. This means that the values of the samples <math>\scriptstyle f(n/2W)</math> determine the Fourier coefficients in the series expansion of <math>\scriptstyle F(\omega).</math> Thus they determine <math>\scriptstyle F(\omega),</math> since <math>\scriptstyle F(\omega)</math> is zero for frequencies greater than ''W'', and for lower frequencies <math>\scriptstyle F(\omega)</math> is determined if its Fourier coefficients are determined. But <math>\scriptstyle F(\omega)</math> determines the original function <math>\scriptstyle f(t)</math> completely, since a function is determined if its spectrum is known. Therefore the original samples determine the function <math>\scriptstyle f(t)</math> completely.
| |
| Shannon's proof of the theorem is complete at that point, but he goes on to discuss reconstruction via sinc functions, what we now call the [[Whittaker–Shannon interpolation formula]] as discussed above. He does not derive or prove the properties of the [[sinc function]], but these would have been familiar to engineers reading his works at the time, since the Fourier pair relationship between [[rectangular function|rect]] (the rectangular function) and sinc was well known. Quoting Shannon:
| |
| | |
| :Let <math>\scriptstyle x_n</math> be the ''n''<sup>th</sup> sample. Then the function <math>\scriptstyle f(t)</math> is represented by:
| |
| | |
| ::<math>f(t) = \sum_{n=-\infty}^{\infty}x_n{\sin \pi(2Wt-n) \over \pi(2Wt-n)}.\,</math>
| |
| | |
| As in the other proof, the existence of the Fourier transform of the original signal is assumed, so the proof does not say whether the sampling theorem extends to bandlimited stationary random processes.
| |
| | |
| ===Notes===
| |
| {{reflist|group=note}}
| |
| | |
| ==Application to multivariable signals and images==
| |
| {{main|Multidimensional sampling}}
| |
| [[File:Moire pattern of bricks small.jpg|thumb|right|205px|Fig. 6: Subsampled image showing a [[Moiré pattern]]]]
| |
| | |
| [[File:Moire pattern of bricks.jpg|thumb|right|205px|Fig. 7]]
| |
| | |
| The sampling theorem is usually formulated for functions of a single variable. Consequently, the theorem is directly applicable to time-dependent signals and is normally formulated in that context. However, the sampling theorem can be extended in a straightforward way to functions of arbitrarily many variables. Grayscale images, for example, are often represented as two-dimensional arrays (or matrices) of real numbers representing the relative intensities of [[pixel]]s (picture elements) located at the intersections of row and column sample locations. As a result, images require two independent variables, or indices, to specify each pixel uniquely — one for the row, and one for the column.
| |
| | |
| Color images typically consist of a composite of three separate grayscale images, one to represent each of the three primary colors — red, green, and blue, or ''RGB'' for short. Other colorspaces using 3-vectors for colors include HSV, CIELAB, XYZ, etc. Some colorspaces such as cyan, magenta, yellow, and black (CMYK) may represent color by four dimensions. All of these are treated as [[vector-valued function]]s over a two-dimensional sampled domain.
| |
| | |
| Similar to one-dimensional discrete-time signals, images can also suffer from aliasing if the sampling resolution, or pixel density, is inadequate. For example, a digital photograph of a striped shirt with high frequencies (in other words, the distance between the stripes is small), can cause aliasing of the shirt when it is sampled by the camera's [[image sensor]]. The aliasing appears as a [[moiré pattern]]. The "solution" to higher sampling in the spatial domain for this case would be to move closer to the shirt, use a higher resolution sensor, or to optically blur the image before acquiring it with the sensor.
| |
| | |
| Another example is shown to the right in the brick patterns. The top image shows the effects when the sampling theorem's condition is not satisfied. When software rescales an image (the same process that creates the thumbnail shown in the lower image) it, in effect, runs the image through a low-pass filter first and then [[downsampling|downsamples]] the image to result in a smaller image that does not exhibit the [[moiré pattern]]. The top image is what happens when the image is downsampled without low-pass filtering: aliasing results.
| |
| | |
| The application of the sampling theorem to images should be made with care. For example, the sampling process in any standard image sensor (CCD or CMOS camera) is relatively far from the ideal sampling which would measure the image intensity at a single point. Instead these devices have a relatively large sensor area at each sample point in order to obtain sufficient amount of light. In other words, any detector has a finite-width [[point spread function]]. The analog optical image intensity function which is sampled by the sensor device is not in general bandlimited, and the non-ideal sampling is itself a useful type of low-pass filter, though not always sufficient to remove enough high frequencies to sufficiently reduce aliasing. When the area of the sampling spot (the size of the pixel sensor) is not large enough to provide sufficient [[spatial anti-aliasing]], a separate anti-aliasing filter (optical low-pass filter) is typically included in a camera system to further blur the optical image. Despite images having these problems in relation to the sampling theorem, the theorem can be used to describe the basics of down and up sampling of images.
| |
| | |
| [[File:CriticalFrequencyAliasing.svg|thumb|right|Fig. 8: A family of sinusoids at the critical frequency, all having the same sample sequences of alternating +1 and –1. That is, they all are aliases of each other, even though their frequency is not above half the sample rate.]]
| |
| | |
| ==Critical frequency==
| |
| To illustrate the necessity of ''f<sub>s</sub>'' > 2''B'', consider the family of sinusoids (depicted in Fig. 8) generated by different values of θ in this formula''':'''
| |
| | |
| :<math>x(t) = \frac{\cos(2 \pi B t + \theta )}{\cos(\theta )}\ = \ \cos(2 \pi B t) - \sin(2 \pi B t)\tan(\theta ), \quad -\pi/2 < \theta < \pi/2.</math>
| |
| | |
| With ''f<sub>s</sub>'' = 2''B'' or equivalently ''T'' = 1/(2''B''), the samples are given by''':'''
| |
| | |
| :<math>x(nT) = \cos(\pi n) - \underbrace{\sin(\pi n)}_{0}\tan(\theta ) = (-1)^n</math>
| |
| | |
| <u>regardless of the value of θ</u>. That sort of ambiguity is the reason for the ''strict'' inequality of the sampling theorem's condition.
| |
| | |
| ==Sampling of non-baseband signals==
| |
| As discussed by Shannon:<ref name="Shannon49"/>
| |
| | |
| :<blockquote>A similar result is true if the band does not start at zero frequency but at some higher value, and can be proved by a linear translation (corresponding physically to [[single-sideband modulation]]) of the zero-frequency case. In this case the elementary pulse is obtained from sin(''x'')/''x'' by single-side-band modulation.</blockquote>
| |
| | |
| That is, a sufficient no-loss condition for sampling [[signal (information theory)|signal]]s that do not have [[baseband]] components exists that involves the ''width'' of the non-zero frequency interval as opposed to its highest frequency component. See ''[[Sampling (signal processing)]]'' for more details and examples.
| |
| | |
| A bandpass condition is that ''X''(''f'') = 0, for all nonnegative ''f'' outside the open band of frequencies:
| |
| ::<math> \left(\frac{N}2f_\mathrm{s},\frac{N+1}2f_\mathrm{s}\right), </math>
| |
| for some nonnegative integer ''N''. This formulation includes the normal baseband condition as the case ''N''=0.
| |
| | |
| The corresponding interpolation function is the impulse response of an ideal brick-wall [[bandpass filter]] (as opposed to the ideal [[brick-wall filter|brick-wall]] [[lowpass filter]] used above) with cutoffs at the upper and lower edges of the specified band, which is the difference between a pair of lowpass impulse responses: | |
| | |
| ::<math>(N+1)\,\operatorname{sinc} \left(\frac{(N+1)t}T\right) - N\,\operatorname{sinc}\left( \frac{Nt}T \right).</math>
| |
| | |
| Other generalizations, for example to signals occupying multiple non-contiguous bands, are possible as well. Even the most generalized form of the sampling theorem does not have a provably true converse. That is, one cannot conclude that information is necessarily lost just because the conditions of the sampling theorem are not satisfied; from an engineering perspective, however, it is generally safe to assume that if the sampling theorem is not satisfied then information will most likely be lost.
| |
| | |
| ==Nonuniform sampling==
| |
| The sampling theory of Shannon can be generalized for the case of [[nonuniform sampling]], that is, samples not taken equally spaced in time. The Shannon sampling theory for non-uniform sampling states that a band-limited signal can be perfectly reconstructed from its samples if the average sampling rate satisfies the Nyquist condition.<ref>Nonuniform Sampling, Theory and Practice (ed. F. Marvasti), Kluwer Academic/Plenum Publishers, New York, 2000</ref> Therefore, although uniformly spaced samples may result in easier reconstruction algorithms, it is not a necessary condition for perfect reconstruction.
| |
| | |
| The general theory for non-baseband and nonuniform samples was developed in 1967 by [[Henry Landau|Landau]].<ref>{{cite journal |first=H. J. |last=Landau |title=Necessary density conditions for sampling and interpolation of certain entire functions |journal=Acta Math. |volume=117 |issue=1 |pages=37–52 |year=1967 |doi=10.1007/BF02395039 }}</ref> He proved that, to paraphrase roughly, the average sampling rate (uniform or otherwise) must be twice the ''occupied'' bandwidth of the signal, assuming it is ''a priori'' known what portion of the spectrum was occupied.
| |
| In the late 1990s, this work was partially extended to cover signals of when the amount of occupied bandwidth was known, but the actual occupied portion of the spectrum was unknown.<ref>see, e.g., {{cite book |first=P. |last=Feng |title=Universal minimum-rate sampling and spectrum-blind reconstruction for multiband signals |publisher=Ph.D. dissertation, University of Illinois at Urbana-Champaign |year=1997 }}</ref> In the 2000s, a complete theory was developed
| |
| (see the section [[Nyquist–Shannon sampling theorem#Beyond Nyquist|Beyond Nyquist]] below) using [[compressed sensing]]. In particular, the theory, using signal processing language, is described in this 2009 paper.<ref>{{cite journal | id = {{citeseerx|10.1.1.154.4255}} | title = Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals | first1 = Moshe | last1 = Mishali | first2 = Yonina C. | last2 = Eldar | journal = IEEE Trans. Signal Processing | month = March | year = 2009 | volume = 57 | issue = 3 }}</ref> They show, among other things, that if the frequency locations are unknown, then it is necessary to sample at least at twice the Nyquist criteria; in other words, you must pay at least a factor of 2 for not knowing the location of the [[spectrum]]. Note that minimum sampling requirements do not necessarily guarantee [[Numerical stability|stability]].
| |
| | |
| ==Sampling below the Nyquist rate under additional restrictions==
| |
| {{Main|Undersampling}}
| |
| The Nyquist–Shannon sampling theorem provides a [[necessary and sufficient condition|sufficient condition]] for the sampling and reconstruction of a band-limited signal. When reconstruction is done via the [[Whittaker–Shannon interpolation formula]], the Nyquist criterion is also a necessary condition to avoid aliasing, in the sense that if samples are taken at a slower rate than twice the band limit, then there are some signals that will not be correctly reconstructed. However, if further restrictions are imposed on the signal, then the Nyquist criterion may no longer be a [[necessary and sufficient condition|necessary condition]].
| |
| | |
| A non-trivial example of exploiting extra assumptions about the signal is given by the recent field of [[compressed sensing]], which allows for full reconstruction with a sub-Nyquist sampling rate. Specifically, this applies to signals that are sparse (or compressible) in some domain. As an example, compressed sensing deals with signals that may have a low over-all bandwidth (say, the ''effective'' bandwidth ''EB''), but the frequency locations are unknown, rather than all together in a single band, so that the [[Nyquist–Shannon sampling theorem#Sampling of non-baseband signals|passband technique]] doesn't apply. In other words, the frequency spectrum is sparse. Traditionally, the necessary sampling rate is thus 2''B''. Using compressed sensing techniques, the signal could be perfectly reconstructed if it is sampled at a rate slightly lower than 2''EB''. The downside of this approach is that reconstruction is no longer given by a formula, but instead by the solution to a [[Convex optimization|convex optimization program]] which requires well-studied but nonlinear methods.
| |
| | |
| ==Historical background==
| |
| The sampling theorem was implied by the work of [[Harry Nyquist]] in 1928 ("Certain topics in telegraph transmission theory"), in which he showed that up to 2''B'' independent pulse samples could be sent through a system of bandwidth ''B''; but he did not explicitly consider the problem of sampling and reconstruction of continuous signals. About the same time, [[Karl Küpfmüller]] showed a similar result,<ref>{{cite journal |first=Karl |last=Küpfmüller |title=Über die Dynamik der selbsttätigen Verstärkungsregler |journal=Elektrische Nachrichtentechnik |volume=5 |issue=11 |pages=459–467 |year=1928 }} {{de icon}} [http://ict.open.ac.uk/classics/2.pdf (English translation 2005)].</ref> and discussed the sinc-function impulse response of a band-limiting filter, via its integral, the step response ''[[sine integral|Integralsinus]]''; this bandlimiting and reconstruction filter that is so central to the sampling theorem is sometimes referred to as a ''Küpfmüller filter'' (but seldom so in English).
| |
| | |
| The sampling theorem, essentially a [[duality (mathematics)|dual]] of Nyquist's result, was proved by [[Claude E. Shannon]] in 1949 ("Communication in the presence of noise").
| |
| [[V. A. Kotelnikov]] published similar results in 1933 ("On the transmission capacity of the 'ether' and of cables in electrical communications", translation from the Russian), as did the mathematician [[E. T. Whittaker]] in 1915 ("Expansions of the Interpolation-Theory", "Theorie der Kardinalfunktionen"), J. M. Whittaker in 1935 ("Interpolatory function theory"), and [[Dennis Gabor|Gabor]] in 1946 ("Theory of communication").
| |
| | |
| ===Other discoverers===
| |
| Others who have independently discovered or played roles in the development of the sampling theorem have been discussed in several historical articles, for example by Jerri<ref>[[Abdul Jerri]], [http://web.archive.org/web/20080605171238/http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=1455040 The Shannon Sampling Theorem—Its Various Extensions and Applications: A Tutorial Review], ''Proceedings of the IEEE'', 65:1565–1595, Nov. 1977. See also [http://web.archive.org/web/20090120203809/http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=1455576 Correction to "The Shannon sampling theorem—Its various extensions and applications: A tutorial review"], Proceedings of the IEEE, 67:695, April 1979</ref> and by Lüke.<ref>Hans Dieter Lüke, {{doi-inline|10.1109/35.755459|The Origins of the Sampling Theorem}}, ''IEEE Communications Magazine,'' pp.106–108, April 1999.</ref> For example, Lüke points out that H. Raabe, an assistant to Küpfmüller, proved the theorem in his 1939 Ph.D. dissertation; the term ''Raabe condition'' came to be associated with the criterion for unambiguous representation (sampling rate greater than twice the bandwidth).
| |
| | |
| Meijering<ref name="EM">Erik Meijering, {{doi-inline|10.1109/5.993400|A Chronology of Interpolation From Ancient Astronomy to Modern Signal and Image Processing}}, ''[[Proc. IEEE]],'' 90, 2002.</ref> mentions several other discoverers and names in a paragraph and pair of footnotes:
| |
| | |
| <blockquote>
| |
| As pointed out by Higgins [135], the sampling theorem should really be considered in two parts, as done above: the first stating the fact that a bandlimited function is completely determined by its samples, the second describing how to reconstruct the function using its samples. Both parts of the sampling theorem were given in a somewhat different form by J. M. Whittaker [350, 351, 353] and before him also by Ogura [241, 242]. They were probably not aware of the fact that the first part of the theorem had been stated as early as 1897 by Borel [25].<sup>27</sup> As we have seen, Borel also used around that time what became known as the cardinal series. However, he appears not to have made the link [135]. In later years it became known that the sampling theorem had been presented before Shannon to the Russian communication community by Kotel'nikov [173]. In more implicit, verbal form, it had also been described in the German literature by Raabe [257]. Several authors [33, 205] have mentioned that Someya [296] introduced the theorem in the Japanese literature parallel to Shannon. In the English literature, Weston [347] introduced it independently of Shannon around the same time.<sup>28</sup>
| |
| </blockquote>
| |
| <blockquote>
| |
| <sup>27</sup> Several authors, following Black [16], have claimed that this first part of the sampling theorem was stated even earlier by Cauchy, in a paper [41] published in 1841. However, the paper of Cauchy does not contain such a statement, as has been pointed out by Higgins [135].
| |
| </blockquote>
| |
| <blockquote>
| |
| <sup>28</sup> As a consequence of the discovery of the several independent introductions of the sampling theorem, people started to refer to the theorem by including the names of the aforementioned authors, resulting in such catchphrases as “the Whittaker-Kotel’nikov-Shannon (WKS) sampling theorem" [155] or even "the Whittaker-Kotel'nikov-Raabe-Shannon-Someya sampling theorem" [33]. To avoid confusion, perhaps the best thing to do is to refer to it as the sampling theorem, "rather than trying to find a title that does justice to all claimants" [136].
| |
| </blockquote>
| |
| | |
| ===Why Nyquist?===
| |
| Exactly how, when, or why [[Harry Nyquist]] had his name attached to the sampling theorem remains obscure. The term ''Nyquist Sampling Theorem'' (capitalized thus) appeared as early as 1959 in a book from his former employer, [[Bell Labs]],<ref>{{cite book | title = Transmission Systems for Communications | author = Members of the Technical Staff of Bell Telephone Lababoratories | year = 1959 | publisher = AT&T | pages = 26–4 (Vol.2)}}</ref> and appeared again in 1963,<ref>{{cite book | title = Theory of Linear Physical Systems | publisher = Wiley | year = 1963 | url = http://books.google.com/books?id=jtI-AAAAIAAJ | author = Ernst Adolph Guillemin }}</ref> and not capitalized in 1965.<ref>{{cite book |first=Richard A. |last=Roberts |first2=Ben F. |last2=Barton |title=Theory of Signal Detectability: Composite Deferred Decision Theory |year=1965 }}</ref> It had been called the ''Shannon Sampling Theorem'' as early as 1954,<ref>{{cite book |first=Truman S. |last=Gray |title=Applied Electronics: A First Course in Electronics, Electron Tubes, and Associated Circuits |year=1954 }}</ref> but also just ''the sampling theorem'' by several other books in the early 1950s.
| |
| | |
| In 1958, Blackman and Tukey cited Nyquist's 1928 paper as a reference for ''the sampling theorem of information theory'',<ref>{{cite book |first=R. B. |last=Blackman |first2=J. W. |last=Tukey |title=The Measurement of Power Spectra : From the Point of View of Communications Engineering |location=New York |publisher=Dover |year=1958 }}</ref> even though that paper does not treat sampling and reconstruction of continuous signals as others did. Their glossary of terms includes these entries:
| |
| | |
| :''Sampling theorem'' (''of information theory'')
| |
| | |
| : Nyquist's result that equi-spaced data, with two or more points per cycle of highest frequency, allows reconstruction of band-limited functions. (See ''Cardinal theorem.'')
| |
| | |
| :''Cardinal theorem'' (''of interpolation theory'')
| |
| | |
| : A precise statement of the conditions under which values given at a doubly infinite set of equally spaced points can be interpolated to yield a continuous band-limited function with the aid of the function
| |
| | |
| ::<math>\frac{\sin (x - x_i)}{x - x_i}.</math>
| |
| | |
| Exactly what "Nyquist's result" they are referring to remains mysterious.
| |
| | |
| When Shannon stated and proved the sampling theorem in his 1949 paper, according to Meijering<ref name="EM" /> "he referred to the critical sampling interval ''T'' = 1/(2''W'') as the ''Nyquist interval'' corresponding to the band ''W'', in recognition of Nyquist’s discovery of the fundamental importance of this interval in connection with telegraphy." This explains Nyquist's name on the critical interval, but not on the theorem.
| |
| | |
| Similarly, Nyquist's name was attached to ''[[Nyquist rate]]'' in 1953 by [[Harold Stephen Black|Harold S. Black]]:<ref>{{cite book |first=Harold S. |last=Black |title=Modulation Theory |year=1953 }}</ref>
| |
| | |
| :"If the essential frequency range is limited to ''B'' cycles per second, 2''B'' was given by Nyquist as the maximum number of code elements per second that could be unambiguously resolved, assuming the peak interference is less half a quantum step. This rate is generally referred to as '''signaling at the Nyquist rate''' and 1/(2''B'') has been termed a ''Nyquist interval''." (bold added for emphasis; italics as in the original)
| |
| | |
| According to the [[OED]], this may be the origin of the term ''Nyquist rate''. In Black's usage, it is not a sampling rate, but a signaling rate.
| |
| | |
| ==See also==
| |
| {{commons category|Nyquist Shannon theorem}}
| |
| * [[Balian–Low theorem]], a similar theoretical lower-bound on sampling rates, but which applies to [[Time–frequency analysis|time–frequency transforms]].
| |
| * The [[Cheung–Marks theorem]] specifies conditions where restoration of a signal by the sampling theorem can become ill-posed.
| |
| * [[Hartley's law]]
| |
| * [[Nyquist ISI criterion]]
| |
| * [[Reconstruction from zero crossings]]
| |
| * [[Zero-order hold]]
| |
| | |
| ==Notes==
| |
| {{Reflist|colwidth=30em}}
| |
| | |
| ==References==
| |
| * J. R. Higgins: ''Five short stories about the cardinal series'', Bulletin of the AMS 12(1985)
| |
| *[[V. A. Kotelnikov]], "On the carrying capacity of the ether and wire in telecommunications", Material for the First All-Union Conference on Questions of Communication, Izd. Red. Upr. Svyazi RKKA, Moscow, 1933 (Russian). [http://ict.open.ac.uk/classics/1.pdf (english translation, PDF)]
| |
| *[[Karl Küpfmüller]], "Utjämningsförlopp inom Telegraf- och Telefontekniken", ("Transients in telegraph and telephone engineering"), Teknisk Tidskrift, no. 9 pp. 153–160 and 10 pp. 178–182, 1931. [http://runeberg.org/tektid/1931e/0157.html] [http://runeberg.org/tektid/1931e/0182.html]
| |
| * R.J. Marks II: ''[http://marksmannet.com/RobertMarks/REPRINTS/1999_IntroductionToShannonSamplingAndInterpolationTheory.pdf Introduction to Shannon Sampling and Interpolation Theory]'', Spinger-Verlag, 1991.
| |
| * R.J. Marks II, Editor: [http://marksmannet.com/RobertMarks/REPRINTS/1993_AdvancedTopicsOnShannon.pdf Advanced Topics in Shannon Sampling and Interpolation Theory], Springer-Verlag, 1993.
| |
| * R.J. Marks II, ''Handbook of Fourier Analysis and Its Applications,'' Oxford University Press, (2009), Chapters 5-8. [http://books.google.com/books?id=Sp7O4bocjPAC Google books].
| |
| *[[Harry Nyquist|H. Nyquist]], "Certain topics in telegraph transmission theory", Trans. AIEE, vol. 47, pp. 617–644, Apr. 1928 [http://replay.web.archive.org/20060706192816/http://www.loe.ee.upatras.gr/Comes/Notes/Nyquist.pdf Reprint as classic paper in: ''Proc. IEEE, Vol. 90, No. 2, Feb 2002].
| |
| *{{Citation | last1=Press | first1=WH | last2=Teukolsky | first2=SA | last3=Vetterling | first3=WT | last4=Flannery | first4=BP | year=2007 | title=Numerical Recipes: The Art of Scientific Computing | edition=3rd | publisher=Cambridge University Press | publication-place=New York | isbn=978-0-521-88068-8 | chapter=Section 13.11. Numerical Use of the Sampling Theorem | chapter-url=http://apps.nrbook.com/empanel/index.html#pg=717}}
| |
| *[[Claude E. Shannon|C. E. Shannon]], "Communication in the presence of noise", Proc. Institute of Radio Engineers, vol. 37, no.1, pp. 10–21, Jan. 1949. [http://www.stanford.edu/class/ee104/shannonpaper.pdf Reprint as classic paper in: ''Proc. IEEE'', Vol. 86, No. 2, (Feb 1998)]
| |
| * Michael Unser: ''[http://bigwww.epfl.ch/publications/unser0001.html Sampling-50 Years after Shannon]'', Proc. IEEE, vol. 88, no. 4, pp. 569–587, April 2000
| |
| *[[E. T. Whittaker]], "On the Functions Which are Represented by the Expansions of the Interpolation Theory", Proc. Royal Soc. Edinburgh, Sec. A, vol.35, pp. 181–194, 1915
| |
| *[[J. M. Whittaker]], ''Interpolatory Function Theory'', Cambridge Univ. Press, Cambridge, England, 1935.
| |
| | |
| ==External links==
| |
| *[http://www.vias.org/simulations/simusoft_nykvist.html Learning by Simulations] Interactive simulation of the effects of inadequate sampling
| |
| *[http://spazioscuola.altervista.org/UndersamplingAR/UndersamplingARnv.htm Undersampling and an application of it]
| |
| *[http://web.archive.org/web/20060614125302/http://www.lavryengineering.com/documents/Sampling_Theory.pdf Sampling Theory For Digital Audio]
| |
| *[http://www.stsip.org/ Journal devoted to Sampling Theory]
| |
| *[http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5699377&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5699377 Sampling Theorem with Constant Amplitude Variable Width Pulse]
| |
| *{{cite paper | id = {{citeseerx|10.1.1.163.2887}} | title = "The Origins of the Sampling Theorem" by Hans Dieter Lüke published in "IEEE Communications Magazine" April 1999 }}
| |
| | |
| {{DSP}}
| |
| {{Compression Methods}}
| |
| | |
| {{DEFAULTSORT:Nyquist-Shannon Sampling Theorem}}
| |
| [[Category:Digital signal processing]]
| |
| [[Category:Information theory]]
| |
| [[Category:Theorems in Fourier analysis]]
| |
| [[Category:Articles containing proofs]]
| |
| [[Category:Mathematical theorems in theoretical computer science]]
| |