Bicubic interpolation: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
en>Yobot
m WP:CHECKWIKI error fixes using AWB (10514)
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
{{multiple issues|
The author is recognized by the name of Numbers Wunder. One of the very very best issues in the globe for him is to gather badges but he is having difficulties to find time for it. For many years I've been working as a payroll clerk. South Dakota is her beginning place but she needs to move simply because of her family.<br><br>Here is my web site [http://www.1t4u.com/blog/5640 std testing at home]
{{refimprove|date=February 2011}}
{{technical|date=August 2012}}
}}
 
'''Particle filters''' or '''Sequential Monte Carlo''' (SMC) methods are a set of on-line posterior [[density estimation]] algorithms that estimate the posterior density of the state-space by directly implementing the [[Recursive Bayesian estimation|Bayesian recursion]] equations. SMC methods use a grid-based approach, and use a set of particles to represent the posterior density. These filtering methods make no restrictive assumption about the dynamics of the state-space or the density function. SMC methods provide a well-established methodology for generating samples from the required distribution without requiring assumptions about the state-space model or the state distributions. The state-space model can be non-linear and the initial state and noise distributions can take any form required. However, these methods do not perform well when applied to high-dimensional systems. SMC methods implement the Bayesian recursion equations directly by using an ensemble based approach. The samples from the distribution are represented by a set of particles; each particle has a weight assigned to it that represents the probability of that particle being sampled from the probability density function.
 
Weight disparity leading to weight collapse is a common issue encountered in these filtering algorithms; however it can be mitigated by including a resampling step before the weights become too uneven. In the resampling step, the particles with negligible weights are replaced by new particles in the proximity of the particles with higher weights.       
 
==History==
The first traces of particle filters date back to the 50's; the 'Poor Man's Monte Carlo', that was proposed by Hammersley et al., in 1954, contained hints of the SMC methods used today. Later in the 70's, similar attempts were made in the control community. However it was in 1993, that Gordon et al., published their seminal work 'A novel Approach to nonlinear/non-Gaussian Bayesian State estimation', that provided the first true implementation of the SMC methods used today. The authors named their algorithm 'the bootstrap filter', and demonstrated that compared to other filtering methods, their algorithm does not require any assumption about that state-space or the noise of the system.
 
==Objective==
The objective of a particle filter is to estimate the posterior density of the state variables given the observation variables. The particle filter is designed for a [[hidden Markov Model]], where the system consists of hidden and observable variables. The observable variables (observation process) is related to the hidden variables (state-process) by some functional form that is known. Similarly the dynamical system describing the evolution of the state variables is also known probabilistically.
 
A generic particle filter estimates the posterior distribution of the hidden states using the observation measurement process. Consider a state-space shown in the diagram (Figure 2).
[[File:Hmm temporal bayesian net.svg|thumb|587x587px|Figure 2. The State-Space (The Hidden states given by the vector '''''x''''', and the observation states given by the vector '''''y''''')]]
The objective of the particle filter is to estimate the values of the hidden states '''''x''''', given the values of the observation process ''''' y'''''.
 
The particle filter aims to estimate the sequence of hidden parameters, ''x''<sub>''k''</sub> for ''k''&nbsp;=&nbsp;0,1,2,3,…, based only on the observed data ''y''<sub>''k''</sub> for ''k''&nbsp;=&nbsp;0,1,2,3,….  All Bayesian estimates of ''x''<sub>''k''</sub> follow from the [[posterior distribution]] ''p''(''x''<sub>''k''</sub>&nbsp;|&nbsp;''y''<sub>0</sub>,''y''<sub>1</sub>,…,''y''<sub>''k''</sub>). In contrast, the MCMC or importance sampling approach would model the full posterior ''p''(''x''<sub>0</sub>,''x''<sub>1</sub>,…,''x''<sub>''k''</sub>&nbsp;|&nbsp;''y''<sub>0</sub>,''y''<sub>1</sub>,…,''y''<sub>''k''</sub>).
 
==Model==
Particle methods assume <math>x_k</math> and the observations <math>y_k</math> can be modeled in this form:
 
*<math>x_0, x_1, \ldots</math> is a first order [[Markov process]] such that
*:<math>x_k|x_{k-1} \sim p_{x_k|x_{k-1}}(x|x_{k-1})</math>
and with an initial distribution <math>p(x_0)</math>.
*The observations <math>y_0, y_1, \ldots</math> are conditionally independent provided that <math>x_0, x_1, \ldots</math> are known
:In other words, each <math>y_k</math> only depends on <math>x_k</math>
::<math>y_k|x_k \sim p_{y|x}(y|x_k)</math>
 
One example form of this scenario is
 
:<math>x_k = g(x_{k-1}) + w_k \,</math>
:<math>y_k = h(x_k) + v_k \,</math>
 
where both <math>w_k</math> and <math>v_k</math> are mutually independent and identically distributed sequences with known [[probability density function]]s and <math>g(\cdot)</math> and <math>h(\cdot)</math> are known functions.
These two equations can be viewed as [[state space (controls)|state space]] equations and look similar to the state space equations for the Kalman filter.  If the functions <math>g(\cdot)</math> and <math>h(\cdot)</math> are linear, and if both <math>w_k</math> and <math>v_k</math> are [[Gaussian]], the Kalman filter finds the exact Bayesian filtering distribution.  If not, Kalman filter based methods are a first-order approximation (EKF) or a second-order approximation (UKF in general, but if probability distribution is Gaussian a third-order approximation is possible).  Particle filters are also an approximation, but with enough particles they can be much more accurate.
 
==Monte Carlo approximation==
Particle methods, like all sampling-based approaches (e.g., [[Markov chain Monte Carlo|MCMC]]), generate a set of samples that approximate the filtering distribution <math>p(x_k|y_0,\dots,y_k)</math>.  So, with <math>P</math> samples, expectations with respect to the filtering distribution are approximated by
 
:<math>\int f(x_k)p(x_k|y_0,\dots,y_k) \, dx_k\approx\frac1P\sum_{L=1}^Pf(x_k^{(L)})</math>
 
and <math>f(\cdot)</math>, in the usual way for Monte Carlo, can give all the [[moment (mathematics)|moments]] etc. of the distribution up to some degree of approximation.
 
== Sequential importance resampling (SIR) ==
''Sequential importance [[Resampling (statistics)|resampling]] (SIR)'', the original particle filtering algorithm (Gordon et al. 1993), is a very commonly used
particle filtering algorithm, which approximates the filtering
distribution <math>p(x_k|y_0,\ldots,y_k)</math> by a weighted set
of P particles
 
: <math>\{(w^{(L)}_k,x^{(L)}_k)~:~L\in\{1,\ldots,P\}\}.</math>
 
The ''importance weights'' <math>w^{(L)}_k</math> are approximations to
the relative posterior probabilities (or densities) of the particles
such that <math>\sum_{L=1}^P w^{(L)}_k = 1</math>.
 
SIR is a sequential (i.e., recursive) version of [[importance sampling]].
As in importance sampling, the expectation of a function
<math>f(\cdot)</math> can be approximated as a weighted average
 
: <math>
\int f(x_k) p(x_k|y_0,\dots,y_k) dx_k \approx
\sum_{L=1}^P w^{(L)} f(x_k^{(L)}).
</math>
 
For a finite set of particles, the algorithm performance is dependent on the choice of the
''proposal distribution''
 
: <math>\pi(x_k|x_{0:k-1},y_{0:k})\, </math>.
 
The ''optimal proposal distribution'' is given as the ''target distribution''
: <math>
\pi(x_k|x_{0:k-1},y_{0:k}) = p(x_k|x_{k-1},y_{k}). \,
</math>
 
However, the transition prior probability distribution is often used as importance function, since it is easier to draw particles (or samples) and perform subsequent importance weight calculations:
: <math>
\pi(x_k|x_{0:k-1},y_{0:k}) = p(x_k|x_{k-1}). \,
</math>
''Sequential Importance Resampling'' (SIR) filters with transition prior probability distribution as importance function are commonly known as [[Resampling (statistics)#Bootstrap|bootstrap filter]] and [[condensation algorithm]].
 
''Resampling'' is used to avoid the problem of degeneracy of the
algorithm, that is, avoiding the situation that all but one of the
importance weights are close to zero. The performance of the algorithm
can be also affected by proper choice of resampling method. The
''[[stratified sampling]]'' proposed by Kitagawa (1996) is optimal in
terms of variance.
 
A single step of sequential importance resampling is as follows:
 
:1) For <math>L=1,\ldots,P</math> draw samples from the ''proposal distribution''
 
:: <math>
x^{(L)}_k \sim \pi(x_k|x^{(L)}_{0:k-1},y_{0:k})
</math>
 
:2) For <math>L=1,\ldots,P</math> update the importance weights up to a normalizing constant:
 
: <math>
\hat{w}^{(L)}_k = w^{(L)}_{k-1}
\frac{p(y_k|x^{(L)}_k) p(x^{(L)}_k|x^{(L)}_{k-1})}
{\pi(x_k^{(L)}|x^{(L)}_{0:k-1},y_{0:k})}.
</math>
:: Note that when we use the transition prior probability distribution as the importance function, <math> \pi(x_k^{(L)}|x^{(L)}_{0:k-1},y_{0:k}) = p(x^{(L)}_k|x^{(L)}_{k-1})</math>, this simplifies to the following  :
::: <math> \hat{w}^{(L)}_k = w^{(L)}_{k-1} p(y_k|x^{(L)}_k), </math>
 
:3) For <math>L=1,\ldots,P</math> compute the normalized importance weights:
 
:: <math>
w^{(L)}_k = \frac{\hat{w}^{(L)}_k}{\sum_{J=1}^P \hat{w}^{(J)}_k}
</math>
 
:4) Compute an estimate of the effective number of particles as
 
:: <math>
\hat{N}_\mathit{eff} = \frac{1}{\sum_{L=1}^P\left(w^{(L)}_k\right)^2}
</math>
 
:5) If the effective number of particles is less than a given threshold <math>\hat{N}_\mathit{eff} < N_{thr}</math>, then perform resampling:
 
::a) Draw <math>P</math> particles from the current particle set with probabilities proportional to their weights. Replace the current particle set with this new one.
 
::b) For <math>L=1,\ldots,P</math> set <math>w^{(L)}_k = 1/P.</math>
 
The term ''Sampling Importance Resampling'' is also sometimes used when referring to SIR filters.
 
== Sequential importance sampling (SIS) ==
* Is the same as sequential importance resampling, but without the resampling stage.
 
== "direct version" algorithm==
{{confusing section|date=October 2011}}
The "direct version" algorithm {{citation needed|date=October 2011}} is rather simple (compared to other particle filtering algorithms) and it uses composition and rejection.
To generate a single sample <math>x</math> at <math>k</math> from <math>p_{x_k|y_{1:k}}(x|y_{1:k})</math>:
 
:1) Set n=0 (This will count the number of particles generated so far)
 
:2) [[Uniform distribution (discrete)|Uniformly]] choose an index L from the range <math>\{1,..., P\}</math>
 
:3) Generate a test <math>\hat{x}</math> from the distribution <math>p_{x_k|x_{k-1}}(x|x_{k-1|k-1}^{(L)})</math>
 
:4) Generate the probability of <math>\hat{y}</math> using <math>\hat{x}</math> from <math>p_{y|x}(y_k|\hat{x})</math> where <math>y_k</math> is the measured value
 
:5) Generate another [[Uniform distribution (continuous)|uniform]] u from <math>[0, m_k]</math> where <math>m_k = \sup_x p_{y|x}(y_k|x) </math>
 
:6) Compare u and <math>p\left(\hat{y}\right)</math>
 
::6a) If u is larger then repeat from step 2
 
::6b) If u is smaller then save <math>\hat{x}</math> as <math>x_{k|k}^{(p)}</math> and increment n
 
:7) If n == P then quit
 
The goal is to generate P "particles" at <math>k</math> using only the particles from <math>k-1</math>.
This requires that a Markov equation can be written (and computed) to generate a <math>x_k</math> based only upon <math>x_{k-1}</math>.
This algorithm uses composition of the P particles from <math>k-1</math> to generate a particle at <math>k</math> and repeats (steps 2–6) until P particles are generated at <math>k</math>.
 
This can be more easily visualized if <math>x</math> is viewed as a two-dimensional array.
One dimension is <math>k</math> and the other dimensions is the particle number.
For example, <math>x(k,L)</math> would be the L<sup>th</sup> particle at <math>k</math> and can also be written <math>x_k^{(L)}</math> (as done above in the algorithm).
Step 3 generates a ''potential'' <math>x_k</math> based on a randomly chosen particle (<math>x_{k-1}^{(L)}</math>) at time <math>k-1</math> and rejects or accepts it in step 6.
In other words, the <math>x_k</math> values are generated using the previously generated <math>x_{k-1}</math>.
 
==Other particle filters==
* [[Auxiliary particle filter]] <ref name="apf1999">{{cite journal
| author = Pitt, M.K.
| coauthors = Shephard, N.
| year = 1999
| title = Filtering Via Simulation: Auxiliary Particle Filters
| journal = Journal of the American Statistical Association
| volume = 94
| issue = 446
| pages = 590–591
| url = http://www.questia.com/PM.qst?a=o&se=gglsc&d=5002321997
| accessdate = 2008-05-06
| doi = 10.2307/2670179
| jstor = 2670179
| publisher = American Statistical Association
}}</ref>
* [[Regularized auxiliary particle filter]] <ref name="jliu2011">{{cite journal
| author = Liu, J.
| coauthors = Wang, W., Ma, F.
| year = 2011
| title = A Regularized Auxiliary Particle Filtering Approach for System State Estimation and Battery Life Prediction
| journal = Smart Materials and Structures
| volume = 20
| issue = 7
| pages = 1–9
| doi = 10.1088/0964-1726/20/7/075021
}}</ref>
* Gaussian particle filter
* Unscented particle filter
* Gauss–Hermite particle filter
* Cost Reference particle filter
* Hierarchical/Scalable particle filter <ref name="Canton2011">{{cite journal
| author = Canton-Ferrer, C.
| coauthors = Casas, J.R., Pardàs, M.
| year = 2011
| title = Human Motion Capture Using Scalable Body Models
| journal = Computer Vision and Image Understanding
| volume = 115
| issue = 10
| pages = 1363–1374
| accessdate = 2012-04-18
| doi = 10.1016/j.cviu.2011.06.001
| publisher = Elsevier
}}</ref>
* Rao–Blackwellized particle filter <ref name="rbpf1999">{{cite conference
| id          = {{citeseerx|10.1.1.137.5199}}
| title        = Rao–Blackwellised particle filtering for dynamic Bayesian networks
| author = Doucet, A.
| coauthors = De Freitas, N. and Murphy, K. and Russell, S.
| year          = 2000
| conference    = Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
| pages        = 176–183
| accessdate    = 2012-04-09
}}
</ref>
* [[Rejection sampling|Rejection-sampling]] based optimal particle filter <ref name="optrj2008">{{cite conference
| id          = {{citeseerx|10.1.1.190.7092}}
| title        = An Optimal Filtering Algorithm for Non-Parametric Observation Models in Robot Localization
| author        = Blanco, J.L.
| coauthors    = Gonzalez, J. and Fernandez-Madrigal, J.A.
| year          = 2008
| conference    = IEEE International Conference on Robotics and Automation (ICRA'08)
| pages        = 461–466
}}
</ref><ref name="optrj2010">{{cite journal
| url          = http://ijr.sagepub.com/content/29/14/1726.full.pdf
| title        = Optimal Filtering for Non-Parametric Observation Models: Applications to Localization and SLAM
| author        = Blanco, J.L.
| coauthors    = Gonzalez, J. and Fernandez-Madrigal, J.A.
| year          = 2010
| journal      = The International Journal of Robotics Research (IJRR)
| volume        = 29
| number        = 14
| pages        = 1726–1742
| doi          = 10.1177/0278364910364165
}}
</ref>
 
==See also==
* [[Ensemble Kalman filter]]
* [[Generalized filtering]]
* [[Moving horizon estimation]]
* [[Recursive Bayesian estimation]]
* [[Monte Carlo localization]]
 
==References==
{{Reflist}}
 
== Bibliography ==
* {{cite book
| author = Cappe, O.
| coauthors = Moulines, E.; Ryden, T.
| year = 2005
| title = Inference in Hidden Markov Models
| publisher = Springer
| isbn =
}}
* {{cite book
| author = Liu, J.
| year = 2001
| title = Monte Carlo strategies in Scientific Computing
| publisher = Springer
| isbn =
}}
 
* {{cite book
| author = Ristic, B.
| coauthors = Arulampalam, S.; Gordon, N.
| year = 2004
| title = Beyond the Kalman Filter: Particle Filters for Tracking Applications
| publisher = Artech House
| isbn =
}}
 
* {{cite journal
| author = Doucet, A.
| coauthors = Johansen, A.M.;
| year = December 2008
| title = A tutorial on particle filtering and smoothing: fifteen years later
| journal = Technical report
| publisher = Department of Statistics, University of British Columbia
| volume =
| issue =
| pages =
| url = http://www.cs.ubc.ca/%7Earnaud/doucet_johansen_tutorialPF.pdf
| doi =
}}
 
* {{cite journal
| author = Doucet, A.
| coauthors = Godsill, S.; Andrieu, C.;
| year = 2000
| title = On sequential Monte Carlo sampling methods for Bayesian filtering
| journal = Statistics and Computing
| volume = 10
| issue = 3
| pages = 197–208
| url = http://www.springerlink.com/content/q6452k2x37357l3r/
| doi = 10.1023/A:1008935410038
}}
 
* {{cite journal
| author = Arulampalam, M.S.
| coauthors = Maskell, S.; Gordon, N.; Clapp, T.;
| year = 2002
| title = A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking
| journal = IEEE Transactions on Signal Processing
| volume = 50
| issue = 2
| pages = 174–188
| url = http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=978374
| doi = 10.1109/78.978374
}}
 
* {{cite journal
| author = Cappe, O.
| coauthors =Godsill, S.; Moulines, E.;
| year = 2007
| title = An overview of existing methods and recent advances in sequential Monte Carlo
| journal = Proceedings of IEEE
| volume = 95
| issue = 5
| doi = 10.1109/JPROC.2007.893250
| page = 899
}}
 
* {{cite journal
| author = Kitagawa, G.
| year = 1996
| title =Monte carlo filter and smoother for non-Gaussian nonlinear state space models
| volume = 5
| issue = 1
| journal = Journal of Computational and Graphical Statistics
| pages = 1–25
| doi = 10.2307/1390750
| jstor = 1390750
}}
 
* {{cite journal
| author = Kotecha, J.H.
| coauthors = Djuric, P.;
| year = 2003
| title =Gaussian Particle filtering
| volume = 51
| issue = 10
| journal = IEEE Transactions Signal Processing
}}
 
* {{cite journal
| author = Haug, A.J.
| year = 2005
| title = A Tutorial on Bayesian Estimation and Tracking Techniques Applicable to Nonlinear and Non-Gaussian Processes
| journal = [[MITRE Corporation|The MITRE Corporation]], USA, Tech. Rep., Feb
| url = http://www.mitre-corporation.net/work/tech_papers/tech_papers_05/05_0211/05_0211.pdf
| accessdate = 2008-05-06
}}
 
* {{cite journal
| author = Pitt, M.K.
| coauthors = Shephard, N.
| year = 1999
| title = Filtering Via Simulation: Auxiliary Particle Filters
| journal = Journal of the American Statistical Association
| volume = 94
| issue = 446
| pages = 590–591
| url = http://www.questia.com/PM.qst?a=o&se=gglsc&d=5002321997
| accessdate = 2008-05-06
| doi = 10.2307/2670179
| jstor = 2670179
| publisher = Journal of the American Statistical Association, Vol. 94, No. 446
}}
 
* {{cite journal
| author = Gordon, N. J.
| coauthors = Salmond, D. J. and Smith, A. F. M.
| year = 1993
| title = Novel approach to nonlinear/non-Gaussian Bayesian state estimation
| journal = IEEE Proceedings F on Radar and Signal Processing
| volume = 140
| issue = 2
| pages = 107–113
| url = http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=210672
| accessdate = 2009-09-19
| doi = 10.1049/ip-f-2.1993.0015
}}
 
* {{cite journal
| author = Chen, Z.
| year = 2003
| title = Bayesian Filtering: From Kalman Filters to Particle Filters, and Beyond
| id = {{citeseerx|10.1.1.107.7415}}
| doi =
| accessdate = 2010-10-14
}}
 
* {{cite journal
  | author = Vaswani, N.
  | coauthors = Rathi, Y. Yezzi, A., Tannenbaum, A.
  | year = 2007
  | title = Tracking deforming objects using particle filtering for geometric active contours
  | journal = IEEE Trans. on Pattern Analysis and Machine Intelligence
  | volume = 29
  | issue = 8
  | pages = 1470=1475
}}
==External links==
* [http://www.math.u-bordeaux1.fr/~delmoral/simulinks.html Feynman–Kac models and interacting particle algorithms (a.k.a. Particle Filtering)] Theoretical aspects and a list of application domains of particle filters
* [http://www-sigproc.eng.cam.ac.uk/smc/ Sequential Monte Carlo Methods (Particle Filtering)] homepage on University of Cambridge
* [http://www.cs.washington.edu/ai/Mobile_Robotics/mcl/ Dieter Fox's MCL Animations]
* [http://blogs.oregonstate.edu/hess/code/particles/ Rob Hess' free software]
* [http://www.jstatsoft.org/v30/i06/ SMCTC: A Template Class for Implementing SMC algorithms in C++]
* [http://www.oursland.net/projects/particlefilter/ Java applet on particle filtering]
* [http://www.ece.sunysb.edu/~zyweng/particle.html Particle++ : a simple C++ class template for particle filter]
 
{{DEFAULTSORT:Particle Filter}}
[[Category:Estimation theory]]
[[Category:Monte Carlo methods]]
[[Category:Computational statistics]]
[[Category:Control theory| ]]
[[Category:Nonlinear filters]]
[[Category:Robot control]]

Latest revision as of 10:32, 13 December 2014

The author is recognized by the name of Numbers Wunder. One of the very very best issues in the globe for him is to gather badges but he is having difficulties to find time for it. For many years I've been working as a payroll clerk. South Dakota is her beginning place but she needs to move simply because of her family.

Here is my web site std testing at home