View source for 23S rRNA (guanine2535-N1)-methyltransferase

'''Background subtraction''', also known as Foreground Detection, is a technique in the fields of [[image processing]] and [[computer vision]] wherein an image's foreground is extracted for further processing (object recognition etc.). Generally an image's regions of interest are objects (humans, cars, text etc.) in its foreground. After the stage of image preprocessing (which may include [[image denoising]] etc.) object localisation is required which may make use of this technique. Background subtraction is a widely used approach for detecting moving objects in videos from static cameras. The rationale in the approach is that of detecting 
the moving objects from the difference between the current frame and a reference frame, often called “background image”, or “background model”.<ref name="Piccardi">{{cite conference|last=M. Piccardi|title=Background subtraction techniques: a review|conference=IEEE International Conference on Systems, Man and Cybernetics|date=October 2004|volume=4|pages=3099–3104|doi=10.1109/icsmc.2004.1400815|url=http://profs.sci.univr.it/~cristanm/teaching/sar_files/lezione4/Piccardi.pdf}}</ref> Background subtraction is mostly done if the image in question is a part of a video stream.

==Approaches==
A robust background subtraction algorithm should be able to handle lighting changes, repetitive motions from clutter and long-term scene changes.<ref name="cs.utexas">{{cite web|last=B. Tamersoy|title=Background Subtraction – Lecture Notes|url=http://www.cs.utexas.edu/~grauman/courses/fall2009/slides/lecture9_background.pdf|publisher=University of Texas at Austin|date=September 29, 2009}}</ref>  The following analyses make use of the function of ''V''(''x'',''y'',''t'') as a video sequence where ''t'' is the time dimension, ''x'' and ''y'' are the pixel location variables. e.g. ''V''(1,2,3) is the pixel intensity at (1,2) pixel location of the image at ''t''&nbsp;=&nbsp;3 in the video sequence.

=== Using frame differencing ===
Frame difference (absolute) at time ''t''&nbsp;+&nbsp;1 is

: <math>D(t+1) = |V(x,y,t+1)-V(x,y,t)|</math>

The background is assumed to be the frame at time ''t''. This difference image would only show some intensity for the pixel locations which have changed in the two frames. Though we have seemingly removed the background, this approach will only work for cases where all foreground pixels are moving and all background pixels are static.<ref name="cs.utexas" />

A threshold "Th" is put on this difference image to improve the subtraction (see Image [[Thresholding (image processing)|thresholding]]).

: <math>|V(x,y,t)-V(x,y,t+1)|>\mathrm{Th} \, </math>

(this means that the difference image's pixels' intensities are 'thresholded' or filtered on the basis of value of Th)

The accuracy of this approach is dependent on speed of movement in the scene. Faster movements may require higher thresholds.

=== Mean filter ===
For calculating the image containing only the background, a series of preceding images are averaged. For calculating the background image at the instant&nbsp;''t'',

: <math>B(x,y)={1 \over N} \sum_{i=1}^N V(x,y,t-i)</math>

where ''N'' is the number of preceding images taken for averaging. This averaging refers to averaging corresponding pixels in the given images. ''N'' would depend on the video speed (number of images per second in the video) and the amount of movement in the video.<ref>{{cite conference|last=Y. Benezeth|coauthors=B. Emile; H. Laurent; C. Rosenberger|title=Review and evaluation of commonly-implemented background subtraction algorithms|conference=19th International Conference on Pattern Recognition|date=December 2008|year=2008|pages=1–4 |doi=10.1109/ICPR.2008.4760998|url=http://www.moivre.usherbrooke.ca/sites/default/files/108.pdf}}</ref>  After calculating the background ''B(x,y)'' we can then subtract it from the image ''V(x,y,t)'' at time ''t''=t and threshold it. Thus the foreground is

: <math>|V(x,y,t)-B(x,y)|> \mathrm{Th} \, </math>

where ''Th'' is threshold. Similarly we can also use median instead of mean in the above calculation of ''B''(''x'',''y'').

Usage of global and time-independent Thresholds (same Th value for all pixels in the image) may limit the accuracy of the above two approaches.<ref name="cs.utexas" />

=== Running Gaussian average ===
For this method, Wren et al.<ref>{{cite journal|last=C.R. Wren|coauthors=A. Azarbayejani;T. Darrell; A.P. Pentland|title=Pfinder: real-time tracking of the human body|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|date=July 1997|volume=19|issue=7|pages=780–785|doi=10.1109/34.598236|url=http://www.cvmt.dk/education/teaching/e06/CVG9/CV/CVG9ExercisePapers/pfinder.pdf}}</ref> propose fitting a [[Gaussian function|Gaussian probabilistic density function]] (pdf) on the most recent <math>n</math> frames. In order to avoid fitting the pdf from scratch at each new frame time <math>t</math>, a running (or on-line cumulative) average is computed.

The pdf of every pixel is characterized by [[mean]] <math> \mu_t </math> and [[variance]] <span style = "vertical-align: 15%"> <math> \sigma ^ 2_t </math> </span>. The following is a possible initial condition (assuming that initially every pixel is background):

<math> \mu_0 = I_0 </math>

<math> \sigma^2_0 = <</math> some default value <math>></math>

where <span style="vertical-align:15%;"> <math> I_t </math> </span> is the value of the pixel's intensity at time <math>t</math>. In order to initialize variance, we can, for example, use the variance in x and y from a small window around each pixel.

Note that background may change over time (e.g. due to illumination changes or non-static background objects). To accommodate for that change, at every frame <math>t</math>, every pixel's mean and variance must be updated, as follows:

<math> \mu_t = \rho I_t + (1 - \rho) \mu_ {t-1} </math>

<math> \sigma^2_t = d ^ 2 \rho + (1 - \rho) \sigma ^ 2_ {t-1} </math>

<math> d = |(I_t-\mu_t)| </math>

Where <math> \rho </math> determines the size of the temporal window that is used to fit the pdf (usually <math> \rho = 0.01 </math> ) and <span style="vertical-align:15%;"> <math> d </math> </span> is the Euclidean distance between the mean and the value of the pixel.

[[File:Running gaussian.jpg|thumb|Gaussian distribution for each pixel.]]

We can now classify a pixel as background if its current intensity lies within some [[confidence interval]] of it's distribution's mean:

<math> \frac{|(I_t-\mu_t)|}{\sigma_t} > k \longrightarrow \mathit{ Foreground }</math>

<math>  \frac{|(I_t-\mu_t)|}{\sigma_t} \le  k \longrightarrow \mathit{ Background }</math> 

where the parameter <span style="vertical-align:15%;"><math>  k </math></span> is a free threshold (usually <span style="vertical-align:15%;"> <math> k = 2.5 </math> </span>). A larger value for <math>k</math> allows for more dynamic background, while a smaller <math>k</math> increases the probability of a transition from background to foreground due to more subtle changes.

In a variant of the method, a pixel's distribution is only updated if it is classified as background. This is to prevent newly introduced foreground objects from fading into the background. The update formula for the mean is changed accordingly:

<math> \mu_t =M\mu_{t-1} +(1-M)(I_t\rho +(1-\rho)\mu_{t-1})  </math>

where <span Style="vertical-align:15%;"> <math> M = 1 </math> </span> when <math> I_t </math> is considered foreground and <span style = "vertical-align: 15%"> <math> M = 0 </math> </span> otherwise. So when <span style="vertical-align:15%;"> <math> M = 1 </math> </span>, that is, when the pixel is detected as foreground, the mean will stay the same. As a result, a pixel, once it has become foreground, can only become background again when the intensity value gets close to what it was before turning foreground. This method, however, has several issues: It only works if all pixels are initially background pixels (or foreground pixels are annotated as such). Also, it cannot cope with gradual background changes: If a pixel is categorized as foreground for a too long period of time, the background intensity in that location might have changed (because illumination has changed etc.). As a result, once the foreground object is gone, the new background intensity might not be recognized as such anymore.

=== Background mixture models ===
In this technique, it is assumed that every pixel's intensity values in the video can be modeled using a [[Gaussian mixture model]].<ref name="StaufferGrimson">{{cite conference|last=C. Stauffer, W. E. L. Grimson|title=Adaptive background mixture models for real-time tracking|conference=IEEE Computer Society Conference on Computer Vision and Pattern Recognition|date=August 1999|volume=2|pages=246–252|doi=10.1109/CVPR.1999.784637|url=http://www.ai.mit.edu/projects/vsam/Publications/stauffer_cvpr98_track.pdf}}</ref> A simple heuristic determines which intensities are most probably of the background. Then the pixels which do not match to these are called the foreground pixels.
Foreground pixels are grouped using 2D [[Connected-component labeling|connected component]] analysis.<ref name="StaufferGrimson" />

At any time t, a particular pixel (<math>x_0, y_0</math>)'s history is

: <math>X_1 , \ldots , X_t = \{V (x_0 , y_0 , i) : 1 \leqslant i \leqslant t\} \, </math>

This history is modeled by a mixture of ''K'' Gaussian distributions:

: <math>P(X_t) = \sum_{i=1}^K \omega_{i,t} N \left ( X_t \mid \mu_{i,t} , \Sigma_{i,t} \right )</math>

where

: <math>N \left (X_t \mid \mu_{it}, \Sigma_{i,t} \right ) = \dfrac{1}{(2\pi)^{D/2}} { 1 \over |\Sigma_{i,t}|^{1/2}} \exp\left(-{ 1 \over 2 }( X_t -\mu_{i,t} )^T \Sigma_{i,t}^{-1} \left (X_t -\mu_{i,t} \right )\right)</math>

An on-line [[K-means]] approximation is used to update the Gaussians. Numerous improvements of this original method developed by Stauffer and Grimson <ref name="StaufferGrimson">{{cite conference|last=C. Stauffer, W. E. L. Grimson|title=Adaptive background mixture models for real-time tracking|conference=IEEE Computer Society Conference on Computer Vision and Pattern Recognition|date=6 August 1999|volume=2|pages=246–252|doi=10.1109/CVPR.1999.784637|url=http://www.ai.mit.edu/projects/vsam/Publications/stauffer_cvpr98_track.pdf}}</ref>  have been proposed and a complete survey can be found in Bouwmans et al.<ref name="Bouwmans">{{cite journal|last=T. Bouwmans|coauthors= F. El Baf; B. Vachon|title=Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey|journal=Recent Patents on Computer Science|date=November 2008|volume=1|pages=219–237|url=http://www.academia.edu/691072/Background_modeling_using_mixture_of_gaussians_for_foreground_detection_A_survey.pdf}}</ref>

=== Surveys ===
Several surveys which concern categories or sub-categories of models can be found as follows:

* MOG Background Subtraction <ref name="Bouwmans">{{cite journal|last=T. Bouwmans|coauthors= F. El Baf; B. Vachon|title=Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey|journal=Recent Patents on Computer Science|date=November 2008|volume=1|pages=219–237|url=http://www.academia.edu/691072/Background_modeling_using_mixture_of_gaussians_for_foreground_detection_A_survey.pdf}}</ref>

* Subspace Learning Background Subtraction <ref name="Bouwmans1">{{cite journal|last=T. Bouwmans|title=Subspace Learning for Background Modeling: A Survey|journal=Recent Patents on Computer Science|date=November 2009|volume=2|pages=223–234|url=https://www.academia.edu/691079/Subspace_learning_for_background_modeling_A_survey.pdf}}</ref>

* Statistical Background Subtraction <ref name="Bouwmans2">{{cite journal|last=T. Bouwmans|title=Statistical Background Modeling for Foreground Detection: A Survey|journal=Chapter 3 in the Handbook of Pattern Recognition and Computer Vision, World Scientific Publishing|date=January 2010|pages=181–199|url=http://www.worldscientific.com/worldscibooks/10.1142/7297#t=aboutBook}}</ref><ref name="Bouwmans3">{{cite journal|last=T. Bouwmans|title=Recent Advanced Statistical Background Modeling for Foreground Detection: A Systematic Survey|journal=Recent Patents on Computer Science|date=September 2011|volume=4|pages=147–176|url=http://www.academia.edu/938519/Recent_Advanced_Statistical_Background_Modeling_for_Foreground_Detection_A_Systematic_Survey.pdf}}</ref>

* Fuzzy Background Subtraction <ref name="Bouwmans4">{{cite journal|last=T. Bouwmans|title=Background Subtraction For Visual Surveillance: A Fuzzy Approach|journal=Chapter 5 in Handbook on Soft Computing for Video Surveillance|date=March 2012|pages=103–134|url=http://www.crcnetbase.com/doi/abs/10.1201/b11631-6}}</ref>

* RPCA Background Subtraction <ref name="Bouwmans5">{{cite journal|last=T. Bouwmans|title=Robust Principal Component Analysis for Background Subtraction: Systematic Evaluation and Comparative Analysis|journal=Chapter 12 in Book on Principal Component Analysis, INTECH|date=March 2012|pages=223–238|url=http://cdn.intechopen.com/pdfs/30443/InTech-Robust_principal_component_analysis_for_background_subtraction_systematic_evaluation_and_comparative_analysis.pdf}}</ref><ref name="Bouwmans6">{{cite journal|last=T. Bouwmans|coauthors= E. Zahzah|title=Robust PCA via Principal Component Pursuit: A Review for a Comparative Evaluation in Video Surveillance|journal=Special Isssue on Background Models Challenge, Computer Vision and Image Understanding|date=2014}}</ref>

* Traditional and Recent Approaches for Background Subtraction <ref name="TBouwmans1">{{cite journal|last=T. Bouwmans|title=Traditional Approaches in Background Modeling for Static Cameras|journal=Chapter 1 in Handbook on "Background Modeling and Foreground Detection for Video Surveillance", CRC Press, Taylor and Francis Group|url=http://www.crcpress.com/product/isbn/9781482205374}}</ref> <ref name="TBouwmans2">{{cite journal|last=T. Bouwmans|title=Recent Approaches in Background Modeling for Static Cameras|journal=Chapter 2 in Handbook on "Background Modeling and Foreground Detection for Video Surveillance", CRC Press, Taylor and Francis Group|url=http://www.crcpress.com/product/isbn/9781482205374}}</ref>

== Resources, Datasets and Codes ==

===  Book  === 

T. Bouwmans, F. Porikli, B. Horferlin, A. Vacavant, Handbook on "Background Modeling and Foreground Detection for Video Surveillance: Traditional and Recent Approaches, Implementations, Benchmarking and Evaluation", CRC Press, Taylor and Francis Group, June 2014. (For more information: http://www.crcpress.com/product/isbn/9781482205374)

===  Special Issue  === 

T. Bouwmans, L. Davis, J. Gonzalez, M. Piccardi, C. Shan, Special Issue on “Background Modeling for Foreground Detection in Real-World Dynamic Scenes”, Special Issue in Machine Vision and Applications, July 2014.

=== BGS Web Site ===
The Background Subtraction Web Site (T. Bouwmans, Univ. La Rochelle, France) contains a full list of the references in the field, links to available datasets and codes. (For more information: http://sites.google.com/site/backgroundsubtraction/Home)

=== BGS Datasets ===
* ChangeDetection.net (For more information: http://www.changedetection.net/)
* Background Models Challenge (For more information: http://bmc.univ-bpclermont.fr/)
* Stuttgart Artificial Background Subtraction Dataset (For more information: http://www.vis.uni-stuttgart.de/index.php?id=sabs)

=== BGS Library ===
The BGS Library (A. Sobral) provides a C++ framework to perform background subtraction algorithms. The code works either on Windows or on Linux. Currently the library offers 29 BGS algorithms. (For more information: http://code.google.com/p/bgslibrary/)

== Applications ==
* Video Surveillance
* Optical Motion Capture
* Human Computer Interaction
* Content based Video Coding

== See also ==
* [[ViBe]]
* PBAS
* SOBS

== References ==
<!--- See http://en.wikipedia.org/wiki/Wikipedia:Footnotes on how to create references using <ref></ref> tags which will then appear here automatically -->
{{Reflist|2}}
* {{cite pmid|23476028}}

[[Category:Articles created via the Article Wizard]]
[[Category:Mathematical examples]]
[[Category:Image processing]]
[[Category:Computer vision]]