|
|
| Line 1: |
Line 1: |
| '''Bayesian experimental design''' provides a general probability-theoretical framework from which other theories on [[Design of experiments|experimental design]] can be derived. It is based on [[Bayesian inference]] to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.
| |
|
| |
|
| The theory of Bayesian experimental design is to a certain extent based on the theory for making [[optimal decision|optimal decisions under uncertainty]]. The aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g. the [[Shannon information]] or the negative [[variance]]), but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.
| |
|
| |
|
| ==Relations to more specialized optimal design theory==
| | I woke up another day and realized - [http://lukebryantickets.asiapak.net luke bryan concert schedule] Today I have been solitary for some time and following much intimidation from friends I today locate myself signed up for web dating. They assured me that there are plenty of fun, sweet and [http://Photo.net/gallery/tag-search/search?query_string=regular+people regular people] to meet up, therefore the pitch is gone by here!<br>My friends and fam are wonderful and spending time together at tavern gigabytes [http://okkyunglee.com discount concert tickets] or dinners is obviously imperative. As I discover that one may never have a good dialogue with the sound I have never been into nightclubs. I additionally have two very adorable and definitely cheeky canines that are invariably eager to meet up new folks.<br>I try to maintain as physically fit as potential coming to the gymnasium many times weekly. I love my athletics and endeavor to play or watch since many a potential. I [http://lukebryantickets.citizenswebcasting.com information on luke bryan] am going to frequently at Hawthorn fits being wintertime. Note: I have observed the carnage of wrestling fits at stocktake sales, If you considered purchasing a sport I don't brain.<br><br>Feel free to surf to my webpage: luke bryan 2014 tour [http://www.netpaw.org luke brian Concerts] tickets ([http://www.ladyhawkshockey.org www.ladyhawkshockey.org]) |
| ===Linear theory===
| |
| If the model is linear, the prior [[probability density function]] (PDF) is homogeneous and observational errors are [[Multivariate normal distribution|normally distributed]], the theory simplifies to the classical [[optimal design|optimal experimental design theory]].
| |
| | |
| ===Approximate normality===
| |
| In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior PDFs will be approximately normal. This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters, an approach reviewed in {{Harvtxt|Chaloner|Verdinelli|1995}}. Caution must however be taken when applying this method, since approximate normality of all possible posteriors is difficult to verify, even in cases of normal observational errors and uniform prior PDF.
| |
| | |
| ===Posterior distribution===
| |
| Recently, increased computational resources allow inference of the [[posterior distribution]] of model parameters, which can directly be used for experiment design. {{Harvtxt|Vanlier|Tiemann|Hilbers|van Riel|2012}} proposed an approach that uses the [[posterior predictive distribution]] to assess the effect of new measurements on prediction uncertainty, while {{Harvtxt|Liepe|Filippi|Komorowski|Stumpf|2013}} suggest maximizing the mutual information between parameters, predictions and potential new experiments.
| |
| | |
| ==Mathematical formulation==
| |
| | |
| {| border=1 align="right" cellpadding="0" cellspacing="0" bordercolor="#111111" width="270px"
| |
| |
| |
| {| border=0
| |
| |+ '''Notation'''
| |
| |-valign="top"
| |
| |nowrap| <math>\theta\,</math>
| |
| | parameters to be determined
| |
| |-valign="top"
| |
| |nowrap| <math>y\,</math>
| |
| | observation or data
| |
| |-valign="top"
| |
| |nowrap| <math>\xi\,</math>
| |
| | design
| |
| |-valign="top"
| |
| |nowrap| <math>p(y|\theta,\xi)\,</math>
| |
| | PDF for making observation <math>y</math>, given parameter values <math>\theta</math> and design <math>\xi</math>
| |
| |-valign="top"
| |
| |nowrap| <math>p(\theta)\,</math>
| |
| | prior PDF
| |
| |-valign="top"
| |
| |nowrap| <math>p(y|\xi)\,</math>
| |
| | marginal PDF in observation space
| |
| |-valign="top"
| |
| |nowrap| <math>p(\theta | y, \xi)\,</math>
| |
| | posterior PDF
| |
| |-valign="top"
| |
| |nowrap| <math>U(\xi)\,</math>
| |
| | utility of the design <math>\xi</math>
| |
| |-valign="top"
| |
| |nowrap| <math>U(y, \xi)\,</math>
| |
| | utility of the experiment outcome after observation <math>y</math> with design <math>\xi</math>
| |
| |}
| |
| |}
| |
| | |
| Given a vector <math>\theta</math> of parameters to determine, a [[Prior probability|prior PDF]] <math>p(\theta)</math> over those parameters and a PDF <math>p(y|\theta,\xi)</math> for making observation <math>y</math>, given parameter values <math>\theta</math> and an experiment design <math>\xi</math>, the posterior PDF can be calculated using [[Bayes' theorem]]
| |
| :<math>p(\theta | y, \xi) = \frac{p(y | \theta, \xi) p(\theta)}{p(y | \xi)} \, ,</math> | |
| | |
| where <math>p(y|\xi)</math> is the marginal probability density in observation space
| |
| :<math>p(y|\xi) = \int{p(\theta)p(y|\theta,\xi)d\theta}\, .</math>
| |
| | |
| The expected utility of an experiment with design <math>\xi</math> can then be defined
| |
| :<math>U(\xi)=\int{p(y|\xi)U(y,\xi)dy}\, ,</math>
| |
| where <math>U(y,\xi)</math> is some real-valued functional of the [[Posterior probability|posterior PDF]] <math>p(\theta | y, \xi)</math> after making observation <math>y</math> using an experiment design <math>\xi</math>.
| |
| | |
| ===Gain in Shannon information as utility===
| |
| | |
| Utility may be defined as the prior-posterior gain in [[Differential entropy|Shannon information]]
| |
| :<math> U(y, \xi) = \int{\log(p(\theta | y, \xi))p(\theta | y, \xi)d\theta} - \int{\log(p(\theta))p(\theta)d\theta} \, .</math>
| |
| Note also that
| |
| :<math>U(y, \xi) = D_{KL}(p(\theta|y,\xi) \| p(\theta|\xi)) \, ,</math>
| |
| the [[Kullback–Leibler divergence]] of the prior from the posterior distribution.
| |
| {{Harvtxt|Lindley|1956}} noted that the expected utility will then be coordinate-independent and can be written in two forms
| |
| :<math>
| |
| \begin{alignat}{2}
| |
| U(\xi) & = \int{\int{\log(p(\theta | y,\xi))p(\theta, y | \xi)d\theta}dy} - \int{\log(p(\theta))p(\theta)d\theta} \\
| |
| & = \int{\int{\log(p(y | \theta,\xi))p(\theta, y | \xi)dy}d\theta} - \int{\log(p(y| \xi))p(y| \xi)dy} ,
| |
| \end{alignat}
| |
| \, </math>
| |
| | |
| of which the latter can be evaluated without the need for evaluating individual posterior PDFs
| |
| <math>p(\theta | y,\xi)</math> for all possible observations <math>y</math>. Worth noting is that the first term on the second equation line will not depend on the design <math>\xi</math>, as long as the observational uncertainty doesn't. On the other hand, the integral of <math>p(\theta) \log p(\theta)</math> in the first form is constant for all <math>\xi</math>, so if the goal is to choose the design with the highest utility, the term need not be computed at all. Several authors have considered numerical techniques for evaluating and optimizing this criterion, e.g. {{Harvtxt|van den Berg|Curtis|Trampert|2003}} and {{Harvtxt|Ryan|2003}}. Note that
| |
| :<math>U(\xi) = I(\theta;y)\, ,</math>
| |
| the expected '''information gain''' being exactly the '''[[mutual information]]''' between the parameter ''θ'' and the observation ''y''. {{Harvtxt|Kelly|1956}} also derived just such a utility function for a gambler seeking to [[Kelly criterion|profit maximally]] from [[Gambling and information theory|side information in a horse race]]; Kelly's situation is identical to the foregoing, with the side information, or "private wire" taking the place of the experiment.
| |
| | |
| ==See also==
| |
| | |
| *[[Optimal design|Optimal Designs]]
| |
| *[[Active_learning_(machine_learning)|Active Learning]]
| |
| | |
| {{No footnotes|date=March 2011}}
| |
| ==References==
| |
| | |
| * {{Citation
| |
| | last1 = Vanlier
| |
| | last2 = Tiemann
| |
| | last3 = Hilbers
| |
| | last4 = van Riel
| |
| | year = 2012
| |
| | title = A Bayesian approach to targeted experiment design
| |
| | journal = Bioinformatics
| |
| | volume = 28
| |
| | pages = 1136–1142
| |
| | url = http://bioinformatics.oxfordjournals.org/content/28/8/1136.full.pdf
| |
| | doi = 10.1093/bioinformatics/bts092
| |
| | issue = 8
| |
| }}
| |
| | |
| * {{Citation
| |
| | last1 = Liepe
| |
| | last2 = Filippi
| |
| | last3 = Komorowski
| |
| | last4 = Stumpf
| |
| | year = 2013
| |
| | title = Maximizing the Information Content of Experiments in Systems Biology
| |
| | journal = PLoS computational biology
| |
| | volume = 9
| |
| | pages = e1002888
| |
| | url = http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002888
| |
| | doi = 10.1371/journal.pcbi.1002888
| |
| | issue = 1
| |
| }}
| |
| | |
| * {{Citation
| |
| | last1 = van den Berg
| |
| | last2 = Curtis
| |
| | last3 = Trampert
| |
| | year = 2003
| |
| | title = Optimal nonlinear Bayesian experimental design: an application to amplitude versus offset experiments
| |
| | journal = Geophysical Journal International | |
| | volume = 55
| |
| | pages = 411–421
| |
| | url = http://www.geos.ed.ac.uk/homes/acurtis/Papers/vandenBerg_etal2003.pdf
| |
| | issue = 2
| |
| }}
| |
| | |
| * {{Citation
| |
| | last1 = Chaloner
| |
| | last2 = Verdinelli
| |
| | year = 1995
| |
| | title = Bayesian experimental design: a review
| |
| | journal = Statistical Science
| |
| | volume = 10
| |
| | pages = 273–304
| |
| | url = http://www.stat.uiowa.edu/~gwoodwor/AdvancedDesign/Chaloner%20Verdinelli.pdf
| |
| | doi = 10.1214/ss/1177009939
| |
| | issue = 3
| |
| | first1 = Kathryn
| |
| | first2 = Isabella
| |
| }}
| |
| | |
| * {{Citation| author=DasGupta, A.
| |
| | chapter=Review of optimal Bayes designs
| |
| | url = http://www.stat.purdue.edu/~dasgupta/publications/tr95-04.pdf
| |
| | pages=1099–1148
| |
| | title=Design and Analysis of Experiments
| |
| | series=Handbook of Statistics
| |
| | volume=13
| |
| |editor=Ghosh, S. and [[Calyampudi Radhakrishna Rao|Rao, C. R.]]
| |
| | publisher=North-Holland| year=1996| isbn=0-444-82061-2
| |
| }}
| |
| | |
| * {{Citation
| |
| | last = Lindley
| |
| | year = 1956
| |
| | title = On a measure of information provided by an experiment
| |
| | journal = Annals of Mathematical Statistics
| |
| | volume = 27
| |
| | pages = 986–1005
| |
| | url = http://projecteuclid.org/handle/euclid.aoms/1177728069
| |
| | doi = 10.1214/aoms/1177728069
| |
| | issue = 4
| |
| | first1 = D. V.
| |
| }}
| |
| | |
| * {{Citation
| |
| | last = Ryan
| |
| | first= K. J.
| |
| | year = 2003
| |
| | title = Estimating Expected Information Gains for Experimental Designs With Application to the Random Fatigue-Limit Model
| |
| | journal = Journal of Computational and Graphical Statistics
| |
| | volume = 12
| |
| | pages = 585–603
| |
| | doi = 10.1198/1061860032012
| |
| | issue = 3
| |
| }}
| |
| | |
| {{Experimental design|state=expanded}}
| |
| {{Statistics|collection|state=collapsed}}
| |
| | |
| [[Category:Bayesian statistics|Experimental design]]
| |
| [[Category:Design of experiments]]
| |
| [[Category:Statistical methods]]
| |
| [[Category:Optimal decisions]]
| |
| [[Category:Operations research]]
| |
| [[Category:Industrial engineering]]
| |
| [[Category:Systems engineering]]
| |
| [[Category:Information, knowledge, and uncertainty]]
| |