HKA test

Template:Bayesian statisticsTemplate:Probability fundamentals Bayesian Programming is a formalism and a methodology to specify probabilistic models and solve problems when all the necessary information is not available.

Edwin T. Jaynes proposed that probability could be considered as an alternative and an extension of logic for rational reasoning with incomplete and uncertain information. In his founding book Probability Theory: The Logic of Science he developed this theory and proposed what he called “the robot,” which was not a physical device, but an inference engine to automate probabilistic reasoning — a kind of Prolog for probability instead of logic. Bayesian Programming is a formal and concrete implementation of this "robot".

Bayesian programming may also be seen as an algebraic formalism to specify graphical models such as, for instance, Bayesian networks, Dynamic Bayesian networks, Kalman filters or Hidden Markov Models. Indeed, Bayesian Programming is more general than Bayesian networks and has a power of expression equivalent to probabilistic factor graphs.

Formalism

A Bayesian program is a means of specifying a family of probability distributions.

The constituent elements of a Bayesian program are presented below:

$Program{\begin{cases}Description{\begin{cases}Specification(\pi ){\begin{cases}Variables\\Decomposition\\Forms\\\end{cases}}\\Identification\ (based\ on\ \delta )\end{cases}}\\Question\end{cases}}$

1. A program is constructed from a description and a question.
2. A description is constructed using some specification ($\pi$) as given by the programmer and an identification or learning process for the parameters not completely specified by the specification, using a data set ($\delta$).
3. A specification is constructed from a set of pertinent variables, a decomposition and a set of forms.
4. Forms are either parametric forms or questions to other Bayesian programs.
5. A question specifies which probability distribution has to be computed.

Description

The purpose of a description is to specify an effective method of computing a joint probability distribution on a set of variables $\left\{X_{1},X_{2},\cdots ,X_{N}\right\}$ given a set of experimental data $\delta$ and some specification $\pi$. This joint distribution is denoted as: $P\left(X_{1}\wedge X_{2}\wedge \cdots \wedge X_{N}|\delta \wedge \pi \right)$.

To specify preliminary knowledge $\pi$, the programmer must undertake the following:

1. Define the set of relevant variables $\left\{X_{1},X_{2},\cdots ,X_{N}\right\}$ on which the joint distribution is defined.
2. Decompose the joint distribution (break it into relevant independant or conditional probabilities).
3. Define the forms each of the distributions (e.g., for each variable, one of the list of probability distributions).

Decomposition

Given a partition $\left\{X_{1},X_{2},\cdots ,X_{N}\right\}$ containing $K$ subsets, $K$ variables are defined $L_{1},\cdots ,L_{K}$, each corresponding to one of these subsets. Each variable $L_{k}$ is obtained as the conjunction of the variables $\left\{X_{k_{1}},X_{k_{2}},\cdots \right\}$ belonging to the $k^{th}$ subset. Recursive application of Bayes' theorem leads to:

${\begin{array}{rl}&P\left(X_{1}\wedge X_{2}\wedge \cdots \wedge X_{N}|\delta \wedge \pi \right)\\=&P\left(L_{1}\wedge \cdots \wedge L_{K}|\delta \wedge \pi \right)\\=&P\left(L_{1}|\delta \wedge \pi \right)\times P\left(L_{2}|L_{1}\wedge \delta \wedge \pi \right)\times \cdots \times P\left(L_{K}|L_{K-1}\wedge \cdots \wedge L_{1}\wedge \delta \wedge \pi \right)\end{array}}$

Conditional independence hypotheses then allow further simplifications. A conditional independence hypothesis for variable $L_{k}$ is defined by choosing some variable $X_{n}$ among the variables appearing in the conjunction $L_{k-1}\wedge \cdots \wedge L_{2}\wedge L_{1}$, labelling $R_{k}$ as the conjunction of these chosen variables and setting:

$P\left(L_{k}|L_{k-1}\wedge \cdots \wedge L_{1}\wedge \delta \wedge \pi \right)=P\left(L_{k}|R_{k}\wedge \delta \wedge \pi \right)$

We then obtain:

${\begin{array}{rl}&P\left(X_{1}\wedge X_{2}\wedge \cdots \wedge X_{N}|\delta \wedge \pi \right)\\=&P\left(L_{1}|\delta \wedge \pi \right)\times P\left(L_{2}|R_{2}\wedge \delta \wedge \pi \right)\times \cdots \times P\left(L_{K}|R_{K}\wedge \delta \wedge \pi \right)\end{array}}$

Such a simplification of the joint distribution as a product of simpler distributions is called a decomposition, derived using the chain rule.

This ensures that each variable appears at the most once on the left of a conditioning bar, which is the necessary and sufficient condition to write mathematically valid decompositionsPotter or Ceramic Artist Truman Bedell from Rexton, has interests which include ceramics, best property developers in singapore developers in singapore and scrabble. Was especially enthused after visiting Alejandro de Humboldt National Park..

Forms

Each distribution $P\left(L_{k}|R_{k}\wedge \delta \wedge \pi \right)$ appearing in the product is then associated with either a parametric form (i.e., a function $f_{\mu }\left(L_{k}\right)$) or a question to another Bayesian program $P\left(L_{k}|R_{k}\wedge \delta \wedge \pi \right)=P\left(L|R\wedge {\widehat {\delta }}\wedge {\widehat {\pi }}\right)$.

When it is a form $f_{\mu }\left(L_{k}\right)$, in general, $\mu$ is a vector of parameters that may depend on $R_{k}$ or $\delta$ or both. Learning takes place when some of these parameters are computed using the data set $\delta$.

An important feature of Bayesian Programming is this capacity to use questions to other Bayesian programs as components of the definition of a new Bayesian program. $P\left(L_{k}|R_{k}\wedge \delta \wedge \pi \right)$ is obtained by some inferences done by another Bayesian program defined by the specifications ${\widehat {\pi }}$ and the data ${\widehat {\delta }}$. This is similar to calling a subroutine in classical programming and provides an easy way to build hierarchical models.

Question

Given a description (i.e., $P\left(X_{1}\wedge X_{2}\wedge \cdots \wedge X_{N}|\delta \wedge \pi \right)$), a question is obtained by partitioning $\left\{X_{1},X_{2},\cdots ,X_{N}\right\}$ into three sets: the searched variables, the known variables and the free variables.

The 3 variables $Searched$, $Known$ and $Free$ are defined as the conjunction of the variables belonging to these sets.

A question is defined as the set of distributions:

$P\left(Searched|Known\wedge \delta \wedge \pi \right)$

made of many "instantiated questions" as the cardinal of $Known$, each instantiated question being the distribution:

$P\left(Searched|Known\wedge \delta \wedge \pi \right)$

Inference

Given the joint distribution $P\left(X_{1}\wedge X_{2}\wedge \cdots \wedge X_{N}|\delta \wedge \pi \right)$, it is always possible to compute any possible question using the following general inference:

${\begin{array}{cl}&P\left(Searched|Known\wedge \delta \wedge \pi \right)\\=&\sum _{Free}\left[P\left(Searched\wedge Free|Known\wedge \delta \wedge \pi \right)\right]\\=&{\frac {\displaystyle \sum _{Free}\left[P\left(Searched\wedge Free\wedge Known|\delta \wedge \pi \right)\right]}{\displaystyle P\left(Known|\delta \wedge \pi \right)}}\\=&{\frac {\displaystyle \sum _{Free}\left[P\left(Searched\wedge Free\wedge Known|\delta \wedge \pi \right)\right]}{\displaystyle \sum _{Free\wedge Searched}\left[P\left(Searched\wedge Free\wedge Known|\delta \wedge \pi \right)\right]}}\\=&{\frac {1}{Z}}\times \sum _{Free}\left[P\left(Searched\wedge Free\wedge Known|\delta \wedge \pi \right)\right]\end{array}}$

where the first equality results from the marginalization rule, the second results from Bayes' theorem and the third corresponds to a second application of marginalization. The denominator appears to be a normalization term and can be replaced by a constant $Z$.

Theoretically, this allows to solve any Bayesian inference problem. In practice, however, the cost of computing exhaustively and exactly $P\left(Searched|Known\wedge \delta \wedge \pi \right)$ is too great in almost all cases.

Replacing the joint distribution but its decomposition we get:

${\begin{array}{cl}&P\left(Searched|Known\wedge \delta \wedge \pi \right)\\=&{\frac {1}{Z}}\sum _{Free}\left[\prod _{k=1}^{K}\left[P\left(L_{i}|K_{i}\wedge \pi \right)\right]\right]\end{array}}$

which is usually a much simpler expression to compute, as the dimensionality of the problem is considerably reduced by the decomposition into a product of lower dimension distributions.

Example

Bayesian spam detection

The purpose of Bayesian spam filtering is to eliminate junk e-mails.

The problem is very easy to formulate. E-mails should be classified into one of two categories: non-spam or spam. The only available information to classify the e-mails is their content: a set of words. Using these words without taking the order into account is commonly called a bag of words model.

The classifier should furthermore be able to adapt to its user and to learn from experience. Starting from an initial standard setting, the classifier should modify its internal parameters when the user disagrees with its own decision. It will hence adapt to the user’s criteria to differentiate between non-spam and spam. It will improve its results as it encounters increasingly classified e-mails.

Variables

The variables necessary to write this program are as follows:

1. $Spam$: a binary variable, false if the e-mail is not spam and true otherwise.
2. $W_{0},W_{1},...,W_{N-1}$: $N$ binary variables. $W_{n}$ is true if the $n^{th}$ word of the dictionary is present in the text.

These $N+1$ binary variables sum up all the information about an e-mail.

Decomposition

Starting from the joint distribution and applying recursively Bayes' theorem we obtain:

${\begin{array}{ll}&P\left(Spam\wedge W_{0}\wedge \cdots \wedge W_{N-1}\right)\\=&P\left(Spam\right)\times P\left(W_{0}|Spam\right)\times P\left(W_{1}|Spam\wedge W_{0}\right)\\&\times \cdots \\&\times P\left(W_{N-1}|Spam\wedge W_{0}\wedge \cdots \wedge W_{N-2}\right)\end{array}}$

This is an exact mathematical expression.

It can be drastically simplified by assuming that the probability of appearance of a word knowing the nature of the text (spam or not) is independent of the appearance of the other words. This is the naive Bayes assumption and this makes this spam filter a naive Bayes model.

For instance, the programmer can assume that:

$P(W_{1}|Spam\land W_{0})=P(W_{1}|Spam)$

to finally obtain:

$P(Spam\land W_{0}\land \ldots \land W_{N-1})=P(Spam)\prod _{n=0}^{N-1}[P(W_{n}|Spam)]$

This kind of assumption is known as the naive Bayes' assumption. It is "naive" in the sense that the independence between words is clearly not completely true. For instance, it completely neglects that the appearance of pairs of words may be more significant than isolated appearances. However, the programmer may assume this hypothesis and may develop the model and the associated inferences to test how reliable and efficient it is.

Parametric forms

To be able to compute the joint distribution, the programmer must now specify the $N+1$ distributions appearing in the decomposition:

1. $P(Spam)$ is a prior defined, for instance, by $P([Spam=1])=0.75$
2. Each of the $N$ forms $P(W_{n}|Spam)$ may be specified using Laplace rule of succession (this is a pseudocounts-based smoothing technique to counter the zero-frequency problem of words never-seen-before):
   1. $P(W_{n}|[Spam=false])={\frac {1+a_{f}^{n}}{2+a_{f}}}$
   2. $P(W_{n}|[Spam=true])={\frac {1+a_{t}^{n}}{2+a_{t}}}$

where $a_{f}^{n}$ stands for the number of appearances of the $n^{th}$ word in non-spam e-mails and $a_{f}$ stands for the total number of non-spam e-mails. Similarly, $a_{t}^{n}$ stands for the number of appearances of the $n^{th}$ word in spam e-mails and $a_{t}$ stands for the total number of spam e-mails.

Identification

The $N$ forms $P(W_{n}|Spam)$ are not yet completely specified because the $2N+2$ parameters $a_{f}^{n=0,\ldots ,N-1}$, $a_{t}^{n=0,\ldots ,N-1}$, $a_{f}$ and $a_{t}$ have no values yet.

The identification of these parameters could be done either by batch processing a series of classified e-mails or by an incremental updating of the parameters using the user's classifications of the e-mails as they arrive.

Both methods could be combined: the system could start with initial standard values of these parameters issued from a generic database, then some incremental learning customizes the classifier to each individual user.

Question

The question asked to the program is: "what is the probability for a given text to be spam knowing which words appear and don't appear in this text?" It can be formalized by: $P\left(Spam|w_{0}\wedge \cdots \wedge w_{N-1}\right)$

which can be computed as follows:

${\begin{array}{ll}&P\left(Spam|w_{0}\wedge \cdots \wedge w_{N-1}\right)\\=&{\frac {\displaystyle P\left(Spam\right)\prod _{n=0}^{N-1}\left[P\left(w_{n}|Spam\right)\right]}{\displaystyle \sum _{Spam}\left[P\left(Spam\right)\prod _{n=0}^{N-1}\left[P\left(w_{n}|Spam\right)\right]\right]}}\end{array}}$

The denominator appears to be a normalization constant. It is not necessary to compute it to decide if we are dealing with spam. For instance, an easy trick is to compute the ratio:

${\begin{array}{ll}&{\frac {P\left(\left[Spam=true\right]|w_{0}\wedge \cdots \wedge w_{N-1}\right)}{P\left(\left[Spam=false\right]|w_{0}\wedge \cdots \wedge w_{N-1}\right)}}\\=&{\frac {P\left(\left[Spam=true\right]\right)}{P\left(\left[Spam=false\right]\right)}}\times \prod _{n=0}^{N-1}\left[{\frac {P\left(w_{n}|\left[Spam=true\right]\right)}{P\left(w_{n}|\left[Spam=false\right]\right)}}\right]\end{array}}$

This computation is faster and easier because it requires only $2N$ products.

Bayesian program

The Bayesian spam filter program is completely defined by:

$Pr{\begin{cases}Ds{\begin{cases}Sp(\pi ){\begin{cases}Va:Spam,W_{0},W_{1}\ldots W_{N-1}\\Dc:{\begin{cases}P(Spam\land W_{0}\land \ldots \land W_{n}\land \ldots \land W_{N-1})\\=P(Spam)\prod _{n=0}^{N-1}P(W_{n}|Spam)\end{cases}}\\Fo:{\begin{cases}P(Spam):{\begin{cases}P([Spam=false])=0.25\\P([Spam=true])=0.75\end{cases}}\\P(W_{n}|Spam):{\begin{cases}P(W_{n}|[Spam=false])\\={\frac {1+a_{f}^{n}}{2+a_{f}}}\\P(W_{n}|[Spam=true])\\={\frac {1+a_{t}^{n}}{2+a_{t}}}\end{cases}}\\\end{cases}}\\\end{cases}}\\Identification\ (based\ on\ \delta )\end{cases}}\\Qu:P(Spam|w_{0}\land \ldots \land w_{n}\land \ldots \land w_{N-1})\end{cases}}$

Bayesian filter, Kalman filter and Hidden Markov model

Bayesian filters (often called Recursive Bayesian estimation) are generic probabilistic models for time evolving processes. Numerous models are particular instances of this generic approach, for instance: the Kalman filter or the Hidden Markov model.

Variables

• Variables $S^{0},\cdots ,S^{T}$ are a time series of state variables considered to be on a time horizon ranging from $0$ to $T$.
• Variables $O^{0},\cdots ,O^{T}$ are a time series of observation variables on the same horizon.

Decomposition

The decomposition is based:

• on $P\left(S^{t}|S^{t-1}\right)$, called the system model, transition model or dynamic model, which formalizes the transition from the state at time $t-1$ to the state at time $t$;
• on $P\left(O^{t}|S^{t}\right)$, called the observation model, which expresses what can be observed at time $t$ when the system is in state $S^{t}$;
• on an initial state at time $0$: $P\left(S^{0}\wedge O^{0}\right)$.

Parametrical forms

The parametrical forms are not constrained and different choices lead to different well-known models: see Kalman filters and Hidden Markov models just below.

Question

The question usually asked of these models is $P\left(S^{t+k}|O^{0}\wedge \cdots \wedge O^{t}\right)$: what is the probability distribution for the state at time $t+k$ knowing the observations from instant $0$ to $t$?

The most common case is Bayesian filtering where $k=0$, which means that one searches for the present state, knowing the past observations.

However it is also possible to do a prediction $(k>0)$, where one tries to extrapolate a future state from past observations, or to do smoothing $(k<0)$, where one tries to recover a past state from observations made either before or after that instant.

Some more complicated questions may also be asked as shown below in the HMM section.

Bayesian filters $(k=0)$ have a very interesting recursive property, which contributes greatly to their attractiveness. $P\left(S^{t}|O^{0}\wedge \cdots \wedge O^{t}\right)$ may be computed simply from $P\left(S^{t-1}|O^{0}\wedge \cdots \wedge O^{t-1}\right)$ with the following formula:

${\begin{array}{ll}&P\left(S^{t}|O^{0}\wedge \cdots \wedge O^{t}\right)\\=&P\left(O^{t}|S^{t}\right)\times \sum _{S^{t-1}}\left[P\left(S^{t}|S^{t-1}\right)\times P\left(S^{t-1}|O^{0}\wedge \cdots \wedge O^{t-1}\right)\right]\end{array}}$

Another interesting point of view for this equation is to consider that there are two phases: a prediction phase and an estimation phase:

• During the prediction phase, the state is predicted using the dynamic model and the estimation of the state at the previous moment:

${\begin{array}{ll}&P\left(S^{t}|O^{0}\wedge \cdots \wedge O^{t-1}\right)\\=&\sum _{S^{t-1}}\left[P\left(S^{t}|S^{t-1}\right)\times P\left(S^{t-1}|O^{0}\wedge \cdots \wedge O^{t-1}\right)\right]\end{array}}$

• During the estimation phase, the prediction is either confirmed or invalidated using the last observation:

${\begin{array}{ll}&P\left(S^{t}|O^{0}\wedge \cdots \wedge O^{t}\right)\\=&P\left(O^{t}|S^{t}\right)\times P\left(S^{t}|O^{0}\wedge \cdots \wedge O^{t-1}\right)\end{array}}$

Bayesian program

$Pr{\begin{cases}Ds{\begin{cases}Sp(\pi ){\begin{cases}Va:\\S^{0},\cdots ,S^{T},O^{0},\cdots ,O^{T}\\Dc:\\{\begin{cases}&P\left(S^{0}\wedge \cdots \wedge S^{T}\wedge O^{0}\wedge \cdots \wedge O^{T}|\pi \right)\\=&P\left(S^{0}\wedge O^{0}\right)\times \prod _{t=1}^{T}\left[P\left(S^{t}|S^{t-1}\right)\times P\left(O^{t}|S^{t}\right)\right]\end{cases}}\\Fo:\\{\begin{cases}P\left(S^{0}\wedge O^{0}\right)\\P\left(S^{t}|S^{t-1}\right)\\P\left(O^{t}|S^{t}\right)\end{cases}}\end{cases}}\\Id\end{cases}}\\Qu:\\{\begin{cases}{\begin{array}{l}P\left(S^{t+k}|O^{0}\wedge \cdots \wedge O^{t}\right)\\\left(k=0\right)\equiv Filtering\\\left(k>0\right)\equiv Prediction\\\left(k<0\right)\equiv Smoothing\end{array}}\end{cases}}\end{cases}}$

Kalman Filter

The very well-known Kalman filters are a special case of Bayesian filters.

They are defined by the following Bayesian program:

$Pr{\begin{cases}Ds{\begin{cases}Sp(\pi ){\begin{cases}Va:\\S^{0},\cdots ,S^{T},O^{0},\cdots ,O^{T}\\Dc:\\{\begin{cases}&P\left(S^{0}\wedge \cdots \wedge O^{T}|\pi \right)\\=&\left[{\begin{array}{c}P\left(S^{0}\wedge O^{0}|\pi \right)\\\prod _{t=1}^{T}\left[P\left(S^{t}|S^{t-1}\wedge \pi \right)\times P\left(O^{t}|S^{t}\wedge \pi \right)\right]\end{array}}\right]\end{cases}}\\Fo:\\{\begin{cases}P\left(S^{t}|S^{t-1}\wedge \pi \right)\equiv G\left(S^{t},A\bullet S^{t-1},Q\right)\\P\left(O^{t}|S^{t}\wedge \pi \right)\equiv G\left(O^{t},H\bullet S^{t},R\right)\end{cases}}\end{cases}}\\Id\end{cases}}\\Qu:\\P\left(S^{T}|O^{0}\wedge \cdots \wedge O^{T}\wedge \pi \right)\end{cases}}$

• Variables are continuous.
• The transition model $P\left(S^{t}|S^{t-1}\wedge \pi \right)$ and the observation model $P\left(O^{t}|S^{t}\wedge \pi \right)$ are both specified using Gaussian laws with means that are linear functions of the conditioning variables.

With these hypotheses and by using the recursive formula, it is possible to solve the inference problem analytically to answer the usual $P\left(S^{T}|O^{0}\wedge \cdots \wedge O^{T}\wedge \pi \right)$ question. This leads to an extremely efficient algorithm, which explains the popularity of Kalman filters and the number of their everyday applications.

When there are no obvious linear transition and observation models, it is still often possible, using a first-order Taylor's expansion, to treat these models as locally linear. This generalization is commonly called the Extended Kalman filter.

Hidden Markov model

Hidden Markov models (HMMs) are another very popular specialization of Bayesian filters.

They are defined by the following Bayesian program:

$Pr{\begin{cases}Ds{\begin{cases}Sp(\pi ){\begin{cases}Va:\\S^{0},\cdots ,S^{T},O^{0},\cdots ,O^{T}\\Dc:\\{\begin{cases}&P\left(S^{0}\wedge \cdots \wedge O^{T}|\pi \right)\\=&\left[{\begin{array}{c}P\left(S^{0}\wedge O^{0}|\pi \right)\\\prod _{t=1}^{T}\left[P\left(S^{t}|S^{t-1}\wedge \pi \right)\times P\left(O^{t}|S^{t}\wedge \pi \right)\right]\end{array}}\right]\end{cases}}\\Fo:\\{\begin{cases}P\left(S^{0}\wedge O^{0}|\pi \right)\equiv Matrix\\P\left(S^{t}|S^{t-1}\wedge \pi \right)\equiv Matrix\\P\left(O^{t}|S^{t}\wedge \pi \right)\equiv Matrix\end{cases}}\end{cases}}\\Id\end{cases}}\\Qu:\\Max_{S^{1}\wedge \cdots \wedge S^{T-1}}\left[P\left(S^{1}\wedge \cdots \wedge S^{T-1}|S^{T}\wedge O^{0}\wedge \cdots \wedge O^{T}\wedge \pi \right)\right]\end{cases}}$

• Variables are treated as being discrete.
• The transition model $P\left(S^{t}|S^{t-1}\wedge \pi \right)$ and the observation model $P\left(O^{t}|S^{t}\wedge \pi \right)$ are

both specified using probability matrices.

• The question most frequently asked of HMMs is:

$Max_{S^{1}\wedge \cdots \wedge S^{T-1}}\left[P\left(S^{1}\wedge \cdots \wedge S^{T-1}|S^{T}\wedge O^{0}\wedge \cdots \wedge O^{T}\wedge \pi \right)\right]$ What is the most probable series of states that leads to the present state, knowing the past observations?

This particular question may be answered with a specific and very efficient algorithm called the Viterbi algorithm.

A specific learning algorithm called the Baum–Welch algorithm has also been developed for HMMs.

Applications

Academic applications

For the last 15 years, Bayesian programming approach has been used in various universities to develop both robotics applications and life sciences models.

Robotics

In robotics, Bayesian Programming has been applied to autonomous robotics, robotic CAD systems, Advanced driver assistance systems, robotic arm control, mobile robotics, Human-robots interactions, Human-vehicle interactions (Bayesian autonomous driver models) video game avatar programming and training and real-time strategy games (AI).

Life sciences

In life sciences, Bayesian Programming has been used in vision to reconstruct shape from motion, to model visuo-vestibular interaction and to study saccadic eye movements; in speech perception and control to study early acquisition of speech and the emergence of articulatory-acoustic systems; and to model handwriting perception and control.

Bayesian programming versus possibility theories

The comparison between probabilistic approaches (not only Bayesian programming) and possibility theories has been debated for a long time and is, unfortunately, a very controversial matter.

Possibility theories like, for instance, fuzzy sets, Fuzzy logic and Possibility theory propose different alternatives to probability to model uncertainty. They argue that probability is insufficient or inconvenient to model certain aspects of incomplete and uncertain knowledge.

The defense of probability is mainly based on Cox's theorem which, starting from four postulates concerning rational reasoning in the presence of uncertainty, demonstrates that the only mathematical framework that satisfies these postulates is probability theory. The argument then goes like this: if you use a different approach than probability, then you necessarily infringe on one of these postulates. Let us see which one and discuss its utility.

Bayesian programming versus probabilistic programming

The purpose of probabilistic programming is to unify the scope of classical programming languages with probabilistic modeling (especially Bayesian networks) in order to be able to deal with uncertainty but still profit from the power of expression of programming languages to describe complex models.

The extended classical programming languages can be logical languages as proposed in Probabilistic Horn Abduction, Independent Choice Logic, PRISM, and ProbLog which propose an extension of Prolog.

It can also be extensions of functional programming languages (essentially Lisp and Scheme) such as IBAL or CHURCH. The inspiring programming languages can even be object oriented like in BLOG and FACTORIE or more standard ones like in CES and FIGARO.

The purpose of Bayesian programming is different. Jaynes' precept of "probability as logic" defends that probability is an extension of and an alternative to logic above which a complete theory of rationality, computation and programming can be rebuilt. Bayesian programming does not search to extend classical languages but rather to replace them by a new programming approach based on probability and taking fully into account incompleteness and uncertainty.

The precise comparison between the semantic and power of expression of Bayesian and probabilistic programming is still an open question.

See also

Sportspersons Hyslop from Nicolet, usually spends time with pastimes for example martial arts, property developers condominium in singapore singapore and hot rods. Maintains a trip site and has lots to write about after touring Gulf of Porto: Calanche of Piana. Template:Columns-list

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

External links

• A companion site to the Bayesian programming book where to download ProBT an inference engine dedicated to Bayesian programming.
• The Bayesian-programming.org site for the promotion of Bayesian programming with detailed information and numerous publications.