|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| In [[probability theory]], a '''continuous-time Markov chain''' ('''CTMC'''<ref>{{cite doi|10.1007/3-540-48320-9_12}}</ref> or '''continuous-time Markov process'''<ref>{{cite doi|10.1287/opre.27.3.616}}</ref>) is a mathematical model which takes values in some finite or countable set and for which the time spent in each state takes non-negative [[real number|real value]]s and has an [[exponential distribution]]. It is a [[continuous-time stochastic process]] with the [[Markov property]] which means that future behaviour of the model (both remaining time in current state and next state) depends only on the current state of the model and not on historical behaviour. The model is a continuous-time version of the [[Markov chain]] model, named because the output from such a process is a sequence (or chain) of states.
| | Our very personal Singapore Real Property Information - Surf this section for the varieties of flats, condos, flats and houses in Singapore. Additionally take pleasure in our particular part of well-liked condominiums and property pictures, or you possibly can do the search of your favourite properties in our extensive property classifieds.<br><br>This website is visited by many Tiong Bahru property hunters EVERYDAY! My goal is to draw local & overseas patrons who are desirous about shopping for Tiong Bahru properties & real estates. I'm committed to offering high quality service to all my clients. And with my native & overseas database, I'm all the time eager to inform & replace my patrons on the most recent and best buy properties and actual estates in Singapore. Simply simply call me or refill my Selling / Leasing Enquiry form when you find yourself ready to put up your property on the market.Your property will receive its own internet page which incorporates full property info and pictures, hyperlinks to maps, community info and / or streaming video & Virtual Excursions.<br><br>Consisting of one other 5 condominiums - The Marina Collection, Turquoise, Seascape , The Pinnacle Collection, Seven Palms and two islands, most builders bought land in Sentosa South Cove for much greater costs than the sooner Sentosa North Cove. Sentosa South Cove spans from the jap fringe of the island to the southern tip the place the seashores are found – and this is what makes it so engaging. Cafes Enterprise For Sale, Espresso Retailers Enterprise For Sale Low takeover fees beneath $15k for a completely operational dessert cafeRental @ 2K/ month negotiable Takeover Comes with- Signage - Furniture- Standing freezer & chiller- Cake show chiller- Counter chiller - Counter freezer- Waffle irons- Accessories comparable to utensil Elementary school in Singapore?<br><br>A licensed and proactive actual estate professional, Jack goes beyond simply sourcing and advertising real estate properties. Efficient and approachable, he makes the entire expertise straightforward and stress-free. D14) Eunos / Geylang / Paya Lebar Condo Widespread Room with Aircon D22) Boon Lay / Jurong / Tuas Condominium Common Room with Aircon Initially I was shocked and angry that the property agent and ex-owner did not inform us about it,' mentioned the proprietor. Mr Koh, 38, who has been an agent for six years, stated a colleague as soon as lost a sale as a result of he came clear. I would additionally go to the temples to wish and seek steerage within the sale. Builders typically bid at greater and higher prices in a bullish market, resulting in [http://www.dada.wagenseller.on-rev.com/tiki-view_forum_thread.php?comments_parentId=7849&forumId=1 property agent in singapore] prices increasing in tandem. Previous Property Auction Lists<br><br>Companies Business For Sale, Postal & Communication Enterprise For Sale, Publish Offices Enterprise For Sale Funding Enterprise For Sale, Training Business For Sale, Well being Care Business For Sale Providers Enterprise For Sale, Well being & Beauty Business For Sale, Hair & Magnificence Salons Business For Sale 2 years previous nail salon for takeover.- 3 pedicure seat & 1 manicure desk.- Full outfitted, stocked and ready for enterprise immediately.- Include customer information base.- Fb Web page Up to 600 Likes- Rental 1.4k-Takeover fee 12.2k + 2.8k 2 rental deposit(able to take again) (If Eating places Business For Sale, Retailers Business For Sale, Italian Restaurants Enterprise For Sale D18) Pasir Ris / Tampines Condo Master Room with Aircon Might-June Job Hunters (At the moment here in SG)<br><br>D12) Balestier / Toa Payo HDB Widespread Room with Aircon D13) Potong Pasir / Machpherson HDB Frequent Room with Aircon D16) Bedok / Higher East Coast HDB Frequent Room with Aircon D20) Ang Mo Kio / Bishan / Thomson HDB Common Room with Aircon D15) East Coast / Marine Parade HDB Widespread Room with Aircon D14) Eunos / Geylang / Paya Lebar HDB Frequent Room with Aircon D10) Holland / Tanglin HDB Common Room with Aircon D02) Chinatown / Tanjong Pagar HDB Frequent Room with Aircon D08) Farrer Park / Serangoon Highway HDB Common Room with Aircon D18) Pasir Ris / Tampines HDB Common Room with Aircon D23) Bukit Batok / Bukit Panjiang HDB Common Room with Aircon D27) Sembawang / Yishun HDB Frequent Room with Aircon D20) Ang Mo Kio / Bishan / Thomson HDB Master Room |
| | |
| ==Definitions==
| |
| | |
| A continuous-time Markov chain (''X''<sub>''t''</sub>)<sub>''t'' ≥ 0</sub> is defined by a finite or countable state space ''S'', a [[transition rate matrix]] ''Q'' with dimensions equal to that of the state space and initial probability distribution defined on the state space. For ''i'' ≠ ''j'', the elements ''q''<sub>''ij''</sub> are non-negative and describe the rate the process transitions from state ''i'' to state ''j''. The elements ''q''<sub>''ii''</sub> are chosen such that each row of the transition rate matrix sums to zero.
| |
| | |
| There are three equivalent definitions of the process.<ref name="norris1">{{cite doi|10.1017/CBO9780511810633.004}}</ref>
| |
| | |
| ===Infinitesimal definition===
| |
| | |
| Let ''X''<sub>''t''</sub> be the random variable describing the state of the process at time ''t'', and assume that the process is in a state ''i'' at time ''t''. Then ''X''<sub>''t'' + h</sub> is independent of previous values (''X''<sub>''s''</sub> : ''s''≤ ''t'') and as ''h'' → 0 uniformly in ''t'' for all ''j''
| |
| | |
| :<math>\Pr(X(t+h) = j | X(t) = i) = \delta_{ij} + q_{ij}h + o(h)</math>
| |
| | |
| using [[little-o notation]]. The ''q''<sub>''ij''</sub> can be seen as measuring how quickly the transition from ''i'' to ''j'' happens
| |
| | |
| ===Jump chain/holding time definition===
| |
| | |
| Define a discrete-time Markov chain ''Y''<sub>''n''</sub> to describe the ''n''th jump of the process and variables ''S''<sub>1</sub>, ''S''<sub>2</sub>, ''S''<sub>3</sub>, ... to describe holding times in each of the states where the distribution of ''S''<sub>''i''</sub> is given by −''q''<sub>''Y''<sub>''i''</sub>''Y''<sub>''i''</sub></sub>.
| |
| | |
| ===Transition probability definition===
| |
| | |
| For any value ''n'' = 0, 1, 2, 3, ... and times indexed up to this value of ''n'': ''t''<sub>0</sub>, ''t''<sub>1</sub>, ''t''<sub>2</sub>, ... and all states recorded at these times ''i''<sub>0</sub>, ''i''<sub>1</sub>, ''i''<sub>2</sub>, ''i''<sub>3</sub>, ... it holds that
| |
| | |
| :<math>\Pr(X_{t_{n+1}} = i_{n+1} | X_{t_0} = i_0 , X_{t_1} = i_1 , \ldots, X_{t_n} = i_n ) = p_{i_n i_{n+1}}( t_{n+1} - t_n)</math>
| |
| | |
| where ''p''<sub>''ij''</sub> is the solution of the [[forward equation]] (a [[first-order differential equation]])
| |
| | |
| :<math>P'(t) = P(t) Q</math>
| |
| | |
| with initial condition P(0) is the [[identity matrix]].
| |
| | |
| ==Properties==
| |
| | |
| ===Irreducibility===
| |
| | |
| The state space ''S'' can be [[partition of a set|partitioned]] into communicating classes. ''i'' and ''j'' are said to communicate (and therefore be in the same communicating class) if it is possible to get to state ''j'' from state ''i'', that is if | |
| :<math>\operatorname{Pr}_i(X_t=j \text{ for some } t \geq 0)>0.</math>
| |
| | |
| A CTMC is irreducible if there is a single communicating class.<ref>{{cite doi|10.1017/CBO9780511810633.003}}</ref><ref name="norris1" />
| |
| | |
| ===Recurrence and transience===
| |
| | |
| A state ''i'' is recurrent if, starting in state ''i'', the probability the process returns unboundedly many times to the state is 1, that is<ref name="norris2" />
| |
| | |
| :<math>\operatorname{Pr}_i(\{t \geq 0 : X_t = i \} \text{ is unbounded}) = 1</math>
| |
| | |
| and a state ''i'' transient if this quantity has probability zero,<ref name="norris2" /> | |
| | |
| :<math>\operatorname{Pr}_i(\{t \geq 0 : X_t = i \} \text{ is unbounded}) = 0.</math>
| |
| | |
| If the expected return time (the time starting in state ''i'' until the next visit to state ''i'') is finite the state is positive recurrent, otherwise it is null recurrent.
| |
| | |
| ==Transient behaviour==
| |
| | |
| Write P(''t'') for the matrix with entries ''p''<sub>''ij''</sub> = P(''X''<sub>''t''</sub> = ''j'' | ''X''<sub>0</sub> = ''i''). Then the matrix P(''t'') satisfies the forward equation, a [[first-order differential equation]]
| |
| | |
| :<math>P'(t) = P(t) Q</math>
| |
| | |
| where the prime denotes differentiation with respect to ''t''. The solution to this equation is given by a [[matrix exponential]]
| |
| | |
| :<math>P(t) = e^{tQ}</math>
| |
| | |
| In a simple case such as a CTMC on the state space {1,2}. The general ''Q'' matrix for such a process is the following 2 × 2 matrix with ''α'',''β'' > 0
| |
| | |
| :<math>Q = \begin{pmatrix} -\alpha & \alpha \\ \beta & -\beta \end{pmatrix}.</math>
| |
| | |
| The above relation for forward matrix can be solved explicitly in this case to give
| |
| :<math>P(t) = \begin{pmatrix}
| |
| \frac{\beta}{\alpha+\beta} + \frac{\alpha}{\alpha+\beta}e^{-(\alpha+\beta)t} &
| |
| \frac{\alpha}{\alpha+\beta} - \frac{\alpha}{\alpha+\beta}e^{-(\alpha+\beta)t} \\
| |
| \frac{\beta}{\alpha+\beta} - \frac{\beta}{\alpha+\beta}e^{-(\alpha+\beta)t} &
| |
| \frac{\alpha}{\alpha+\beta} + \frac{\beta}{\alpha+\beta}e^{-(\alpha+\beta)t}
| |
| \end{pmatrix}</math>
| |
| | |
| However, direct solutions are complicated to compute for larger matrices. The fact that ''Q'' is the generator for a [[semigroup]] of matrices
| |
| | |
| :<math>P(t+s) = e^{Q(t+s)} = e^{tQ} e^{sQ} = P(t) P(s)</math>
| |
| | |
| is used.
| |
| | |
| ==Stationary distribution==
| |
| | |
| The stationary distribution for an irreducible recurrent CTMC is the probability distribution to which the process converges for large values of ''t''. Observe that for the two-state process considered earlier with P(''t'') given by
| |
| | |
| :<math>P(t) = \begin{pmatrix}
| |
| \frac{\beta}{\alpha+\beta} + \frac{\alpha}{\alpha+\beta}e^{-(\alpha+\beta)t} &
| |
| \frac{\alpha}{\alpha+\beta} - \frac{\alpha}{\alpha+\beta}e^{-(\alpha+\beta)t} \\
| |
| \frac{\beta}{\alpha+\beta} - \frac{\beta}{\alpha+\beta}e^{-(\alpha+\beta)t} &
| |
| \frac{\alpha}{\alpha+\beta} + \frac{\beta}{\alpha+\beta}e^{-(\alpha+\beta)t}
| |
| \end{pmatrix}</math>
| |
| | |
| as ''t'' → ∞ the distribution tends to
| |
| | |
| :<math>P_\pi = \begin{pmatrix}
| |
| \frac{\beta}{\alpha+\beta} &
| |
| \frac{\alpha}{\alpha+\beta} \\
| |
| \frac{\beta}{\alpha+\beta} &
| |
| \frac{\alpha}{\alpha+\beta}
| |
| \end{pmatrix}</math>
| |
| | |
| Observe that each row has the same distribution as this does not depend on starting state. The row vector ''π'' may be found by solving<ref name="norris2">{{cite doi|10.1017/CBO9780511810633.005}}</ref>
| |
| | |
| :<math>\pi Q = 0.</math>
| |
| | |
| with the additional constraint that
| |
| | |
| :<math>\sum_{i \in S} \pi_i = 1.</math>
| |
| | |
| ===Example===
| |
| | |
| [[File:Financial Markov process.svg|thumb|250px|right|Directed graph representation of a continuous-time Markov chain describing the state of financial markets (note: numbers are made-up).]]
| |
| The image to the right describes a continuous-time Markov chain with state-space {Bull market, Bear market, Stagnant market} and [[transition rate matrix]]
| |
| ::<math>Q=\begin{pmatrix} | |
| -0.025 & 0.02 & 0.005 \\
| |
| 0.3 & -0.5 & 0.2 \\
| |
| 0.02 & 0.4 & -0.42
| |
| \end{pmatrix}.</math>
| |
| | |
| The stationary distribution of this chain can be found by solving ''π'' ''Q'' = 0 subject to the contraint that elements must sum to 1 to obtain
| |
| | |
| :<math>\pi = \begin{pmatrix}0.885 & 0.071 & 0.044 \end{pmatrix}.</math>
| |
| | |
| ==Hitting times==
| |
| | |
| {{Main|phase-type distribution}}
| |
| The hitting time is the time, starting in a given set of states until the chain arrives in a given state or set of states. The distribution of such a time period has a phase type distribution. The simplest such distribution is that of a single exponentially distributed transition.
| |
| | |
| ===Expected hitting times===
| |
| For a subset of states ''A'' ⊆ ''S'', the vector ''k''<sup>''A''</sup> of hitting times (where element ''k''<sup>''A''</sup><sub>''i''</sub> represents the [[expected value]], starting in state ''i'' that the chain enters one of the states in the set ''A'') is the minimal non-negative solution to<ref name="norris2" /> | |
| | |
| :<math>\begin{align}
| |
| k_i^A = 0 & \text{ for } i \in A\\
| |
| -\sum_{j \in S} q_{ij} k_j^A = 1&\text{ for } i \notin A. | |
| \end{align}</math>
| |
| | |
| ==Time reversal==
| |
| | |
| For a CTMC ''X''<sub>''t''</sub>, the time-reversed process is defined to be <math>\scriptstyle \hat X_t = X_{T-t}</math>.
| |
| By [[Kelly's lemma]] this process has the same stationary distribution as the forward process.
| |
| | |
| A chain is said to be reversible if the reversed process is the same as the forward process. [[Kolmogorov's criterion]] states that the necessary and sufficient condition for a process to be reversible is that the product of transition rates around a closed loop must be the same in both directions.
| |
| | |
| ==Embedded Markov chain==
| |
| <!-- Embedded Markov chain redirects here -->
| |
| One method of finding the [[stationary probability distribution]], ''π'', of an [[ergodic]] continuous-time Markov chain, ''Q'', is by first finding its embedded Markov chain (EMC). Strictly speaking, the EMC is a regular discrete-time Markov chain, sometimes referred to as a '''[[jump process]]'''. Each element of the one-step transition probability matrix of the EMC, ''S'', is denoted by ''s''<sub>''ij''</sub>, and represents the [[conditional probability]] of transitioning from state ''i'' into state ''j''. These conditional probabilities may be found by
| |
| :<math>
| |
| s_{ij} = \begin{cases}
| |
| \frac{q_{ij}}{\sum_{k \neq i} q_{ik}} & \text{if } i \neq j \\
| |
| 0 & \text{otherwise}.
| |
| \end{cases}
| |
| </math>
| |
| | |
| From this, ''S'' may be written as
| |
| | |
| :<math>S = I - \left( \operatorname{diag}(Q) \right)^{-1} Q</math>
| |
| | |
| where ''I'' is the [[identity matrix]] and diag(''Q'') is the [[diagonal matrix]] formed by selecting the [[main diagonal]] from the matrix ''Q'' and setting all other elements to zero.
| |
| | |
| To find the stationary probability distribution vector, we must next find <math>\phi</math> such that
| |
| | |
| :<math>\phi S = \phi, \, </math>
| |
| | |
| with <math>\phi</math> being a row vector, such that all elements in <math>\phi</math> are greater than 0 and [[Norm (mathematics)|<math>||\phi||_1</math>]] = 1. From this, ''π'' may be found as | |
| | |
| :<math>\pi = {-\phi (\operatorname{diag}(Q))^{-1} \over \left\| \phi (\operatorname{diag}(Q))^{-1} \right\|_1}. </math>
| |
| | |
| Note that ''S'' may be periodic, even if ''Q'' is not. Once ''π'' is found, it must be normalized to a [[unit vector]].
| |
| | |
| Another discrete-time process that may be derived from a continuous-time Markov chain is a '''δ-skeleton'''—the (discrete-time) Markov chain formed by observing ''X''(''t'') at intervals of δ units of time. The random variables ''X''(0), ''X''(δ), ''X''(2δ), ... give the sequence of states visited by the δ-skeleton.
| |
| | |
| ==Applications==
| |
| Markov chains are used to describe physical processes where a system evolves in constant time. Sometimes, rather than a single systems, they are applied to an ensemble of identical, independent systems, and the probabilities are used to find how many members of the ensemble are in a given state. A [[master equation]] treatment is often used to analyse systems that evolve as Markov chains {{citation needed|date=May 2012}}, with [[System size expansion|approximations]] possible for complicated systems {{citation needed|date=May 2012}}.
| |
| | |
| ===Chemical reactions===
| |
| Imagine a large number ''n'' of molecules in solution in state A, each of which can undergo a chemical reaction to state B with a certain average rate. Perhaps the molecule is an enzyme, and the states refer to how it is folded. The state of any single enzyme follows a Markov chains, and since the molecules are essentially independent of each other, the number of molecules in state A or B at a time is ''n'' times the probability a given molecule is in that state.
| |
| | |
| ===Queueing theory===
| |
| {{Main|Queueing theory}}
| |
| Numerous queueing models use continuous-time Markov chains. For example, an [[M/M/1 queue]] is a CTMC on the non-negative integers where upward transitions from ''i'' to ''i'' + 1 occur at rate ''λ'' according to a [[Poisson process]] and describe job arrivals, while transitions from ''i'' to ''i'' – 1 (for ''i'' > 1) occur at rate ''μ'' (job service times are exponentially distributed) and describe completed services (departures) from the queue.
| |
| | |
| ==Extensions==
| |
| | |
| A time dependent (time heterogeneous) CTMC is as above, but with the transition rate matrix a function of time Q(''t'').
| |
| | |
| ==See also==
| |
| * [[Master equation]] (physics)
| |
| * [[Semi-Markov process]]
| |
| * [[Variable-order Markov model]]
| |
| * [[Spectral expansion solution]]
| |
| * [[Matrix geometric solution method]]
| |
| | |
| ==References==
| |
| <references/>
| |
| * {{cite book|title=Introduction to the Numerical Solution of Markov Chains|author=William J. Stewart|year=1994|publisher=Princeton University Press|pages=17–23|isbn=0-691-03699-3}}
| |
| | |
| {{Queueing theory}}
| |
| | |
| {{DEFAULTSORT:Continuous-Time Markov Process}}
| |
| [[Category:Markov processes]]
| |
Our very personal Singapore Real Property Information - Surf this section for the varieties of flats, condos, flats and houses in Singapore. Additionally take pleasure in our particular part of well-liked condominiums and property pictures, or you possibly can do the search of your favourite properties in our extensive property classifieds.
This website is visited by many Tiong Bahru property hunters EVERYDAY! My goal is to draw local & overseas patrons who are desirous about shopping for Tiong Bahru properties & real estates. I'm committed to offering high quality service to all my clients. And with my native & overseas database, I'm all the time eager to inform & replace my patrons on the most recent and best buy properties and actual estates in Singapore. Simply simply call me or refill my Selling / Leasing Enquiry form when you find yourself ready to put up your property on the market.Your property will receive its own internet page which incorporates full property info and pictures, hyperlinks to maps, community info and / or streaming video & Virtual Excursions.
Consisting of one other 5 condominiums - The Marina Collection, Turquoise, Seascape , The Pinnacle Collection, Seven Palms and two islands, most builders bought land in Sentosa South Cove for much greater costs than the sooner Sentosa North Cove. Sentosa South Cove spans from the jap fringe of the island to the southern tip the place the seashores are found – and this is what makes it so engaging. Cafes Enterprise For Sale, Espresso Retailers Enterprise For Sale Low takeover fees beneath $15k for a completely operational dessert cafeRental @ 2K/ month negotiable Takeover Comes with- Signage - Furniture- Standing freezer & chiller- Cake show chiller- Counter chiller - Counter freezer- Waffle irons- Accessories comparable to utensil Elementary school in Singapore?
A licensed and proactive actual estate professional, Jack goes beyond simply sourcing and advertising real estate properties. Efficient and approachable, he makes the entire expertise straightforward and stress-free. D14) Eunos / Geylang / Paya Lebar Condo Widespread Room with Aircon D22) Boon Lay / Jurong / Tuas Condominium Common Room with Aircon Initially I was shocked and angry that the property agent and ex-owner did not inform us about it,' mentioned the proprietor. Mr Koh, 38, who has been an agent for six years, stated a colleague as soon as lost a sale as a result of he came clear. I would additionally go to the temples to wish and seek steerage within the sale. Builders typically bid at greater and higher prices in a bullish market, resulting in property agent in singapore prices increasing in tandem. Previous Property Auction Lists
Companies Business For Sale, Postal & Communication Enterprise For Sale, Publish Offices Enterprise For Sale Funding Enterprise For Sale, Training Business For Sale, Well being Care Business For Sale Providers Enterprise For Sale, Well being & Beauty Business For Sale, Hair & Magnificence Salons Business For Sale 2 years previous nail salon for takeover.- 3 pedicure seat & 1 manicure desk.- Full outfitted, stocked and ready for enterprise immediately.- Include customer information base.- Fb Web page Up to 600 Likes- Rental 1.4k-Takeover fee 12.2k + 2.8k 2 rental deposit(able to take again) (If Eating places Business For Sale, Retailers Business For Sale, Italian Restaurants Enterprise For Sale D18) Pasir Ris / Tampines Condo Master Room with Aircon Might-June Job Hunters (At the moment here in SG)
D12) Balestier / Toa Payo HDB Widespread Room with Aircon D13) Potong Pasir / Machpherson HDB Frequent Room with Aircon D16) Bedok / Higher East Coast HDB Frequent Room with Aircon D20) Ang Mo Kio / Bishan / Thomson HDB Common Room with Aircon D15) East Coast / Marine Parade HDB Widespread Room with Aircon D14) Eunos / Geylang / Paya Lebar HDB Frequent Room with Aircon D10) Holland / Tanglin HDB Common Room with Aircon D02) Chinatown / Tanjong Pagar HDB Frequent Room with Aircon D08) Farrer Park / Serangoon Highway HDB Common Room with Aircon D18) Pasir Ris / Tampines HDB Common Room with Aircon D23) Bukit Batok / Bukit Panjiang HDB Common Room with Aircon D27) Sembawang / Yishun HDB Frequent Room with Aircon D20) Ang Mo Kio / Bishan / Thomson HDB Master Room