Markov Matrices

Each day for a mid-morning snack I either have an apple or a banana. If I have an apple on one day then the probability that I have an apple on the next day is $\frac 7 {10}$, otherwise I have a banana.  If I have a banana on one day then the probablity that I have a banana on the next day is $\frac 4 {10}$.

This information could be displayed as a diagram:

Work out the probability that, if I have an apple on Monday, I have:

1. An apple on Wednesday
2. A banana on Wednesday

Hint: There are two different ways you can go from an apple on Monday to an apple on Wednesday, and similarly for going from an apple on Monday to a banana on Wednesday.

The information could also be displayed as a matrix (sometimes called a Transition matrix).

$${\bf M} = \begin{pmatrix} 0.7 & 0.6 \\ 0.3 & 0.4 \end{pmatrix}$$

You can use the transition matrix to find probabilities.  If you start on an apple you can say that your initial state is $\begin{pmatrix} 1 \\ 0\end{pmatrix}$.  You can then find the probability of having an apple or banana next time by calculating:

$$\begin{pmatrix} 0.7 & 0.6 \\ 0.3 & 0.4 \end{pmatrix}\begin{pmatrix} 1 \\ 0\end{pmatrix} = \begin{pmatrix} 0.7 \\ 0.3\end{pmatrix}$$

Calculate ${\bf M}^2$.  What do you notice about the first column of your matrix?

Calculate ${\bf M}^3$ and ${\bf M}^4$.  What do you think will happen to ${\bf M}^n$ as $n \to \infty$?

On average, what proportion of the days will I choose an apple?

You may find these Matrix Power calculators useful:

• Matrix Power Calculator (decimal version)
• Matrix Power Calculator (fraction version)

Now, instead of just apples and bananas I can choose a cake (or clementine if you want to be healthy!).  The diagram below shows the new probabilities.

Write down a transition matrix for this situation.

Use your matrix to work out the probabilities that I have an apple, banana or cake on Friday given that I had an apple on Monday.

On average, what proportion of the days will I eat cake?

There are more matrix problems in this feature.

NOTES AND BACKGROUND

A Markov Chain is a model showing a sequence of events where the probability of an event happening depends only on the current state (this is sometimes called the memorylessness property).

Markov chains have many applications, including Quantum Mechanics, Population Dynamics, Data Compression, Stock Price modelling and ranking the relevance of web pages.