Category Archives: Markov Models

Markov Transition Matrix Equilibrium made simple in Python

As a refresher on Markov Models, I’ve been watching Professor Scott E. Page’s excellent videos on YouTube. He does all the computation by hand – and now I’ve written some code to perform it faster.

In video 2, we are given a transition matrix of 1 hypothetical student in a classroom transitioning between alertness and boredom:

Alert: t Bored: t
Alert: t+1  0.8  0.25
Bored: t+1  0.2  0.75

This can be represented in Python as:

The vector of students is again, a simple list:

Let’s calculate one stage:

Calling this produces:

Or, let’s try to find the equilibrium state by looping  markov_stage until the values basically stop changing (to a certain decimal place accuracy):

This produces, with an accuracy set at 2 decimal places:

I probably could have written this in Numpy – which would calculate faster using less memory (and probably has built-in functions for the vector-matrix row multiplication), but it was fun just doing this. I’ll try and extend the markov_equilibrium  to give some more detailed stats such as the “churn” as mentioned by Prof. Page.