Suggest edit — Markov Chain

Title

Name

Note

---
title: "Markov Chain"
type: concept
related: [Markov Chain, Eigenvalue]
source: https://www.jemoka.com/posts/kbhmarkov_chain/
confidence: high
status: active
---

A Markov Chain is a chain of \(N\) states, with an \(N \times N\) transition matrix.
at each step, we are in exactly one of those states the matrix \(P_{ij}\) tells us \(P(j|i)\), the probability of going to state \(j\) given you are at state \(i\) And therefore:
\begin{equation} \sum_{j=1}^{N} P_{ij} = 1 \end{equation}
Ergotic Markov Chain a markov chain is Ergotic if&hellip;
you have a path from any one state to any other for any start state, after some time \(T_0\), the probability of being in any state at any \(T &gt; T_0\) is non-zero Every Ergotic Markov Chain has a long-term visit rate:
i.e. a steady state visitation count exists. We usually call it:
\begin{equation} \pi = \qty(\pi_{i}, \dots, \pi_{n}) \end{equation}
Computing steady state Fact:
let&rsquo;s declare that \(\pi\) is the steady state to a transition matrix \(T\); recall that the FROM states are the rows, which means that \(\pi\) has to be a row vector; \(\pi\) being a steady state makes:
\begin{equation} \pi T = \pi \end{equation}
This is a left e.v. with eigenvalue \(1\), which is the principle eigenvector of \(T\) as transition matricies always have eigenvector eigenvalue to \(1\).