Marginal Stationary Distributions and Convergence Rates of Higher Order Markov Chains

For a regular higher order Markov chain, its reduced first order chain may be reducible.

When the reduced first order chain is reducible, it does not have a unique stationary distribution.

However, it has been shown that the probability distribution of the current state of a regular higher order chain converges to a unique limiting probability distribution.

In this paper, we interpret this limiting distribution as a marginal stationary distribution and prove some properties of marginal stationary distributions.

We then establish some convergence rate results for such type of convergence.

Finally, we introduce two types of marginal mixing times, which extend the notions of mixing times for first order Markov chains to higher order chains.