Tag Archives: MCMC

Monte Carlo methods in TensorFlow

The Markov Chain Monte Carlo (MCMC) is a sampling method to sample from a probability distribution when direct sampling is not feasible.

The implementation of Monte Carlo in the TensorFlow Probability package included sample to run the Hamiltonian MCMC, which is a variation with input from the Hamiltonian dynamics to avoid slow exploration of state space.

While running the samples, my Python reported errors on importing tensorflow_probability:tfp1

The problem is resolved by uninstalling and re-installing the tensorflow-estimator package:

pip uninstall tensorflow_estimator
pip install tensorflow_estimator

Finally the samples run fine with expected results.


The results from the sample run.

Using Nelder-Mead to solve for Markov chain stationary distribution

Taking the same problem from previous installments, another approach using the Nelder-Mead algorithm is tried and successfully obtained the same answer. After formulating the equation for passing to the Nelder-Mead program in TI Nspire, which required the same algebraic manipulation as in the previous linsolve() approach, another more intuitive formula is tried. The second one directly calculates matrix in the minimization equation and therefore render the algebraic manipulation unnecessary.

The first setting is as below.

While the second setting that directly calculates on the matrix is shown below.

Finding stationary distribution in Markov chain with TI Nspire

For Markov chain calculation with a transition matrix P:

attempt to find the stationary distribution using TI Nspire solve function apparently is not appropriate:

But no worry, the question can be solved using the linsolve function because this problem can be broken down into a series of linear equations by the properties of stationary distribution of Markov chain. In TI Nspire, a pop-up menu style wizard will guide through the linear solver:

When finished setting up the template, the equations can be entered directly in the Calculator screen.markov_stationarydist4

A shortcut to this in the TI Nspire can be referenced here as a one-liner making use of matrices operations.