On importance sampling and independent Metropolis-Hastings with an unbounded weight function

Importance sampling and independent Metropolis-Hastings are among the fundamental building blocks of Monte Carlo methods.

Both require a proposal distribution that globally approximates the target distribution, and pointwise evaluation of the Radon-Nikodym derivative of the target distribution relative to the proposal, also called the weight function.

We study the bias of importance sampling and independent Metropolis-Hastings, without assuming that the weight function is bounded.

We show that the common random numbers coupling of independent Metropolis-Hastings is maximal.

Using that coupling, we derive polynomial bounds on the total variation distance of the chain to its target distribution.

We further consider bias removal techniques using couplings, and provide conditions under which the resulting unbiased estimators have finite moments, and under which their efficiency is comparable to that of importance sampling.

Experiments illustrate unbiased estimators of the inverse of a normalizing constant, estimators of nested expectations, and combination of importance sampling with robust mean estimation methods.