Dispersion of Gaussian Sources with Memory and an Extension to Abstract Sources

We consider finite blocklength lossy compression of information sources whose components are independent but non-identically distributed.

Crucially, Gaussian sources with memory can be cast in this form.

We show that under the operational constraint of exceeding distortion $d$ with probability at most $\epsilon$, the minimum achievable rate at blocklength $n$ satisfies $R(n, d, \epsilon)=\mathbb{R}_n(d)+\sqrt{\frac{\mathbb{V}_n(d)}{n}}Q^{-1}(\epsilon)+O \left(\frac{\log n}{n}\right)$, where $Q^{-1}(\cdot)$ is the inverse $ Q$-function, while $\mathbb{R}_n(d)$ and $\mathbb{V}_n(d)$ are fundamental characteristics of the source computed using its $n$-letter joint distribution and the distortion measure, called the $n$th-order informational rate-distortion function and the source dispersion, respectively.

This result generalizes the existing dispersion result for abstract sources with i.i.d. components.

The key novel technical tool in our analysis is the point-mass product proxy measure, which enables the construction of typical sets.

This proxy generalizes the empirical distribution beyond the i.i.d. setting by preserving additivity across coordinates and facilitating a typicality analysis for sums of independent, non-identical terms.

Furthermore, for Gaussian autoregressive sources, we quantify how fast $\mathbb{R}_n(d)$ and $\mathbb{V}_n(d)$ approach their limiting values as the blocklength $n$ tends to infinity, by approximating the eigenvalues of the $n$th-letter covariance matrix.

Using these convergence results, we sharpen and extend the only known dispersion result for a source with memory, namely the scalar Gauss-Markov source, to more general Gaussian autoregressive sources with finite memory.