Variational Tail Bounds for Norms of Random Vectors and Matrices

We propose a variational tail bound for norms of random vectors and matrices under moment assumptions on their one-dimensional marginals.

A simplified version of the bound that parametrizes the ``aggregating distribution'' using a certain pushforward of the Gaussian distribution is also provided.

We apply the proposed method to reproduce some of the well-known bounds on norms of Gaussian random vectors, and also obtain dimension-free tail bounds for the Euclidean norm of random vectors with arbitrary moment profiles.

Furthermore, we reproduce a dimension-free concentration inequality for sum of independent and identically distributed positive semidefinite matrices with sub-exponential marginals, and obtain a concentration inequality for the sample covariance matrix of sub-exponential random vectors.

We also obtain a tail bound for the operator norm of a random matrix series whose random coefficients may have arbitrary moment profiles.

Furthermore, we use coupling to formulate an abstraction of the proposed approach that applies more broadly.

As a corollary, we derive a PAC-Bayesian-style bound in terms of a certain combination of the KL and Rényi divergences between the prior and posterior distributions.