Fourier-Diagonalized Natural Gradients and Sobolev Mirror Descent

We study natural-gradient updates whose metric operators are diagonalized by the Fourier transform and relate them to Sobolev mirror descent.

Translation-invariant Fisher geometries and Sobolev mirror geometries share a common inverse-map structure in the spectral domain.

The Fisher metric is represented by a positive Fourier symbol, while Sobolev mirror geometry corresponds to the specific Bessel-potential symbol associated with the Sobolev norm.

When these symbols coincide, the natural-gradient and mirror-descent updates are identical; otherwise, Sobolev mirror descent provides a canonical spectral preconditioner for the Fisher inverse geometry.

This gives a mathematical lens through which spectral filtering and truncation techniques in PDE and operator learning can be viewed as natural actions of inverse metric geometry.

We introduce Spectral Natural Gradient, an FFT-based implementation of these geometric updates.