Non-vanishing of multiple correlation sequences

We resolve in the negative a conjecture of Frantzikinakis and Kuca concerning the vanishing of multiple correlation sequences in nilsystems.

Specifically, we prove the existence of an ergodic $3$-step nilsystem $(G/\Gamma, \mu_{G/\Gamma}, R_\alpha)$ and bounded functions $f_0, f_1, f_2 \in L^\infty(\mu_{G/\Gamma})$ orthogonal to the Conze--Lesigne factor $L^2(G/G_3\Gamma)$, whose associated multiple correlation sequence $$a(n) = \int_{G/\Gamma} f_0(x) f_1(\alpha^n x) f_2(\alpha^{2n} x) \, d\mu_{G/\Gamma}(x)$$ does not decay to zero.

The same counterexample also refutes another conjecture of Frantzikinakis and Kuca and a conjecture of Leibman.

To construct this counterexample, we develop a framework for Fourier analysis on $G/\Gamma$ where $G$ is the free $3$-step nilpotent Lie group on $4$ generators, a methodology that extends naturally to general nilsystems.