Signal Space-Transformed Expectation Propagation for Symbol Detection in ISI Channels

Iterative message passing detection based on expectation propagation (EP) has demonstrated near-optimum performance in many signal processing and communication scenarios.

The method remains feasible even for channel impulse responses (CIRs), where the optimal Bahl-Cocke-Jelinek-Raviv (BCJR) detector is infeasible.

However, significant performance degradation occurs for channels with strong inter-symbol interference (ISI), where the initial linear minimum mean square error (LMMSE) estimate is inaccurate.

We propose an EP-based detector that operates in a transformed signal space.

Specifically, instead of the conventional approach that iterates between an LMMSE estimator and a non-linear symbol-wise demapper, the proposed method iterates between a linear channel shortening filter-based estimator and a non-linear BCJR detector with reduced memory compared to the actual channel.

Additionally, we propose a deliberate mismatch between the initialized messages and the initialized covariance used in the linear estimator in the first iteration for faster convergence.

The proposed approach is evaluated for the well-known Proakis-C ISI channel and for CIRs from a wireless measurement campaign.

We demonstrate improvements of up to 6 dB at 2 bits per channel use and an improved performance-complexity trade-off over conventional EP-based detection