Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
Abstract
Selective State-Space Models (SSMs) such as Mamba have become central to long-sequence modeling.
Still, their stability is poorly understood: their state-space coefficients are modulated online by a token-dependent gating signal, making the recurrence neither linear time-invariant nor classically nonlinear.
We study continuous-time selective SSMs through passivity, dissipativity, and Input-to-State Stability (ISS), explicitly separating the selection signal $x(\cdot)$ from the driving input $u(\cdot)$.
We obtain four results: exponential forgetting under strict dissipativity; a canonical $\mathrm{AUC}_{\mathrm{loc}}$ quadratic storage for the frozen-selection subsystem that accommodates discontinuous gating; a parametric LMI together with universal kernel constraints and "irreversible forgetting" under universal quadratic storage; and sufficient conditions for global ISS uniformly over admissible selection schedules.
We then bridge to practice by deriving a sampled block LMI for the Mamba selective-scan core, which is used as a differentiable training-time regularizer.
Across seven standard time-series datasets and four prediction horizons, the regularizer reduces sampled Mamba-core LMI violations by roughly $92\%$ in $28/28$ pairs at a clean-MSE cost of less than $0.018\%$.
It improves internal Mamba passivity and state-norm diagnostics under injected perturbations.
Our results turn classical control-theoretic tools into verifiable structural and training criteria for selective SSMs, while honestly scoping which guarantees transfer to a deep selective-scan architecture.
이 뉴스, 어떠셨어요?
한 번의 탭으로 반응을 남겨요 · 로그인 불필요